7209 Commits (4de51f99537ea20913672836d533e4f532a06143)

 

All Branches   

Search

Author	SHA1	Message	Date
Robert Fratto	4de51f9953	chore(bloomgateway): update bloom gateway to test for structured metadata blooms (#14130) ... Signed-off-by: Robert Fratto <robertfratto@gmail.com> Signed-off-by: Christian Haudum <christian.haudum@gmail.com> Co-authored-by: Christian Haudum <christian.haudum@gmail.com>	9 months ago
Sandeep Sukhani	89c4282654	chore: avoid some allocations in tsdb while reading posting offset table (#14111)	9 months ago
Sven Grossmann	63e84b476a	fix(aggregated-metrics): correctly create logfmt string (#14124)	9 months ago
Robert Fratto	53cfef3bca	chore(storage/bloom/v1): add ExtractTestableLabelMatchers (#14119) ... Signed-off-by: Robert Fratto <robertfratto@gmail.com>	9 months ago
Christian Haudum	e64124ec1a	chore(blooms): Expose series/stream metadata when iterating over blooms (#14097) ... The bloom block format stores metadata from indexing in the index part of the block (series file), which contains information, such as the fields (metadata names) that have been indexed in the bloom filter. This will be needed in the read path (bloom gateway) later on, to check whether a provided label name from the label filter expression was actually indexed or not. In this PR, the newly available metadata is used by the bloom inspect tool (located in ./tools/bloom/inspect/) to list the fields along the series, as well as the individual pages of the series that contain the actual scalable bloom filters. --- Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	9 months ago
George Robinson	18c27f9d4e	feat: revert "feat: add functions to common.libsonnet for warpstream" (#14129)	9 months ago
George Robinson	2bde071872	feat: add functions to common.libsonnet for warpstream (#14123)	9 months ago
Paul Rogers	f226b598a4	chore: Final batch of linting cleanup for Go 1.23 and newer golangci (#14120)	9 months ago
Sven Grossmann	93009d4e8c	fix(detected_fields): always return empty array as `null` (#14112)	9 months ago
Cyril Tovena	4da035b6b7	fix(kafka): Fixes writer initialization for arm32 (#14115)	9 months ago
Christian Haudum	8307c42c54	fix(blooms): Fix panic in initialisation of the bloom planner and builder (#14110) ... The targets `bloom-planner` and `bloom-builder` panic at startup on branch `main`. ```console $ make loki && ./cmd/loki/loki -target=bloom-builder CGO_ENABLED=0 go build -ldflags "-extldflags \"-static\" -s -w -X github.com/grafana/loki/v3/pkg/util/build.Branch=main -X github.com/grafana/loki/v3/pkg/util/build.Version=main-b29b4b4 -X github.com/grafana/loki/v3/pkg/util/build.Revision=b29b4b4bb -X github.com/grafana/loki/v3/pkg/util/build.BuildUser=christian@grafana1210 -X github.com/grafana/loki/v3/pkg/util/build.BuildDate=2024-09-11T08:30:11Z" -tags netgo -o cmd/loki/loki ./cmd/loki panic: runtime error: invalid memory address or nil pointer dereference [signal SIGSEGV: segmentation violation code=0x1 addr=0x10 pc=0x287b590] goroutine 1 [running]: github.com/grafana/dskit/ring.(*BasicLifecycler).GetInstanceID(...) /home/christian/sandbox/grafana/loki/vendor/github.com/grafana/dskit/ring/basic_lifecycler.go:132 github.com/grafana/loki/v3/pkg/bloombuild/builder.New({{0x6400000, 0x6400000, {0x0, 0x0}, 0x0, 0x0, 0x0, {0x5f5e100, 0x2540be400, 0xa}, ...}, ...}, ...) /home/christian/sandbox/grafana/loki/pkg/bloombuild/builder/builder.go:92 +0x470 github.com/grafana/loki/v3/pkg/loki.(*Loki).initBloomBuilder(0xc002984008) /home/christian/sandbox/grafana/loki/pkg/loki/modules.go:1750 +0x4b6 github.com/grafana/dskit/modules.(*Manager).initModule(0xc000d2e6f0, {0x7ffe36edcb45, 0xd}, 0xc0027a22f8, 0xc0013ce960) /home/christian/sandbox/grafana/loki/vendor/github.com/grafana/dskit/modules/modules.go:136 +0x1ea github.com/grafana/dskit/modules.(*Manager).InitModuleServices(0xc000d2e6f0, {0xc000dbeed0, 0x1, 0x7510c18f88e2e5ce?}) /home/christian/sandbox/grafana/loki/vendor/github.com/grafana/dskit/modules/modules.go:108 +0xe8 github.com/grafana/loki/v3/pkg/loki.(*Loki).Run(0xc002984008, {0x0?, {0x4?, 0x2?, 0x6457fc0?}}) /home/christian/sandbox/grafana/loki/pkg/loki/loki.go:497 +0x97 main.main() /home/christian/sandbox/grafana/loki/cmd/loki/main.go:129 +0x131e ```` This is because `t.indexGatewayRingManager` is not nil and therefore incorrectly tries to initialize the ring watcher, which fails. --- Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	9 months ago
Grot (@grafanabot)	4d274f091a	chore( operator): community release 0.6.2 (#13106)	9 months ago
Joao Marcal	b29b4b4bbe	chore(operator): prepare community release v0.6.2 (#14080) ... Co-authored-by: Periklis Tsirakidis <periklis@redhat.com>	9 months ago
J Stickler	795455f59c	docs: Update Release Notes for 3.1.1 (#14098)	9 months ago
J Stickler	b9a4ded98f	docs: Update Release Notes for 3.0.1 (#14099)	9 months ago
J Stickler	7a15fa241f	docs: Update Release Notes for 2.9.10 (#14100)	9 months ago
Christian Haudum	a2fbaa8e09	feat(blooms)!: Index structured metadata into blooms (#14061) ... Instead of indexing n-grams of the log line content, we index the plain values of the structured metadata keys and values. Resulting tokens are: * `name` * `chunkPrefix + name` * `value` * `chunkPrefix + value` * `name + '=' value` * `chunkPrefix + name + '=' + value` Indexed fields (metadata name) are also extracted into the series metadata. Indexed metadata values cannot be queried with the bloom gateways yet. This PR does not cleanup unused code used for ngram-tokenization, it is scope for a follow up. --- Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	9 months ago
dropbigfish	5f78784d4f	chore: fix some function names (#14020) ... Signed-off-by: dropbigfish <fillfish@foxmail.com> Co-authored-by: Christian Haudum <christian.haudum@gmail.com> Co-authored-by: Cyril Tovena <cyril.tovena@gmail.com>	9 months ago
Kason Braley	c5083c7f1f	perf(logql): Micro-optimizations for IP filter (#14072) ... Co-authored-by: Cyril Tovena <cyril.tovena@gmail.com>	9 months ago
George Robinson	4cd77b9747	chore: use quartz in metastore (#13900)	9 months ago
renovate[bot]	e9cec1d159	fix(deps): update module github.com/hashicorp/raft to v1.7.1 (#14005) ... Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>	9 months ago
renovate[bot]	7c18642521	fix(deps): update github.com/grafana/dskit digest to 931a021 (#14032) ... Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>	9 months ago
renovate[bot]	b7eecc7a69	fix(deps): update k8s.io/utils digest to 702e33f (#14033) ... Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>	9 months ago
Cyril Tovena	d178f4c7e2	feat: Implement WAL segment ingestion via Kafka with partition ring (#14043) ... Co-authored-by: George Robinson <george.robinson@grafana.com>	9 months ago
Ravishankar	55e374e85e	fix: skipping label if it contains special symbol (#14068) ... Co-authored-by: Vladyslav Diachenko <82767850+vlad-diachenko@users.noreply.github.com>	9 months ago
Vladyslav Diachenko	c65721e7ad	feat: track discarded data by usageTracker (#14081) ... Signed-off-by: Vladyslav Diachenko <vlad.diachenko@grafana.com>	9 months ago
Paul Rogers	20ac1bacd8	chore(deps): Downgrade fake-gcs-server (#14078) ... Co-authored-by: Trevor Whitney <trevorjwhitney@gmail.com>	9 months ago
Sven Grossmann	0780456662	feat(max-allowed-line-length): add config to set `max-allowed-line-length` in pattern ingester (#14070) ... Co-authored-by: Trevor Whitney <trevorjwhitney@gmail.com>	9 months ago
Trevor Whitney	8c38d462f5	fix: downgrade grpc to fix regression (#14065)	9 months ago
Periklis Tsirakidis	d02f435d3b	feat(operator): Add support for Loki OTLP limits config (#13446) ... Co-authored-by: Robert Jacob <rojacob@redhat.com>	9 months ago
Trevor Whitney	1b3ba530b8	fix: correct _extracted logic in detected fields (#14064) ... Co-authored-by: Sven Grossmann <svennergr@gmail.com>	9 months ago
Paul Rogers	18cef217fc	chore: Linting updates in prep for Go 1.23 (#14071)	9 months ago
Paul Rogers	73cbbb0f22	feat: add retries for s3 ObjectExists calls (#14062) ... **What this PR does / why we need it**: It was determined via careful examination that Loki would issue retries for S3 `GetObject` calls, but would not retry calls to `ObjectExists`. For transient issues (e.g. rate-limiting), it makes sense to retry the command to see if an object does exist. If the object is not found (meaning, a 404 return code, and a successful query of the S3 storage to show the object does not exist), the retries are not leveraged, as it is not needed.	9 months ago
Christian Haudum	5395daf898	chore(blooms)!: Introduce a new block schema (V3) (#14038) ... A new schema version V3 for bloom blocks that is incompatible with old schemas. It simplifies some parts of the code and makes the schema more extensible, laying the groundwork for indexing structured metadata fields into bloom filters. ⚠️ This PR breaks the bloom filter functionality! Until the read path is implemented, bloom blocks created with the new schema cannot be queries in the bloom gateways. --- Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	9 months ago
Romain Gaillard	84e2b78a47	docs: update template_functions.md to change argument order for `contains` (#14048)	9 months ago
renovate[bot]	bbbd82bc73	fix(deps): update module github.com/spf13/afero to v1.11.0 (#14060) ... Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>	9 months ago
Trevor Whitney	4f962ef7af	fix: enable service detection for otlp endoint (#14036) ... Co-authored-by: Sven Grossmann <Svennergr@gmail.com> Co-authored-by: Ashwanth <iamashwanth@gmail.com> Co-authored-by: JordanRushing <rushing.jordan@gmail.com>	9 months ago
Sven Grossmann	aa1ac99f4d	fix(`detected_fields`): return parsed labels when parsers are passed (#14047) ... Co-authored-by: Galen Kistler <109082771+gtk-grafana@users.noreply.github.com> Co-authored-by: Trevor Whitney <trevorjwhitney@gmail.com>	9 months ago
Periklis Tsirakidis	7ae1588200	feat(operator): Update Loki operand to v3.1.1 (#14042)	9 months ago
renovate[bot]	677d217533	fix(deps): update module github.com/ibm/go-sdk-core/v5 to v5.17.5 (#14045) ... Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>	9 months ago
renovate[bot]	7fb34b4884	fix(deps): update module github.com/baidubce/bce-sdk-go to v0.9.189 (#14044) ... Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>	9 months ago
renovate[bot]	5d61885f85	chore(deps): update terraform aws to ~> 5.65.0 (#14046) ... Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>	9 months ago
renovate[bot]	a57ad983fa	chore(deps): update prom/alertmanager docker tag to v0.27.0 (#14039) ... Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>	9 months ago
Paul Rogers	02205f480e	chore: Remove call to deprecated rand.Seed() (#14041)	9 months ago
renovate[bot]	24ad3bac21	chore(deps): update terraform google to v6 (#14040) ... Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>	9 months ago
Christian Haudum	08e61ca4db	feat(helm): Replace bloom compactor with bloom planner and builder (#14003) ... This is PR is part of the process of removing the bloom compactor and replace it with bloom planner and bloom builder. Part of https://github.com/grafana/loki/issues/13957 Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	9 months ago
renovate[bot]	51f9376847	fix(deps): update module github.com/minio/minio-go/v7 to v7.0.76 (#14006) ... Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>	9 months ago
renovate[bot]	6f491233ca	fix(deps): update aws-sdk-go-v2 monorepo (#13986) ... Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>	9 months ago
Paweł Wilczyński	4e17ad8756	docs: rename v3.0.md to v3-0.md (#14024)	9 months ago
Sandeep Sukhani	4e4174400f	fix: do not retain span logger created with index set initialized at query time (#14027)	9 months ago