2762 Commits (1f653f02ed40d8d21e3fad01c0aaf5a98b21faf7)

Author	SHA1	Message	Date
Christian Haudum	9af191f3d4	feat(bloom-gw): Add `metrics.go` style log line to bloom gateway `FilterChunks` call (#12354) ... This PR adds a "metrics.go" style log line to the bloom gateway that contains latencies for queueing, processing, post-processing, as well as number of chunks/series requested/filtered. This log line should give operators a better understanding where time is spent for individual requests. Example log line: ``` ts=2024-03-26T13:16:11.869619964Z caller=spanlogger.go:109 component=bloom-gateway tenant=XXX method=bloomgateway.FilterChunkRefs user=XXX traceID=6239d49e1e88f88645bd7f1f6f1b85c8 level=info status=success tasks=1 series_requested=35 series_filtered=31 chunks_requested=103 chunks_filtered=93 queue_time=4.108128936s metas_fetch_time=22.040408ms blocks_fetch_time=1.284575ms processing_time=30.215863002s post_processing_time=16.084µs ``` Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	1 year ago
Salva Corts	5c6a031c08	fix(blooms): Not filters should always match (#12358)	1 year ago
Ashwanth	64c7812015	fix(codec): inject disable wrappers in ctx (#12352)	1 year ago
Owen Diehl	a36483b731	feat(blooms): bloom integration in query planning (#12208) ... Signed-off-by: Owen Diehl <ow.diehl@gmail.com>	1 year ago
Ashwanth	774b01dac0	fix(ruler): pass noop analyseRules to rules manager (#12353)	1 year ago
Travis Patterson	04398a1562	fix: log state of 'disable pipeline wrappers' (#12345)	1 year ago
Christian Haudum	c74d0eb107	fix(bloom-gateway): Close all block queriers in case of error (#12338) ... This PR fixes the logic for closing block queriers retrieved from the `store.FetchBlocks()` call in case of an error. Since `concurrency.ForEach()` returns on first errors, it happened that the remaining block queriers did not get closed. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	1 year ago
Travis Patterson	c8d2c32002	feat: Introduce special header that tells Loki not to modify query results (#12327)	1 year ago
Callum Styan	ab17a50b90	fix: don't shard `quantile_over_time` if it's not the top level aggregation for a query (#12326) ... Signed-off-by: Callum Styan <callumstyan@gmail.com>	1 year ago
renovate[bot]	6665685888	fix(deps): update golang.org/x/exp digest to a685a6e (main) (#11155) ... Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> Co-authored-by: Michel Hollands <42814411+MichelHollands@users.noreply.github.com>	1 year ago
Salva Corts	e4a5733ada	fix(blooms): Extract only line filters before line format expressions (#12334)	1 year ago
Dylan Guedes	51c54adc4f	feat: Memcached: Add TLS support (#12318)	1 year ago
Callum Styan	611cb01b70	fix: quantile sharding; fix bug where sharding would always happen (#12283) ... Signed-off-by: Callum Styan <callumstyan@gmail.com>	1 year ago
Christian Haudum	a54f2dba7a	perf(blooms): Optimize memory usage for blocks that have no compression (#12316) ... Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	1 year ago
Travis Patterson	d175184fe5	chore: remove shards from wrapper signature (#12322)	1 year ago
Travis Patterson	1d99b06514	fix: explicitly serialize and deserialize noop label filters (#12124)	1 year ago
Travis Patterson	fed7566ba9	fix: Pass query to pipeline wrappers from the AST (#12305)	1 year ago
Christian Haudum	07fab4b118	fix(bloomstore): Fix race conditions (#12320) ... Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	1 year ago
Ashwanth	3b5ed8c3ec	chore: pass shard count to injected pipelines and extractors (#12213)	1 year ago
BiKangNing	75e031d8e8	chore: fix function names in doc strings (#12313) ... Signed-off-by: depthlending <bikangning@outlook.com>	1 year ago
Christian Haudum	863acc0152	chore(blooms): Make block query concurrency configurable (#12292) ... A concurrency of 10 per worker seems to be too high for a reasonable 8GiB memory limit on bloom gateways. So this PR makes the concurrency configurable and uses a default of 4. Memory usage of loading block pages into memory therefore being: ``` worker_concurrency x block_query_concurrency x max_block_page_size x 2 ``` With current defaults: ``` 4 x 4 x 32MiB x 2 = 1GiB ``` Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	1 year ago
Owen Diehl	0f43bf6958	fix(blooms): fix typo in pool decl (#12302)	1 year ago
Owen Diehl	fe1b357cc2	fix(blooms): leaky ref (#12300)	1 year ago
Christian Haudum	62b3f358a6	feat(bloomstore): Wire up new blocks cache (#12275) ... Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	1 year ago
Christian Haudum	eb03974086	feat(blooms): Make bloom block compression configurable (#12293) ... Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	1 year ago
Christian Haudum	2edbe1280f	fix(blooms): Correctly skip block page in case it exceeds the max page size for querying (#12297) ... Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	1 year ago
Karsten Jeschkies	243e6f813a	refactor: Pass context to usage tracker. (#12272)	1 year ago
Paulin Todev	d54e0871d4	chore(deps): Update Prometheus in Loki and Promtail (#12245) ... This PR updates the Prometheus dependency in Loki and Promtail. The PR is required so that we can also update Grafana Agent to the latest Prometheus. Unfortunately, Promtail and Loki share the same go.mod file.	1 year ago
Ashwanth	ef742c4838	fix(querytags): propagate query tags to ingesters (#12267) ... Co-authored-by: Travis Patterson <travis.patterson@grafana.com>	1 year ago
Christian Haudum	0f48ce8f18	feat(bloomstore): Introduce specialised cache for blocks (#12257) ... The blocks cache is an in-memory cache that represents the single source of truth of what blocks are available on the file system and manages their end-of-life behaviour. There is a follow-up branch to wire up the new cache, based on this branch. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	1 year ago
Travis Patterson	9e66e05e68	chore: make error labels injectable via 'Add' (#12251)	1 year ago
Travis Patterson	fcd3e3342a	fix: pass correctly formatted query tags from the ruler to the querier (#12265)	1 year ago
Karsten Jeschkies	9a2e00bc17	refactor: Replace TenantConfig method with interface. (#12260) ... **What this PR does / why we need it**: This change allows the retrieval of the `TenantConfig` or the `DefaultConfig`. It also defines an interface instead of a tenant config function which makes it a little easier to grok the code. Eventually this will enable us to introduce GEL specific configurations. **Checklist** - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [ ] Documentation added - [x] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](d10549e3ec) - [ ] If the change is deprecating or removing a configuration option, update the `deprecated-config.yaml` and `deleted-config.yaml` files respectively in the `tools/deprecated-config-checker` directory. [Example PR](0d4416a4b0)	1 year ago
Cyril Tovena	cdb934ccee	feat(blooms): Dedupe download queue items to reduce queue size (#12222)	1 year ago
Christian Haudum	4aec20f36b	perf(blooms): Recycle bloom page buffers (#12256) ... Every time when reading a bloom page into memory, a new byte slice with the size of the decoded page was allocated. This PR changes bloom page decoder to use a byte slice from the `BlockPool`. The caller (`LazyBloomIter` of the block querier) is responsible for closing the page whenever a new one is loaded, so the byte slice is put back to its pool. Additionally, this PR changes the max bucket of the `BlockPool` byte slice pool to 64MB so that pages bigger than 16MB are correctly put back to the pool, since [slices exceeding the capacity of the largest bucket are not put back](https://github.com/grafana/loki/blob/main/vendor/github.com/prometheus/prometheus/util/pool/pool.go#L74-L87). --- Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	1 year ago
Bryan Boreham	9621d5c5ee	test: Fix race condition in LogQL test (#12247)	1 year ago
Cyril Tovena	c235fbfaad	fix(bloom-compactor): Correctly disable the bloom-compactor. (#12196) ... Co-authored-by: Salva Corts <salva.corts@grafana.com>	1 year ago
Michel Hollands	4ce5fa8954	feat: update logcli so it tries to load the latest version of the schemaconfig (#11852) ... Signed-off-by: Michel Hollands <michel.hollands@gmail.com> Signed-off-by: Michel Hollands <michel.hollands@grafana.com> Co-authored-by: Dylan Guedes <djmgguedes@gmail.com>	1 year ago
Paul Rogers	0d1a3694f5	test: Don't start server until all configured (#12226)	1 year ago
Poyzan	3dd5b706aa	fix: correct use of atomic int (#12225)	1 year ago
Poyzan	b9266883a0	fix: data race in unit test (#12224)	1 year ago
Poyzan	60dcee1b34	fix: data race in seriesIterator, use atomic int (#12223)	1 year ago
Salva Corts	2544f003cf	chore: Add summary for received filters in bloom gateway (#12158)	1 year ago
Salva Corts	63c88489d0	refactor: Add RF and Tokens to Loki ring Cfg and allow overwriting docs. (#12142) ... Co-authored-by: Owen Diehl <ow.diehl@gmail.com>	1 year ago
Christian Haudum	8abfc29e44	chore(bloom-gw): Process blocks in parallel (#12203) ... While processing a single block is rather fast, processing a lot of blocks sequentially can lead to problem that single slowly processed blocks lead to high tail latency. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	1 year ago
Salva Corts	4b28f8221b	fix: Read all series in NewTSDBSeriesIter and close index file (#12211)	1 year ago
Travis Patterson	1331dc5bdf	fix: add metadata resource and source attrs to stats (#12201)	1 year ago
Sandeep Sukhani	74e94cd1c5	fix: avoid wiping out delete request list cache in queriers when compactor is down (#12210)	1 year ago
Sandeep Sukhani	8b9d8b0226	feat: Make list of otel blessed attributes configurable (#12180)	1 year ago
Cyril Tovena	19f5f3fd44	fix: Fixes compactor shutdown log (#12195)	1 year ago