308 Commits (dea5d7883a05856c99e18db976ff88e25a40ac31)

Author	SHA1	Message	Date
Shantanu Alshi	d3d31f1842	chore(aggregated_metrics): Add metrics to aggreated metrics (#14986)	6 months ago
renovate[bot]	e0ac0d0efd	fix(deps): update module github.com/grpc-ecosystem/go-grpc-middleware to v2 (#15045) ... Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> Co-authored-by: Paul Rogers <paul.rogers@grafana.com>	6 months ago
renovate[bot]	33590be53e	fix(deps): update module k8s.io/apimachinery to v0.31.2 (#14959) ... Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> Co-authored-by: Paul Rogers <paul.rogers@grafana.com>	6 months ago
Cyril Tovena	90c5dbf0d3	chore: Revert syntax module (#14831)	7 months ago
Cyril Tovena	28c9645d10	chore: Introduce logql/syntax go module (#14790)	7 months ago
Karsten Jeschkies	7b53f20f70	feat: Introduce shardable probabilistic topk for instant queries. (#14243) ... Signed-off-by: Callum Styan <callumstyan@gmail.com> Co-authored-by: Callum Styan <callumstyan@gmail.com>	7 months ago
Christian Haudum	0760095a3e	chore: Add bool to metrics.go log line whether query used bloom filters (#14645) ... Better observability of which queries use bloom filters for chunk filtering. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	7 months ago
Ashwanth	8bca2e7608	feat(ruler): enables ruler store that uses clients from thanos-io/objstore pkg (#11713)	7 months ago
Callum Styan	f411a0795a	fix: fix bug in query result marshaling for invalid utf8 characters (#14585) ... Signed-off-by: Callum Styan <callumstyan@gmail.com>	7 months ago
Joao Marcal	b87224647d	feat(storage): AWS backend using thanos.io/objstore (#11221) ... Co-authored-by: Ashwanth Goli <iamashwanth@gmail.com>	7 months ago
Vladyslav Diachenko	a962edba33	feat(loki): include structured_metadata size while asserting rate limit (#14571) ... Signed-off-by: Vladyslav Diachenko <vlad.diachenko@grafana.com>	7 months ago
Salva Corts	70508975fd	feat: Configurable list of json fields to mine patterns (#14528)	7 months ago
Christian Haudum	f568dda136	chore: Improve logging of jumphash server selector (#14306) ... Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	7 months ago
Trevor Whitney	b5462b6639	chore: remove copied loop vars (#14383)	8 months ago
Ashwanth	900751c3bb	fix: report correct status code for metric and log queries in metrics.go (#12102)	8 months ago
Sven Grossmann	63e84b476a	fix(aggregated-metrics): correctly create logfmt string (#14124)	9 months ago
Cyril Tovena	d178f4c7e2	feat: Implement WAL segment ingestion via Kafka with partition ring (#14043) ... Co-authored-by: George Robinson <george.robinson@grafana.com>	9 months ago
Paul Rogers	18cef217fc	chore: Linting updates in prep for Go 1.23 (#14071)	9 months ago
Christian Haudum	b75eacc288	chore(blooms)!: Remove bloom compactor component (#13969) ... **What this PR does / why we need it**: This commit removes the code related to the bloom compactor which is superseded by the bloom planner and builders. A handful of CLI arguments of per-tenant settings changed their prefix from `-bloom-compactor.*` to `-bloom-build.*`. Other settings for the compactor component itself, also prefixed with `-bloom-compactor.*` were removed. See upgrade docs for further information. **Special notes for your reviewer**: ✔️ Part of https://github.com/grafana/loki/issues/13957 📔 Documentation update https://github.com/grafana/loki/pull/13965 ⚠️ Integrating the bloom planner and builder into the `backend` target is done in a follow-up PR. --- Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	9 months ago
Paul Rogers	8fb7b488bd	chore(deps): update dskit 20240819 (#13924)	9 months ago
Trevor Whitney	913e9f9347	feat: aggregate byte and count metrics (#13731)	9 months ago
Trevor Whitney	c6ab6b31e8	chore: remove initial metric aggregation experiment (#13729)	10 months ago
Ed Welch	0bf8df1b88	chore: treat datasample queries the same as limited queries (#13700) ... Signed-off-by: Edward Welch <edward.welch@grafana.com>	10 months ago
Christian Haudum	ce71f1cf95	feat(api)!: Fail log queries when executed on instant query endpoint (#13421) ... ### What this PR does / why we need it **Background** A log selector expression is a LogQL expression that returns logs, in contrast to a sample expressions, which returns metrics (samples). The simplest form of log selector expressions are label matchers, e.g. `{env="prod"}`. **Change** This PR changes the behaviour of Loki so that the instant query endpoint `/api/v1/query` does not allow sending a log selector expression as query any more. Instead, it returns a status code 400 (Bad Request) with the error message **"log queries are not supported as an instant query type, please change you query to a range query type"**. **Why** Previously this API endpoint allowed these types of log queries, but returned inconsistent results, which where a major cause for confusion. Returning a concise error helps the user understand that they likely selected the wrong query type in Grafana when executing the query. --- Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	11 months ago
Christian Haudum	b1adfce5d0	chore(mempool): Wait for buffer instead of returning error (#13298) ... Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	11 months ago
Christian Haudum	61a9854eb1	perf(mempool): Replace `sync.Mutex` with `sync.Once` (#13293) ... `sync.Once` uses an atomic integer as gate to check if the slab is already initialized and therefore yields a ~9% performance improvement for `get` operations. ``` goos: linux goarch: amd64 pkg: github.com/grafana/loki/v3/pkg/util/mempool cpu: 11th Gen Intel(R) Core(TM) i7-1185G7 @ 3.00GHz │ bench.old │ bench.new │ │ sec/op │ sec/op vs base │ Slab/1KB-8 138.5n ± 2% 125.1n ± 2% -9.71% (p=0.000 n=10) Slab/1MB-8 140.5n ± 3% 128.1n ± 2% -8.83% (p=0.000 n=10) Slab/128MB-8 143.0n ± 3% 130.4n ± 3% -8.81% (p=0.000 n=10) geomean 140.7n 127.8n -9.12% ``` Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	11 months ago
Owen Diehl	eb1cd4c855	perf(blooms): mempool no longer zeroes out buffers unnecessarily (#13282)	11 months ago
Owen Diehl	d36e1d580a	fix(blooms): improves mempool metrics (#13283)	11 months ago
Christian Haudum	4117b6ca98	perf: Re-introduce fixed size memory pool for bloom querier (#13172) ... This PR re-introduces the fixed size memory pool that was originally introduced with #13039 but reverted with #13162 Additionally, it removes the package-global variable to store the type of allocator that should be used. Instead, the allocator type is passed during the bloom store creation. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	11 months ago
Trevor Whitney	467eb1bb1b	feat: collect and serve pre-aggregated bytes and counts (#13020) ... Co-authored-by: Cyril Tovena <cyril.tovena@gmail.com>	11 months ago
JordanRushing	e7689b248d	feat: add recalculateOwnedStreams to check stream ownership if the ring is changed (#13103) ... Signed-off-by: JordanRushing <rushing.jordan@gmail.com> Signed-off-by: Vladyslav Diachenko <vlad.diachenko@grafana.com> Co-authored-by: Vladyslav Diachenko <vlad.diachenko@grafana.com> Co-authored-by: Vladyslav Diachenko <82767850+vlad-diachenko@users.noreply.github.com>	12 months ago
Callum Styan	691b174138	fix: properly return http status codes from ingester to querier for RPC function calls (#13134) ... Signed-off-by: Callum Styan <callumstyan@gmail.com>	12 months ago
Owen Diehl	41c5ee21fc	fix(regression): reverts grafana/loki#13039 to prevent use-after-free corruptions (#13162)	12 months ago
Christian Haudum	fc264310ce	perf: Introduce fixed size memory pool for bloom querier (#13039) ... This PR introduces a fixed size memory pool for bloom pages that are loaded in the block querier. The aim of a fixed size pool of `[]byte` buffers is to reduce the amount of allocations, as well as to control the maximum heap size to prevent OOMing of the bloom gateways. Also, with the current usage of a `sync.Pool`, the query parallelism (`bloom-gateway.worker-concurrency` and `bloom-gateway.block-query-concurrency`) was defined by the `bloom.max-query-page-size` (and vice versa), because the max page size could be loaded `workers * concurrency` times at the same time. Most of the time, though, smaller pages are loaded, and therefore concurrency is not optimized. With the new fixed size memory pool, this problem should be solved. The pool is divided into slabs of different sizes holding different amounts of buffers; a larger amount of small sized buffers and a smaller amount of large sized buffers. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	12 months ago
Salva Corts	3195036435	refactor(bloom planner): Compute gaps and build tasks from metas and TSDBs (#12994)	1 year ago
Owen Diehl	9c25985b97	feat(blooms): record time spent resolving shards (#12636)	1 year ago
Christian Haudum	2f24b670cb	chore: Restructure `indexgateway` component package (#12630) ... This PR moves code for the index gateway server and client components into the `pkg/indexgateway` directory, so it that the index gateway has its own top-level package like other components, such as ingester, distributor, querier, ... Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	1 year ago
Christian Haudum	58c8c61905	perf(blooms): Use jumpmap for client side sharding of the filter requests (#12470) ... **What this PR does / why we need it**: Use the same client-side sharding mechanism for bloom gateway filter requests that we use for client side sharding in the memcached client. This uses DNS discovery to resolve a list of addresses uses as buckets for the jumphash algorithm to shard GroupedChunkRefs across bloom gateway servers. The ring component is still part of the bloom gateway, but it is not used any more. **Special notes for your reviewer**: https://arxiv.org/pdf/1406.2294.pdf Signed-off-by: Christian Haudum <christian.haudum@gmail.com> Co-authored-by: Owen Diehl <ow.diehl@gmail.com>	1 year ago
Cyril Tovena	491d2513e7	feat: add log patterns ingestion and query API. (#12403) ... Signed-off-by: Edward Welch <edward.welch@grafana.com> Co-authored-by: Anton Kolesnikov <anton.e.kolesnikov@gmail.com> Co-authored-by: Sven Grossmann <svennergr@gmail.com> Co-authored-by: Edward Welch <edward.welch@grafana.com>	1 year ago
Travis Patterson	cf04ec1320	feat: Enable warnings in Loki query responses (#12425)	1 year ago
Ed Welch	4c88be0ef2	chore: update loki modules for 3.0 release (#12433) ... Signed-off-by: Edward Welch <edward.welch@grafana.com>	1 year ago
Shantanu Alshi	5190dda64b	feat(detected_labels): Initial skeleton for the API (#12390) ... Co-authored-by: Cyril Tovena <cyril.tovena@gmail.com>	1 year ago
Ashwanth	c8c8477c93	chore(stats): add pipeline_wrapper_filtered_lines (#12399)	1 year ago
crazeteam	3202e34729	chore: fix misorganized annotations (#12351) ... Signed-off-by: crazeteam <lilujing@outlook.com>	1 year ago
Trevor Whitney	5b4bfa54b6	feat(detected_fields): initial plumbing of new endpoint (#12385)	1 year ago
Owen Diehl	a36483b731	feat(blooms): bloom integration in query planning (#12208) ... Signed-off-by: Owen Diehl <ow.diehl@gmail.com>	1 year ago
Travis Patterson	c8d2c32002	feat: Introduce special header that tells Loki not to modify query results (#12327)	1 year ago
BiKangNing	75e031d8e8	chore: fix function names in doc strings (#12313) ... Signed-off-by: depthlending <bikangning@outlook.com>	1 year ago
Paulin Todev	d54e0871d4	chore(deps): Update Prometheus in Loki and Promtail (#12245) ... This PR updates the Prometheus dependency in Loki and Promtail. The PR is required so that we can also update Grafana Agent to the latest Prometheus. Unfortunately, Promtail and Loki share the same go.mod file.	1 year ago
Ashwanth	ef742c4838	fix(querytags): propagate query tags to ingesters (#12267) ... Co-authored-by: Travis Patterson <travis.patterson@grafana.com>	1 year ago