153 Commits (9fb2f385529c2444be98161743aa0cd8ad420bf2)

Author	SHA1	Message	Date
Shantanu Alshi	867ce3d711	chore(level_detection): Make log level detection configurable (#14784)	7 months ago
Karsten Jeschkies	7b53f20f70	feat: Introduce shardable probabilistic topk for instant queries. (#14243) ... Signed-off-by: Callum Styan <callumstyan@gmail.com> Co-authored-by: Callum Styan <callumstyan@gmail.com>	7 months ago
Trevor Whitney	c1fde26730	feat: move metric aggregation to a per-tenant config (#14709)	7 months ago
Trevor Whitney	f2a16f43b2	feat: add app_name as a service label (#13660)	7 months ago
Ashwanth	8bca2e7608	feat(ruler): enables ruler store that uses clients from thanos-io/objstore pkg (#11713)	7 months ago
Vladyslav Diachenko	a962edba33	feat(loki): include structured_metadata size while asserting rate limit (#14571) ... Signed-off-by: Vladyslav Diachenko <vlad.diachenko@grafana.com>	7 months ago
Salva Corts	673ede16a5	feat: New bloom planning using chunk size TSDB stats (#14547)	7 months ago
Salva Corts	70508975fd	feat: Configurable list of json fields to mine patterns (#14528)	7 months ago
Salva Corts	ab5e6eaaee	feat: Extract task computing into a strategy interface (#13690)	7 months ago
Cyril Tovena	1a4436c417	feat(ingester): implement partition shuffle sharding for ingester (#14304)	8 months ago
Christian Haudum	f52f8ad540	chore: Clarify compression package (#14257) ... This PR renames "encoding" to "codec" in the compression package to remove the cognitive dissonance. It also removes the `Enc` prefix for codec identifiers, so that they adhere Go's best practice of naming conventions, e.g. `compression.EncGZIP` becomes `compression.GZIP` when used in a different package. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	8 months ago
Christian Haudum	2840d48b92	chore: Clean up unused bloom filter related code (#14183) ... This PR tries to eliminate dead code in the bloom package, mainly related to line tokenization. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	8 months ago
Christian Haudum	963779064b	chore: Move compression utilities into separate package (#14167) ... Compression tooling has been part of the `chunkenc` (chunk encoding) package in the past for legacy reasons. Since more components use this now, it's easier to keep it in a separate package. This also eliminates the confusion around "encoding", since this has been incorrectly used synonymously with "compression" in the past. --- Signed-off-by: Christian Haudum <christian.haudum@gmail.com> Co-authored-by: Robert Fratto <robertfratto@gmail.com>	8 months ago
Trevor Whitney	4f962ef7af	fix: enable service detection for otlp endoint (#14036) ... Co-authored-by: Sven Grossmann <Svennergr@gmail.com> Co-authored-by: Ashwanth <iamashwanth@gmail.com> Co-authored-by: JordanRushing <rushing.jordan@gmail.com>	9 months ago
Christian Haudum	b75eacc288	chore(blooms)!: Remove bloom compactor component (#13969) ... **What this PR does / why we need it**: This commit removes the code related to the bloom compactor which is superseded by the bloom planner and builders. A handful of CLI arguments of per-tenant settings changed their prefix from `-bloom-compactor.*` to `-bloom-build.*`. Other settings for the compactor component itself, also prefixed with `-bloom-compactor.*` were removed. See upgrade docs for further information. **Special notes for your reviewer**: ✔️ Part of https://github.com/grafana/loki/issues/13957 📔 Documentation update https://github.com/grafana/loki/pull/13965 ⚠️ Integrating the bloom planner and builder into the `backend` target is done in a follow-up PR. --- Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	9 months ago
Salva Corts	b5ac6a0258	feat: Limit to block ingestion until configured date (#13958)	9 months ago
Owen Diehl	4220737a52	feat(exporter): include boolean values in limit exporter (#13466)	11 months ago
Salva Corts	47f6ea53fc	feat(loki): Do not enforce max-query-bytes-read and max-querier-bytes-read in limited tripperware (#13406)	11 months ago
Salva Corts	f7887269f8	refactor(blooms): Limit task retries in bloom planner (#13139)	12 months ago
Salva Corts	cd64d6ddbd	refactor(blooms): Implement retry mechanisms in planner (#13064)	12 months ago
Vladyslav Diachenko	1111595179	feat: new stream count limiter (#13006) ... Signed-off-by: Vladyslav Diachenko <vlad.diachenko@grafana.com> Co-authored-by: JordanRushing <rushing.jordan@gmail.com>	1 year ago
Salva Corts	efd8f5dc1b	refactor(blooms): Add queue to bloom planner and enqueue tasks (#13005)	1 year ago
Salva Corts	3195036435	refactor(bloom planner): Compute gaps and build tasks from metas and TSDBs (#12994)	1 year ago
Sandeep Sukhani	7cc9a9386a	fix: fix parsing of default per tenant otlp config (#12836)	1 year ago
Dylan Guedes	afd9e36306	chore: Call `shardstreams.Config` by value instead of by reference (#12915) ... Use `shardstreams.Config` by value instead of by reference to fix docs generation. Our `docs-generator` tool relies on the struct address/references to assume that flags are present. Using this config by value fixes it.	1 year ago
Shantanu Alshi	6904a6520d	fix: Fix log level detection (#12651)	1 year ago
Owen Diehl	7bbd8b5087	feat(blooms): ingester aware bounded impl (#12840)	1 year ago
Owen Diehl	bc78d13d9b	feat(blooms): compute chunks once (#12664) ... Signed-off-by: Owen Diehl <ow.diehl@gmail.com>	1 year ago
Owen Diehl	eac56224b8	feat(blooms): limit bloom size during creation (#12796)	1 year ago
Func	4a05964d55	fix: Use to the proper config names in warning messages (#12114) ... Co-authored-by: Callum Styan <callumstyan@gmail.com>	1 year ago
Cyril Tovena	397aa56e15	feat: Enable log volume endpoint by default (#12628)	1 year ago
Callum Styan	ec81991f4d	feat: include the stream we failed to create in the stream limit error message (#12437) ... Signed-off-by: Callum Styan <callumstyan@gmail.com>	1 year ago
Christian Haudum	a9345d0fb6	chore(blooms): Clean up settings and make reasonable defaults (#12483) ... Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	1 year ago
Cyril Tovena	9e6e6d22e0	feat: Detect level as structured metadata by default (#12479)	1 year ago
Ed Welch	7607f04484	feat!: enable structure metadata by default. (#12453) ... Signed-off-by: Edward Welch <edward.welch@grafana.com>	1 year ago
Ed Welch	4c88be0ef2	chore: update loki modules for 3.0 release (#12433) ... Signed-off-by: Edward Welch <edward.welch@grafana.com>	1 year ago
Sandeep Sukhani	b66c343383	feat: add support for discovering and adding log levels as structured metadata (#12428)	1 year ago
Sandeep Sukhani	288c0061bf	feat: discovery of name of services emitting the logs (#12392)	1 year ago
crazeteam	3202e34729	chore: fix misorganized annotations (#12351) ... Signed-off-by: crazeteam <lilujing@outlook.com>	1 year ago
Owen Diehl	a36483b731	feat(blooms): bloom integration in query planning (#12208) ... Signed-off-by: Owen Diehl <ow.diehl@gmail.com>	1 year ago
Christian Haudum	eb03974086	feat(blooms): Make bloom block compression configurable (#12293) ... Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	1 year ago
Sandeep Sukhani	8b9d8b0226	feat: Make list of otel blessed attributes configurable (#12180)	1 year ago
Owen Diehl	8a8756aa7a	chore(blooms): honors bloom_compactor_enabled for tenants (#12190)	1 year ago
Christian Haudum	38e2d7be4a	chore(blooms): Various bloom gateway fixes/improvements (#12120) ... * Response order of `FetchBlocks` is deterministic The response keeps the order of the refs from the request. * Update reference documentation * Skip not found blocks * Unify error messages with object key in bloom client * Change default value for bloom gateway shard size to 0 * Use only 16 tokens per ring instance * Add debug log line to each request in the bloom gateway client Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	1 year ago
Kaviraj Kanagaraj	fac5997b18	feat: Support split align and caching for instant metric query results (#11814) ... Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com>	1 year ago
Owen Diehl	85f7baaeda	Blooms/integration fixes (#11979)	1 year ago
Owen Diehl	5d1798f16d	Blooms/misc fixes (#11974)	1 year ago
Ashwanth	9e7725b31b	feat(metadata): introduce a separate split interval for recent query window (#11897)	1 year ago
Salva Corts	7a95cb8663	Configurable bloom tokenizer and block settings (#11889) ... **What this PR does / why we need it**: This PR makes the max block size configurable and wires up the already existing nGramLen and nGramSkip settings. **Special notes for your reviewer**: **Checklist** - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](d10549e3ec) - [ ] If the change is deprecating or removing a configuration option, update the `deprecated-config.yaml` and `deleted-config.yaml` files respectively in the `tools/deprecated-config-checker` directory. [Example PR](0d4416a4b0)	1 year ago
Sandeep Sukhani	e625d7f1ca	otel: add support for per tenant configuration for mapping otlp data to loki format (#11143) ... **What this PR does / why we need it**: In OTEL, we pick select Resource Attributes to identify the streams and store all the other attributes as Structured Metadata, as explained here. The problem however is that the list of Resource Attributes that are picked as Stream labels are hardcoded, and there is no way to drop unwanted data. This PR adds support for configuring how data is mapped from OTEL to Loki format per tenant. It also adds support for dropping unwanted data. We decided to make the config look similar to Prometheus's relabling config to make it familiar. **Special notes for your reviewer**: Opening a draft PR to get some initial feedback. I will add documentation in a separate PR. **Checklist** - [x] Tests updated - [x] `CHANGELOG.md` updated	1 year ago