124 Commits (b95bfabd422a9b458dbbbde3eda33a95ed768a3a)

Author	SHA1	Message	Date
Ed Welch	37eddabac7	chore: Revert "perf: Reduce allocations for JSON and logfmt parser (#18637)" (#19244)	4 months ago
Sophie Waldman	98b411a649	chore(linter): Fix issues flagged by updated golangci-lint version (#19206)	4 months ago
Karsten Jeschkies	adb27ff177	perf: Reduce allocations for JSON and logfmt parser (#18637) ... **What this PR does / why we need it**: The JSON and logfmt parsers suffer from a high number of allocations when the parsed labels have a high cardinality. This change reduces the allocations quite a bit by copying the labels bytes to a byte buffer. Note: The benchmarks ran without a validation step that calls `LabelResult.Labels()` as that method uses a lot of allocation but is used only once in production code.	4 months ago
Sandeep Sukhani	d379de501f	fix: while validating logql expression, detect and validate expression in label_replace (#18470)	6 months ago
Karsten Jeschkies	c2573fea5a	refactor(stringlabels): Support stringlabels in util, engine and logql. (#17998) ... This is a next step of support Prometheus `stringlabels` implementation in Loki. It adds support in the `util`, `engine` and `logql` packages. The tests should compile with build tag `stringlabels`.	7 months ago
Charles Korn	545fb1597c	refactor(stringlabels): Support stringlabels in `logql/log` package (#17838) ... This is another step towards supporting Prometheus' stringlabels implementation in Loki. It adds support in the `logql/log` package. Part of https://github.com/grafana/loki/issues/17122 The tests should now compile and pass with `-tags stringlabels`. --- Co-authored-by: Karsten Jeschkies <karsten.jeschkies@grafana.com>	7 months ago
Trevor Whitney	3d2286a840	chore: implement multi-variant extractor that can return multiple samples (#17149)	8 months ago
Trevor Whitney	577d501839	refactor: extractors can return multiple samples (#17064) ... Signed-off-by: Trevor Whitney <trevorjwhitney@gmail.com>	8 months ago
Karsten Jeschkies	a864f8b986	refactor(stringlabels): Support stringlabels in log.StreamPipeline (#17216)	9 months ago
Christian Haudum	0a3230f457	chore(ast): Update WalkFn signature to return bool (#16989) ... The boolean return value indicates whether walking/traversing should be continued with child nodes. This PR does not change any existing traversal behaviour. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	9 months ago
Trevor Whitney	507d269832	feat: return the jsonpath of json detected fields (#16861)	10 months ago
Paul Rogers	27431b7e7e	chore: Linting update for new golangci (#16572)	10 months ago
Trevor Whitney	40d7b979a4	chore: implement multi-variant queries (#16426)	11 months ago
Trevor Whitney	5b16c0be7d	refactor: allow sample expressions to have multiple extractors (#16224)	11 months ago
Trevor Whitney	417d0a5252	chore: add multi-variant query support to LogQL (#16196)	11 months ago
Christian Haudum	1187a73667	chore: LogQL syntax and AST simplification and cosistency (#16028) ... This PR aims for making the grammar file (syntax.y) for the LogQL syntax more readable and the AST and its expressions more consistent in naming. You could say these are cosmetics, and yes they are, but if they can help developers better and faster understand the code it's a win. Personally I found the grammar yacc file intimidating until I understood that it was way more complicated that it needed to be. I hope it enables future additions to the syntax to be easier to implement. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	11 months ago
Christian Haudum	bebd1532b7	ci: Fix `gochecksumtype` linter error (#16096) ... Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	11 months ago
Christian Haudum	3888866616	fix(engine): Fix `Walk()` function implementation on various `Expr` implementations (#16033) ... The `Walk(f WalkFn)` implementation expects to first visit the current node and then invoke `Walk(f)` on all its children if they are not `nil`. This also fixes the `checkIntervalLimit(syntax.SampleExpr, time.Duration)` function, which did not visit the expression if it was a `*ConcatSampleExpr`. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	11 months ago
Sven Grossmann	9182addea6	fix: prevent fast ascii comparison if char is not letter (#15774)	12 months ago
Trevor Whitney	17f197206d	fix: allow comparison against zero bytes (#15217) ... Co-authored-by: George Robinson <george.robinson@grafana.com>	1 year ago
Cyril Tovena	90c5dbf0d3	chore: Revert syntax module (#14831)	1 year ago
Cyril Tovena	28c9645d10	chore: Introduce logql/syntax go module (#14790)	1 year ago
Karsten Jeschkies	7b53f20f70	feat: Introduce shardable probabilistic topk for instant queries. (#14243) ... Signed-off-by: Callum Styan <callumstyan@gmail.com> Co-authored-by: Callum Styan <callumstyan@gmail.com>	1 year ago
Trevor Whitney	b5462b6639	chore: remove copied loop vars (#14383)	1 year ago
Paul Rogers	04f87ca933	chore: Lint in prep for golangci compatible with Go 1.23 (#14289)	1 year ago
Christian Haudum	14b468b236	chore: Add `has_labelfilter_before_parser` to metrics.go log line (#14244) ... The value of this field indicates that the query could potentially benefit from bloom filters on structured metadata, if the queried field is a structured metadata field. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	1 year ago
Robert Fratto	53cfef3bca	chore(storage/bloom/v1): add ExtractTestableLabelMatchers (#14119) ... Signed-off-by: Robert Fratto <robertfratto@gmail.com>	1 year ago
Paul Rogers	18cef217fc	chore: Linting updates in prep for Go 1.23 (#14071)	1 year ago
Johan Lindvall	40f4f14791	fix: Improve execution speed for queries with label filters (#13922) ... Co-authored-by: Travis Patterson <travis.patterson@grafana.com>	1 year ago
Marco Pracucci	b88583da7d	feat: upgrade prometheus (#13671) ... Signed-off-by: Marco Pracucci <marco@pracucci.com> Signed-off-by: Vladyslav Diachenko <vlad.diachenko@grafana.com> Co-authored-by: Vladyslav Diachenko <vlad.diachenko@grafana.com>	1 year ago
jackyin	6dd6b65139	fix: ast left cycular reference result in oom (#13501) ... Co-authored-by: Travis Patterson <travis.patterson@grafana.com>	1 year ago
Ashwanth	65697676e6	chore: do not shard log queries with empty filter (#13568)	2 years ago
Sven Grossmann	9981e9e40d	fix(orFilters): fix multiple or filters would get wrong filtertype (#13169)	2 years ago
benclive	6df81db978	feat: Support negative numbers in LogQL (#13091)	2 years ago
Karsten Jeschkies	f66172eed1	feat: improve performance of `first_over_time` and `last_over_time` queries by sharding them (#11605) ... Signed-off-by: Callum Styan <callumstyan@gmail.com> Co-authored-by: Callum Styan <callumstyan@gmail.com>	2 years ago
Kaviraj Kanagaraj	e9b6ce9de4	fix(logql): Linefilter behaviour with chained `or` (#12463) ... Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com>	2 years ago
Ed Welch	4c88be0ef2	chore: update loki modules for 3.0 release (#12433) ... Signed-off-by: Edward Welch <edward.welch@grafana.com>	2 years ago
Anton Kolesnikov	36c703dae8	feat: add pattern match line filter (#12398)	2 years ago
Anton Kolesnikov	cc941fe42a	chore: refactor line filter MatchType (#12388)	2 years ago
Callum Styan	ab17a50b90	fix: don't shard `quantile_over_time` if it's not the top level aggregation for a query (#12326) ... Signed-off-by: Callum Styan <callumstyan@gmail.com>	2 years ago
Travis Patterson	1d99b06514	fix: explicitly serialize and deserialize noop label filters (#12124)	2 years ago
Salva Corts	4fa5148eb5	refactor: Pass query plan down to bloom gateway (#12037)	2 years ago
Karsten Jeschkies	ddaa497554	Prettify concat and downstream expressions. (#11877) ... **What this PR does / why we need it**: Prettifying `ConcatSampleExpr` et al will simplify debugging the shard mapper. **Checklist** - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [ ] Documentation added - [x] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](d10549e3ec) - [ ] If the change is deprecating or removing a configuration option, update the `deprecated-config.yaml` and `deleted-config.yaml` files respectively in the `tools/deprecated-config-checker` directory. [Example PR](0d4416a4b0)	2 years ago
Poyzan	6d66ea382c	[Fix] OR statements not being evaluated as part of nested line filters (#11735) ... **What this PR does / why we need it**: When nester line filters were being evaluated, the `OR` statements in right expressions were omitted and only `LineFilters` were returned. Resulting in only first value being returned. It can be reproduced in stringer unit test failing with, ``` Error Trace: /Users/poyzannur/workspace/loki/pkg/logql/syntax/ast_test.go:535 Error: Not equal: expected: "{app=\"foo\"} |= \"foo\" or \"bar\" |= \"baz\" or \"bal\"" actual : "{app=\"foo\"} |= \"foo\" or \"bar\" |= \"baz\"" ``` We now return the `OR` expression as part of nested line filters. Thanks a million to @ashwanthgoli for help with debugging, and extra unit test. **Which issue(s) this PR fixes**: Fixes https://github.com/grafana/support-escalations/issues/9042 **Special notes for your reviewer**: **Checklist** - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [ ] Documentation added - [x] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](d10549e3ec) - [ ] If the change is deprecating or removing a configuration option, update the `deprecated-config.yaml` and `deleted-config.yaml` files respectively in the `tools/deprecated-config-checker` directory. [Example PR](0d4416a4b0)	2 years ago
Ed Welch	e21b525674	Loki: increase the limit for query length from 5k to 128k (#11701) ... **What this PR does / why we need it**: We originally added this limit from fuzz testing and realizing there should be some maximum limit to an allowed query size. The original limit was 5120 based on some internet searching and a best estimate of what a reasonable limit would be. We have seen use cases with queries containing a lot of filter expressions or long expanded variable names where this limit was too small. Apparently the spec does not specify a limit, and more internet searching suggests almost all browsers will handle 100k+ length urls without issue Some limit here still seems prudent however, so the new limit is now 128k. **Which issue(s) this PR fixes**: Fixes #<issue number> **Special notes for your reviewer**: **Checklist** - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](d10549e3ec) - [ ] If the change is deprecating or removing a configuration option, update the `deprecated-config.yaml` and `deleted-config.yaml` files respectively in the `tools/deprecated-config-checker` directory. [Example PR](0d4416a4b0) --------- Signed-off-by: Edward Welch <edward.welch@grafana.com>	2 years ago
Christian Haudum	62176339c4	Bloom querying: Propagate line filters from querier to bloom filter (#11492) ... In order to query the bloom filters with the search strings from the user query, we need to propagate the line filters through the different implementations of the store, namely through the GetChunks function. In order to avoid converting data structures, the line filter from the syntax tree is used as custom protobuf type, because it is passed from querier to index gateway and from index gateway to bloom gateway. So far, the line filters request have not been tokenized to n-grams. This PR adds the ability to read the n-gram length from the block schema at query time and tokenizes the line filters from the input just ahead of searching the block. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
Karsten Jeschkies	27d21067e1	Clone filterer of label filter (#11514)	2 years ago
Karsten Jeschkies	d048e93f1a	Always set matcher in label filter deserialization. (#11497) ... **What this PR does / why we need it**: Originally we would not set the matcher of a label filter if the name or value was empty. However , `post != ""` is a valid filter expression with an empty value. **Checklist** - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [ ] Documentation added - [x] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](d10549e3ec) - [ ] If the change is deprecating or removing a configuration option, update the `deprecated-config.yaml` and `deleted-config.yaml` files respectively in the `tools/deprecated-config-checker` directory. [Example PR](0d4416a4b0)	2 years ago
Christian Haudum	cd74ddf4b4	Implement default DFS traversal visitor for LogQL expressions (#11489) ... This can be used as a default visitor for use cases where you only want to visit specific expression without the needing to implement the traversal logic of each and every expression, see implementation in test case. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
Karsten Jeschkies	f67fff3eb2	Approximate `quantile_over_time` (#10417) ... **What this PR does / why we need it**: This change shards `quantile_over_time` queries using t-digest or DDSketch approximations. It can be enabled with `querier.shard_aggregations=quantile_over_time`. Outstanding - [x] Replace generic return type of `StepEvaluator` with interface `StepResult`. - [x] Send mapped query with quantile sketch expression from frontend to querier over the wire. - [x] Serialize sketches. See https://github.com/influxdata/tdigest/issues/34 - [x] Add feature flag. **Checklist** - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [x] Documentation added - [x] Tests updated - [x] `CHANGELOG.md` updated - [x] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [x] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](d10549e3ec) --------- Signed-off-by: Callum Styan <callumstyan@gmail.com> Co-authored-by: Callum Styan <callumstyan@gmail.com>	2 years ago