apitech/loki - loki - Gitea: Git with a cup of tea

Commit Graph

Author	SHA1	Message	Date
Karsten Jeschkies	f66172eed1	feat: improve performance of `first_over_time` and `last_over_time` queries by sharding them (#11605 ) Signed-off-by: Callum Styan <callumstyan@gmail.com> Co-authored-by: Callum Styan <callumstyan@gmail.com>	2 years ago
Ed Welch	4c88be0ef2	chore: update loki modules for 3.0 release (#12433 ) Signed-off-by: Edward Welch <edward.welch@grafana.com>	2 years ago
Karsten Jeschkies	88e957af58	Skip optimization for DownstreamSampleExpr (#11651 ) This fixes a four year old bug. The `optimizeSampleExpr` was switching on the wrong `DownstreamSampleExpr` type. This bug only surfaced because of the recent fix how binop expressions are executed.	2 years ago
Karsten Jeschkies	f67fff3eb2	Approximate `quantile_over_time` (#10417 ) What this PR does / why we need it: This change shards `quantile_over_time` queries using t-digest or DDSketch approximations. It can be enabled with `querier.shard_aggregations=quantile_over_time`. Outstanding - [x] Replace generic return type of `StepEvaluator` with interface `StepResult`. - [x] Send mapped query with quantile sketch expression from frontend to querier over the wire. - [x] Serialize sketches. See https://github.com/influxdata/tdigest/issues/34 - [x] Add feature flag. Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [x] Documentation added - [x] Tests updated - [x] `CHANGELOG.md` updated - [x] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [x] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) --------- Signed-off-by: Callum Styan <callumstyan@gmail.com> Co-authored-by: Callum Styan <callumstyan@gmail.com>	2 years ago
Karsten Jeschkies	1c56aa91ef	Check switches on syntax.Expr for exhaustiveness. (#11113 ) What this PR does / why we need it: In preparation to serlialize the LogQL AST as JSON we are declaring the `syntax.Expr` as a "sum type" and test all switch statements on it to be exhaustive. Special notes for your reviewer: Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) - [ ] If the change is deprecating or removing a configuration option, update the `deprecated-config.yaml` and `deleted-config.yaml` files respectively in the `tools/deprecated-config-checker` directory. [Example PR](`0d4416a4b0`)	2 years ago
Ashwanth	f9fc1df4c0	logfmt: add --strict flag to enable strict parsing, perform nostrict parsing by default (#9626 ) What this PR does / why we need it: Introduces `--strict` flag for logfmt parser to enable strict parsing. With strict parsing enabled, logfmt parser stops scanning the log line and returns early with an error if it encounters any poorly formatted key=value pair. This pr updates the parser to perform non-strict parsing by default where it skips over the invalid tokens. This offers the flexibility to parse semi-structured log lines. Also adds `--keep-empty` flag to retain standalone keys(with empty value) as labels with value set to empty string Special notes for your reviewer: Checklist - [X] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [X] Documentation added - [X] Tests updated - [x] `CHANGELOG.md` updated - [x] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` --------- Co-authored-by: Michel Hollands <42814411+MichelHollands@users.noreply.github.com>	3 years ago
Bayan Taani	68a9fd6c0a	Add logfmt selective label extraction (#6675 ) <!-- Thanks for sending a pull request! Before submitting: 1. Read our CONTRIBUTING.md guide 2. Name your PR as `<Feature Area>: Describe your change`. a. Do not end the title with punctuation. It will be added in the changelog. b. Start with an imperative verb. Example: Fix the latency between System A and System B. c. Use sentence case, not title case. d. Use a complete phrase or sentence. The PR title will appear in a changelog, so help other people understand what your change will be. 3. Rebase your PR if it gets out of sync with main --> What this PR does / why we need it: This PR introduces extracting labels from a log line in `logfmt`, with an extra option to rename them. For example, this query: ``` {app=”foo”} \| logfmt msg,status ``` will extract the labels `msg` and `status` from the following logfmt line: ``` level=error ts=2021-02-12T19:18:10.037940878Z caller=client.go:294 component=client host=observability-loki-gateway msg="final error sending batch" status=400 error="server returned HTTP status 400 Bad Request (400): entry with timestamp 2021-02-12 19:18:08.917452 +0000 UTC ignored, reason: 'entry out of order' for stream..." ``` With the following results: ``` msg="final error sending batch" status=”400” ``` -------------- Another possible scenario with label renaming: ``` {app=”foo”} \| logfmt message="msg", status ``` That produces the following results: ``` message="final error sending batch" status=”400” ``` Which issue(s) this PR fixes: Fixes #3355 Special notes for your reviewer: <!-- Note about CHANGELOG entries, if a change adds: * an important feature * fixes an issue present in a previous release, * causes a change in operation that would be useful for an operator of Loki to know then please add a CHANGELOG entry. For documentation changes, build changes, simple fixes etc please skip this step. We are attempting to curate a changelog of the most relevant and important changes to be easier to ingest by end users of Loki. Note about the upgrade guide, if this changes: * default configuration values * metric names or label names * changes existing log lines such as the metrics.go query output line * configuration parameters * anything to do with any API * any other change that would require special attention or extra steps to upgrade Please document clearly what changed AND what needs to be done in the upgrade guide. --> Checklist - [x] Documentation added - [x] Tests updated - [x] Is this an important fix or new feature? Add an entry in the `CHANGELOG.md`. - [x] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` --------- Signed-off-by: Christian Haudum <christian.haudum@gmail.com> Co-authored-by: Christian Haudum <christian.haudum@gmail.com>	3 years ago
Danny Kopping	a649e0d7c4	Naïve fix for optimize bug with `line_format` and `json` expression parser (#6375 ) * Naïve fix Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * CHANGELOG Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Update CHANGELOG.md Co-authored-by: Christian Haudum <christian.haudum@gmail.com> Co-authored-by: Christian Haudum <christian.haudum@gmail.com>	4 years ago
Cyril Tovena	79f0e349f5	Refactor LogQL syntax into its own package (#5539 ) * Refactor LogQL syntax into its own package Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Merge upstream Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Update pkg/loghttp/series.go Co-authored-by: Susana Ferreira <ssncferreira@gmail.com> * lint Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * lint Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> Co-authored-by: Susana Ferreira <ssncferreira@gmail.com>	4 years ago
Garrett	a046d79261	make LogQL syntax scope from private to public (#4446 ) * make LogQL syntax scope from private to public The follow-up for https://github.com/grafana/loki/pull/3991, it makes using LogQL syntax outside possible, it also makes LogRange implements Expr interface. * make constructor also public * Revert "make constructor also public" This reverts commit `ab9d0a942c`.	4 years ago
Periklis Tsirakidis	cb337c3b71	Add LogQL AST walker (#3991 ) * Make expressions public scoped * Add walkable implementation * Use walkable in logql/optimize	5 years ago
Cyril Tovena	eff4ff1231	Improve matchers validations. (#3564 ) 1- It currently 500. 2- Happens too late, at ingester level and querier. (better at the frontend) 3- Improve API surface level, we don't need to expose this validation for anything else than ParseLogSelector. see #3216 Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago
Sandeep Sukhani	b152bc2cf3	check for stream selectors to have atleast one equality matcher (#3216 ) * check for stream selectors to have atleast one equality matcher * move equality matchers check to logql	5 years ago
Cyril Tovena	c971e8e65a	Optimizes SampleExpr to remove unnecessary line_format. (#3065 ) * Optimizes SampleExpr to remove unnessary line_format. When doing count_over_time and rate(), it doesn't make sense to have a line_format since you're counting line, you don't really care about what the line contains. At the same time this set the foundation of for further optimization, like parsing only required labels during label parsing. I've also found a bug where `topk by(foo) (1...` where not correctly parsed. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * For all range vector operations. Except for bytes one which count bytes in lines. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Update pkg/logql/optimize_test.go Co-authored-by: Owen Diehl <ow.diehl@gmail.com> Co-authored-by: Owen Diehl <ow.diehl@gmail.com>	5 years ago

14 Commits (bbbd82bc73322d662ba81efeda3884efcdc09708)