apitech/loki - loki - Gitea: Git with a cup of tea

Commit Graph

Author	SHA1	Message	Date
Bayan Taani	68a9fd6c0a	Add logfmt selective label extraction (#6675 ) <!-- Thanks for sending a pull request! Before submitting: 1. Read our CONTRIBUTING.md guide 2. Name your PR as `<Feature Area>: Describe your change`. a. Do not end the title with punctuation. It will be added in the changelog. b. Start with an imperative verb. Example: Fix the latency between System A and System B. c. Use sentence case, not title case. d. Use a complete phrase or sentence. The PR title will appear in a changelog, so help other people understand what your change will be. 3. Rebase your PR if it gets out of sync with main --> What this PR does / why we need it: This PR introduces extracting labels from a log line in `logfmt`, with an extra option to rename them. For example, this query: ``` {app=”foo”} \| logfmt msg,status ``` will extract the labels `msg` and `status` from the following logfmt line: ``` level=error ts=2021-02-12T19:18:10.037940878Z caller=client.go:294 component=client host=observability-loki-gateway msg="final error sending batch" status=400 error="server returned HTTP status 400 Bad Request (400): entry with timestamp 2021-02-12 19:18:08.917452 +0000 UTC ignored, reason: 'entry out of order' for stream..." ``` With the following results: ``` msg="final error sending batch" status=”400” ``` -------------- Another possible scenario with label renaming: ``` {app=”foo”} \| logfmt message="msg", status ``` That produces the following results: ``` message="final error sending batch" status=”400” ``` Which issue(s) this PR fixes: Fixes #3355 Special notes for your reviewer: <!-- Note about CHANGELOG entries, if a change adds: * an important feature * fixes an issue present in a previous release, * causes a change in operation that would be useful for an operator of Loki to know then please add a CHANGELOG entry. For documentation changes, build changes, simple fixes etc please skip this step. We are attempting to curate a changelog of the most relevant and important changes to be easier to ingest by end users of Loki. Note about the upgrade guide, if this changes: * default configuration values * metric names or label names * changes existing log lines such as the metrics.go query output line * configuration parameters * anything to do with any API * any other change that would require special attention or extra steps to upgrade Please document clearly what changed AND what needs to be done in the upgrade guide. --> Checklist - [x] Documentation added - [x] Tests updated - [x] Is this an important fix or new feature? Add an entry in the `CHANGELOG.md`. - [x] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` --------- Signed-off-by: Christian Haudum <christian.haudum@gmail.com> Co-authored-by: Christian Haudum <christian.haudum@gmail.com>	3 years ago
Aditya C S	8df5803d9a	feat(logql): Support drop labels in logql pipeline (#7975 ) This PR introduces `drop` stage in logql pipeline. Fixes #7870, Fixes #7368	3 years ago
Christian Haudum	089ec1b05f	Fix various linter errors These changes have been generated using `make lint`. ```console $ golangci-lint --version golangci-lint has version 1.50.0 built from 704109c6 on 2022-10-04T10:25:07Z ``` Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	3 years ago
Kaviraj Kanagaraj	cbd6ec15ce	feat(logql): Supporting prettifying LogQL expressions (#7906 ) What this PR does / why we need it: Changes 1. Added `Pretty()` method to `Expr` interface. So that every expression would know how to render themselfs as pretty :) 2. Implemented `Pretty()` for every single LogQL expressions 3. Exposed `/api/v1/format_query` endpoint that takes `query` argument and returns formatted version. 4. Integerated into `logcli`. echo `'<query>' \| logcli fmt` would format the query. Why? 1. Readability, helpful in debugging and more importantly, to understand the execution flow example before ``` quantile_over_time(0.99,{container="ingress-nginx",service="hosted-grafana"}\| json\| unwrap response_latency_seconds\| __error__=""[1m]) by (cluster) ``` ``` sum(rate({job="loki", namespace="loki", cluster="loki-dev-us"} \|= "err" [5m])) + sum(rate({job="loki-dev", namespace="loki", cluster="loki-dev-eu"}\|logfmt \| level != "info" [5m])) / sum(rate({job="loki-prod", namespace="loki", cluster="loki-prod-us"} \|logfmt \| level="error"[5m])) ``` ``` label_replace(rate({job="api-server",service="a:c"}\|= "err" [5m]), "foo", "$1", "service", "(.):.") ``` after ``` quantile_over_time( 0.99, {container="ingress-nginx", service="hosted-grafana"} \| json \| unwrap response_latency_seconds \| __error__="" [1m] ) by (cluster) ``` ``` sum( rate( {job="loki", namespace="loki", cluster="loki-dev-us"} \|= "err" [5m] ) ) + sum( rate( {job="loki-dev", namespace="loki", cluster="loki-dev-eu"} \| logfmt \| level!="info" [5m] ) ) / sum( rate( {job="loki-prod", namespace="loki", cluster="loki-prod-us"} \| logfmt \| level="error" [5m] ) ) ``` ``` label_replace( rate({job="api-server", service="a:c"} \|= "err"[5m]), "foo", "$1", "service", "(.):." ) ``` You can find more examples in the `prettier_test.go` Future plans * Integerate into LogQL analyzer * Integrate into Grafana UI. Which issue(s) this PR fixes: Fixes # NA Special notes for your reviewer: This whole idea was inspired from last [PromCon lighting talk](https://youtu.be/pjkWzDVxWk4?t=24469) Checklist - [x] Reviewed the `CONTRIBUTING.md` guide - [x] Documentation added - [x] Tests updated - [x] `CHANGELOG.md` updated Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> Co-authored-by: Christian Haudum <christian.haudum@gmail.com> Co-authored-by: Christian Simon <simon@swine.de>	3 years ago

4 Commits (Alex3k-patch-3)