apitech/loki - loki - Gitea: Git with a cup of tea

Commit Graph

Author	SHA1	Message	Date
Trevor Whitney	1a66d2ddab	feat: ability to send query context for limit enforcement (#19900 )	1 month ago
benclive	a751c15201	chore: Fix ordering when merging frontend responses (#19565 )	2 months ago
Sophie Waldman	98b411a649	chore(linter): Fix issues flagged by updated golangci-lint version (#19206 )	3 months ago
Trevor Whitney	fdcc12da4d	feat: query persisted patterns (#17980 ) Signed-off-by: Trevor Whitney <trevorjwhitney@gmail.com>	7 months ago
Oleg Zaytsev	c8a15f451c	refactor: Use OTel tracing library (#17859 ) Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>	7 months ago
Karsten Jeschkies	d0c11a689e	feat(api): Support Parquet as a query response format. (#15408 ) What this PR does / why we need it: This changes adds support for the header `Accept: application/vnd.apacher.parquet`. If this header is set the response for metric and stream queries will response with a Parquet file. A metrics response has the columns `timestamp`, `labels` and `value`. A stream response has the columns `timestamp`, `labels` and `line`. Co-authored-by: Christian Haudum <christian.haudum@gmail.com>	1 year ago
Eng Zer Jun	062d1e9ca4	refactor: replace experimental `maps` and `slices` with stdlib (#15051 ) Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>	1 year ago
Trevor Whitney	7983f94b15	feat: detected field values (#14350 )	1 year ago
Sven Grossmann	f8d9143eea	fix(log-to-span): timestamp.Time should be called with milliseconds (#14196 )	1 year ago
Paul Rogers	18cef217fc	chore: Linting updates in prep for Go 1.23 (#14071 )	1 year ago
Trevor Whitney	b1dc0763d6	feat: ability to ignore cache for volume queries (#13945 )	1 year ago
Trevor Whitney	c6ab6b31e8	chore: remove initial metric aggregation experiment (#13729 )	1 year ago
Trevor Whitney	467eb1bb1b	feat: collect and serve pre-aggregated bytes and counts (#13020 ) Co-authored-by: Cyril Tovena <cyril.tovena@gmail.com>	2 years ago
benclive	740551bb31	fix: Correctly encode step when translating proto to http internally (#13171 )	2 years ago
Kaviraj Kanagaraj	88e545fc95	feat(cache): Add `Cache-Control: no-cache` support for Loki instant queries. (#12896 ) Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com>	2 years ago
Shantanu Alshi	e7fdeb974a	perf: Improve Detected labels API (#12816 )	2 years ago
Owen Diehl	bc78d13d9b	feat(blooms): compute chunks once (#12664 ) Signed-off-by: Owen Diehl <ow.diehl@gmail.com>	2 years ago
benclive	7b8533e435	feat: Add step param to Patterns Query API (#12703 )	2 years ago
Shantanu Alshi	587a6d20e9	feat: Detected labels from store (#12441 )	2 years ago
Trevor Whitney	f68d1f7faf	fix: make detected fields work for both json and proto (#12682 )	2 years ago
Trevor Whitney	6c33809015	feat: split detected fields queries (#12491 )	2 years ago
Cyril Tovena	491d2513e7	feat: add log patterns ingestion and query API. (#12403 ) Signed-off-by: Edward Welch <edward.welch@grafana.com> Co-authored-by: Anton Kolesnikov <anton.e.kolesnikov@gmail.com> Co-authored-by: Sven Grossmann <svennergr@gmail.com> Co-authored-by: Edward Welch <edward.welch@grafana.com>	2 years ago
Travis Patterson	cf04ec1320	feat: Enable warnings in Loki query responses (#12425 )	2 years ago
Cyril Tovena	fbb8686d03	fix: Fixes detected labels API wiring (#12442 )	2 years ago
Trevor Whitney	a53a0cc792	feat: return real data in detected fields endpoint (#12421 ) parse log lines in the querier to return data about detected fields. this is not sustainable long term, but a short term solution to validate the data we want for the frontend.	2 years ago
Ed Welch	4c88be0ef2	chore: update loki modules for 3.0 release (#12433 ) Signed-off-by: Edward Welch <edward.welch@grafana.com>	2 years ago
Callum Styan	018856c1c1	fix: fix span logging based on changes to request types timestamps (#12393 ) Signed-off-by: Callum Styan <callumstyan@gmail.com>	2 years ago
Shantanu Alshi	5190dda64b	feat(detected_labels): Initial skeleton for the API (#12390 ) Co-authored-by: Cyril Tovena <cyril.tovena@gmail.com>	2 years ago
Kaviraj Kanagaraj	7480468e0f	fix: (Bug) correct resultType when storing instant query results in cache (#12312 ) Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com>	2 years ago
Trevor Whitney	246623f7d0	fix(detected_fields): fix issues with frontend integration (#12406 ) This PRs fixes issues we found when integrating with the frontend * the `/experimental` api made it difficult to interact with using the existing datasource, so move to `v1/detected_fields` * the config flag was considered cumbersome as the only potential negative impact of the endpoint is when it is used, and nothing is currently using it * the use of an enum in the protobuf produced unexpected results in the json, so type was converted to string	2 years ago
Trevor Whitney	5b4bfa54b6	feat(detected_fields): initial plumbing of new endpoint (#12385 )	2 years ago
Ashwanth	64c7812015	fix(codec): inject disable wrappers in ctx (#12352 )	2 years ago
Owen Diehl	a36483b731	feat(blooms): bloom integration in query planning (#12208 ) Signed-off-by: Owen Diehl <ow.diehl@gmail.com>	2 years ago
Travis Patterson	c8d2c32002	feat: Introduce special header that tells Loki not to modify query results (#12327 )	2 years ago
Kaviraj Kanagaraj	ce574485cf	feat(caching): Support caching `/series` and `/labels` query results (#11539 ) What this PR does / why we need it: Add support for caching metadata queries (both series and labels). caching happens after splitting similar to other types of queries. This pr adds the following configs to enable them. ``` cache_series_results: true\|false (default false) cache_label_results: true\|false (default false) ``` And the cache backend for them can be configured using `series_results_cache` and `label_results_cache` blocks under the `query_range` section. Currently the split interval for metadata queries is fixed and defaults to 24h, this pr makes it configurable by introducing `split_metadata_queries_by_interval` Which issue(s) this PR fixes: Fixes #<issue number> Special notes for your reviewer: Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [x] Documentation added - [x] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) - [ ] If the change is deprecating or removing a configuration option, update the `deprecated-config.yaml` and `deleted-config.yaml` files respectively in the `tools/deprecated-config-checker` directory. [Example PR](`0d4416a4b0`) --------- Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> Co-authored-by: Ashwanth Goli <iamashwanth@gmail.com>	2 years ago
Danny Kopping	18ea38883f	Do not reflect label names in request metrics' "route" label (#11551 ) Which issue(s) this PR fixes: Fixes #11549	2 years ago
Karsten Jeschkies	9aae25008d	Optimize series response format by using repeated fileds. (#11498 ) What this PR does / why we need it: The Protobuf map type is encodied as a repeated field of map entries. Decoding them to a slice is much faster than decoding them into a map. Since Loki is not using the fast key check for a map we can use the slice decoding. This change also allows us to decode the JSON directly into the right protobuf struct. This doulbes the JSON decoding speed and reduces the memory pressure by ~40%. ``` › go test -bench=. -run=^$ -count=10 ./pkg/querier/queryrange > before.log › go test -bench=. -run=^$ -count=10 ./pkg/querier/queryrange > after.log › benchstat before.log after.log goos: darwin goarch: arm64 pkg: github.com/grafana/loki/pkg/querier/queryrange │ before.log │ after.log │ │ sec/op │ sec/op vs base │ ResponseMerge/mergeStreams_unlimited-10 32.36m ± 0% 32.63m ± 2% ~ (p=0.393 n=10) ResponseMerge/mergeOrderedNonOverlappingStreams_unlimited-10 1.050m ± 1% 1.080m ± 3% +2.84% (p=0.005 n=10) ResponseMerge/mergeStreams_limited-10 33.02m ± 0% 32.60m ± 1% -1.29% (p=0.004 n=10) ResponseMerge/mergeOrderedNonOverlappingStreams_limited-10 15.11m ± 0% 15.07m ± 0% ~ (p=0.075 n=10) _CodecDecodeLogs-10 4.395m ± 1% 4.364m ± 0% -0.72% (p=0.005 n=10) _CodecDecodeSamples-10 16.97m ± 0% 16.84m ± 2% -0.77% (p=0.023 n=10) _CodecDecodeSeries/application/vnd.google.protobuf-10 745.8µ ± 8% 736.8µ ± 12% ~ (p=0.739 n=10) _CodecDecodeSeries/application/json;_charset=utf-8-10 15.37m ± 1% 10.60m ± 0% -31.03% (p=0.000 n=10) _MergeResponses-10 1186.9m ± 2% 149.8m ± 1% -87.38% (p=0.000 n=10) _UnwrapSeries-10 9.399m ± 1% 4.049m ± 0% -56.92% (p=0.000 n=10) _DecodeMergeEncodeCycle-10 666.0m ± 3% 194.6m ± 0% -70.79% (p=0.000 n=10) geomean 18.87m 12.51m -33.70% │ before.log │ after.log │ │ B/op │ B/op vs base │ _CodecDecodeLogs-10 3.649Mi ± 0% 3.649Mi ± 0% ~ (p=0.364 n=10) _CodecDecodeSamples-10 18.12Mi ± 0% 18.12Mi ± 0% ~ (p=0.926 n=10) _CodecDecodeSeries/application/vnd.google.protobuf-10 7.647Mi ± 0% 7.647Mi ± 0% ~ (p=0.587 n=10) _CodecDecodeSeries/application/json;_charset=utf-8-10 27.94Mi ± 0% 16.99Mi ± 0% -39.18% (p=0.000 n=10) _MergeResponses-10 2.362Mi ± 0% 2.408Mi ± 0% +1.98% (p=0.000 n=10) _UnwrapSeries-10 19.495Mi ± 0% 8.595Mi ± 0% -55.91% (p=0.000 n=10) _DecodeMergeEncodeCycle-10 772.3Mi ± 0% 772.3Mi ± 0% ~ (p=0.912 n=10) geomean 17.50Mi 14.54Mi -16.91% │ before.log │ after.log │ │ allocs/op │ allocs/op vs base │ _CodecDecodeLogs-10 41.10k ± 0% 41.10k ± 0% ~ (p=1.000 n=10) ¹ _CodecDecodeSamples-10 411.9k ± 0% 411.9k ± 0% ~ (p=1.000 n=10) _CodecDecodeSeries/application/vnd.google.protobuf-10 32.00 ± 0% 32.00 ± 0% ~ (p=1.000 n=10) ¹ _CodecDecodeSeries/application/json;_charset=utf-8-10 304.2k ± 0% 298.1k ± 0% -2.01% (p=0.000 n=10) _MergeResponses-10 100.1k ± 0% 100.1k ± 0% -0.00% (p=0.002 n=10) _UnwrapSeries-10 201.1k ± 0% 198.0k ± 0% -1.54% (p=0.000 n=10) _DecodeMergeEncodeCycle-10 203.1k ± 0% 203.1k ± 0% ~ (p=0.621 n=10) geomean 48.95k 48.70k -0.51% ¹ all samples are equal ``` Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [x] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) - [ ] If the change is deprecating or removing a configuration option, update the `deprecated-config.yaml` and `deleted-config.yaml` files respectively in the `tools/deprecated-config-checker` directory. [Example PR](`0d4416a4b0`)	2 years ago
Karsten Jeschkies	8dde7b9d49	Support tail requests with protobuf encoding. (#11426 ) What this PR does / why we need it: The `TailRequest` would use `bytes plan` instead of `Plan plan`. The former is not well supported in gogoprotobuf. Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [x] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) - [ ] If the change is deprecating or removing a configuration option, update the `deprecated-config.yaml` and `deleted-config.yaml` files respectively in the `tools/deprecated-config-checker` directory. [Example PR](`0d4416a4b0`)	2 years ago
Salva Corts	489ac8d529	Extract results cache into new pkg (#11343 ) What this PR does / why we need it: This extracts the results cache from `queryrangebase` into its own pkg so we can reuse it in other components such as the bloom-gateway without having to import `queryrangebase`. - Most of the logic inside `pkg/querier/queryrange/queryrangebase/results_cache.go` now lives in `pkg/storage/chunk/cache/results_cache/cache.go`. - Some of the tests in `pkg/querier/queryrange/queryrangebase/results_cache.go` are moved into pkg/storage/chunk/cache/results_cache/cache_test.go. - Note that here we don't have access to the types we use in `queryrangebase` so we created a new set of mock request/response types to test with.	2 years ago
Karsten Jeschkies	5b97fcfd93	Send query plan to querier. (#11246 ) What this PR does / why we need it: Following https://github.com/grafana/loki/pull/11123 and in order to enable https://github.com/grafana/loki/pull/10417 the query frontend should send the serialized LogQL AST instead of the query string to the queriers. This enables the frontend to change the AST and inject expressions that are not expressible in LogQL. Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [x] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) - [ ] If the change is deprecating or removing a configuration option, update the `deprecated-config.yaml` and `deleted-config.yaml` files respectively in the `tools/deprecated-config-checker` directory. [Example PR](`0d4416a4b0`) --------- Signed-off-by: Callum Styan <callumstyan@gmail.com> Co-authored-by: Callum Styan <callumstyan@gmail.com>	2 years ago
Karsten Jeschkies	8d34f857bc	Propagate trace ID with HTTP gRPC request. (#11251 ) The changes in https://github.com/grafana/loki/pull/10688 did not propage the trace ID from the context. `Frontend.RoundTripGRPC` would inject the trace ID into the request. That's not done in `Frontend.Do`. This changes extends the `codec.EncodeRequest` to inject the trace ID there. This is more inline with other metadata.	2 years ago
Owen Diehl	265018b729	populate empty resonse for IndexStatsResponse and VolumeResponse (#11209 ) Fixes a panic when the underlying response struct is accessed but not populated after an out-of-bounds request is short-circuited via `NewEmptyResponse` [here](https://github.com/grafana/loki/blob/main/pkg/querier/queryrange/limits.go#L155-L164). This PR ensures the embedded types are not nil in accordance with the other variants in this function.	2 years ago
Karsten Jeschkies	05c4b77ed0	Define QueryResponse and QueryRequest protobufs. (#10956 ) What this PR does / why we need it: This will be a follow up to https://github.com/grafana/loki/pull/10688 finally switching over to protos. Hang on tight! Everything is behind the feature flag `-frontend.encoding=protobuf` which will disable the transcoding to HTTP and HTTPgRPC. Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [x] Tests updated - [x] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) --------- Signed-off-by: Callum Styan <callumstyan@gmail.com> Co-authored-by: Callum Styan <callumstyan@gmail.com> Co-authored-by: Danny Kopping <dannykopping@gmail.com>	2 years ago
Karsten Jeschkies	979530b6b9	Propagate query metrics and cache num information. (#11176 ) What this PR does / why we need it: https://github.com/grafana/loki/pull/10858 removed the extraction of the query time header on the querier side and the generation of the cache number. This change adds them back and uses the headers of the response format. Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) - [ ] If the change is deprecating or removing a configuration option, update the `deprecated-config.yaml` and `deleted-config.yaml` files respectively in the `tools/deprecated-config-checker` directory. [Example PR](`0d4416a4b0`) --------- Signed-off-by: Edward Welch <edward.welch@grafana.com> Co-authored-by: Edward Welch <edward.welch@grafana.com>	2 years ago
Karsten Jeschkies	6d95a86ae2	Extract query tags from HTTP requests. (#11147 ) What this PR does / why we need it: https://github.com/grafana/loki/pull/10858 removed the code that would inject query tags into the context of the querier. This change adds an extraction in the decode method.	2 years ago
Karsten Jeschkies	4ed3b117d2	Use newest build image 0.31.2 with golangci-lint update. (#11118 ) What this PR does / why we need it: This just updates the Loki build imaage changed in #11114. Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) - [ ] If the change is deprecating or removing a configuration option, update the `deprecated-config.yaml` and `deleted-config.yaml` files respectively in the `tools/deprecated-config-checker` directory. [Example PR](`0d4416a4b0`)	2 years ago
Karsten Jeschkies	33b7e51342	Bring back RED metrics for querier when processing scheduler requests. (#11097 ) What this PR does / why we need it: A previous change removed the RED metrics for the querier. This adds them back as part of a middleware. Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) - [ ] If the change is deprecating or removing a configuration option, update the `deprecated-config.yaml` and `deleted-config.yaml` files respectively in the `tools/deprecated-config-checker` directory. [Example PR](`0d4416a4b0`)	2 years ago
Danny Kopping	34b9b9a11f	Improve observability of index queries (#11064 ) What this PR does / why we need it: Requests to the following APIs previously had limited or incorrect logging, which made understanding their behaviour at runtime difficult. [GET /loki/api/v1/labels](https://grafana.com/docs/loki/latest/reference/api/#list-labels-within-a-range-of-time) [GET /loki/api/v1/label/<name>/values](https://grafana.com/docs/loki/latest/reference/api/#list-label-values-within-a-range-of-time) [GET /loki/api/v1/series](https://grafana.com/docs/loki/latest/reference/api/#list-series) [GET /loki/api/v1/index/stats](https://grafana.com/docs/loki/latest/reference/api/#index-stats) [GET /loki/api/v1/index/volume](https://grafana.com/docs/loki/latest/reference/api/#volume) [GET /loki/api/v1/index/volume_range](https://grafana.com/docs/loki/latest/reference/api/#volume) All of these APIs now have querier and query-frontend logs; sharding and time-based splitting are applied to these requests, and it's valuable to see all requests.	2 years ago
Karsten Jeschkies	aedf6dfc21	Remove `GetHeaders` from `logproto.VolumeResponse`. (#11049 ) What this PR does / why we need it: Replaces https://github.com/grafana/loki/pull/11048/. We would return `logproto.VolumeResponse` but not match it in the codec. Special notes for your reviewer: Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) - [ ] If the change is deprecating or removing a configuration option, update the `deprecated-config.yaml` and `deleted-config.yaml` files respectively in the `tools/deprecated-config-checker` directory. <!-- TODO(salvacorts): Add example PR -->	2 years ago
Callum Styan	0d7b6b2622	Return empty responses for logproto request types from (#11029 ) We need to return empty responses for logproto request types from NewEmptyResponse based on types that can be returned from DecodeRequest. --------- Signed-off-by: Callum Styan <callumstyan@gmail.com>	2 years ago

1 2 3

119 Commits (main)