apitech/loki - loki - Gitea: Git with a cup of tea

Commit Graph

Author	SHA1	Message	Date
Kaviraj Kanagaraj	88e545fc95	feat(cache): Add `Cache-Control: no-cache` support for Loki instant queries. (#12896 ) Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com>	2 years ago
Shantanu Alshi	e7fdeb974a	perf: Improve Detected labels API (#12816 )	2 years ago
Owen Diehl	93aaf29e68	fix(spans): corrects early-close for a few spans (#12887 )	2 years ago
Owen Diehl	5a643c7115	chore: reduces span footprint + double recording (#12864 ) Trying to remove some cruft from traces.	2 years ago
Owen Diehl	bc78d13d9b	feat(blooms): compute chunks once (#12664 ) Signed-off-by: Owen Diehl <ow.diehl@gmail.com>	2 years ago
benclive	7b8533e435	feat: Add step param to Patterns Query API (#12703 )	2 years ago
Trevor Whitney	af09f53a33	ci: release workflows get build image from makefile (#12778 )	2 years ago
Shantanu Alshi	587a6d20e9	feat: Detected labels from store (#12441 )	2 years ago
Trevor Whitney	f68d1f7faf	fix: make detected fields work for both json and proto (#12682 )	2 years ago
Trevor Whitney	6c33809015	feat: split detected fields queries (#12491 )	2 years ago
Owen Diehl	9c25985b97	feat(blooms): record time spent resolving shards (#12636 )	2 years ago
Ed Welch	670041368a	chore: add more validation at startup to make sure we have a proper schema, and necessary directories are created (#12496 ) Signed-off-by: Edward Welch <edward.welch@grafana.com> Co-authored-by: Dylan Guedes <djmgguedes@gmail.com>	2 years ago
Cyril Tovena	491d2513e7	feat: add log patterns ingestion and query API. (#12403 ) Signed-off-by: Edward Welch <edward.welch@grafana.com> Co-authored-by: Anton Kolesnikov <anton.e.kolesnikov@gmail.com> Co-authored-by: Sven Grossmann <svennergr@gmail.com> Co-authored-by: Edward Welch <edward.welch@grafana.com>	2 years ago
Ed Welch	bc26198c00	feat: enable caching of index stats results, volume results, series results and label results by default (#12452 ) Signed-off-by: Edward Welch <edward.welch@grafana.com>	2 years ago
Travis Patterson	cf04ec1320	feat: Enable warnings in Loki query responses (#12425 )	2 years ago
Cyril Tovena	fbb8686d03	fix: Fixes detected labels API wiring (#12442 )	2 years ago
Ed Welch	824f5aa20a	chore: fix submodule for v3 (#12438 ) Signed-off-by: Edward Welch <edward.welch@grafana.com>	2 years ago
Trevor Whitney	a53a0cc792	feat: return real data in detected fields endpoint (#12421 ) parse log lines in the querier to return data about detected fields. this is not sustainable long term, but a short term solution to validate the data we want for the frontend.	2 years ago
Ed Welch	4c88be0ef2	chore: update loki modules for 3.0 release (#12433 ) Signed-off-by: Edward Welch <edward.welch@grafana.com>	2 years ago
Callum Styan	018856c1c1	fix: fix span logging based on changes to request types timestamps (#12393 ) Signed-off-by: Callum Styan <callumstyan@gmail.com>	2 years ago
Shantanu Alshi	5190dda64b	feat(detected_labels): Initial skeleton for the API (#12390 ) Co-authored-by: Cyril Tovena <cyril.tovena@gmail.com>	2 years ago
Kaviraj Kanagaraj	7480468e0f	fix: (Bug) correct resultType when storing instant query results in cache (#12312 ) Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com>	2 years ago
Trevor Whitney	246623f7d0	fix(detected_fields): fix issues with frontend integration (#12406 ) This PRs fixes issues we found when integrating with the frontend * the `/experimental` api made it difficult to interact with using the existing datasource, so move to `v1/detected_fields` * the config flag was considered cumbersome as the only potential negative impact of the endpoint is when it is used, and nothing is currently using it * the use of an enum in the protobuf produced unexpected results in the json, so type was converted to string	2 years ago
Ashwanth	c8c8477c93	chore(stats): add pipeline_wrapper_filtered_lines (#12399 )	2 years ago
Owen Diehl	6e1547fe62	fix(blooms): logql shard embeds tsdb (#12396 )	2 years ago
Anton Kolesnikov	cc941fe42a	chore: refactor line filter MatchType (#12388 )	2 years ago
Trevor Whitney	5b4bfa54b6	feat(detected_fields): initial plumbing of new endpoint (#12385 )	2 years ago
Ashwanth	64c7812015	fix(codec): inject disable wrappers in ctx (#12352 )	2 years ago
Owen Diehl	a36483b731	feat(blooms): bloom integration in query planning (#12208 ) Signed-off-by: Owen Diehl <ow.diehl@gmail.com>	2 years ago
Travis Patterson	c8d2c32002	feat: Introduce special header that tells Loki not to modify query results (#12327 )	2 years ago
Salva Corts	2544f003cf	chore: Add summary for received filters in bloom gateway (#12158 )	2 years ago
Paul Rogers	0ddf6fc49d	test: Fix for a few data races (#12129 )	2 years ago
Danny Kopping	609fdb3214	feat(querier/query-frontend): track and log congestion control latency (#12058 ) Signed-off-by: Danny Kopping <danny.kopping@grafana.com>	2 years ago
Kaviraj Kanagaraj	fac5997b18	feat: Support split align and caching for instant metric query results (#11814 ) Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com>	2 years ago
Ashwanth	9e7725b31b	feat(metadata): introduce a separate split interval for recent query window (#11897 )	2 years ago
Christian Haudum	1c43991ddc	Remove `bloomcompactor.DayTable` in favour of `config.DayTime` (#11917 ) Both structs shared the same semantics. This PR moves additional functionality from the `DayTable` to the `DayTime` struct. To get the table name of a day (ordinal number of day since unix epoch), call `DayTime.Table()`. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
Paul Rogers	472496feb7	Parse JSON String arrays properly so string elements can be retrieved (#11921 ) What this PR does / why we need it: This PR imports the newly forked grafana/jsonparser over the buger/jsonparser module. The latter has seemingly been abandoned. PR 10690 introduces a fix to the jsonparser module, which has been incorporated into the grafana fork of the module. The PR is designed to fix accessing string array elements from within a JSON structure. For example, with the following JSON: `{"log":{"message":{"content":{"misses":["a","b","c","d"]}}}}` The Loki code, before this PR, when searching for `json misses = "log.message.content.misses[0]" ` will result in an "Unknown value type error". After this PR is merged, the result will assign `a` to the `misses` variable. Which issue(s) this PR fixes: Fixes #[9179](https://github.com/grafana/loki/issues/9179) https://github.com/grafana/loki/pull/10690 Special notes for your reviewer: Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [x] Tests updated - [x] `CHANGELOG.md` updated - [x] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) - [ ] If the change is deprecating or removing a configuration option, update the `deprecated-config.yaml` and `deleted-config.yaml` files respectively in the `tools/deprecated-config-checker` directory. [Example PR](`0d4416a4b0`)	2 years ago
Karsten Jeschkies	483ee56139	Inject downstream accumulator. (#11863 ) What this PR does / why we need it: We inject the downstream accumulator into the downstreamer instead of constructing it on the first arrival of the first result. Since we know the query type before executing the first query we can pass the correct accumulator. This will allow us to define special `topk` or `sum by` accumulators in the future. Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) - [ ] If the change is deprecating or removing a configuration option, update the `deprecated-config.yaml` and `deleted-config.yaml` files respectively in the `tools/deprecated-config-checker` directory. [Example PR](`0d4416a4b0`) --------- Co-authored-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
Trevor Whitney	406b10c675	accidentally merged this from ##11333 (#11788 ) What this PR does / why we need it: I did not intend to merge this change, reverting to what it was before merging #11333	2 years ago
Trevor Whitney	a064f5d670	feat: add release please config (#11333 ) What this PR does / why we need it: The new release process is being developed in another repo for now as it enables much faster feedback loops. We are at the stage where we are pulling the Loki repo and building images, so it's time to move the `release-please` config into Loki, and configure it for the various packages we will be releasing. This PR adds the two basic configs that `release-please` requires. This PR does not change anything about the release process, it just enables us to keep iterating out of band until we're ready to move over. Which issue(s) this PR fixes: Re #11332	2 years ago
Danny Kopping	e7b782b186	Query-frontend: use the same query split interval for generated cache keys (#11679 ) What this PR does / why we need it: Follow up from #11535 (specifically from [this thread](https://github.com/grafana/loki/pull/11535#issuecomment-1887151515)), this PR modifies the results cache implementation to use the same interval for generating cache keys.	2 years ago
Ashwanth	51899b521b	feat(metadata cache): adds max_metadata_cache_freshness (#11682 ) What this PR does / why we need it: Adds `max_metadata_cache_freshness` to limit the metadata requests that get cached. When configured, only metadata requests with end time before `now - max_metadata_cache_freshness` are cacheable. _reason for setting the default to 24h?_ metric results cache can [extract samples for the desired time range from an extent](`b6e64e1ef1/pkg/querier/queryrange/queryrangebase/results_cache.go (L78)`) since the samples are associated with a timestamp. But the same is not true for metadata caching, it is not possible to extract a subset of labels/series from a cached extent. As a result, we could return inaccurate results, more that what was requested. for ex: returning results from an entire 1h extent for a 5m query Setting `max_metadata_cache_freshness` to 24h should help us avoid caching recent data. For anything older, we would report cached metadata results at a granularity controlled by `split_metadata_queries_by_interval` Which issue(s) this PR fixes: Fixes #<issue number> Special notes for your reviewer: Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [x] Documentation added - [x] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) - [ ] If the change is deprecating or removing a configuration option, update the `deprecated-config.yaml` and `deleted-config.yaml` files respectively in the `tools/deprecated-config-checker` directory. [Example PR](`0d4416a4b0`)	2 years ago
Danny Kopping	bcd03150c9	Query-frontend: customisable query splitting for queries overlapping `query_ingester_within` window (#11535 ) What this PR does / why we need it: The config option `query_ingesters_within` defines the window during which logs _could_ be present on ingesters, and as such queriers will send queries to ingesters instead. `split_queries_by_interval` is defined to split queries into subqueries for increased parallelism. Aggressive query splitting within the `query_ingesters_within` window can result in overloading ingesters with unnecessarily large numbers of subqueries, which perversely can impact writes. `query_ingesters_within` is set to 3h by default. In Grafana Cloud Logs we set `split_queries_by_interval` as low as 15m (defaults to 1h), which would result in result in 3*60/15=12 requests. Every querier queries every ingester during this window, so that's 12 requests _per ingester per query_ which has the `query_ingesters_within` window in its time range _(i.e. a query from now to now-7d would include the `query_ingesters_within` window as well, now-3h to now-7d would not)_. However, we _do_ want to split queries so an ingester won't have to handle a query for a full `query_ingesters_within` window - this could involve a large amount of data. To account for this, this PR introduces a new option `split_ingester_queries_by_interval` on the query-frontend; this setting is disabled by default. ![image](https://github.com/grafana/loki/assets/373762/2e671bd8-9e8d-4bf3-addf-bebcfc25e8d7)	2 years ago
Ashwanth	e915efc7f8	fix(log results cache): compose empty response based on the request (#11657 ) What this PR does / why we need it: Log results cache when handling a hit composes an empty response based on the cached request. But the limit or direction fields in the cached request need not match with the current request being served. This causes the log results cache to return a response with incorrect limit. This incorrect limit could then get applied when merging responses upstream (split by interval mw for ex.) This pr fixes this by composing the response based on the request being served. I also thought about updating the cache key to include both limit and direction to have a clear separation, but I left it as is for the following reason: if a time range contains no log lines, that result would not change irrespective of a different limit or direction Which issue(s) this PR fixes: Fixes #<issue number> Special notes for your reviewer: Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [x] Tests updated - [x] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) - [ ] If the change is deprecating or removing a configuration option, update the `deprecated-config.yaml` and `deleted-config.yaml` files respectively in the `tools/deprecated-config-checker` directory. [Example PR](`0d4416a4b0`)	2 years ago
Danny Kopping	627e09365b	Restore previous behaviour of no trailing newlines in HTTP error response bodies (#11606 ) What this PR does / why we need it: #11487 introduced a regression which added trailing newlines to HTTP error response bodies, which may affect client integrations. This regression hasn't made it into the 2.9.x or 2.8.x releases yet so no backporting is necessary.	2 years ago
Ashwanth	852becf227	results cache: add new stat `query_length_served` to measure cache effectiveness (#11589 ) What this PR does / why we need it: cache hit rate that is currently being measured using metrics and stats does not account for the fact that a cache hit could return partial results. When we query the cache for a key, we get back a list of extents and these need no cover the entire (split) range of the cache key. This pr adds a new cache stat called `query_length_served` to better measure the cache effectiveness. ``` query_length - sum(length of downstream queries) ``` Which issue(s) this PR fixes: Fixes #<issue number> Special notes for your reviewer: Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [x] Documentation added - [x] Tests updated - [x] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) - [ ] If the change is deprecating or removing a configuration option, update the `deprecated-config.yaml` and `deleted-config.yaml` files respectively in the `tools/deprecated-config-checker` directory. [Example PR](`0d4416a4b0`)	2 years ago
Kaviraj Kanagaraj	ce574485cf	feat(caching): Support caching `/series` and `/labels` query results (#11539 ) What this PR does / why we need it: Add support for caching metadata queries (both series and labels). caching happens after splitting similar to other types of queries. This pr adds the following configs to enable them. ``` cache_series_results: true\|false (default false) cache_label_results: true\|false (default false) ``` And the cache backend for them can be configured using `series_results_cache` and `label_results_cache` blocks under the `query_range` section. Currently the split interval for metadata queries is fixed and defaults to 24h, this pr makes it configurable by introducing `split_metadata_queries_by_interval` Which issue(s) this PR fixes: Fixes #<issue number> Special notes for your reviewer: Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [x] Documentation added - [x] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) - [ ] If the change is deprecating or removing a configuration option, update the `deprecated-config.yaml` and `deleted-config.yaml` files respectively in the `tools/deprecated-config-checker` directory. [Example PR](`0d4416a4b0`) --------- Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> Co-authored-by: Ashwanth Goli <iamashwanth@gmail.com>	2 years ago
Karsten Jeschkies	8d01cbf0c5	Merge quantile sketches as they arrive (#11544 )	2 years ago
Danny Kopping	18ea38883f	Do not reflect label names in request metrics' "route" label (#11551 ) Which issue(s) this PR fixes: Fixes #11549	2 years ago
Karsten Jeschkies	afe146c884	Benchmark and improve sketch join (#11534 ) What this PR does / why we need it: We saw that `JoinQuantileSketchVector` was using too much memory. This is a small step to improve the allocated memory. ``` pkg: github.com/grafana/loki/pkg/logql │ main.log │ pool.log │ │ sec/op │ sec/op vs base │ JoinQuantileSketchVector-10 3.603µ ± 4% 2.631µ ± 0% -26.99% (p=0.002 n=6) │ main.log │ pool.log │ │ B/op │ B/op vs base │ JoinQuantileSketchVector-10 8.344Ki ± 0% 2.539Ki ± 0% -69.57% (p=0.002 n=6) │ main.log │ pool.log │ │ allocs/op │ allocs/op vs base │ JoinQuantileSketchVector-10 109.0 ± 0% 104.0 ± 0% -4.59% (p=0.002 n=6) ``` Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) - [ ] If the change is deprecating or removing a configuration option, update the `deprecated-config.yaml` and `deleted-config.yaml` files respectively in the `tools/deprecated-config-checker` directory. [Example PR](`0d4416a4b0`) --------- Signed-off-by: Callum Styan <callumstyan@gmail.com> Co-authored-by: Callum Styan <callumstyan@gmail.com>	2 years ago

1 2 3 4 5 ...

330 Commits (88e545fc952d6ff55c61d079db920f00abc04865)