apitech/loki - loki - Gitea: Git with a cup of tea

Commit Graph

Author	SHA1	Message	Date
loki-gh-app[bot]	4309f47681	chore: Add bool to metrics.go log line whether query used bloom filters (backport k227) (#14751 ) Co-authored-by: Christian Haudum <christian.haudum@gmail.com>	7 months ago
Trevor Whitney	b5462b6639	chore: remove copied loop vars (#14383 )	8 months ago
Owen Diehl	9c25985b97	feat(blooms): record time spent resolving shards (#12636 )	1 year ago
Travis Patterson	cf04ec1320	feat: Enable warnings in Loki query responses (#12425 )	1 year ago
Ed Welch	4c88be0ef2	chore: update loki modules for 3.0 release (#12433 ) Signed-off-by: Edward Welch <edward.welch@grafana.com>	1 year ago
Ashwanth	c8c8477c93	chore(stats): add pipeline_wrapper_filtered_lines (#12399 )	1 year ago
Owen Diehl	a36483b731	feat(blooms): bloom integration in query planning (#12208 ) Signed-off-by: Owen Diehl <ow.diehl@gmail.com>	1 year ago
Danny Kopping	609fdb3214	feat(querier/query-frontend): track and log congestion control latency (#12058 ) Signed-off-by: Danny Kopping <danny.kopping@grafana.com>	1 year ago
Kaviraj Kanagaraj	fac5997b18	feat: Support split align and caching for instant metric query results (#11814 ) Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com>	1 year ago
Ashwanth	852becf227	results cache: add new stat `query_length_served` to measure cache effectiveness (#11589 ) What this PR does / why we need it: cache hit rate that is currently being measured using metrics and stats does not account for the fact that a cache hit could return partial results. When we query the cache for a key, we get back a list of extents and these need no cover the entire (split) range of the cache key. This pr adds a new cache stat called `query_length_served` to better measure the cache effectiveness. ``` query_length - sum(length of downstream queries) ``` Which issue(s) this PR fixes: Fixes #<issue number> Special notes for your reviewer: Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [x] Documentation added - [x] Tests updated - [x] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) - [ ] If the change is deprecating or removing a configuration option, update the `deprecated-config.yaml` and `deleted-config.yaml` files respectively in the `tools/deprecated-config-checker` directory. [Example PR](`0d4416a4b0`)	1 year ago
Kaviraj Kanagaraj	ce574485cf	feat(caching): Support caching `/series` and `/labels` query results (#11539 ) What this PR does / why we need it: Add support for caching metadata queries (both series and labels). caching happens after splitting similar to other types of queries. This pr adds the following configs to enable them. ``` cache_series_results: true\|false (default false) cache_label_results: true\|false (default false) ``` And the cache backend for them can be configured using `series_results_cache` and `label_results_cache` blocks under the `query_range` section. Currently the split interval for metadata queries is fixed and defaults to 24h, this pr makes it configurable by introducing `split_metadata_queries_by_interval` Which issue(s) this PR fixes: Fixes #<issue number> Special notes for your reviewer: Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [x] Documentation added - [x] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) - [ ] If the change is deprecating or removing a configuration option, update the `deprecated-config.yaml` and `deleted-config.yaml` files respectively in the `tools/deprecated-config-checker` directory. [Example PR](`0d4416a4b0`) --------- Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> Co-authored-by: Ashwanth Goli <iamashwanth@gmail.com>	1 year ago
Sandeep Sukhani	0e5da205a6	structured metadata: track usage of structured metadata in queries (#11505 ) What this PR does / why we need it: Adds support for tracking queries referencing structured metadata. It tracks the following kinds of references: * Fields in structured metadata used as label filters, `drop`/`keep` stages etc. in the query. * Fields in structured metadata used in metric queries like `uwrap` and `by` or `without` clauses in aggregations. Checklist - [x] Tests updated	1 year ago
Vladyslav Diachenko	00a3c5b026	renamed `non indexed labels` feature to `structured metadata` (#10432 ) Loki team has made a decision to rename the feature because the new name is more precise. --------- Signed-off-by: Vladyslav Diachenko <vlad.diachenko@grafana.com> Co-authored-by: Salva Corts <salva.corts@grafana.com>	2 years ago
Trevor Whitney	a05744a385	Enable results cache for volume queries (#10403 ) This PR enables the results cache for volume queries to the `index/volume` and `index/volume_range` endpoints	2 years ago
Danny Kopping	c725de6035	Querier: add chunk refs fetch time to metrics (#10362 ) What this PR does / why we need it: Adds an entry to the `metrics.go` log line to track how long a query waits for its chunk references to be returned by the index. This is useful when trying to understand where processing time is being spent.	2 years ago
Salva Corts	3f161f5c1a	Improve observability for non-indexed labels usage (#9993 ) What this PR does / why we need it: In https://github.com/grafana/loki/pull/9700, we support encoding and decoding metadata for each entry into the chunks. In this PR we: - Update the bytes processed stats to account for the bytes from those non-indexed labels - Add new stats for bytes processed for those non-indexed labels - Add new ingestion metrics to track ingested non-indexed bytes Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [x] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`)	2 years ago
Ed Welch	fa6f9c638a	Loki: add 'post_filter_lines' stat for tracking how many lines are match a queries filter expression(s) (#9983 ) What this PR does / why we need it: To better understand query behavior and labeling strategies of users, logging the number of log lines "post filtering" can be very useful, we already log the total_lines processed in a query so this will allow us to see how many lines in the label selector matched what the query was looking for. Which issue(s) this PR fixes: Fixes #<issue number> Special notes for your reviewer: Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) Signed-off-by: Edward Welch <edward.welch@grafana.com>	2 years ago
Salva Corts	6cc581bd26	Add back cache stats for index stats requests (#9816 ) What this PR does / why we need it: In https://github.com/grafana/loki/pull/9536, we added cache stats for index stats requests. That PR had a bug that inflated the query stats due to reusing the stats context in the query engine. Therefore, we had to revert the PR at https://github.com/grafana/loki/pull/9721. This PR brings back the changes from https://github.com/grafana/loki/pull/9536 but fixes the inflated starts by no longer reusing the same context in the query engine, but rather creating a new one for the shard resolver. I tested it on a dev cluster and seems to be working fine. here's the output for the same query: Stats with the bug from #9536: ``` ... Cache.StatsResult.Requests 980 Cache.StatsResult.EntriesRequested 490 Cache.StatsResult.EntriesFound 0 Cache.StatsResult.EntriesStored 490 Cache.StatsResult.BytesSent 0 B Cache.StatsResult.BytesReceived 0 B ... Summary.BytesProcessedPerSecond 43 GB Summary.LinesProcessedPerSecond 93305142 Summary.TotalBytesProcessed 945 GB Summary.TotalLinesProcessed 2059694183 ``` Stats from _main_ ``` ... Summary.BytesProcessedPerSecond 1.6 GB Summary.LinesProcessedPerSecond 3403718 Summary.TotalBytesProcessed 95 GB Summary.TotalLinesProcessed 207971404 ``` Stats with fix in this PR ``` .. Cache.StatsResult.Requests 132 Cache.StatsResult.EntriesRequested 66 Cache.StatsResult.EntriesFound 0 Cache.StatsResult.EntriesStored 66 Cache.StatsResult.BytesSent 0 B Cache.StatsResult.BytesReceived 0 B ... Summary.BytesProcessedPerSecond 4.3 GB Summary.LinesProcessedPerSecond 9468900 Summary.TotalBytesProcessed 95 GB Summary.TotalLinesProcessed 207793816 ``` As can be seen, with the changes in this PR, the summary stats are no longer inflated. Which issue(s) this PR fixes: Fixes https://github.com/grafana/loki/pull/9536 Special notes for your reviewer: I think it's ok to skip reviewing the changes from the commit cherry-picking the changes from https://github.com/grafana/loki/pull/9536 Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`)	2 years ago
Salva Corts	b7359c5d53	Revert "Add summary stats and metrics for stats cache (#9536 )" (#9721 ) This reverts commit `af287ac3eb`. There is a bug in this PR that inflates the stats returned for the query since we reuse the stats ctx in the query execution engine.	2 years ago
Salva Corts	af287ac3eb	Add summary stats and metrics for stats cache (#9536 ) What this PR does / why we need it: When a query finishes, we return (and log) the following stats: ```go Cache.Chunk.Requests 0 Cache.Chunk.EntriesRequested 0 Cache.Chunk.EntriesFound 0 Cache.Chunk.EntriesStored 0 Cache.Chunk.BytesSent 0 B Cache.Chunk.BytesReceived 0 B Cache.Chunk.DownloadTime 0s Cache.Index.Requests 0 Cache.Index.EntriesRequested 0 Cache.Index.EntriesFound 0 Cache.Index.EntriesStored 0 Cache.Index.BytesSent 0 B Cache.Index.BytesReceived 0 B Cache.Index.DownloadTime 0s Cache.Result.Requests 13 Cache.Result.EntriesRequested 13 Cache.Result.EntriesFound 13 Cache.Result.EntriesStored 0 Cache.Result.BytesSent 0 B Cache.Result.BytesReceived 2.5 kB Cache.Result.DownloadTime 4.600266ms ``` In addition to that, we log the following in metrics.go: ``` level=info ts=2023-05-29T09:17:10.93029945Z caller=metrics.go:152 component=frontend org_id=145265 traceID=52d59b78fe6b9221 sampled=true latency=fast query="{cluster=\"dev-us-central-0\", namespace=~\"loki.\", container=~\"distributor\|ingester \|promtail\|index-gateway\|compactor\"} \|= \"thislinewillnotexist\"" query_hash=1194136170 query_type=filter range_type=range length=3h0m0s start_delta=165h37m24.930289434s end_delta=162h37m24.930289612s step=43s duration=2.473055ms status=200 lim it=30 returned_lines=0 throughput=0B total_bytes=0B lines_per_second=0 total_lines=0 total_entries=0 store_chunks_download_time=0s queue_time=0s splits=13 shards=0 cache_chunk_req=0 cache_chunk_hit=0 cache_chunk_bytes_stored=0 cache_chunk_bytes _fetched=0 cache_chunk_download_time=0s cache_index_req=0 cache_index_hit=0 cache_index_download_time=0s cache_result_req=13 cache_result_hit=13 cache_result_download_time=4.600266ms ``` With the goal of being able to better monitor how the stats cache is performing; this PR adds stats for the index stats cache, similarly to how it's done for the results cache. Here's an example of the new stats being returned and printed: ```go ... Cache.StatsResult.Requests 180 Cache.StatsResult.EntriesRequested 129 Cache.StatsResult.EntriesFound 129 Cache.StatsResult.EntriesStored 51 Cache.StatsResult.BytesSent 0 B Cache.StatsResult.BytesReceived 75 kB ... ``` And the new stats from metrics.go ``` ... caller=metrics.go:155 ... cache_stats_results_req=129 cache_stats_results_hit=129 cache_stats_results_download_ti me=156.864429ms ... ``` Special notes for your reviewer: - Blocked by https://github.com/grafana/loki/pull/9535 - Note the new`stats.GetOrCreateContext` func. It's used inside the `query.Exec` method so we don't overwrite the stats added in the stats middleware. Checklist* - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [x] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`)	2 years ago
Paul Rogers	14370bb8ce	Revert "Augment statistics.." PR 9400. (#9430 ) What this PR does / why we need it: This PR reverts PR 9400. The data collected within that PR was not sufficient. When queries are done, they are filtered before the merge iterator, resulting in an inability to collect an accurate count of duplicated data. Which issue(s) this PR fixes: Special notes for your reviewer: Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	2 years ago
Paul Rogers	1671751cbd	Augment statistics to note how many bytes are in duplicate lines due to replicas (#9400 ) What this PR does / why we need it: This PR is for counting the number of bytes of log lines that were marked as duplicates. This will be utilized to collect better statistics.	2 years ago
Ed Welch	a4eb536fb2	Loki: remove `subqueries` from metrics.go logging and replace it with separate split and shard counters (#8761 ) What this PR does / why we need it: Currently the `metrics.go` log line emitted after every query includes a metric called "subqueries". This currently tracks the number of queries created by the split_by_time operations done in Loki, but does not include any counts for subqueries created as a result of sharding. It becomes difficult to make a single subqueries counter that gives useful information to determine how much a query is split by time and sharded by a shard factor, especially now that sharding in TSDB indexes is dynamic. This PR removes and deprecates the `subqueries` stat and instead creates a `splits` and `shards` statistic which records how much a query was split_by_time and the total number of shards created as well. Which issue(s) this PR fixes: Fixes #<issue number> Special notes for your reviewer: Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [x] Tests updated - [x] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` --------- Signed-off-by: Edward Welch <edward.welch@grafana.com>	2 years ago
Danny Kopping	a4f306399a	Add store & cache download statistics (#7982 )	2 years ago
Danny Kopping	36e0979cf5	Add cache usage statistics (#6317 ) * Adding cache statistics Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Adding metrics to metrics.go Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Creating new stats context for use in metric queries middleware Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Clean up unnecessary log fields Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Fixing tests Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Adding stats tests Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * CHANGELOG entry Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Appeasing the linter Documenting function Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Moving CHANGELOG entry to appropriate section Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Implementing a stats collector cache wrapper to simplify stats collection If we keep the stats collection in pkg/storage/chunk/cache/instrumented.go, then any implementation that wraps it will cause the stats collected to be incomplete. For example: NewBackground(cacheName, cfg.Background, Instrument(cacheName, cache, reg), reg)) - the background cache requests are not collected Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Fixing tests Signed-off-by: Danny Kopping <danny.kopping@grafana.com>	3 years ago
Kaviraj Kanagaraj	1389857846	`metrics.go` support for metadata queries(labels and series) (#5971 ) * Add different functions to record labels and series queries * Add tests for logging series and labels queries * Add statistics to the Series and Labels proto response types * Plug in the statistics recorder on the query handler * Remove duplicate imports and cleanup comments * Remove the need to add usage statistics for metadata queries. * Add status based on returned error from the api * Fixing lint error by fixin return values order * Use explicit logger * Remove unused recorders * Add `total_entries` representing length of total result returned * Inject statscollector middleware for metadata queries * Fix query frontend stats middleware Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Fix tests with change in extra `totalEntriesReturned` field Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * `make check-generated-files` Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Fix typo Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Fix tripperware middleware for label values endpoint Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Fix bug with label extraction Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Add changelog entry Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Fix `total_entries` value for range query Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com>	3 years ago
Cyril Tovena	9b6f17f168	Add the total number of subqueries to query statistics (#5397 ) * Add subqueries count Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes tests Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	3 years ago
Dylan Guedes	cf419d1cb8	Implement `LegacySample`, `LegacyLabelPair` proto messages (#5301 ) * Add LegacySample and modify protos to use it. * [Skip this commit] Regenerate protos. * Modify codebase to use LegacySample instead of Sample. For reference, take a look at PR https://github.com/grafana/loki/pull/5151. * Implement LegacyLabelPair. * Use new Legacy implementations accordingly. * Use LegacySample in compat tests. - Since `compat.go` was forked from Cortex, we can reliably change all mentions from `Sample` to `LegacySample` there.	3 years ago
Dylan Guedes	3f496e7336	Copy `cortex/pkg/cortexpb` package dependency into Loki (#5151 ) * Fork cortexpb into Cortex `pkg/logproto`. * Use `pkg/logproto` instead of Cortex `pkg/cortexpb`. * Rename `cortexpb` usage to `logproto`. * Fix encoding/decoding test. * Replace cortexpb usage in the ruler/base package. * Remove exemplars from logproto.	3 years ago
Dylan Guedes	0e3df1938f	Copy querier subpackages dependency into Loki (#5068 ) * Fork cortex `querier/series`. * Fork cortex `querier/chunkstore`. * Fork cortex `querier/lazyquery`. * Use Loki forked cortex code. * Copy cortex `querier/queryrange` as `querier/queryrange/queryrangebase`. * Fork cortex `pkg/querier/queryrange` to `pkg/querier/queryrange/queryrangebase`. * Tell protoc to use gogo.types instead of pb.Any. * Use `pkg/querier/queryrange/queryrangebase`. * Remove cortex `queryrange` vendoring. * Fix lint. * Use `_ = ...` syntax instead of comment to fix lint.	3 years ago
Susana Ferreira	191a8a6f9d	Store metrics queueTime in seconds (#5052 ) * Store metrics queueTime in seconds * Address comments * Helper function to convert seconds to nanoseconds	3 years ago
Susana Ferreira	82f8f3c1e5	Log time in queue per request (#4949 ) * Log time in queue per request * Add querierID to log message * Add unescape request url and tenant_ids to log message * Add query enqueue time to metrics.go * Remove "querier request dequeued" log from scheduler * Address comments * Rename enqueue_time to queue_time * Store queue time in nanoseconds (int64) * Use CanonicalMIMEHeaderKey for setting the httpgrpc header	3 years ago
Cyril Tovena	d81f757ecb	Refactor and simplify stats package. (#4712 ) * Refactor and simplify stats package. This PR refactors the stats package and make it smaller and simpler to work with. - It doesn't expose the context which can causes race when mutating it. - Uses concurrency primitive to ensure correctness - Replace trailers with in-batch statistics to propagate ingesters stats. Fixes #3366 Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * fix tests and lint issues. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes moar tests. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * mod-check Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * More fixes. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	4 years ago
Cyril Tovena	1efeace644	Fixes instant queries in the frontend. (#4091 ) * Fixes instant queries in the frontend. This is a set of fixes for instant queries: - correctly propagate statistics. - correctly convert sampleStream to vector. - correctly marshal to json. - correctly encode the time for the request. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes tests. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	4 years ago

34 Commits (23b5fc2c9b1a77b8776eac70279018956a458fc6)