Like Prometheus, but for logs.

prometheushacktoberfestgrafanacloudnativeloggingloki

You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

Salva Corts 6cc581bd26 Add back cache stats for index stats requests (#9816) ... **What this PR does / why we need it**: In https://github.com/grafana/loki/pull/9536, we added cache stats for index stats requests. That PR had a bug that inflated the query stats due to reusing the stats context in the query engine. Therefore, we had to revert the PR at https://github.com/grafana/loki/pull/9721. This PR brings back the changes from https://github.com/grafana/loki/pull/9536 but fixes the inflated starts by no longer reusing the same context in the query engine, but rather creating a new one for the shard resolver. I tested it on a dev cluster and seems to be working fine. here's the output for the same query: **Stats with the bug from #9536**: ``` ... Cache.StatsResult.Requests 980 Cache.StatsResult.EntriesRequested 490 Cache.StatsResult.EntriesFound 0 Cache.StatsResult.EntriesStored 490 Cache.StatsResult.BytesSent 0 B Cache.StatsResult.BytesReceived 0 B ... Summary.BytesProcessedPerSecond 43 GB Summary.LinesProcessedPerSecond 93305142 Summary.TotalBytesProcessed 945 GB Summary.TotalLinesProcessed 2059694183 ``` **Stats from _main_** ``` ... Summary.BytesProcessedPerSecond 1.6 GB Summary.LinesProcessedPerSecond 3403718 Summary.TotalBytesProcessed 95 GB Summary.TotalLinesProcessed 207971404 ``` **Stats with fix in this PR** ``` .. Cache.StatsResult.Requests 132 Cache.StatsResult.EntriesRequested 66 Cache.StatsResult.EntriesFound 0 Cache.StatsResult.EntriesStored 66 Cache.StatsResult.BytesSent 0 B Cache.StatsResult.BytesReceived 0 B ... Summary.BytesProcessedPerSecond 4.3 GB Summary.LinesProcessedPerSecond 9468900 Summary.TotalBytesProcessed 95 GB Summary.TotalLinesProcessed 207793816 ``` As can be seen, with the changes in this PR, the summary stats are no longer inflated. **Which issue(s) this PR fixes**: Fixes https://github.com/grafana/loki/pull/9536 **Special notes for your reviewer**: I think it's ok to skip reviewing the changes from the commit cherry-picking the changes from https://github.com/grafana/loki/pull/9536 **Checklist** - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](d10549e3ec)		2 years ago
..
log	logfmt: add --strict flag to enable strict parsing, perform nostrict parsing by default (#9626)	2 years ago
syntax	logfmt: add --strict flag to enable strict parsing, perform nostrict parsing by default (#9626)	2 years ago
vector	Update Prometheus dependency (#9205)	2 years ago
blocker.go	Querier: block query by hash (#8953)	2 years ago
blocker_test.go	Querier: block query by hash (#8953)	2 years ago
downstream.go	Upgrade `golangci-lint` and fix linting errors (#9601)	2 years ago
downstream_test.go	Fix instant query summary split stats (#9773)	2 years ago
engine.go	Revert "Add summary stats and metrics for stats cache (#9536)" (#9721)	2 years ago
engine_test.go	Upgrade `golangci-lint` and fix linting errors (#9601)	2 years ago
evaluator.go	Upgrade `golangci-lint` and fix linting errors (#9601)	2 years ago
evaluator_test.go	Update Prometheus dependency (#9205)	2 years ago
limits.go	Upgrade `golangci-lint` and fix linting errors (#9601)	2 years ago
mapper_metrics.go	query_frontend_shard_factor uses exponential buckets (#6689)	3 years ago
mapper_stats.go	Fix instant query summary split stats (#9773)	2 years ago
matchers.go	Refactor LogQL syntax into its own package (#5539)	3 years ago
matchers_test.go	Regex line/label filters return original regex when `String` is called (#9584)	2 years ago
matrix.go	Update Prometheus dependency (#9205)	2 years ago
matrix_test.go	logql tests: use abstractions over Labels (#9533)	2 years ago
metrics.go	Add back cache stats for index stats requests (#9816)	2 years ago
metrics_test.go	Loki: log all trace ID's but include `sampled=true` when the trace is logged (#9039)	2 years ago
optimize.go	logfmt: add --strict flag to enable strict parsing, perform nostrict parsing by default (#9626)	2 years ago
optimize_test.go	fix panics when cloning a special query (#8531)	2 years ago
range_vector.go	Upgrade `golangci-lint` and fix linting errors (#9601)	2 years ago
range_vector_test.go	Update Prometheus dependency (#9205)	2 years ago
rangemapper.go	logfmt: add --strict flag to enable strict parsing, perform nostrict parsing by default (#9626)	2 years ago
rangemapper_test.go	Fix instant query summary split stats (#9773)	2 years ago
shardmapper.go	Loki: Add a limit for the [range] value on range queries (#8343)	2 years ago
shardmapper_test.go	Regex line/label filters return original regex when `String` is called (#9584)	2 years ago
step_evaluator.go	Upgrade `golangci-lint` and fix linting errors (#9601)	2 years ago
test_utils.go	Upgrade `golangci-lint` and fix linting errors (#9601)	2 years ago
vector.go	Update Prometheus dependency (#9205)	2 years ago