Like Prometheus, but for logs.

prometheushacktoberfestgrafanacloudnativeloggingloki

You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

Salva Corts 6cc581bd26 Add back cache stats for index stats requests (#9816) ... **What this PR does / why we need it**: In https://github.com/grafana/loki/pull/9536, we added cache stats for index stats requests. That PR had a bug that inflated the query stats due to reusing the stats context in the query engine. Therefore, we had to revert the PR at https://github.com/grafana/loki/pull/9721. This PR brings back the changes from https://github.com/grafana/loki/pull/9536 but fixes the inflated starts by no longer reusing the same context in the query engine, but rather creating a new one for the shard resolver. I tested it on a dev cluster and seems to be working fine. here's the output for the same query: **Stats with the bug from #9536**: ``` ... Cache.StatsResult.Requests 980 Cache.StatsResult.EntriesRequested 490 Cache.StatsResult.EntriesFound 0 Cache.StatsResult.EntriesStored 490 Cache.StatsResult.BytesSent 0 B Cache.StatsResult.BytesReceived 0 B ... Summary.BytesProcessedPerSecond 43 GB Summary.LinesProcessedPerSecond 93305142 Summary.TotalBytesProcessed 945 GB Summary.TotalLinesProcessed 2059694183 ``` **Stats from _main_** ``` ... Summary.BytesProcessedPerSecond 1.6 GB Summary.LinesProcessedPerSecond 3403718 Summary.TotalBytesProcessed 95 GB Summary.TotalLinesProcessed 207971404 ``` **Stats with fix in this PR** ``` .. Cache.StatsResult.Requests 132 Cache.StatsResult.EntriesRequested 66 Cache.StatsResult.EntriesFound 0 Cache.StatsResult.EntriesStored 66 Cache.StatsResult.BytesSent 0 B Cache.StatsResult.BytesReceived 0 B ... Summary.BytesProcessedPerSecond 4.3 GB Summary.LinesProcessedPerSecond 9468900 Summary.TotalBytesProcessed 95 GB Summary.TotalLinesProcessed 207793816 ``` As can be seen, with the changes in this PR, the summary stats are no longer inflated. **Which issue(s) this PR fixes**: Fixes https://github.com/grafana/loki/pull/9536 **Special notes for your reviewer**: I think it's ok to skip reviewing the changes from the commit cherry-picking the changes from https://github.com/grafana/loki/pull/9536 **Checklist** - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](d10549e3ec)		2 years ago
..
analytics	Upgrade `golangci-lint` and fix linting errors (#9601)	2 years ago
canary	Upgrade `golangci-lint` and fix linting errors (#9601)	2 years ago
chunkenc	Upgrade `golangci-lint` and fix linting errors (#9601)	2 years ago
configs	Update prometheus/prometheus from 2.41 to 2.42 (#8571)	2 years ago
distributor	Failures logging: avoid logging generic errors (#9796)	2 years ago
ingester	Failures logging: avoid logging generic errors (#9796)	2 years ago
iter	Reuse allocated buffer space in merge iterator. (#9538)	2 years ago
logcli	Upgrade `golangci-lint` and fix linting errors (#9601)	2 years ago
loghttp	Log push failures per-entry instead of in batches (#9720)	2 years ago
logproto	Log Volume: Do the prometheus-format conversion in dedicated middleware (#9776)	2 years ago
logql	Add back cache stats for index stats requests (#9816)	2 years ago
logqlanalyzer	fixed logql-analyzer (#8258)	2 years ago
logqlmodel	Add back cache stats for index stats requests (#9816)	2 years ago
loki	Add per-tenant request counter metric to index gateway server (#9797)	2 years ago
lokifrontend	Upgrade `golangci-lint` and fix linting errors (#9601)	2 years ago
prom1/storage/metric	Copy `cortex/pkg/prom1` package dependency into Loki (#5196)	3 years ago
push	Bump golang.org/x/net from 0.4.0 to 0.7.0 in /pkg/push (#8562)	2 years ago
querier	Add back cache stats for index stats requests (#9816)	2 years ago
ruler	Remove some unused code (#9611)	2 years ago
runtime	Loki: Implement initial phase of `limited_log_push_errors` (#9556)	2 years ago
scheduler	Upgrade `golangci-lint` and fix linting errors (#9601)	2 years ago
server	Configurable TLS MinVersion and CipherSuites (#7227)	3 years ago
sizing	Upgrade `golangci-lint` and fix linting errors (#9601)	2 years ago
storage	Add per-tenant request counter metric to index gateway server (#9797)	2 years ago
tracing	Promtail: Add basic tracing support (#7414)	3 years ago
util	Add back cache stats for index stats requests (#9816)	2 years ago
validation	adds tsdb-max-bytes-per-shard limit (#9811)	2 years ago