Like Prometheus, but for logs.

prometheushacktoberfestgrafanacloudnativeloggingloki

You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

Salva Corts ee69f2bd37 Split index request in 24h intervals (#8909) ... **What this PR does / why we need it**: At https://github.com/grafana/loki/pull/8670, we applied a time split of 24h intervals to all index stats requests to enforce the `max_query_bytes_read` and `max_querier_bytes_read` limits. When the limit is surpassed, the following message get's displayed:  As can be seen, the reported bytes read by the query are not the same as those reported by Grafana in the lower right corner of the query editor. This is because: 1. The index stats request for enforcing the limit is split in subqueries of 24h. The other index stats rquest is not time split. 2. When enforcing the limit, we are not displaying the bytes in powers of 2, but powers of 10 ([see here][2]). I.e. 1KB is 1000B vs 1KiB is 1024B. This PR adds the same logic to all index stats requests so we also time split by 24 intervals all requests that hit the Index Stats API endpoint. We also use powers of 2 instead of 10 on the message when enforcing `max_query_bytes_read` and `max_querier_bytes_read`.  Note that the library we use under the hoot to print the bytes rounds up and down to the nearest integer ([see][3]); that's why we see 16GiB compared to the 15.5GB in the Grafana query editor. **Which issue(s) this PR fixes**: Fixes https://github.com/grafana/loki/issues/8910 **Special notes for your reviewer**: - I refactored the`newQuerySizeLimiter` function and the rest of the _Tripperwares_ in `rountrip.go` to reuse the new IndexStatsTripperware. So we configure the split-by-time middleware only once. **Checklist** - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [x] Documentation added - [x] Tests updated - [x] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` [1]: https://grafana.com/docs/loki/latest/api/#index-stats [2]: https://github.com/grafana/loki/blob/main/pkg/querier/queryrange/limits.go#L367-L368 [3]: https://github.com/dustin/go-humanize/blob/master/bytes.go#L75-L78		2 years ago
..
astmapper	Restoring lint checks that were previously disabled (#7340)	3 years ago
queryrange	Split index request in 24h intervals (#8909)	2 years ago
series	Ruler: remote rule evaluation (#8744)	2 years ago
stats	Update Prometheus dependency to latest release (v2.40.4) (#7826)	3 years ago
worker	retry+backoff querier->qf response (#8285)	2 years ago
http.go	Add initial implementation of per-query limits (#8727)	2 years ago
http_test.go	Loki: Improve query timeouts behavior (#7230)	3 years ago
ingester_querier.go	ingester_querier: early exit queries on reaching quorum (#7172)	3 years ago
ingester_querier_test.go	ingester_querier: early exit queries on reaching quorum (#7172)	3 years ago
metrics.go	Add tailing metrics (#6059)	3 years ago
multi_tenant_querier.go	LogQL: [optimization] syntax: Replace "panic" in "/pkg/logql/syntax" with "error" (#7208)	2 years ago
multi_tenant_querier_test.go	Update Prometheus dependency to latest release (v2.39.1) (#7425)	3 years ago
querier.go	Print duration in error messages with more readable. (#8816)	2 years ago
querier_mock_test.go	ingester_querier: early exit queries on reaching quorum (#7172)	3 years ago
querier_test.go	Print duration in error messages with more readable. (#8816)	2 years ago
tail.go	Iterators: re-implement mergeEntryIterator using loser.Tree for performance (#8637)	2 years ago
tail_mock_test.go	Improve protobuf serialization (#2031)	5 years ago
tail_test.go	Iterators: re-implement mergeEntryIterator using loser.Tree for performance (#8637)	2 years ago
testutils.go	Removing duplicated limits file and refactored references (#8210)	2 years ago
worker_service.go	Remove hardwired '127.0.0.1' values in default addresses (#7784)	3 years ago
worker_service_test.go	Querier worker/inflight metrics (#5124)	3 years ago