Like Prometheus, but for logs.

prometheushacktoberfestgrafanacloudnativeloggingloki

You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

Salva Corts ee69f2bd37 Split index request in 24h intervals (#8909) ... **What this PR does / why we need it**: At https://github.com/grafana/loki/pull/8670, we applied a time split of 24h intervals to all index stats requests to enforce the `max_query_bytes_read` and `max_querier_bytes_read` limits. When the limit is surpassed, the following message get's displayed:  As can be seen, the reported bytes read by the query are not the same as those reported by Grafana in the lower right corner of the query editor. This is because: 1. The index stats request for enforcing the limit is split in subqueries of 24h. The other index stats rquest is not time split. 2. When enforcing the limit, we are not displaying the bytes in powers of 2, but powers of 10 ([see here][2]). I.e. 1KB is 1000B vs 1KiB is 1024B. This PR adds the same logic to all index stats requests so we also time split by 24 intervals all requests that hit the Index Stats API endpoint. We also use powers of 2 instead of 10 on the message when enforcing `max_query_bytes_read` and `max_querier_bytes_read`.  Note that the library we use under the hoot to print the bytes rounds up and down to the nearest integer ([see][3]); that's why we see 16GiB compared to the 15.5GB in the Grafana query editor. **Which issue(s) this PR fixes**: Fixes https://github.com/grafana/loki/issues/8910 **Special notes for your reviewer**: - I refactored the`newQuerySizeLimiter` function and the rest of the _Tripperwares_ in `rountrip.go` to reuse the new IndexStatsTripperware. So we configure the split-by-time middleware only once. **Checklist** - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [x] Documentation added - [x] Tests updated - [x] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` [1]: https://grafana.com/docs/loki/latest/api/#index-stats [2]: https://github.com/grafana/loki/blob/main/pkg/querier/queryrange/limits.go#L367-L368 [3]: https://github.com/dustin/go-humanize/blob/master/bytes.go#L75-L78		2 years ago
..
canary	Canary: support filtering / parsing logs with LogQL (#8871)	2 years ago
chunkenc	Use 0.28.1 build image and update go and alpine versions (#8583)	2 years ago
configs	Update prometheus/prometheus from 2.41 to 2.42 (#8571)	2 years ago
distributor	Distributor: add detail to stream rates failure (#8900)	2 years ago
ingester	WAL: remove ePool that is unused (#8669)	2 years ago
iter	Iterators: re-implement mergeEntryIterator using loser.Tree for performance (#8637)	2 years ago
logcli	Define `RequiredLabels` query limit. (#8851)	2 years ago
loghttp	Loki: add a 'since' query parameter for easier querying using relative time (#7964)	2 years ago
logproto	Cosmetic lint warning fixes (#8363)	2 years ago
logql	Short circuit parsing when label matchers are present (#8890)	2 years ago
logqlanalyzer	fixed logql-analyzer (#8258)	2 years ago
logqlmodel	Allow math on errors (#8823)	2 years ago
loki	feat(storage): add support for IBM cloud object storage as storage client (#8826)	2 years ago
lokifrontend	Extend scheduler queue metrics with enqueue/dequeue counters (#8891)	2 years ago
prom1/storage/metric	Copy `cortex/pkg/prom1` package dependency into Loki (#5196)	3 years ago
push	Bump golang.org/x/net from 0.4.0 to 0.7.0 in /pkg/push (#8562)	2 years ago
querier	Split index request in 24h intervals (#8909)	2 years ago
ruler	feat(storage): add support for IBM cloud object storage as storage client (#8826)	2 years ago
runtime	Loki/Promtail: Client Refactor (#3623)	4 years ago
scheduler	Extend scheduler queue metrics with enqueue/dequeue counters (#8891)	2 years ago
server	Configurable TLS MinVersion and CipherSuites (#7227)	3 years ago
sizing	Add OVHcloud to sizing tool (#8423)	2 years ago
storage	feat(storage): add support for IBM cloud object storage as storage client (#8826)	2 years ago
tracing	Promtail: Add basic tracing support (#7414)	3 years ago
usagestats	Usage stats reporter: fix to remove duplicate if block (#8392)	2 years ago
util	Iterators: re-implement mergeEntryIterator using loser.Tree for performance (#8637)	2 years ago
validation	Max bytes read limit (#8670)	2 years ago