3177 Commits (30528c9f6695d7e55aa904b83df1308becc93d21)

Author	SHA1	Message	Date
Callum Styan	4d6992982d	feat: add query user and query source to "executing query" log lines (#14320) ... Signed-off-by: Callum Styan <callumstyan@gmail.com>	8 months ago
Christian Haudum	f52f8ad540	chore: Clarify compression package (#14257) ... This PR renames "encoding" to "codec" in the compression package to remove the cognitive dissonance. It also removes the `Enc` prefix for codec identifiers, so that they adhere Go's best practice of naming conventions, e.g. `compression.EncGZIP` becomes `compression.GZIP` when used in a different package. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	8 months ago
Paul Rogers	84788ad2ae	chore: Update for failing lint issue (#14301)	8 months ago
Trevor Whitney	39119c0c9f	chore: refactor detected fields handler (#14288) ... Co-authored-by: Paul Rogers <paul.rogers@grafana.com>	8 months ago
Emad Mohamadi	9267ee3561	fix(canary): Reconnect immediately upon tail max duration (#14287)	8 months ago
Robert Jacob	59ff1ece1d	fix(config): Copy Alibaba and IBM object storage configuration from common (#14297)	8 months ago
Ashwanth	a584fb713d	chore: sync tables without acquiring read lock the whole time (#14179)	8 months ago
Ashwanth	17c472d9ab	fix(sharding): apply offset to both `from` and `through` in shard request (#14256)	8 months ago
George Robinson	6354deda90	fix: missing dep PartitionRing for Ingester (#14292)	8 months ago
Ravishankar	2d4792a54f	fix: allow rename of structuremetadata labels (#13955) ... Co-authored-by: Christian Haudum <christian.haudum@gmail.com>	8 months ago
Paul Rogers	04f87ca933	chore: Lint in prep for golangci compatible with Go 1.23 (#14289)	8 months ago
benclive	3c36ba949d	feat: Implement owned streams calculation using Partition Ring (#14282)	8 months ago
Cyril Tovena	beca6f3366	feat(kafka): enqueue commit offset only once per batch process (#14278) ... Co-authored-by: George Robinson <george.robinson@grafana.com>	8 months ago
Paul Rogers	c874d2cc44	chore: Wrap error to surface more context (#14284)	8 months ago
Cyril Tovena	f80d68a1ed	feat(distributors): Use a pool of worker to push to ingesters. (#14245)	8 months ago
Christian Haudum	6143a548f1	chore: Disable bloom filtering for bounded sharding strategy (#14272) ... This commit temporarily disables bloom filtering chunk refs in the `GetShards()` call on the index gateway when using the `bounded` sharding strategy. This will result in chunks only being filtered once by the bloom gateway, reducing the amount of checks that need to be done on the bloom filters for a single request.Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	8 months ago
Dylan Guedes	ac422b3bc3	feat: Introduce new `ObjectExistsWithSize` API to (#14268) ... **What this PR does / why we need it**: Introduce a new `ObjectExistsWithSize` API to our object storage interface. This is the same as `ObjectExists` but with the object size as part of the return value. This is useful to compare the object size present.	8 months ago
Trevor Whitney	c001a1d93a	fix: allow any level for aggregated metrics (#14255)	8 months ago
Trevor Whitney	36ace66b73	feat: move detected field logic to query frontend (#14212)	8 months ago
Ashwanth	758364c777	fix: remove usage of unsafe string in label adapter unmarshal (#14216)	8 months ago
Christian Haudum	14b468b236	chore: Add `has_labelfilter_before_parser` to metrics.go log line (#14244) ... The value of this field indicates that the query could potentially benefit from bloom filters on structured metadata, if the queried field is a structured metadata field. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	8 months ago
Cyril Tovena	ae955ed30d	fix(pattern): Fixes latency metric namespace for tee to pattern (#14241)	8 months ago
Cyril Tovena	3f47233079	fix(kafka): Fixes partition selection in distributors (#14242)	8 months ago
Cyril Tovena	b6e9945f83	feat(kafka): Add Ingestion from Kafka in Ingesters (#14192) ... Co-authored-by: Joao Marcal <joao.marcal12@gmail.com> Co-authored-by: benclive <ben.clive@grafana.com>	8 months ago
Paul Rogers	a4642b55e9	fix: Update AWS storage timeout error for Go 1.23 behavior (#14226)	8 months ago
Paul Rogers	71d7291c9c	fix: Expand matching for additional variations (#14221)	8 months ago
Paul Rogers	8322e518e6	fix: Add additional validation for timeout while retrieving headers (#14217)	8 months ago
benclive	7c78232ad3	feat: Optionally require writes to kafka on Push requests (#14186)	8 months ago
Cyril Tovena	a4aee4f4ff	fix: Wait for OwnedStreams service in Ingester startup (#14208)	8 months ago
Christian Haudum	cdf084fdae	perf(blooms): Remove compression of `.tar` archived bloom blocks (#14159) ... Decompression is a CPU intensive task, especially un-gzipping. The gain of compressing a tar archive of storage optimized binary blocks is neglectable (question: is it?). In this example, the block of ~170MiB is ~3.3MiB bigger when not compressed, which is a ratio of ~2% ```bash $ ls -las 2bc017f79711e12a-2bffc5dcc0e8e964_1726004114913-1726106283939-bc42f529.tar 177048 -rw-rw-r-- 1 christian christian 181293056 Sep 18 13:49 2bc017f79711e12a-2bffc5dcc0e8e964_1726004114913-1726106283939-bc42f529.tar $ ls -las 2bc017f79711e12a-2bffc5dcc0e8e964_1726004114913-1726106283939-bc42f529.tar.gz 173732 -rw-rw-r-- 1 christian christian 177897689 Sep 18 13:49 2bc017f79711e12a-2bffc5dcc0e8e964_1726004114913-1726106283939-bc42f529.tar.gz $ qalc '(181293056 - 177897689) / 1024/ 1024' ((181293056 − 177897689) / 1024) / 1024 ≈ 3.238074303 $ qalc '181293056 / 177897689' 181293056 / 177897689 ≈ 1.019086066 ``` After some consideration, we decided to store the encoding of the bloom block in the `BlockRef`. This means, that the changes in this PR do not break compatibility with existing blocks compressed with gzip, although new blocks will not be compressed any more. However, the PR adds support for different compression algorithms, such as gzip, snappy, lz4, flate, and zstd. Compression is not configurable yet. --- Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	8 months ago
Sven Grossmann	f8d9143eea	fix(log-to-span): timestamp.Time should be called with milliseconds (#14196)	8 months ago
Paul Rogers	fc90a63636	feat: implement IsRetryableErr for S3ObjectClient (#14174) ... **What this PR does / why we need it**: Adds support for `IsRetryableErr`in the `S3ObjectClient`, similar to what is found in the `GCSObjectClient`	8 months ago
Christian Haudum	2840d48b92	chore: Clean up unused bloom filter related code (#14183) ... This PR tries to eliminate dead code in the bloom package, mainly related to line tokenization. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	8 months ago
Ashwanth	900751c3bb	fix: report correct status code for metric and log queries in metrics.go (#12102)	8 months ago
Dylan Guedes	a47515300a	fix: Report PSRL error message correctly (#14187) ... **What this PR does / why we need it**: Modify our `validateEntries` code to report a log failure when a PSRL is identified.	8 months ago
Vladyslav Diachenko	31368806a9	feat: add backoff mechanism to the retention process (#14182) ... Signed-off-by: Vladyslav Diachenko <vlad.diachenko@grafana.com>	8 months ago
Christian Haudum	963779064b	chore: Move compression utilities into separate package (#14167) ... Compression tooling has been part of the `chunkenc` (chunk encoding) package in the past for legacy reasons. Since more components use this now, it's easier to keep it in a separate package. This also eliminates the confusion around "encoding", since this has been incorrectly used synonymously with "compression" in the past. --- Signed-off-by: Christian Haudum <christian.haudum@gmail.com> Co-authored-by: Robert Fratto <robertfratto@gmail.com>	8 months ago
Robert Fratto	1b7086632e	chore(storage/bloom/v1): update comments on bloom test (#14168)	8 months ago
Christian Haudum	02b26a85df	chore: Remove outdate TODO file for bloom package (#14158) ... Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	8 months ago
Cyril Tovena	44958c4b8b	chore(kafka): Refactor configurations and remove old modules (#14161)	8 months ago
Karsten Jeschkies	48e4a346b8	chore: Remove unused quantile merge step evaluator. (#14160)	8 months ago
Christian Haudum	6e36041c08	chore(blooms): Remove excessive logging in fused querier (#14152) ... The FusedQuerier emits a large amount of log lines at level WARN if a bloom is empty. Since the introduction of structured metadata blooms, this happens every time a series does not have structured metadata in any of its entries. In the future, an optimisation could be to not write blooms for series at all, if they contain no data. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	8 months ago
benclive	78b275bf10	fix: Propagate query stats from quantile & topk queries (#13831) ... Signed-off-by: Callum Styan <callumstyan@gmail.com> Co-authored-by: Callum Styan <callumstyan@gmail.com>	8 months ago
Christian Haudum	39fa9b47e9	chore: Fix linter errors (#14145) ... Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	8 months ago
Christian Haudum	5bffc10d85	chore(blooms): Replace extracting line filters with extracting label filters (#14137) ... Replace the logic for extracting line filters with extracting label filters on the bloom gateway client, because otherwise the bloom gateway server would receive requests with no label matchers. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	9 months ago
Robert Fratto	ebe0bb875e	chore: replace remaining references to ExtractTestableLineFilters (#14135)	9 months ago
Robert Fratto	4de51f9953	chore(bloomgateway): update bloom gateway to test for structured metadata blooms (#14130) ... Signed-off-by: Robert Fratto <robertfratto@gmail.com> Signed-off-by: Christian Haudum <christian.haudum@gmail.com> Co-authored-by: Christian Haudum <christian.haudum@gmail.com>	9 months ago
Sandeep Sukhani	89c4282654	chore: avoid some allocations in tsdb while reading posting offset table (#14111)	9 months ago
Sven Grossmann	63e84b476a	fix(aggregated-metrics): correctly create logfmt string (#14124)	9 months ago
Robert Fratto	53cfef3bca	chore(storage/bloom/v1): add ExtractTestableLabelMatchers (#14119) ... Signed-off-by: Robert Fratto <robertfratto@gmail.com>	9 months ago