Like Prometheus, but for logs.

prometheushacktoberfestgrafanacloudnativeloggingloki

You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

Christian Haudum cdf084fdae perf(blooms): Remove compression of `.tar` archived bloom blocks (#14159) ... Decompression is a CPU intensive task, especially un-gzipping. The gain of compressing a tar archive of storage optimized binary blocks is neglectable (question: is it?). In this example, the block of ~170MiB is ~3.3MiB bigger when not compressed, which is a ratio of ~2% ```bash $ ls -las 2bc017f79711e12a-2bffc5dcc0e8e964_1726004114913-1726106283939-bc42f529.tar 177048 -rw-rw-r-- 1 christian christian 181293056 Sep 18 13:49 2bc017f79711e12a-2bffc5dcc0e8e964_1726004114913-1726106283939-bc42f529.tar $ ls -las 2bc017f79711e12a-2bffc5dcc0e8e964_1726004114913-1726106283939-bc42f529.tar.gz 173732 -rw-rw-r-- 1 christian christian 177897689 Sep 18 13:49 2bc017f79711e12a-2bffc5dcc0e8e964_1726004114913-1726106283939-bc42f529.tar.gz $ qalc '(181293056 - 177897689) / 1024/ 1024' ((181293056 − 177897689) / 1024) / 1024 ≈ 3.238074303 $ qalc '181293056 / 177897689' 181293056 / 177897689 ≈ 1.019086066 ``` After some consideration, we decided to store the encoding of the bloom block in the `BlockRef`. This means, that the changes in this PR do not break compatibility with existing blocks compressed with gzip, although new blocks will not be compressed any more. However, the PR adds support for different compression algorithms, such as gzip, snappy, lz4, flate, and zstd. Compression is not configurable yet. --- Signed-off-by: Christian Haudum <christian.haudum@gmail.com>		8 months ago
..
bloomgateway.go	chore(bloomgateway): update bloom gateway to test for structured metadata blooms (#14130)	8 months ago
bloomgateway_test.go	perf(blooms): Remove compression of `.tar` archived bloom blocks (#14159)	8 months ago
cache.go	fix(blooms): Deduplicate filtered series and chunks (#12791)	1 year ago
cache_test.go	fix(blooms): Fully deduplicate chunks from FilterChunkRef responses (#12807)	1 year ago
client.go	feat: upgrade prometheus (#13671)	10 months ago
client_pool.go	perf(blooms): Resolve bloom blocks on index gateway and shard by block address (#12720)	1 year ago
client_pool_test.go	test: Add mutex and setter/getter for addresses in test (#12793)	1 year ago
client_test.go	fix(blooms): Suppress error from resolving server addresses for blocks (#13385)	11 months ago
config.go	chore(blooms): Clean up settings and make reasonable defaults (#12483)	1 year ago
metrics.go	chore(bloomgateway): update bloom gateway to test for structured metadata blooms (#14130)	8 months ago
multiplexing.go	chore(bloomgateway): update bloom gateway to test for structured metadata blooms (#14130)	8 months ago
multiplexing_test.go	chore(bloomgateway): update bloom gateway to test for structured metadata blooms (#14130)	8 months ago
processor.go	chore(bloomgateway): update bloom gateway to test for structured metadata blooms (#14130)	8 months ago
processor_test.go	chore(bloomgateway): update bloom gateway to test for structured metadata blooms (#14130)	8 months ago
querier.go	chore: replace remaining references to ExtractTestableLineFilters (#14135)	8 months ago
querier_test.go	chore: replace remaining references to ExtractTestableLineFilters (#14135)	8 months ago
resolver.go	perf: Re-introduce fixed size memory pool for bloom querier (#13172)	11 months ago
resolver_test.go	fix(blooms): Fix a regression introduced with #12774 (#12776)	1 year ago
stats.go	chore(bloomgateway): update bloom gateway to test for structured metadata blooms (#14130)	8 months ago
util.go	refactor(blooms): more easily testable day partitioning (#13331)	11 months ago
util_test.go	refactor(blooms): more easily testable day partitioning (#13331)	11 months ago
worker.go	chore: Remove excessive logging from bloom gateway (#13603)	10 months ago