2773 Commits (c9142722684ad6aca2bb09ae5487d5a2af087d0f)

Author	SHA1	Message	Date
Poyzan	3dd5b706aa	fix: correct use of atomic int (#12225)	1 year ago
Poyzan	b9266883a0	fix: data race in unit test (#12224)	1 year ago
Poyzan	60dcee1b34	fix: data race in seriesIterator, use atomic int (#12223)	1 year ago
Salva Corts	2544f003cf	chore: Add summary for received filters in bloom gateway (#12158)	1 year ago
Salva Corts	63c88489d0	refactor: Add RF and Tokens to Loki ring Cfg and allow overwriting docs. (#12142) ... Co-authored-by: Owen Diehl <ow.diehl@gmail.com>	1 year ago
Christian Haudum	8abfc29e44	chore(bloom-gw): Process blocks in parallel (#12203) ... While processing a single block is rather fast, processing a lot of blocks sequentially can lead to problem that single slowly processed blocks lead to high tail latency. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	1 year ago
Salva Corts	4b28f8221b	fix: Read all series in NewTSDBSeriesIter and close index file (#12211)	1 year ago
Travis Patterson	1331dc5bdf	fix: add metadata resource and source attrs to stats (#12201)	1 year ago
Sandeep Sukhani	74e94cd1c5	fix: avoid wiping out delete request list cache in queriers when compactor is down (#12210)	1 year ago
Sandeep Sukhani	8b9d8b0226	feat: Make list of otel blessed attributes configurable (#12180)	1 year ago
Cyril Tovena	19f5f3fd44	fix: Fixes compactor shutdown log (#12195)	1 year ago
Sandeep Sukhani	2659e3597c	fix: fix middlewares being applied for series request when querier is running as a standalone service (#12194)	1 year ago
Sandeep Sukhani	25f15cb326	feat: add support for setting custom grpc interceptors for requests sent to index-gateway (#12193)	1 year ago
Owen Diehl	8a8756aa7a	chore(blooms): honors bloom_compactor_enabled for tenants (#12190)	1 year ago
Paul Rogers	bd3f83a45d	test: Add synchronization for fakeTailer responses (#12186)	1 year ago
Christian Haudum	552d1f613f	chore(bloom-gw): Parallelise bloom gateway queries (#12182) ... Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	1 year ago
Christian Haudum	a0fce39169	feat(bloom-gw): Make num multiplexing tasks configurable, and other adjustments (#12169) ... * We want to be able to change the amount of dequeued items per iterations to test different configurations. * Ignore block pages that are magnitues bigger than the target page size. * Increase download queue capacity to avoid blocking when enqueueing Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	1 year ago
Owen Diehl	e95cc6cc09	chore(blooms): Adds basic tracing implementation to bloom-gw (#12175) ... Also * removes unused `SyncMap` * adds a `BlockRefFrom` helper	1 year ago
Christian Haudum	f6cfff5356	refactor(bloomstore): Introduce fetch option for blocks fetcher (#12160) ... Requests to the bloom gateway should not wait for blocks to be downloaded into cache first, but should operate on readily available blocks. This PR introduces fetch options for the bloom store that specify how the store should behave when requesting blocks from block refs. `WithIgnoreMissing`: Ignore errors from blocks that could not be found in object storage, and instead return a `nil` value in the response. `WithFetchAsync`: Return only the blocks that are available locally in cache and dispatch downloading of blocks to the blocks downloader, that operates in the background, and make them available for subsequent requests. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	1 year ago
Owen Diehl	f4b2c5d69b	fix(blooms): bloom-gw chunk merging improvements (#12162)	1 year ago
Salva Corts	49f8f7b3ee	refactor: Chunkref as alias to logproto.ShortRef (#12159)	1 year ago
Owen Diehl	9a993ca690	chore(blooms): Ordering & Unless iterator (#12154)	1 year ago
Owen Diehl	8f3767c4ad	chore(blooms): block_query_latency_seconds rename (#12152)	1 year ago
Owen Diehl	3658dc922a	perf(blooms): removes pendingTasks impl with an atomic.Int64 (#12151)	1 year ago
Owen Diehl	30a240b470	fix(blooms): filter out nil blocks (#12150)	1 year ago
Owen Diehl	fd702f6691	chore(blooms): remove unused helper from bloomgateway (#12148)	1 year ago
Owen Diehl	102e3ea07f	chore(blooms): use the right iter reference in blockLoadingIter.loadNext (#12146)	1 year ago
Travis Patterson	bf11a7e2b4	fix: make it possible to wire up the request wrapper and created object store clients from enterprise logs (#12134)	1 year ago
Christian Haudum	1556cc0c68	chore(blooms): Remove unnecessary sorting of fingerprints (#12139) ... Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	1 year ago
Owen Diehl	9235229b9e	refactor(bloom-gw): todos and comments (#12135)	1 year ago
Paul Rogers	0ddf6fc49d	test: Fix for a few data races (#12129)	1 year ago
Owen Diehl	63e0bcb84b	fix(blooms): embeddedcache size panic (#12130)	1 year ago
Salva Corts	f7710ad7aa	feat: (blooms) Configurable number of tokens (#12125)	1 year ago
Christian Haudum	38e2d7be4a	chore(blooms): Various bloom gateway fixes/improvements (#12120) ... * Response order of `FetchBlocks` is deterministic The response keeps the order of the refs from the request. * Update reference documentation * Skip not found blocks * Unify error messages with object key in bloom client * Change default value for bloom gateway shard size to 0 * Use only 16 tokens per ring instance * Add debug log line to each request in the bloom gateway client Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	1 year ago
Paul Rogers	97ae790f36	test: Update chunk/cache/mock to have a GetKeys() function (#12122)	1 year ago
Owen Diehl	17988d5965	fix(blooms): measurable compaction progress (#12101)	1 year ago
Travis Patterson	b0478e61c8	feat: add the ability to inject log parsers to the distributor (#12029)	1 year ago
Christian Haudum	4cae003ece	chore(blooms): Make blocks cache preloading less verbose (#12107) ... The warnings suggest something going wrong where it's not the case. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	1 year ago
Salva Corts	a8aa3f68cb	feat: Bloom filter regexes (#12096)	1 year ago
Christian Haudum	90bcaea724	chore(blooms): Add more instrumentation to bloom gateway and bloom store (#12048) ... * Add metric for size of metas/blocks fetched Getting the distribution of the uncompressed size of metas and blocks, grouped by their source (cache/filesystem/storage). * Move blocks/metas fetched metrics into bloomstore * Add minimal logging to bloom gateway processor * Observe queue duration of tasks The histogram metric gives us the insight about how long tasks spend their time enqueued waiting to be processed. * Remove queueMetrics field from bloom gateway struct The metrics are only used by the queue instance, which is part of the bloom gateway struct as well. * Move various bloom gateway metrics into separate file This commit introduces a new metrics struct that contains both the metrics specific the server/handler and the metrics specific to the worker. This has the advantage that metrics can be instantiated and accessed from one single place. --- Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	1 year ago
Paul Rogers	77a2580052	fix: Add data race protection for the tm.wg var (#12100)	1 year ago
Christian Haudum	184d1de4db	chore(blooms): Use `bloomshipper.Interval` instead of `model.Interval` (#12079) ... Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	1 year ago
Christian Haudum	637c292cd8	fix(blooms): Fix issue when loading cache on startup if directory does not exist (#12080) ... The check `!dirEntry.IsDir()` panicked in case `dirEntry` is `nil`, which is the case when the root directory does not exist. The PR also adds more logging. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	1 year ago
Danny Kopping	b5f488caec	fix(query/query-frontend): correct congestion control stats aggregation (#12072) ... Signed-off-by: Danny Kopping <danny.kopping@grafana.com>	1 year ago
Danny Kopping	609fdb3214	feat(querier/query-frontend): track and log congestion control latency (#12058) ... Signed-off-by: Danny Kopping <danny.kopping@grafana.com>	1 year ago
Shantanu Alshi	b1a61b3df6	chore: Change local setup under dev to use tsdb and v13 schema (#12057) ... Co-authored-by: Christian Haudum <christian.haudum@gmail.com>	1 year ago
Owen Diehl	60f75f3395	fix(blooms): closes chunk iterator (#12053)	1 year ago
Christian Haudum	cb7e016d64	chore(blooms): Populate blocks cache on startup (#12056) ... Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	1 year ago
Karsten Jeschkies	6578a00481	feat: Support usage trackers for received and discarded bytes. (#11840)	1 year ago
Christian Haudum	2f54f8d5be	chore(blooms): Implement BloomStore as a service (#12044) ... The bloomstore service is instantiated once at startup and used as dependency for both the bloom compactor and the bloom gateway. This will prevent instantiating the store in different ways. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	1 year ago