14 Commits (a00f1f1b0b8f536f2cdac2f8857eb40c716aa696)

Author	SHA1	Message	Date
Ed Welch	4c88be0ef2	chore: update loki modules for 3.0 release (#12433) ... Signed-off-by: Edward Welch <edward.welch@grafana.com>	2 years ago
Salva Corts	cf71ac7114	fix(blooms): Remove blocks not matching any series in task (#12401)	2 years ago
Salva Corts	49f8f7b3ee	refactor: Chunkref as alias to logproto.ShortRef (#12159)	2 years ago
Owen Diehl	fd702f6691	chore(blooms): remove unused helper from bloomgateway (#12148)	2 years ago
Christian Haudum	184d1de4db	chore(blooms): Use `bloomshipper.Interval` instead of `model.Interval` (#12079) ... Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
Salva Corts	4fa5148eb5	refactor: Pass query plan down to bloom gateway (#12037)	2 years ago
Christian Haudum	fb728a6781	(chore) Cleanup duplicate functions/strucs in bloomgateway package (#11978) ... Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
Christian Haudum	1c43991ddc	Remove `bloomcompactor.DayTable` in favour of `config.DayTime` (#11917) ... Both structs shared the same semantics. This PR moves additional functionality from the `DayTable` to the `DayTime` struct. To get the table name of a day (ordinal number of day since unix epoch), call `DayTime.Table()`. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
Christian Haudum	6871f3c555	Bloom Gateway: Partition requests into multiple tasks before enqueuing (#11768) ... **What this PR does / why we need it**: This PR changes at which point a request is split into multiple tasks in case the chunks boundaries of the request overlap multiple days. This solves the problem that the request handler needs to know for how many responses from the block querier needs to wait. Previously, a single task that spanned across multiple days was executed multiple times (once for each matching day). --------- Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
Christian Haudum	600e390b16	Extend `TestPartitionFingerprintRange` test case (#11750) ... **What this PR does / why we need it**: The test cases now also cover partially out of bounds blocks as well as inconsecutive block ranges. --------- Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
Christian Haudum	62176339c4	Bloom querying: Propagate line filters from querier to bloom filter (#11492) ... In order to query the bloom filters with the search strings from the user query, we need to propagate the line filters through the different implementations of the store, namely through the GetChunks function. In order to avoid converting data structures, the line filter from the syntax tree is used as custom protobuf type, because it is passed from querier to index gateway and from index gateway to bloom gateway. So far, the line filters request have not been tokenized to n-grams. This PR adds the ability to read the n-gram length from the block schema at query time and tokenizes the line filters from the input just ahead of searching the block. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
Christian Haudum	c4f5a57bc8	Bloom Compactor: Optimize check for fingerprint ownership (#11389) ... Calling `c.sharding.OwnsFingerprint(tenant, uint64(fingerprint))` for each Series of a TSDB index is very expensive, because it not only creates the tenant's sub-ring but also needs to check the fingerprint against it. Instead, we can pre-calculate the current instance's token ranges and check if the (uint32 converted) fingerprint is contained within these ranges. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
Christian Haudum	79693d79ef	Fix `getFromThrough()` function for ShortRefs (#11386) ... **What this PR does / why we need it**: Since chunks don't necessarily have the same duration, we cannot assume that the last item of a slice sorted by `From` time also has the greatest `Through` time. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
Christian Haudum	d62d4e37d1	Bloom Gateway: Implement chunk filtering using workers that multiplex requests (#11181) ... This change adds an internal request queue to the bloom gateway. Instead of executing every single request individually, which involves resolving bloom blocks, downloading them if needed and executing the chunk filtering, requests are now enqueued to the internal, per-tenant queue. The queue implements the same shuffle sharding mechanism as the queue in the query scheduler component. Workers then dequeue a batch of requests for a single tenant and multiplex them into a single processing task for each day. This has the big advantage that the chunks of multiple requests can be processed in a single sequential scan through a set a bloom blocks, without needing to skip back and forth within the binary stream of the block. --------- Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago