2 Commits (a5aa8b315da22ef94763f9e048b1a77840e724fd)

Author	SHA1	Message	Date
Christian Haudum	62176339c4	Bloom querying: Propagate line filters from querier to bloom filter (#11492) ... In order to query the bloom filters with the search strings from the user query, we need to propagate the line filters through the different implementations of the store, namely through the GetChunks function. In order to avoid converting data structures, the line filter from the syntax tree is used as custom protobuf type, because it is passed from querier to index gateway and from index gateway to bloom gateway. So far, the line filters request have not been tokenized to n-grams. This PR adds the ability to read the n-gram length from the block schema at query time and tokenizes the line filters from the input just ahead of searching the block. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	1 year ago
Christian Haudum	d62d4e37d1	Bloom Gateway: Implement chunk filtering using workers that multiplex requests (#11181) ... This change adds an internal request queue to the bloom gateway. Instead of executing every single request individually, which involves resolving bloom blocks, downloading them if needed and executing the chunk filtering, requests are now enqueued to the internal, per-tenant queue. The queue implements the same shuffle sharding mechanism as the queue in the query scheduler component. Workers then dequeue a batch of requests for a single tenant and multiplex them into a single processing task for each day. This has the big advantage that the chunks of multiple requests can be processed in a single sequential scan through a set a bloom blocks, without needing to skip back and forth within the binary stream of the block. --------- Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago