6268 Commits (a8aa3f68cb46289baf912b22e427b1a18a520bcc)

 

All Branches   

Search

Author	SHA1	Message	Date
Salva Corts	a8aa3f68cb	feat: Bloom filter regexes (#12096)	2 years ago
Christian Haudum	90bcaea724	chore(blooms): Add more instrumentation to bloom gateway and bloom store (#12048) ... * Add metric for size of metas/blocks fetched Getting the distribution of the uncompressed size of metas and blocks, grouped by their source (cache/filesystem/storage). * Move blocks/metas fetched metrics into bloomstore * Add minimal logging to bloom gateway processor * Observe queue duration of tasks The histogram metric gives us the insight about how long tasks spend their time enqueued waiting to be processed. * Remove queueMetrics field from bloom gateway struct The metrics are only used by the queue instance, which is part of the bloom gateway struct as well. * Move various bloom gateway metrics into separate file This commit introduces a new metrics struct that contains both the metrics specific the server/handler and the metrics specific to the worker. This has the advantage that metrics can be instantiated and accessed from one single place. --- Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
Paul Rogers	77a2580052	fix: Add data race protection for the tm.wg var (#12100)	2 years ago
J Stickler	b24d48d2f6	docs: Update READMEs (#12061)	2 years ago
J Stickler	0ac42e78c2	docs: restore links, update procedure in Quick Start (#12068) ... Co-authored-by: Jack Baldry <jack.baldry@grafana.com>	2 years ago
Trevor Whitney	c1fbab5228	chore(release-2.8.x): release 2.8.9 and 2.8.10 (#12089) ... Co-authored-by: loki-gh-app[bot] <160051081+loki-gh-app[bot]@users.noreply.github.com>	2 years ago
J Stickler	c39786bd51	docs: update storage topic to include azure (#12063)	2 years ago
Christian Haudum	184d1de4db	chore(blooms): Use `bloomshipper.Interval` instead of `model.Interval` (#12079) ... Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
Christian Haudum	637c292cd8	fix(blooms): Fix issue when loading cache on startup if directory does not exist (#12080) ... The check `!dirEntry.IsDir()` panicked in case `dirEntry` is `nil`, which is the case when the root directory does not exist. The PR also adds more logging. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
Danny Kopping	b5f488caec	fix(query/query-frontend): correct congestion control stats aggregation (#12072) ... Signed-off-by: Danny Kopping <danny.kopping@grafana.com>	2 years ago
Danny Kopping	609fdb3214	feat(querier/query-frontend): track and log congestion control latency (#12058) ... Signed-off-by: Danny Kopping <danny.kopping@grafana.com>	2 years ago
Shantanu Alshi	b1a61b3df6	chore: Change local setup under dev to use tsdb and v13 schema (#12057) ... Co-authored-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
Owen Diehl	60f75f3395	fix(blooms): closes chunk iterator (#12053)	2 years ago
Christian Haudum	cb7e016d64	chore(blooms): Populate blocks cache on startup (#12056) ... Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
Karsten Jeschkies	6578a00481	feat: Support usage trackers for received and discarded bytes. (#11840)	2 years ago
Christian Haudum	2f54f8d5be	chore(blooms): Implement BloomStore as a service (#12044) ... The bloomstore service is instantiated once at startup and used as dependency for both the bloom compactor and the bloom gateway. This will prevent instantiating the store in different ways. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
Owen Diehl	7c8992f4e4	fix(blooms): fix reference leak and resulting race condition in BloomPageDecoder (#12050)	2 years ago
Salva Corts	8193063a4a	feat: Convert line filter expressions to a set of bloom tests (#12035)	2 years ago
Salva Corts	4fa5148eb5	refactor: Pass query plan down to bloom gateway (#12037)	2 years ago
Trevor Whitney	782b938334	test: split unit and integrations suites to parallelize (#12039)	2 years ago
Christian Haudum	dbea8ba85b	chore(blooms): Various fixes in blooms read path (#12036) ... * Wire up bloom querier metrics to fix nil pointer panic * Use `bloomutils.TokenRangesForInstance` instead of `subRing.GetTokenRangesForInstance` to avoid `zone not set` error Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
Christian Haudum	b9ce005ec1	fix: Ensure working dir for bloomstore exists (#12019) ... Fail startup if directory does not exist or there are not enough permissions. This prevents the bloomstore to fail later in the process once it tries to download and extract blocks. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
Sheikh-Abubaker	817c516bd2	feat: area/helm: Added missing default values to support ServerSideApply (#11567) ... Signed-off-by: Sheikh-Abubaker <sheikhabubaker761@gmail.com> Co-authored-by: Michel Hollands <42814411+MichelHollands@users.noreply.github.com> Co-authored-by: Karsten Jeschkies <karsten.jeschkies@grafana.com>	2 years ago
Christian Haudum	32a9a3f45d	chore(blooms): Improve how block directory is extracted (#12030) ... This PR contains two changes: 1) Extracting the block without an intermediate temp file. The commit is cherry-picked from https://github.com/grafana/loki/pull/12021, but that PR got merged into the wrong branch 🙈 2) Strip `.tar.gz` suffix from block file when extracting it into a directory. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
Christian Haudum	1f56da206d	refactor: Simplify fingerprint partitioning in bloom gateway client (#12018) ... Use the recently introduced `tokenRangesForInstance()` function to determine the the ranges for each replication set. For this purpose, the function moves into the `bloomutils` packages. This PR eliminates the structs `addrsWithBounds` and `instanceWithFingerprints` (which had somewhat similar usage) and unifies them into `rsWithRanges` that holds both the fingerprint ranges and the assigned GroupedChunkRefs. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
Paul Rogers	edc9a44f66	test: Finish sync work for mockIngester.pushed var (#12028)	2 years ago
Christian Haudum	b47f2bfbc6	refactor: Limit processing of blocks to requested fingerprint ranges in bloom gateway (#11987) ... This PR limits the amount of data being processed for a single multiplexed iteration to the union of the fingerprint bounds of its requests, instead of looking at blocks from the complete fingerprint range. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
J Stickler	6e4a9f3bdd	docs: Restore link to flog (#12011) ... Co-authored-by: Trevor Whitney <trevorjwhitney@gmail.com>	2 years ago
Owen Diehl	1dce7b512c	fix(blooms): race condition in `BloomPageDecoder` (#12027)	2 years ago
Owen Diehl	e835b6192f	refactor(blooms): refactors `blockLoadingIter`, exposing more error cases + additional metrics (#12015)	2 years ago
Robert Jacob	aeaefe6aab	feat(operator): Extend Azure secret validation (#12007) ... Co-authored-by: Periklis Tsirakidis <periklis@redhat.com>	2 years ago
Christian Haudum	b9d8eefe27	test: Use table prefix in bloomstore tests (#12022) ... Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
Robert Jacob	1f3e64bbe1	fix(operator): Support using multiple buckets with AWS STS (#12008) ... Co-authored-by: Periklis Tsirakidis <periklis@redhat.com>	2 years ago
Trevor Whitney	65b3ec441b	chore: update to latest release action (#11808)	2 years ago
Owen Diehl	09aa0f80b5	Blooms/contiguous bounds union (#12009)	2 years ago
Callum Styan	b91bcecb3a	quantile sharding; fix bug related to # of steps (instant query mem usage) and use CollapsingLowestDense store for ddsketch (#11905) ... Signed-off-by: Callum Styan <callumstyan@gmail.com>	2 years ago
Owen Diehl	166229818d	[Blooms] Consistent hashing via tokens for bloomcompactor (#12002)	2 years ago
Danny Kopping	eb1379afb9	Remove unnecessary ingester fields from metrics.go which only work in single binary (#12005) ... Signed-off-by: Danny Kopping <danny.kopping@grafana.com>	2 years ago
Christian Haudum	71fa802c86	Map ring token keyspace (uint32) into fingerprint keyspace (uint64) (#11975) ... In order to compare the keyspace from the bloom gateway's ring with the fingerprints, they need to use the same keyspace. The ring, however, uses tokens within a `uint32` keyspace, whereas the fingerprints use a `uint64` keyspace. Therefore the ranges from the ring tokens need to be mapped into the `uint64` keyspace. This is done bit-shifting the Min value 32 bits to the left and the Max value 32 bits to the left plus adding `(1<<32)-1` to "fill" the remaining values up to the next Min value. The structs used to combined ring instance information and token/fingerprint ranges are yet to be simplified. However, this is not goal of this PR. Signed-off-by: Christian Haudum <christian.haudum@gmail.com> Co-authored-by: Owen Diehl <ow.diehl@gmail.com>	2 years ago
Salva Corts	620488656f	Set tracing.profiling-enabled to true by default (#12004)	2 years ago
Danny Kopping	bdea0b6df1	Add o11y for chunk data retrieved from ingesters (#12003) ... Signed-off-by: Danny Kopping <danny.kopping@grafana.com>	2 years ago
Kaviraj Kanagaraj	fac5997b18	feat: Support split align and caching for instant metric query results (#11814) ... Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com>	2 years ago
Salva Corts	b7cb85f92b	Tail queue (#11930)	2 years ago
renovate[bot]	0660cfc9df	fix(deps): update github.com/axiomhq/hyperloglog digest to 24bca3a (main) (#11756) ... Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>	2 years ago
Christian Haudum	38c09a09bd	Bloom gateway: Fix metric for fetched blocks (#11988) ... This PR fixes the metric for fetched blocks Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
Owen Diehl	85f7baaeda	Blooms/integration fixes (#11979)	2 years ago
Salva Corts	ffc61fbbf4	Remove blooms jsonnet components (#11984)	2 years ago
Alexander Lindeskär	50b51dd6ca	Helm: Add alibabacloud to isUsingObjectStorage. (#11946) ... Co-authored-by: Michel Hollands <42814411+MichelHollands@users.noreply.github.com>	2 years ago
Christian Haudum	9f86473b0a	Various bloom gateway instrumentation changes (#11983) ... See individual commit messages for details. * Improve help text for chunk removals metric * End counter metric with _total * Fix incorrect observation of failed tasks * Observe chunk removals from block querier * Move chunks/series metrics into bloom querier * Observe total/filtered series and total/filtered chunks in bloom gateway Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
Travis Patterson	918e7365db	Set compactor memory limits (#11970)	2 years ago