apitech/loki - loki - Gitea: Git with a cup of tea

Commit Graph

Author	SHA1	Message	Date
George Robinson	5394a4cae1	fix: fix lint failure in retention.go (#17894 )	7 months ago
Sandeep Sukhani	267f4dc0ec	feat(compactor HS): add job queue and deletion job builder (#17843 )	7 months ago
Sandeep Sukhani	e8a94efcad	fix: set delete request batch to empty when we load to fail requests (#17810 ) Signed-off-by: Sandeep Sukhani <sandeep.d.sukhani@gmail.com>	8 months ago
Sandeep Sukhani	fdf3637ccf	chore: improve duplicate series detection while iterating table for retention/deletion (#17682 )	8 months ago
Sandeep Sukhani	a5e95bacee	fix: move the check for duplicate series up the call stack to fix an issue when retention is enabled (#17663 )	8 months ago
Sandeep Sukhani	47961f802d	feat: Compactor deletion manifest builder (#17474 )	8 months ago
Karsten Jeschkies	bf6f130cdf	refactor(stringlabels): Support stringlabels in chunkenc (#17263 )	8 months ago
Sandeep Sukhani	32c50888b0	fix: concurrent map access for processedSeries in delete requests manager (#17469 )	8 months ago
Footur	ba8830b080	docs: Document the default value for the working_directory for compactor (#17075 ) Signed-off-by: Footur <3769085+Footur@users.noreply.github.com> Co-authored-by: J Stickler <julie.stickler@grafana.com>	9 months ago
Christian Haudum	525b41155d	chore(engine): Add expression evaluator to evaluate physical expressions against a given record batch (#17257 ) The evaluator takes a physcial expressions and an arrow.Record as input and return a ColumnVector. This PR only implements the evaluation of the LiteralExpr and ColumnExpr. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	9 months ago
Karsten Jeschkies	8142f123f5	refactor(stringlabels): Support stringlabels in filter.Func (#17233 )	9 months ago
Karsten Jeschkies	a864f8b986	refactor(stringlabels): Support stringlabels in log.StreamPipeline (#17216 )	9 months ago
Sandeep Sukhani	288ec8c64d	fix: fix timeout and series progress marker for same requests with different shards (#17125 )	9 months ago
Sandeep Sukhani	db59d0e9b2	fix: wire up deletion series progress tracking (#17099 )	9 months ago
Sandeep Sukhani	e3e1f096ea	feat: store details of processed streams while processing delete requests (#16825 )	10 months ago
Paul Rogers	24c260e3e4	chore(deps): Update dskit (#16782 )	10 months ago
Sandeep Sukhani	3277321e21	feat: when using sqlite for storing delete requests, use the stored completion time to reduce the requests considered for querytime filtering (#16600 )	10 months ago
Paul Rogers	27431b7e7e	chore: Linting update for new golangci (#16572 )	10 months ago
Dylan Guedes	ea13730e63	feat: Snapshopt stream overrides values on each request (#16523 ) What this PR does / why we need it: Caches a snapshot of resolved policies and retention temporarily to guarantee consistent results on the same request. We evaluate retention and policies in a map when emitting some metrics so having policies or retention inconsistent along the chain for the same push request can cause us to miscalculate things or worse, interact with a nil map.	10 months ago
Sandeep Sukhani	5b33e65bcd	feat: add support for using sqlite for storing delete requests (#16437 )	10 months ago
Eng Zer Jun	982b16b404	refactor: replace min/max helpers with built-in min/max (#16438 ) Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>	10 months ago
Cyril Tovena	dbf2befc1d	feat: Distributed Operational UI (#16097 )	11 months ago
Sandeep Sukhani	5e6d0b5f12	fix: remove unused variable in delete requests store code (#16241 ) Signed-off-by: Sandeep Sukhani <sandeep.d.sukhani@gmail.com>	11 months ago
Sandeep Sukhani	bd0ca9387d	chore: refactor delete requests store interface to prepare for replacing boltdb with sqlite for storing delete requests (#16181 )	11 months ago
Sandeep Sukhani	89c60542a1	chore: avoid doing some unnecessary work while listing or merging sharded delete requests (#16125 )	11 months ago
Dylan Guedes	5c8e832260	feat: Introduce policy stream mapping (#15982 ) What this PR does / why we need it: Introduces the idea of policies to Loki, which are recognizable based on the given stream selectors. This is an improved version of https://github.com/grafana/loki/pull/15561 and built on top of https://github.com/grafana/loki/pull/15875. A policy mapping can be configured the following way: ```yaml 12345: policy_stream_mapping: policy6: - selector: `{env="prod"}` priority: 2 - selector: `{env=~"prod\|staging"}` priority: 1 - selector: `{team="finance"}` priority: 4 policy7: - selector: `{env=~"prod\|dev"}` priority: 3 ``` With that configuration, pushes to tenant `12345` with the labels `{env="prod", team="finance"}` would be assigned to policy6 because the third mapping for policy6 matches these labels and has higher priority than any other matching.	11 months ago
Sandeep Sukhani	c99771efb8	chore: upload delete requests table only when there are updates since last upload (#16008 )	11 months ago
Cyril Tovena	4df8c60b6b	chore(querier): Refactor the store and querier interface. (#15969 )	11 months ago
Dylan Guedes	0bc23db95d	refactor: Add retention hours to `discarded` metrics (#15875 ) What this PR does / why we need it: We have no distinction between the type of data that gets discarded. On this PR I'm adding `retention_hours` to our discarded metrics so that we can better look at what data was discarded.	11 months ago
Sandeep Sukhani	07ccf7c561	chore: merge back delete request when we are done processing all its splits (#15968 )	11 months ago
Sandeep Sukhani	81940c86d5	feat: dedupe duplicate delete requests while loading them for processing in compactor (#15852 )	12 months ago
Sandeep Sukhani	00d4fcb712	fix: do not update delete requests tracking metrics for users whose delete requests are not supposed to be processed (#15855 )	12 months ago
Sandeep Sukhani	905caa11cf	chore: align delete request splits to shard interval (#15790 )	12 months ago
Paul Rogers	f2c2a22bdc	chore: Preparation for incoming static code analysis CI check (#15164 ) Co-authored-by: Danny Cooper <danny.cooper@grafana.com>	1 year ago
Trevor Whitney	b5462b6639	chore: remove copied loop vars (#14383 )	1 year ago
Christian Haudum	f52f8ad540	chore: Clarify compression package (#14257 ) This PR renames "encoding" to "codec" in the compression package to remove the cognitive dissonance. It also removes the `Enc` prefix for codec identifiers, so that they adhere Go's best practice of naming conventions, e.g. `compression.EncGZIP` becomes `compression.GZIP` when used in a different package. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	1 year ago
Vladyslav Diachenko	31368806a9	feat: add backoff mechanism to the retention process (#14182 ) Signed-off-by: Vladyslav Diachenko <vlad.diachenko@grafana.com>	1 year ago
Christian Haudum	963779064b	chore: Move compression utilities into separate package (#14167 ) Compression tooling has been part of the `chunkenc` (chunk encoding) package in the past for legacy reasons. Since more components use this now, it's easier to keep it in a separate package. This also eliminates the confusion around "encoding", since this has been incorrectly used synonymously with "compression" in the past. --- Signed-off-by: Christian Haudum <christian.haudum@gmail.com> Co-authored-by: Robert Fratto <robertfratto@gmail.com>	1 year ago
Paul Rogers	18cef217fc	chore: Linting updates in prep for Go 1.23 (#14071 )	1 year ago
Marco Pracucci	b88583da7d	feat: upgrade prometheus (#13671 ) Signed-off-by: Marco Pracucci <marco@pracucci.com> Signed-off-by: Vladyslav Diachenko <vlad.diachenko@grafana.com> Co-authored-by: Vladyslav Diachenko <vlad.diachenko@grafana.com>	1 year ago
Christian Haudum	d8cc1cea83	chore: Make dedicated iterator package (#13273 ) Iterators of various types are widely used throughout the Loki code base. With the recent code additions of the bloom filters, a new set of utility functions for iterators emerged in `github.com/grafana/loki/v3/pkg/storage/bloom/v1`. The package defines interfaces for various common types, but also provides implementations to create and compose new iterators that implement these interfaces. This new package uses Go Generics. - However, at the current state, there are multiple iterator interfaces and implementations, which adds cognitive overhead in understanding how they work and which ones should be used. The idea is to unify them into a single iterator lib for Loki. - As a first step towards a single iterator library for the Loki code base, this PR moves the utilities from the `pkg/storage/bloom/v1` package to the `pkg/iter/v2` package. - Second, it changes the existing `EntryIterator` and `SampleIterator` iterators to "inherit" `v2.CloseIterator` in order to expose the same function names. - And lastly, naming conventions of iterator interfaces and structs are unified. The basic iterator interface (defined in `pkg/iter/v2/interface.go`) looks like so: ```go // Usage: // // for it.Next() { // curr := it.At() // // do something // } // if it.Err() != nil { // // do something // } type Iterator[T any] interface { Next() bool Err() error At() T } ``` --- Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
Paul Rogers	40ee766724	feat: Collect duplicate log line metrics (#13084 )	2 years ago
Paul Rogers	d16a3bf139	test: First pass at unregistering metrics so that we can run multiple tests (#12927 )	2 years ago
Ed Welch	4c563f7823	docs: unhide compactor GRPC config (#12587 ) Signed-off-by: Edward Welch <edward.welch@grafana.com> Co-authored-by: DylanGuedes <djmgguedes@gmail.com>	2 years ago
Ed Welch	705379cc81	chore: remove experimental flags from features and configs which are no longer so. (#12481 ) Signed-off-by: Edward Welch <edward.welch@grafana.com>	2 years ago
Ed Welch	4c88be0ef2	chore: update loki modules for 3.0 release (#12433 ) Signed-off-by: Edward Welch <edward.welch@grafana.com>	2 years ago
Sandeep Sukhani	0b7ff48175	chore: delete request processing improvements (#12259 )	2 years ago
Poyzan	b9266883a0	fix: data race in unit test (#12224 )	2 years ago
Salva Corts	63c88489d0	refactor: Add RF and Tokens to Loki ring Cfg and allow overwriting docs. (#12142 ) Co-authored-by: Owen Diehl <ow.diehl@gmail.com>	2 years ago
Sandeep Sukhani	74e94cd1c5	fix: avoid wiping out delete request list cache in queriers when compactor is down (#12210 )	2 years ago

1 2

68 Commits (60c76318e66d33a726b41d8542e71663e581c92a)