518 Commits (bbbd82bc73322d662ba81efeda3884efcdc09708)

Author	SHA1	Message	Date
Paul Rogers	02205f480e	chore: Remove call to deprecated rand.Seed() (#14041)	1 year ago
benclive	a0c75987a2	fix: Propagate headers/warnings/stats from quantile downstreams (#13881)	1 year ago
Travis Patterson	3bf7fa9f15	fix: sanatize structured metadata at query time (#13983)	1 year ago
Ashwanth	ae938d06d9	fix: record datasample queries are limited query type (#13930)	1 year ago
Johan Lindvall	40f4f14791	fix: Improve execution speed for queries with label filters (#13922) ... Co-authored-by: Travis Patterson <travis.patterson@grafana.com>	1 year ago
Trevor Whitney	c6ab6b31e8	chore: remove initial metric aggregation experiment (#13729)	1 year ago
Ned Andreev	5ef83a741b	fix: panic when parsing and extracting JSON key values (#13790)	1 year ago
Marco Pracucci	b88583da7d	feat: upgrade prometheus (#13671) ... Signed-off-by: Marco Pracucci <marco@pracucci.com> Signed-off-by: Vladyslav Diachenko <vlad.diachenko@grafana.com> Co-authored-by: Vladyslav Diachenko <vlad.diachenko@grafana.com>	1 year ago
jackyin	6dd6b65139	fix: ast left cycular reference result in oom (#13501) ... Co-authored-by: Travis Patterson <travis.patterson@grafana.com>	1 year ago
Kaviraj Kanagaraj	94bd3a042a	chore(lint-error): remove unused `stepCount` in `engine.go` (#13709) ... Signed-off-by: Kaviraj Kanagaraj <kavirajkanagaraj@gmail.com>	1 year ago
Callum Styan	29a37d5dcd	fix: special case the return values from a sharded first/last_over_time query (#13578) ... Signed-off-by: Callum Styan <callumstyan@gmail.com>	1 year ago
Ashwanth	65697676e6	chore: do not shard log queries with empty filter (#13568)	2 years ago
Karsten Jeschkies	08615bf751	fix: Return empty vector instead of nil for empty evaluator. (#13485)	2 years ago
Christian Haudum	e0ca67dd45	fix(query engine): Include lines with ts equal to end timestamp of the query range when executing range aggregations (#13448) ... **Background** When performing range vector aggregations, such as `count_over_time({env="dev"}[1h])`, the query range is divided into multiple steps at which the aggregation operation (e.g. counting the log lines) is evaluated. Each step starts at `current step - step interval` and ends at `current step`, as depicted in the following chart. The select range for the logs is extended by the `step interval` into the past, in order to select logs for calculating the first step.  However, the select range for logs is `start` inclusive and `end` exclusive (written as `[start, end)`), but the evaluation of the steps for the range aggregation is `start` exclusive and `end` inclusive (written as `(start, end]`). This leads to the problem that the very first timestamp at the beginning of the select range and the very last timestamp at the end of the select range are not included in the range aggregation. The "missing" last timestamp is not a problem, because a) in an instant query it is not supposed to be included anyway because of the `[start, end)` inclusivity of the query range and b) in a range query the last point of the previous step will be part of the next step evaluation. **Issue** The missing first timestamp, however, gets problematic when executing an instant query and the log timestamps are exactly at the start of the query range. This can happen when the query is split in the query frontend into multiple smaller time ranges, e.g. `1h`, `30m`, ... Since the sub queries are executed independently on the queriers, all logs that have a timestamp exactly a multiple of the split interval, e.g. 00:00, 01:00, 02:00, ... for a 1h interval, are dismissed and therefore missing in the query result over the full time range of the original query. **Fix** In order to avoid the missing logs that have a timestamp a multiple of the split interval in instant queries, we need to adjust the query range for logs to also include the `end` timestamp (written as `[start, end]`). This is done by adding a "leap nanosecond" to the `end` timestamp of the log select range. This ensures that the included `end` timestamp of the step evaluation is also included in the log selection. --- Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
Christian Haudum	845359d4ab	test: Add range aggregation test for instant queries (#13447) ... This PR adds test coverage for the `streamRangeVectorIterator`, which is used by instant queries to evaluate the single step of the query. Signed-off-by: Christian Haudum <christian.haudum@gmail.com> Co-authored-by: Ashwanth <iamashwanth@gmail.com>	2 years ago
Christian Haudum	d8cc1cea83	chore: Make dedicated iterator package (#13273) ... Iterators of various types are widely used throughout the Loki code base. With the recent code additions of the bloom filters, a new set of utility functions for iterators emerged in `github.com/grafana/loki/v3/pkg/storage/bloom/v1`. The package defines interfaces for various common types, but also provides implementations to create and compose new iterators that implement these interfaces. This new package uses Go Generics. - However, at the current state, there are multiple iterator interfaces and implementations, which adds cognitive overhead in understanding how they work and which ones should be used. The idea is to unify them into a single iterator lib for Loki. - As a first step towards a single iterator library for the Loki code base, this PR moves the utilities from the `pkg/storage/bloom/v1` package to the `pkg/iter/v2` package. - Second, it changes the existing `EntryIterator` and `SampleIterator` iterators to "inherit" `v2.CloseIterator` in order to expose the same function names. - And lastly, naming conventions of iterator interfaces and structs are unified. The basic iterator interface (defined in `pkg/iter/v2/interface.go`) looks like so: ```go // Usage: // // for it.Next() { // curr := it.At() // // do something // } // if it.Err() != nil { // // do something // } type Iterator[T any] interface { Next() bool Err() error At() T } ``` --- Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
Ashwanth	eb8a363066	fix(sharding): use without() grouping when merging `avg_over_time` shard results (#12176)	2 years ago
Dylan Guedes	4eb45cc588	refactor: Introduce context cause to our code (#13224) ... Introduce the usage of Go new `WithCause` functions to make our context cancellation errors more descriptive.	2 years ago
Dylan Guedes	c1fada9af0	refactor: Remove unnecessary spanlogger usage (#13255) ... Historically, we've been using `spanlogger` to enrich our spans. That isn't the right usage for it (as of now), as they'll be emitting log lines. Our current usage is causing: - Noisy/important log lines less visible - Unnecessary logging volume - Spread of the bad practice. With this PR I'm moving away from `spanlogger` all places where the log message is too bad or nonexistent. That's because these cases are indicating we don't care about that log line at all, and only about the data injected in the span.	2 years ago
Trevor Whitney	467eb1bb1b	feat: collect and serve pre-aggregated bytes and counts (#13020) ... Co-authored-by: Cyril Tovena <cyril.tovena@gmail.com>	2 years ago
Sven Grossmann	9981e9e40d	fix(orFilters): fix multiple or filters would get wrong filtertype (#13169)	2 years ago
benclive	6df81db978	feat: Support negative numbers in LogQL (#13091)	2 years ago
benclive	f6f8babf83	feat: Add utf8 support to Pattern Lexer to support utf8 chars (#13085)	2 years ago
Karsten Jeschkies	6e4555010e	fix: Introduce feature flag for [last|first]_over_time sharding. (#13067)	2 years ago
Kaviraj Kanagaraj	88e545fc95	feat(cache): Add `Cache-Control: no-cache` support for Loki instant queries. (#12896) ... Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com>	2 years ago
Karsten Jeschkies	f66172eed1	feat: improve performance of `first_over_time` and `last_over_time` queries by sharding them (#11605) ... Signed-off-by: Callum Styan <callumstyan@gmail.com> Co-authored-by: Callum Styan <callumstyan@gmail.com>	2 years ago
Owen Diehl	5a643c7115	chore: reduces span footprint + double recording (#12864) ... Trying to remove some cruft from traces.	2 years ago
Owen Diehl	bc78d13d9b	feat(blooms): compute chunks once (#12664) ... Signed-off-by: Owen Diehl <ow.diehl@gmail.com>	2 years ago
Shantanu Alshi	587a6d20e9	feat: Detected labels from store (#12441)	2 years ago
Owen Diehl	9c25985b97	feat(blooms): record time spent resolving shards (#12636)	2 years ago
Travis Patterson	11e768726f	fix(query sharding): Generalize avg -> sum/count sharding using existing binop mapper (#12599)	2 years ago
Cyril Tovena	48dae4417c	feat: improve syntax parser for pattern (#12489)	2 years ago
Kaviraj Kanagaraj	e9b6ce9de4	fix(logql): Linefilter behaviour with chained `or` (#12463) ... Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com>	2 years ago
Travis Patterson	cf04ec1320	feat: Enable warnings in Loki query responses (#12425)	2 years ago
Ed Welch	4c88be0ef2	chore: update loki modules for 3.0 release (#12433) ... Signed-off-by: Edward Welch <edward.welch@grafana.com>	2 years ago
Anton Kolesnikov	36c703dae8	feat: add pattern match line filter (#12398)	2 years ago
Shantanu Alshi	5190dda64b	feat(detected_labels): Initial skeleton for the API (#12390) ... Co-authored-by: Cyril Tovena <cyril.tovena@gmail.com>	2 years ago
Ashwanth	c8c8477c93	chore(stats): add pipeline_wrapper_filtered_lines (#12399)	2 years ago
Owen Diehl	6e1547fe62	fix(blooms): logql shard embeds tsdb (#12396)	2 years ago
Owen Diehl	3922d3864a	fix(tsdb): correctly use bit prefix calculation in tsdb shard matching (#12394)	2 years ago
Anton Kolesnikov	cc941fe42a	chore: refactor line filter MatchType (#12388)	2 years ago
Trevor Whitney	5b4bfa54b6	feat(detected_fields): initial plumbing of new endpoint (#12385)	2 years ago
Travis Patterson	2b8db8b691	fix: attempt non-string label filtering when errors are present (#12378)	2 years ago
Owen Diehl	a36483b731	feat(blooms): bloom integration in query planning (#12208) ... Signed-off-by: Owen Diehl <ow.diehl@gmail.com>	2 years ago
Travis Patterson	04398a1562	fix: log state of 'disable pipeline wrappers' (#12345)	2 years ago
Callum Styan	ab17a50b90	fix: don't shard `quantile_over_time` if it's not the top level aggregation for a query (#12326) ... Signed-off-by: Callum Styan <callumstyan@gmail.com>	2 years ago
Callum Styan	611cb01b70	fix: quantile sharding; fix bug where sharding would always happen (#12283) ... Signed-off-by: Callum Styan <callumstyan@gmail.com>	2 years ago
Travis Patterson	d175184fe5	chore: remove shards from wrapper signature (#12322)	2 years ago
Travis Patterson	1d99b06514	fix: explicitly serialize and deserialize noop label filters (#12124)	2 years ago
Ashwanth	3b5ed8c3ec	chore: pass shard count to injected pipelines and extractors (#12213)	2 years ago