Like Prometheus, but for logs.

prometheushacktoberfestgrafanacloudnativeloggingloki

You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

Christian Haudum e0ca67dd45 fix(query engine): Include lines with ts equal to end timestamp of the query range when executing range aggregations (#13448) ... **Background** When performing range vector aggregations, such as `count_over_time({env="dev"}[1h])`, the query range is divided into multiple steps at which the aggregation operation (e.g. counting the log lines) is evaluated. Each step starts at `current step - step interval` and ends at `current step`, as depicted in the following chart. The select range for the logs is extended by the `step interval` into the past, in order to select logs for calculating the first step.  However, the select range for logs is `start` inclusive and `end` exclusive (written as `[start, end)`), but the evaluation of the steps for the range aggregation is `start` exclusive and `end` inclusive (written as `(start, end]`). This leads to the problem that the very first timestamp at the beginning of the select range and the very last timestamp at the end of the select range are not included in the range aggregation. The "missing" last timestamp is not a problem, because a) in an instant query it is not supposed to be included anyway because of the `[start, end)` inclusivity of the query range and b) in a range query the last point of the previous step will be part of the next step evaluation. **Issue** The missing first timestamp, however, gets problematic when executing an instant query and the log timestamps are exactly at the start of the query range. This can happen when the query is split in the query frontend into multiple smaller time ranges, e.g. `1h`, `30m`, ... Since the sub queries are executed independently on the queriers, all logs that have a timestamp exactly a multiple of the split interval, e.g. 00:00, 01:00, 02:00, ... for a 1h interval, are dismissed and therefore missing in the query result over the full time range of the original query. **Fix** In order to avoid the missing logs that have a timestamp a multiple of the split interval in instant queries, we need to adjust the query range for logs to also include the `end` timestamp (written as `[start, end]`). This is done by adding a "leap nanosecond" to the `end` timestamp of the log select range. This ensures that the included `end` timestamp of the step evaluation is also included in the log selection. --- Signed-off-by: Christian Haudum <christian.haudum@gmail.com>		11 months ago
..
analytics	fix: errors reported by the race detector (#13174)	12 months ago
bloombuild	refactor(blooms): Improve task progress logging in builder (#13394)	11 months ago
bloomcompactor	chore: Make dedicated iterator package (#13273)	11 months ago
bloomgateway	chore: Make dedicated iterator package (#13273)	11 months ago
bloomutils	chore: update loki modules for 3.0 release (#12433)	1 year ago
canary	feat: collect and serve pre-aggregated bytes and counts (#13020)	11 months ago
chunkenc	chore: Improve performance of structured metadata (#13404)	11 months ago
compactor	chore: Make dedicated iterator package (#13273)	11 months ago
configs	chore: update loki modules for 3.0 release (#12433)	1 year ago
distributor	feat: Collect duplicate log line metrics (#13084)	11 months ago
framedstdcopy	fix(promtail): Handle docker logs when a log is split in multiple frames (#12374)	1 year ago
indexgateway	fix(gateway client): Fix nil pointer dereference panic when using ruler in ring mode (#13436)	11 months ago
ingester	chore: Refactor storage interface for rf1 (#13415)	11 months ago
ingester-rf1	feat: RF1 improves replication stategy to support 1 replica (#13469)	11 months ago
iter	chore: Convert iterators in pattern module to `v2.Iterator` interface (#13407)	11 months ago
logcli	chore: Make dedicated iterator package (#13273)	11 months ago
loghttp	feat: collect and serve pre-aggregated bytes and counts (#13020)	11 months ago
logproto	feat: Ingester RF-1 (#13365)	11 months ago
logql	fix(query engine): Include lines with ts equal to end timestamp of the query range when executing range aggregations (#13448)	11 months ago
logqlanalyzer	chore: update loki modules for 3.0 release (#12433)	1 year ago
logqlmodel	feat(api)!: Fail log queries when executed on instant query endpoint (#13421)	11 months ago
loki	fix: Fix panic in BloomStore initialisation (#13457)	11 months ago
lokifrontend	fix: `codec` not initialized in downstream roundtripper (#12873)	1 year ago
pattern	feat: Pattern ingesters add a limiter for high eviction rate (#13464)	11 months ago
push	feat: Ingester RF-1 (#13365)	11 months ago
querier	fix(log results cache): include pipeline wrapper disabled in cache key (#13328)	11 months ago
queue	chore: update loki modules for 3.0 release (#12433)	1 year ago
ruler	chore: Make dedicated iterator package (#13273)	11 months ago
runtime	feat: Collect duplicate log line metrics (#13084)	11 months ago
scheduler	test: First pass at unregistering metrics so that we can run multiple tests (#12927)	1 year ago
server	Upgrade dskit to use packages migrated from weaveworks/common (#10164)	2 years ago
sizing	fix xss vulnerability (#10559)	2 years ago
storage	fix(query engine): Include lines with ts equal to end timestamp of the query range when executing range aggregations (#13448)	11 months ago
tool	feat: Introduce `index audit` to `lokitool` (#13008)	12 months ago
tracing	Set tracing.profiling-enabled to true by default (#12004)	1 year ago
util	feat(api)!: Fail log queries when executed on instant query endpoint (#13421)	11 months ago
validation	feat(exporter): include boolean values in limit exporter (#13466)	11 months ago