3809 Commits (88beefb02acca7651d57e69dd5118598d829a904)

Author	SHA1	Message	Date
Robert Fratto	88beefb02a	fix(logql): Fix inconsistency with parsed field short circuiting (#17104) ... PR #8724 originally changed the behaviour of LogQL so that the first extracted field takes precedence over any later extracted field with the same name. However, this PR had two subtle issues due to the caching that parse stages perform for key lookups: * The precedence logic did not always apply to duplicate fields within the same parse stage (notably for logfmt). * The caching behaviour could incorrectly cause stages to permanently ignore fields for future log lines where that stage should otherwise perform extraction. Additionally, once structured metadata was introduced, it was incorrectly being flagged as an "extracted field." Combined with the caching behaviour described above, this means that parsed fields only take precedence over structured metadata if the first log line encountered by the engine doesn't have that field also set as structured metadata. This can be demonstrated with the following scenario: 1. A log line from a stream that uses structured metadata for `trace_id` is encountered first. This (incorrectly) causes the trace_id field to be ignored in all parse stages. 2. All later log lines from streams without structured metadata can no longer extract or filter on a parsed `trace_id` for the lifetime of the query, due to the caching from step 1. The fix is two-fold: * Decouple skipping previously extracted fields from the logic which caches interned keys. * Only mark parsed labels as extracted fields (previously all labels). Because the check for whether a field is already extracted is now called more times, the list of extracted fields has been updated to a map to allow for faster checks.	2 months ago
Karsten Jeschkies	7312ccb3f3	refactor(stringlabels): Support stringlabels distributor tests (#17123)	2 months ago
Karsten Jeschkies	5bba0c27b7	refactor(stringlabels): Support stringlabels ingester tests (#17121)	2 months ago
benclive	8e421186df	chore: Collect basic stats from dataobj queries (#17111)	2 months ago
George Robinson	d197cdac8d	feat: add enforceLimits to ingest_limits.go (#17117)	2 months ago
George Robinson	b0931b1b52	chore: change the result of exceedsLimits (#17112)	2 months ago
George Robinson	bccededc44	chore: fix private interface had exported methods (#17115)	2 months ago
Karsten Jeschkies	75593e0ed7	refactor(stringlabels): Support stringlabels in loghttp, pattern and ruler tests. (#17102)	2 months ago
aarogoss	e962398462	feat: Optionally enable kafka histograms for read/write latency (#17089)	2 months ago
Periklis Tsirakidis	54ff5dce52	feat(ingest-limits): Add partition id caching (#16964)	2 months ago
Sandeep Sukhani	db59d0e9b2	fix: wire up deletion series progress tracking (#17099)	2 months ago
Karsten Jeschkies	c49eeba326	refactor(stringlabels): Support stringlabels in querier tests (#17088) ... **What this PR does / why we need it**: This is the first step of support Prometheus `stringlabels` implementation in Loki. **Special notes for your reviewer**: The tests should compile with build tag `stringlabels`.	2 months ago
renovate[bot]	6e77d026dd	chore(deps): update dependency typescript to v5.8.3 (main) (#17097) ... Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>	2 months ago
Periklis Tsirakidis	befb5b36bb	chore(ingest-limits): Rollout using both balancers (#17085)	2 months ago
Sandeep Sukhani	8e4b10408d	fix: avoid copying label values from tsdb unless required (#17077)	2 months ago
Christophe Serafin	5625464332	fix: Fix missing series in monolithic mode (#17067) ... In monolithic mode, unlike other API endpoints (query/labels/...), the series endpoint did not retrieve data from storage for the time window between end of `index_cache_validity` and `query_store_max_look_back_period`, and therefore missing series in the response.	2 months ago
Trevor Whitney	940a067a50	chore: apply series limit per variant for multi-variant queries (#16887)	2 months ago
Owen Diehl	2aed4c3da5	fix(json, structured metadata): json-parsing cleanup (#17072)	2 months ago
Hristo Venev	9629d07c60	fix: Unescape JSON strucutred metadata string values (#13919)	2 months ago
Owen Diehl	cf272892e4	revert: "fix(json, structured metadata): correctly handle escaping in json parsing API" (#17071)	2 months ago
Owen Diehl	243b155ff3	fix(json, structured metadata): correctly handle escaping in json parsing API (#17068)	2 months ago
JordanRushing	9000de1aa7	fix(push): Add guard clauses to prevent negative counter values (#17056) ... Signed-off-by: Jordan Rushing <rushing.jordan@gmail.com> Co-authored-by: Trevor Whitney <trevorjwhitney@gmail.com>	2 months ago
Christian Haudum	e297c51d20	chore(metastore): Use constants for label names (#17048) ... This PR contains improvements for: 1. Make label names contants 2. Use `strconv.FormatInt` for encoding timestamp into label value (equivalent to `strconv.ParseInt` when decoding) 3. Use `labels.New` to construct labels, rather then string formatting Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 months ago
JordanRushing	9e9f534460	fix: prevent panic when ContentLength is negative in Distributor RequestBodyTooLarge metrics (#17054)	2 months ago
Robert Fratto	300c1e859e	chore(logql/bench): Fix bug in bench data generation and update naming conventions (#17051) ... This PR fixes a bug found in generating test data for chunks: MemChunks weren't being closed properly before flushing, which caused the last block of every chunk to be omitted from the flush. As a result, benchmark tests against chunks were inaccurate, causing many metric queries to appear 5-10x faster with chunks than they actually are. Additionally, this PR expands on #17006 with another naming convention change: the type of query is now included in the benchmark name to allow developers to only run benchmarks of a specific query type (metric, log). PR #17006 neglected to reflect the naming convention change in the benchmark TUI, which stopped it from working. This is now fixed to match the latest naming convention introduced in this PR, and the TUI will now work again.	2 months ago
Christian Haudum	1705f98a9f	chore(engine): Wire up new execution engine (#17032) ... This PR connects the already existing components of the new query engine and wires them up in the querier API. The feature flag `-querier.engine.enable-v2-engine` controls whether supported queries should be executed using the new engine. Note that this implementation currently does not execute any queries. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 months ago
George Robinson	ffa9656f7f	chore: refactor how distributor checks ingest-limits (#17042)	2 months ago
George Robinson	e90e0c41fb	chore: rename RejectedStreams to ExceedsLimitsResult (#17041)	2 months ago
George Robinson	38c40170fc	chore(ingest-limits): rate should be a uint64 as it cannot be negative (#17040)	2 months ago
Trevor Whitney	bfb935d893	fix: allow boolean numeric values in detected labels (#16997)	2 months ago
Periklis Tsirakidis	406690d26e	fix(ingest-limits): Incorrect ingested bytes counter usage (#17037)	2 months ago
Travis Patterson	d204d7465c	chore: expose partition ring for consumption by GEL (#17036)	2 months ago
Periklis Tsirakidis	5229a2ab65	fix(ingest-limits): Get usage from owing instances only (#16937)	2 months ago
George Robinson	7f1a1c755a	feat: add metric to track ingest limits failures in distributor (#17015)	2 months ago
Christian Haudum	ba61aa88de	chore(engine): Integrate metastore into physical query planning (#17016) ... This PR integrates the data object lookup using the metastore into the query planning. To achieve that, I extended the `Metastore` interface with an additional function that not only returns the data object paths, but also the stream IDs of streams that match the given label matchers. ```go type Metastore interface { ... // StreamsIDs returns object store paths and stream IDs for all matching objects for the given matchers between [start,end] StreamIDs(ctx context.Context, start, end time.Time, matchers ...*labels.Matcher) ([]string, [][]int64, error) ... } ``` Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 months ago
Trevor Whitney	58aa00a5f4	fix: short circuit level detection when already detected (#17028)	2 months ago
Trevor Whitney	a8f65d1db3	ci: skip flaky analytics test (#17009)	2 months ago
George Robinson	532a652c21	chore: some metrics changes to ingest-limits-frontend (#17004)	2 months ago
aarogoss	7fc9026208	feat: Add agent IPs to distributor log (#17011) ... Signed-off-by: aarogoss <32306622+aarogoss@users.noreply.github.com> Co-authored-by: Ed Welch <edward.welch@grafana.com>	2 months ago
Christian Haudum	6824f4e575	chore(engine): Add printer for Mermaid diagram format (#17000) ... Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 months ago
Robert Fratto	5b787c0f5b	chore(logql/bench): use benchstat naming conventions for benchmarks (#17006)	2 months ago
Sven Grossmann	e678b615c1	fix: revert remove colons from level detection (#16999)	2 months ago
Christian Haudum	04d2ba3ca8	chore(engine): Add simple plan optimizer (#16965) ... This PR adds a plan optimizer that pushes down limit and filter predicates to the scan node. ### Example Plan: ``` Limit #0xc0000110f8 offset=0 limit=1000 └── Filter #0xc00039dd10 │ └── Predicate expr=LT(timestamp.builtin, 1742826126000000000) └── Filter #0xc00039ddd0 │ └── Predicate expr=GT(age.metadata, 21) └── SortMerge #0xc00039de60 column=timestamp.builtin order=ASC ├── DataObjScan #0xc00011b420 location=obj1 stream_ids=(1, 2) projections=() direction=0 limit=0 └── DataObjScan #0xc00011b490 location=obj2 stream_ids=(3, 4) projections=() direction=0 limit=0 ``` Optimized plan: ``` Limit #0xc0000110f8 offset=0 limit=1000 └── SortMerge #0xc00039de60 column=timestamp.builtin order=ASC ├── DataObjScan #0xc00011b420 location=obj1 stream_ids=(1, 2) projections=() direction=0 limit=1000 │ ├── Predicate expr=LT(timestamp.builtin, 1742826126000000000) │ └── Predicate expr=GT(age.metadata, 21) └── DataObjScan #0xc00011b490 location=obj2 stream_ids=(3, 4) projections=() direction=0 limit=1000 ├── Predicate expr=LT(timestamp.builtin, 1742826126000000000) └── Predicate expr=GT(age.metadata, 21) ``` Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 months ago
Christian Haudum	40223d7320	chore(engine): Unify `Literal` and `ColumnRef` across logical and physical plan (#16927) ... Literals and column refs are used by both the logical and physical plan. However, the interfaces they need to implement are partly private (`isValue()` and `isExpression()`). Therefore they are wrapped in their own struct that implement these interfaces. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 months ago
Christian Haudum	138093b900	chore(planner): Convert AST into logical plan (#16916) ... Minimal implementation of the logical planner that converts a parsed LogQL statement in AST form into a logical plan in SSA form. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 months ago
Ashwanth	42eed6ded5	chore(dataobj): introduce InPredicate to replace long chain of OR operators (#16919)	2 months ago
Christian Haudum	0a3230f457	chore(ast): Update WalkFn signature to return bool (#16989) ... The boolean return value indicates whether walking/traversing should be continued with child nodes. This PR does not change any existing traversal behaviour. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 months ago
Periklis Tsirakidis	1d99f4d86d	feat(distributor): Add MaxRecvMsgSize config for uncompressed message size limits (#16915)	2 months ago
renovate[bot]	ab8ab01d05	fix(deps): update dependency lucide-react to ^0.486.0 (main) (#16966) ... Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>	2 months ago
renovate[bot]	e1371cb559	chore(deps): update dependency vite to v6.2.4 (main) (#16961) ... Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>	2 months ago