356 Commits (e40777ea3640f6b4bc577c99bc3bc2e7ae5b3627)

Author	SHA1	Message	Date
benclive	e40777ea36	chore: Apply label hash sorting to topk when timestamps match (#22282)	2 days ago
Ashwanth	d62c5906a9	chore(engine): Add stats for measuring data locality (#22271)	3 days ago
Trevor Whitney	b3e9265b6c	feat(engine/physicalpb): Add IndexMerge node marshalling (#22231)	3 days ago
benclive	667121ad33	chore: Support marshalling LabelFmt in physical plan (#22227)	4 days ago
Trevor Whitney	beb732c3c5	fix(engine/workflow): Surface failed-task errors before results EOF (#22232)	4 days ago
Paul Rogers	a7adf30edb	perf(engine): Short-circuit DataObjScan when logs predicate is unsatisfiable (#22162)	5 days ago
Trevor Whitney	5790955aba	feat(dataobj-compactor): Compactor coordinator/planner main loop (#21973) ... Signed-off-by: Trevor Whitney <trevorjwhitney@gmail.com> Co-authored-by: Ivan Kalita <ivan.kalita@grafana.com> Co-authored-by: J Stickler <julie.stickler@grafana.com>	5 days ago
Ivan Kalita	3f62bb7207	chore(compactor): Allow overflowed logsobj builders (#22148) ... I'm changing the behavior of logsobj builders to allow appends to overflowed builders. Before this PR: (1) Processor tries to append a record to a builder (2) If a builder is full it doesn't append and returns an error ErrBuilderFull (3) Processor checks the error and flushes the builder (4) Processor tries to append the same record as in the step (1) to the now empty builder This approach works fine but becomes problematic once we introduce multiple TOC windows builders. The issue is that with a single builder we have atomic "try append - flush if error" behavior guarantee. With multiple builders this becomes a problem as a single kafka record might contain entries that belong to different builders. Example: - we have a record R that has a log entry L0 for TOC window W0 AND a log entry L1 for TOC window W1 - we attempt to append L0 to the first builder B0 (for TOC W0) -> success - we attempt to append L1 to the second builder B1 (for TOC W1) -> not appended, err builder full - we flush all the builders (have to flush all of them because otherwise we can lose data as Kafka offset is committed after flushing) - we attempt to append R again -> builder B0 will receive a duplicate of log L0 To simplify the processing we can allow appends to full builders ("one last time"). For the example above we'd allow the append of L1 to B1 even though B1 is already full. This way the dataobjects might become slightly bigger than the target size but this could already be the case given that the existing size estimates are estimates.	6 days ago
Ivan Kalita	378ad03ee6	chore(compactor): EarliestRecordTime per logsobj builder (#22139) ... I'm changing processor / builder such a way that EarliestRecordTime is set per-builder not per processor. This timestamp is used later by dataobj-index-builder to track the e2e indexing latency. The reason to make it per-builder is that when we introduce builders builder-per-time-window approach we'll need to keep separate EarliestRecordTimes for these builders (one per builder). The change is intended to be a no-op (no behavioral changes).	6 days ago
Ivan Kalita	bd6316d9ac	feat(compactor): Reuse logsobj builder metrics (#22138) ... I'm updating logsobj consumer BuilderMetrics so they are registered outside of the builder and passed down to builders through the factory. The reason is – we are going to create builders dynamically in the future (see #21573) and the current approach where builder metrics are registered every time a builder is created is not going to work (the same metrics will be registered multiple times). The change is intended to be a no-op (no behavioral changes), just shuffled metrics registration around.	7 days ago
Sandeep Sukhani	77b271f9c9	fix: Request source key for collision-renamed labels in metric queries (#22141)	7 days ago
Joe Elliott	40c9d046f0	chore: Remove dead code (unused subsystems and stub files) (#22077) ... Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	1 week ago
Bryan Boreham	107dfbde54	chore: Fix more errors reported by go test -race (#22055)	1 week ago
Paul Rogers	ac1e9877e5	chore(deps): Upgrade to moby from docker (#22034)	2 weeks ago
Trevor Whitney	88cdd1badd	refactor(dataobj-compactor): Split CompactionMerge into IndexMerge + LogMerge (#21930)	3 weeks ago
benclive	13c7181980	chore: Reduce allocs in range agg node (#21924)	3 weeks ago
Robert Fratto	6e9d98616b	chore(engine): Improve log events to observe query lifecycles (#21863)	3 weeks ago
Trevor Whitney	3a3245ba99	feat(dataobj-compactor): Add dataobj-compactor-worker target scaffold (#21873)	4 weeks ago
Trevor Whitney	3dd8056722	feat(dataobj-compactor): Calculate compaction runs/jobs (#21845)	4 weeks ago
Trevor Whitney	12c419b65d	feat(dataobj-compactor): Add CompactionMerge physical-plan node (#21844)	4 weeks ago
Ivan Kalita	fef3db05f1	fix(engine): Revert "chore: Enable unwrap of batching pipeline (#21795)" (#21856)	4 weeks ago
Trevor Whitney	4cd2cf16bd	feat(dataobj-compactor): Add compactor target skeleton, no-op coordinator stub (#21818)	4 weeks ago
Trevor Whitney	4e9a034ef8	feat(engine/workflow): Add dormant taskTypeCompaction admission lane (#21813)	4 weeks ago
benclive	0dcd72c42c	chore: Enable unwrap of batching pipeline (#21795)	4 weeks ago
Robert Fratto	122f168075	chore(dataset): Add dataset/array, dataset/buffer packages (#21586)	1 month ago
Ashwanth	61538e4c6c	chore(aggregation): Without grouping should be performed on shortnames and sorted columns (#21687)	1 month ago
Christian Haudum	aaa2d1137a	chore: Remove unused/dead code (#21724) ... * chore(dataobj): Remove dead code * chore(engine): Remove unused code * chore(ci): Fix PR/commit title for Renovate (#21721) * fix: Incorrect impl of ProduceSync (#21702) * (origin/k304) docs: New usage statistics topic (#21668) * feat(lokitool): Rename `rules lint` to `rules format` (#21671) * chore: single binary without Kafka (#21318)	1 month ago
Salva Corts	904c963d17	feat: Resolve cached tasks in parent task (#21558)	1 month ago
Bryan Boreham	4850f1be4c	fix(columnar): Fix two race conditions in pkg/engine (#21674) ... Co-authored-by: Stas Spiridonov <stanislav@spiridonov.pro>	2 months ago
Sophie Waldman	7be7c9d594	feat(query engine): Implement line_format and label_format with new engine (#20123) ... Signed-off-by: Sophie Waldman <sophie.waldman@grafana.com>	2 months ago
Bryan Boreham	9cdc29fb99	fix(columnar): Don't release Arrow records (#21670) ... Multiple goroutines may still be accessing records, so we cannot Release them. Remove from the real code and from an example.	2 months ago
Bryan Boreham	dc6cb78e74	chore(queries): Log User-Agent on query metrics and finished lines (#21663) ... Add user_agent entry on query metrics log lines and on "finished executing" lines. This can help identify the source of particular queries. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2 months ago
Bryan Boreham	8ca16c4e30	fix(columnar): Copy label values to avoid retaining memory (#21594)	2 months ago
Assel Meher	c0a243b2a1	feat: add per-phase histogram metrics to engine worker pipeline (#21547)	2 months ago
Salva Corts	24ca0e4a2f	feat: Remove task_pruning_cache_fetch_batch_size in favor of existing memcached batching (#21593)	2 months ago
Bryan Boreham	bc90e2620a	chore: try running workers with a smaller buffer (#21584)	2 months ago
Salva Corts	f3673747f6	feat: Batch cache fetches in schedulers during task pruning (#21548)	2 months ago
Dylan Guedes	f4f4bce1de	fix(query-engine): Fix different error messages ending up in different streams (#21552)	2 months ago
Salva Corts	2e0a70dfca	feat: clamp pointerscans to index start/end timeranges (#21534)	2 months ago
Stas Spiridonov	ededf4029c	chore: Range aggregation split (#21497)	2 months ago
Salva Corts	b68de56092	feat: use query logger on metastore workflow (#21511)	2 months ago
Salva Corts	1eae14bcf5	docs: Rename task results cache config (#21503)	2 months ago
Stas Spiridonov	db47671ef1	chore: add statistics for bytes sent over the wire by scheduler (#21483)	2 months ago
Salva Corts	fb6abe5d08	feat: Avoid executing tasks for which we have cached empty results (#21437)	2 months ago
Stas Spiridonov	53ff3023ac	chore: reorder `physical` package files (#21465)	2 months ago
Sophie Waldman	4e187b13dc	feat(query engine): Add workflow optimizer to clamp predicates on scan nodes (#21327) ... Signed-off-by: Sophie Waldman <sophie.waldman@grafana.com> Co-authored-by: Salva Corts <salva.corts@grafana.com>	2 months ago
Sandeep Sukhani	e74e2a20c5	fix: grouping by non-existent label in v2 query engine (#21430)	2 months ago
Salva Corts	c32b57f36b	feat: return caching stats in new engine (#21369)	2 months ago
Sandeep Sukhani	399efb0d67	fix: preserve parsed labels with empty values in samples query response from thor engine (#21351)	2 months ago
Salva Corts	5f06abe88d	perf: short-circuit cache when over max cacheable size (#21277)	2 months ago