4977 Commits (main)

Author	SHA1	Message	Date
George Robinson	c3e909d538	feat: decouple dataobj consumers from the reader service (#20315)	1 day ago
Assel Meher	5663f9c26d	feat(engine): add regexp parser support for log queries (#20286)	2 days ago
Stas Spiridonov	c2116618bd	chore: Improving memory allocations for new query engine (#20321)	2 days ago
Ed Welch	9b0af7c002	fix: track discarded entries and bytes when hitting stream limits using the ingest limits service (#20244)	3 days ago
Robert Fratto	c8a49d4d72	chore(engine): add support for simplifying regular expressions (#20297) ... Signed-off-by: Robert Fratto <robertfratto@gmail.com>	3 days ago
George Robinson	5a5e90e9f7	feat: add histogram loki_dataobj_consumer_flush_duration_seconds (#20304)	3 days ago
George Robinson	fba0c5d8a6	feat: dataobj-consumer add processed bytes metric (#20303)	3 days ago
George Robinson	bd77a47df9	chore: remove use of namespace/subsystem in builder metrics (#20301)	3 days ago
George Robinson	e0e5d78e69	chore: move appends total metric into builder (#20300)	3 days ago
Robert Fratto	f69fa66a71	chore(engine): log number of tasks in execution plan (workflow) (#20305)	3 days ago
Ashwanth	559c7ecd47	chore(xcap): track task queue & admission duration (#20298)	4 days ago
Stas Spiridonov	2203c1d8e8	chore: tweaks for feature parity (#19940)	4 days ago
Noah Krause	993b3ae65d	feat(canary): Support passing arbitrary set of labels to use for the query (#17008) ... Signed-off-by: Noah Krause <krausenoah@gmail.com> Co-authored-by: Ed Welch <ed@edjusted.com>	1 week ago
Ivan Kalita	08e3c4385f	feat(metastore): shard sections queries over index files (#20134) ... Running metastore queriesin a distributed manner using query engine workers: - Build a distributed metastore plan from GetIndexes and execute it via the v2 scheduler/worker pipeline - Split the request into per-index PointersScan tasks, then fan-in and CollectSections to produce final section descriptors - Add physical plan + protobuf support for Merge/PointersScan, improve tracing/cleanup, and add coverage for planner/workflow/proto roundtrips	3 weeks ago
Daniel-Davies	d940433736	[feat] Add config option to override Azure AD endpoint (#20267) ... Co-authored-by: Trevor Whitney <trevorjwhitney@gmail.com>	3 weeks ago
Assel Meher	6d74050e63	chore: update dskit to include LRU cache for partition ring shuffle sharding (#20270)	3 weeks ago
Ashwanth	2b97b207e0	chore: stream sink handle retryable errors (#20264)	3 weeks ago
Ashwanth	be9f5fd729	chore(scheduler): reduce lock contention during task assignment (#20254)	3 weeks ago
Stas Spiridonov	d776c10dff	fix: Scheduler memory leak (#20268)	3 weeks ago
Ivan Kalita	9be17c7a42	feat(metastore): metastore DI (#20253) ... This commit fixes a problem that happens when we instantiate 2 metastores in a single process. Even though this can't happen at the moment it will be the case once we introduce distributed metastore queries - when running both schedulers and workers in the same process all of them will need an instance of metastore. Each instance (before this PR) internally creates and registers metastore metrics which causes panic due to exactly the same metrics were already registered by the same registerer. I attempt to fix this problem via creating and registering metastore metrics only once.	3 weeks ago
Ivan Kalita	e4ec8441d3	feat(metastore): use arrow for scanning and blooms (#20234) ... Changing metastore implementation to work with arrow.RecordBatch without intermediate materialization to structs. metastore.Sections now uses new metastore.GetIndexes, metastore.IndexSectionsReader and metastore.CollectSections. We'd like to run metastore queries in a distributed manner using engine v2. In order to achieve that we need to split the work that metastore.Sections does internally, schedule it on different workers, and collect the results. The engine achieves that via pipelines that produce arrow.RecordBatch-es. In this PR the metastore.Sections is updated such a way to make it easier to integrate metastore with the engine.	3 weeks ago
Stas Spiridonov	b6b7459435	chore: Aggregation groupings for by() and without() (#19928)	3 weeks ago
Stas Spiridonov	377af1aa8b	chore: graceful worker shutdown (#20136)	3 weeks ago
zethis	0c1561d15e	feat(lokitool): Add regex namespace filtering (#20209)	3 weeks ago
Jake Leahy	c524203a6e	feat(logcli): Allow custom headers to be passed (#20231)	3 weeks ago
Ivan Kalita	3a74fe71fe	feat(engine): delegate metastore queries to engine (#20189) ... This is a prerequisite to distributed metastore queries execution. I'm updating the catalog such a way to receive a callback from the caller to call metastore. This is needed to separate the execution of the metastore queries that will be handled by the engine and not by the catalog.	4 weeks ago
Ashwanth	7da7f000ff	chore: use projection for indexobj streams reader (#20205)	4 weeks ago
Robert Fratto	eaeb7afe88	chore(engine)!: Share worker threads across all scheduler connections (#20229) ... Signed-off-by: Robert Fratto <robertfratto@gmail.com>	4 weeks ago
Trevor Whitney	89396c190f	chore: use tenant instead of service_name for pattern_ingester_metric_samples (#20033)	4 weeks ago
Ed Welch	657d9ac55f	chore: add stream count to metrics.go for queriers (#20215)	4 weeks ago
Ashwanth	e08721487c	chore: compute stats summary in query engine scheduler (#20222)	4 weeks ago
Robert Fratto	0c9dd8b7b7	chore(engine): update how scheduler tracks readiness (#20225) ... Signed-off-by: Robert Fratto <robertfratto@gmail.com>	4 weeks ago
benclive	ea88458c03	fix(enginev2): Always compute summary when creating the stats object (#20224)	4 weeks ago
Assel Meher	de092daf0a	feat: Handle state change lock in prepare downscale (#20141)	4 weeks ago
Ashwanth	beb1728870	chore: stop adding plans to span events (#20219)	4 weeks ago
Robert Fratto	c57dc2cac8	chore(querier): fix engine router splitting logic (#20217) ... Signed-off-by: Robert Fratto <robertfratto@gmail.com>	4 weeks ago
Robert Fratto	1272bbb384	chore(engine): send structured errors over the wire (#20211) ... Signed-off-by: Robert Fratto <robertfratto@gmail.com>	4 weeks ago
Robert Fratto	9dc824bf7f	chore(engine): specify invalid time range (#20207) ... Signed-off-by: Robert Fratto <robertfratto@gmail.com>	4 weeks ago
George Robinson	a03e00a3bc	chore: better comments when end offset is zero (#20196)	4 weeks ago
George Robinson	333da739c7	feat: add processed records metric (#20191)	4 weeks ago
benclive	f22e854dea	chore: Dispatch tasks more frequently (#20195)	1 month ago
Ashwanth	833b4e1690	chore: revert change to shrink v2 split (#20190)	1 month ago
Trevor Whitney	43a3f159b3	fix: persist correct goldfish result comparison in database (#20172)	1 month ago
George Robinson	c0c27b3596	feat: add downscalePermittedFunc to check downscale is permitted (#20171)	1 month ago
nicolevanderhoeven	7f47dddb3a	ci: Remove Helm 4 verification and fix flaky ruler test (#20178) ... Signed-off-by: J Stickler <julie.stickler@grafana.com> Co-authored-by: JStickler <4106682+JStickler@users.noreply.github.com> Co-authored-by: J Stickler <julie.stickler@grafana.com>	1 month ago
Ashwanth	c6d6bfc0b6	chore: xcap summary treat duration observation values as seconds (#20175)	1 month ago
Julien Pinsonneau	0c3dd8ce7e	fix: compactor file descriptor leak (#20077)	1 month ago
Ashwanth	5bbb50562a	chore: improvements to v2 engine split range (#20174)	1 month ago
George Robinson	23ef8eccce	feat: add gauge to track in-flight bytes (#20091)	1 month ago
Trevor Whitney	aab9e46bcf	feat: implement query splitting in the query-tee (#20039)	1 month ago