197 Commits (9d0d1084e39fdfd4a58d701b13ff755f4e5f36eb)

Author	SHA1	Message	Date
Robert Fratto	9d0d1084e3	chore(engine): use github.com/grafana/dskit/dns for scheduler discovery (#19835) ... Signed-off-by: Robert Fratto <robertfratto@gmail.com>	2 months ago
Ashwanth	25c8827f56	chore(xcap): export captures as traces (#19880)	2 months ago
Robert Fratto	86ccc65355	chore(engine): report saturation as load when there are no workers (#19925) ... Signed-off-by: Robert Fratto <robertfratto@gmail.com>	2 months ago
Robert Fratto	e9030beedf	chore(engine): track scheduler load/pressure metrics (#19924)	2 months ago
Robert Fratto	2fce7da611	chore(engine): add base set of new scheduler metrics (#19913) ... Signed-off-by: Robert Fratto <robertfratto@gmail.com>	2 months ago
Ashwanth	11b1dcb2a3	chore(engine): introduce execution capture (#19821)	2 months ago
Sophie Waldman	b9a51f0129	feat: Enable support for max, min, max_over_time, min_over_time for new engine (#19841)	2 months ago
Christian Haudum	97fc8b3a50	chore(metastore): Simplify public API of Metastore (#19886) ... Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 months ago
Robert Fratto	e9600aed4b	chore(engine)!: Make scheduler aware of total compute capacity (#19876)	2 months ago
Robert Fratto	03f917a357	refactor(engine): update workflow to register all tasks/streams at once (#19859) ... Signed-off-by: Robert Fratto <robertfratto@gmail.com>	2 months ago
Christian Haudum	f066d378a5	chore(engine): Do not embed `types.Literal` interface in logical and physical Literal (#19858) ... This ensures that logical.Literal and physical.LiteralExpr do not implicitly implement the types.Literal interface, which caused a bug fixed with #19853 Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 months ago
Christian Haudum	0f35890d4b	chore(engine): Fix conversion from logical to physical Literal in math expressions (#19853) ... Since both logical.Literal and physical.LiteralExpr embed the types.Literal interface, they both automatically also implement the interface. This is problematic when creating a physical LiteralExpr from a logical Literal, rather than from the underlying value. The serialization of the physical plan fails, because the pointer value of the interface is then a logical.Literal, rather than a physical.LiteralExpr	2 months ago
Robert Fratto	0f0846adec	chore(engine): make stream/task debug logging configurable (#19842) ... Signed-off-by: Robert Fratto <robertfratto@gmail.com>	2 months ago
Ashwanth	53230deee4	chore(scheduler): send tracing context to workers (#19839) ... Co-authored-by: Robert Fratto <robertfratto@gmail.com>	2 months ago
Ivan Kalita	9b001dfd01	fix(engine): unset write and read deadlines for wire listeners (#19828) ... Wire http2 peers continuously drop connections due to http server write timeouts. I'm unsetting them.	2 months ago
Robert Fratto	9a25c9010f	chore(engine): remove reference to internal type in exported function (#19829) ... Signed-off-by: Robert Fratto <robertfratto@gmail.com>	2 months ago
Robert Fratto	abaeff7fad	chore(engine): query new schedulers over HTTP from query-frontend (#19744) ... Signed-off-by: Robert Fratto <robertfratto@gmail.com> Co-authored-by: Ivan Kalita <ivan.kalita@grafana.com>	2 months ago
renovate-sh-app[bot]	d76b3bf495	fix(deps): update module github.com/apache/arrow-go/v18 to v18.4.1 (main) (#19750) ... Signed-off-by: renovate-sh-app[bot] <219655108+renovate-sh-app[bot]@users.noreply.github.com> Co-authored-by: renovate-sh-app[bot] <219655108+renovate-sh-app[bot]@users.noreply.github.com> Co-authored-by: Paul Rogers <paul.rogers@grafana.com>	2 months ago
Stas Spiridonov	f2c874e6ab	chore: scheduler tasks are aware of their time range (#19709)	2 months ago
Robert Fratto	377e067725	chore(engine): determine proper advertise address for scheduler (#19731) ... Signed-off-by: Robert Fratto <robertfratto@gmail.com>	2 months ago
Robert Fratto	785743214e	chore(engine): propagate worker listen address to scheduler (#19730) ... Signed-off-by: Robert Fratto <robertfratto@gmail.com>	2 months ago
Christian Haudum	c2cf38383c	chore: Wire up new scheduler and worker components (#19728) ... Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 months ago
benclive	36613bd017	fix: Empty keys are returned if desired in v2 engine (#19717)	2 months ago
Ivan Kalita	f3798af153	chore(engine): Add HTTP/2 transport for wire protocol (#19575) ... Implements HTTP/2 transport layer with protobufCodec, HTTP2Listener/Conn. - Listener receives an http request and holds it. The underlying connection is now managed by the listener and will be returned to the next .Accept() caller (wrapped into an http2Conn). - http2Conn implements wire.Conn interface and supports reads and writes of frames. It uses protobufCodec to serialize, write, read, deserialize the data. - there's proto definition for frames and messages that is used by protobufCodec	2 months ago
Christian Haudum	4b118fb729	chore: Admission lanes for workflow tasks (#19584) ... This PR introduces "admission lanes" for dispatching tasks to the runner (scheduler), that are controlled by token buckets, to avoid overflowing the runner with a large amount of tasks. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 months ago
Stas Spiridonov	d02e55c57a	chore: remove limit physical plan node (#19682)	2 months ago
benclive	01cab53447	feat(engine): implement strict and keepEmpty logfmt parsing (#19668)	2 months ago
Ashwanth	b3803113ad	chore(engine): registers functions for logical op (#19704)	2 months ago
Robert Fratto	7473aee3a5	chore(engine): add periodic discovery of remote schedulers (#19695) ... Signed-off-by: Robert Fratto <robertfratto@gmail.com>	2 months ago
Ashwanth	5d961a4cc3	chore(topk): improvements to reduce alloc bytes and alloc space (#19660)	2 months ago
Robert Fratto	016ee7d39e	chore(engine): serializable physical plans (#19672) ... This adds two new packages, `expressionpb` and `physicalpb`, which are serializable representations of `physical.Expression` and `physical.Plan`, respectively. These packages include utility functions to convert between the protobuf representations and the planner types. A translation layer is used due to the complexity of integrating protobuf throughout the engine, as well as difficulties with finding a clean pattern to construct node types. Once we're able to identify a clean and consistent pattern for interacting with the protobuf types throughout the codebase, the translation layer can be removed in favour of using the protobuf types directly. Co-authored-by: Sophie Waldman <sophie.waldman@grafana.com>	2 months ago
Trevor Whitney	dd6b31473e	fix: errors in parse pipeline (#19667) ... Co-authored-by: Ashwanth Goli <iamashwanth@gmail.com>	2 months ago
Ed Welch	c3f12ef3df	chore: track time spent waiting for data from ingesters (#19648)	2 months ago
Robert Fratto	75e5193a97	chore(engine): expose new scheduler and worker (#19633) ... Signed-off-by: Robert Fratto <robertfratto@gmail.com>	2 months ago
Trevor Whitney	8b39d4d077	refactor: implment parse as a projection (#19579) ... Signed-off-by: Trevor Whitney <trevorjwhitney@gmail.com> Co-authored-by: Christian Haudum <christian.haudum@gmail.com>	2 months ago
Christian Haudum	cf596d0260	chore(engine): Remove unused physical plan visitor (#19647) ... The visitor was introduced in the very beginning without knowing whether it would be used in the future. Except for in tests, it hasn't been used so far. The functionality is provided by the Walk function of the dag package.	2 months ago
Christian Haudum	73bc30d9a8	perf(engine): Improve regexp expression evaluation (#19644) ... This PR changes the evaluation of regex-match binary expression evaluations so that the regexp on right hand side of the expression is only compiled once for each batch evaluation, rather than for each row. This saves a ton of allocations and improves general performance of queries with a regexp line or label filter.	2 months ago
Stas Spiridonov	22292439a2	feat(engine): rate() aggregate function (#19589)	2 months ago
Robert Fratto	3c99535b0a	chore(engine): provide worker (#19588) ... The new worker package connects to an instance of a scheduler (#19570) for task assignment and execution. A worker spawns a fixed number of threads, each of which execute one task at a time. Signed-off-by: Robert Fratto <robertfratto@gmail.com>	2 months ago
benclive	9a68900bc1	fix: Sanitize logfmt keys in v2 engine (#19621)	2 months ago
Ashwanth	9a50c8076e	chore: stream injector: propagate orig error (#19616)	2 months ago
Stas Spiridonov	7eda6749a4	feat(engine): basic math expressions (#19407)	2 months ago
Ashwanth	d34cfbd84a	chore: refactor and fixes for optimiser (#19581)	3 months ago
Robert Fratto	53487c7724	chore(engine): provide scheduler (#19570) ... Signed-off-by: Robert Fratto <robertfratto@gmail.com>	3 months ago
benclive	dd52234fc5	chore: Use first-wins semantics for duplicate fields in logfmt/json parsers (#19568)	3 months ago
Christian Haudum	9e76f1d251	chore(engine): Refactor `ColumnVector` (#19549) ... This PR removes the need of the ColumnVector as a result of expression evaluation and instead returns an arrow.Array. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	3 months ago
Robert Fratto	68e7f552d7	chore(engine): add wire package (#19560) ... Signed-off-by: Robert Fratto <robertfratto@gmail.com>	3 months ago
Ivan Kalita	f46ef0eb7c	chore(engine): Refactor streams result builder to use column-oriented processsing (#19505) ... Change streamsResultBuilder.CollectRecord() from row-by-row to column-by-column iteration. This better aligns with Arrow's columnar memory layout, improving CPU cache locality by reading contiguous memory regions. I also remove IsValid checks given that IsNull checks are present. IsValid seems to be an exact opposite of IsNull (based on this code). The slices and label builders for intermediate rows data are reused between CollectRecord.	3 months ago
Stas Spiridonov	160dc2c493	chore: removed arrow-go allocators/retain/release (#19569)	3 months ago
Robert Fratto	053429db21	chore(engine): add information for task updates (#19561) ... Signed-off-by: Robert Fratto <robertfratto@gmail.com>	3 months ago