114 Commits (e08721487c3fc8d537d448052ea2e41fa07ddefb)

Author	SHA1	Message	Date
Robert Fratto	0c9dd8b7b7	chore(engine): update how scheduler tracks readiness (#20225) ... Signed-off-by: Robert Fratto <robertfratto@gmail.com>	4 weeks ago
Robert Fratto	1272bbb384	chore(engine): send structured errors over the wire (#20211) ... Signed-off-by: Robert Fratto <robertfratto@gmail.com>	4 weeks ago
benclive	f22e854dea	chore: Dispatch tasks more frequently (#20195)	1 month ago
Robert Fratto	2d54ddc6d9	chore(engine): adjustments to log task short circuiting (#20121)	1 month ago
Ashwanth	54f9723af1	chore(xcap): adds exporter to summarise capture as a structured log line (#20099)	1 month ago
Ashwanth	356cd36d8e	chore: close stream section reader (#20107)	1 month ago
Ashwanth	728b308edf	chore: update read stats to use xcap (#20095)	1 month ago
Ashwanth	cbad9d4ccb	chore(xcap): support events, status and improve coverage (#20096)	1 month ago
benclive	99ba51e48a	fix(enginev2): Close pipeline before building query results (#20100)	1 month ago
Ashwanth	484299df05	chore(xcap): Adds wrapper around bucket client to track operations (#20094)	1 month ago
Christian Haudum	8b59b944f2	chore(dataobj): Use common base config for indexobj and logsobj builders (#20037) ... Use the same config struct for both the indexobj builder and the logsobj builder, since they are configured identically, only with different default values.	1 month ago
Ashwanth	6a6681d9b7	chore(scheduler): extract trace info from orig query context (#20041)	1 month ago
Stas Spiridonov	5b88bcef3a	chore: Short circuiting for new scheduler (#20000)	1 month ago
Ashwanth	5bb6b54052	chore: compat should also handles metadata collisions (#20005)	2 months ago
Robert Fratto	9d0d1084e3	chore(engine): use github.com/grafana/dskit/dns for scheduler discovery (#19835) ... Signed-off-by: Robert Fratto <robertfratto@gmail.com>	2 months ago
Ashwanth	25c8827f56	chore(xcap): export captures as traces (#19880)	2 months ago
Robert Fratto	86ccc65355	chore(engine): report saturation as load when there are no workers (#19925) ... Signed-off-by: Robert Fratto <robertfratto@gmail.com>	2 months ago
Robert Fratto	e9030beedf	chore(engine): track scheduler load/pressure metrics (#19924)	2 months ago
Robert Fratto	2fce7da611	chore(engine): add base set of new scheduler metrics (#19913) ... Signed-off-by: Robert Fratto <robertfratto@gmail.com>	2 months ago
Ashwanth	11b1dcb2a3	chore(engine): introduce execution capture (#19821)	2 months ago
Sophie Waldman	b9a51f0129	feat: Enable support for max, min, max_over_time, min_over_time for new engine (#19841)	2 months ago
Christian Haudum	97fc8b3a50	chore(metastore): Simplify public API of Metastore (#19886) ... Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 months ago
Robert Fratto	e9600aed4b	chore(engine)!: Make scheduler aware of total compute capacity (#19876)	2 months ago
Robert Fratto	03f917a357	refactor(engine): update workflow to register all tasks/streams at once (#19859) ... Signed-off-by: Robert Fratto <robertfratto@gmail.com>	2 months ago
Christian Haudum	f066d378a5	chore(engine): Do not embed `types.Literal` interface in logical and physical Literal (#19858) ... This ensures that logical.Literal and physical.LiteralExpr do not implicitly implement the types.Literal interface, which caused a bug fixed with #19853 Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 months ago
Christian Haudum	0f35890d4b	chore(engine): Fix conversion from logical to physical Literal in math expressions (#19853) ... Since both logical.Literal and physical.LiteralExpr embed the types.Literal interface, they both automatically also implement the interface. This is problematic when creating a physical LiteralExpr from a logical Literal, rather than from the underlying value. The serialization of the physical plan fails, because the pointer value of the interface is then a logical.Literal, rather than a physical.LiteralExpr	2 months ago
Robert Fratto	0f0846adec	chore(engine): make stream/task debug logging configurable (#19842) ... Signed-off-by: Robert Fratto <robertfratto@gmail.com>	2 months ago
Ashwanth	53230deee4	chore(scheduler): send tracing context to workers (#19839) ... Co-authored-by: Robert Fratto <robertfratto@gmail.com>	2 months ago
Ivan Kalita	9b001dfd01	fix(engine): unset write and read deadlines for wire listeners (#19828) ... Wire http2 peers continuously drop connections due to http server write timeouts. I'm unsetting them.	2 months ago
Robert Fratto	abaeff7fad	chore(engine): query new schedulers over HTTP from query-frontend (#19744) ... Signed-off-by: Robert Fratto <robertfratto@gmail.com> Co-authored-by: Ivan Kalita <ivan.kalita@grafana.com>	2 months ago
renovate-sh-app[bot]	d76b3bf495	fix(deps): update module github.com/apache/arrow-go/v18 to v18.4.1 (main) (#19750) ... Signed-off-by: renovate-sh-app[bot] <219655108+renovate-sh-app[bot]@users.noreply.github.com> Co-authored-by: renovate-sh-app[bot] <219655108+renovate-sh-app[bot]@users.noreply.github.com> Co-authored-by: Paul Rogers <paul.rogers@grafana.com>	2 months ago
Stas Spiridonov	f2c874e6ab	chore: scheduler tasks are aware of their time range (#19709)	2 months ago
Robert Fratto	785743214e	chore(engine): propagate worker listen address to scheduler (#19730) ... Signed-off-by: Robert Fratto <robertfratto@gmail.com>	2 months ago
Christian Haudum	c2cf38383c	chore: Wire up new scheduler and worker components (#19728) ... Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 months ago
benclive	36613bd017	fix: Empty keys are returned if desired in v2 engine (#19717)	2 months ago
Ivan Kalita	f3798af153	chore(engine): Add HTTP/2 transport for wire protocol (#19575) ... Implements HTTP/2 transport layer with protobufCodec, HTTP2Listener/Conn. - Listener receives an http request and holds it. The underlying connection is now managed by the listener and will be returned to the next .Accept() caller (wrapped into an http2Conn). - http2Conn implements wire.Conn interface and supports reads and writes of frames. It uses protobufCodec to serialize, write, read, deserialize the data. - there's proto definition for frames and messages that is used by protobufCodec	2 months ago
Christian Haudum	4b118fb729	chore: Admission lanes for workflow tasks (#19584) ... This PR introduces "admission lanes" for dispatching tasks to the runner (scheduler), that are controlled by token buckets, to avoid overflowing the runner with a large amount of tasks. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 months ago
Stas Spiridonov	d02e55c57a	chore: remove limit physical plan node (#19682)	2 months ago
benclive	01cab53447	feat(engine): implement strict and keepEmpty logfmt parsing (#19668)	2 months ago
Ashwanth	b3803113ad	chore(engine): registers functions for logical op (#19704)	2 months ago
Robert Fratto	7473aee3a5	chore(engine): add periodic discovery of remote schedulers (#19695) ... Signed-off-by: Robert Fratto <robertfratto@gmail.com>	2 months ago
Ashwanth	5d961a4cc3	chore(topk): improvements to reduce alloc bytes and alloc space (#19660)	2 months ago
Robert Fratto	016ee7d39e	chore(engine): serializable physical plans (#19672) ... This adds two new packages, `expressionpb` and `physicalpb`, which are serializable representations of `physical.Expression` and `physical.Plan`, respectively. These packages include utility functions to convert between the protobuf representations and the planner types. A translation layer is used due to the complexity of integrating protobuf throughout the engine, as well as difficulties with finding a clean pattern to construct node types. Once we're able to identify a clean and consistent pattern for interacting with the protobuf types throughout the codebase, the translation layer can be removed in favour of using the protobuf types directly. Co-authored-by: Sophie Waldman <sophie.waldman@grafana.com>	2 months ago
Trevor Whitney	dd6b31473e	fix: errors in parse pipeline (#19667) ... Co-authored-by: Ashwanth Goli <iamashwanth@gmail.com>	2 months ago
Robert Fratto	75e5193a97	chore(engine): expose new scheduler and worker (#19633) ... Signed-off-by: Robert Fratto <robertfratto@gmail.com>	2 months ago
Trevor Whitney	8b39d4d077	refactor: implment parse as a projection (#19579) ... Signed-off-by: Trevor Whitney <trevorjwhitney@gmail.com> Co-authored-by: Christian Haudum <christian.haudum@gmail.com>	2 months ago
Christian Haudum	cf596d0260	chore(engine): Remove unused physical plan visitor (#19647) ... The visitor was introduced in the very beginning without knowing whether it would be used in the future. Except for in tests, it hasn't been used so far. The functionality is provided by the Walk function of the dag package.	2 months ago
Christian Haudum	73bc30d9a8	perf(engine): Improve regexp expression evaluation (#19644) ... This PR changes the evaluation of regex-match binary expression evaluations so that the regexp on right hand side of the expression is only compiled once for each batch evaluation, rather than for each row. This saves a ton of allocations and improves general performance of queries with a regexp line or label filter.	2 months ago
Stas Spiridonov	22292439a2	feat(engine): rate() aggregate function (#19589)	2 months ago
Robert Fratto	3c99535b0a	chore(engine): provide worker (#19588) ... The new worker package connects to an instance of a scheduler (#19570) for task assignment and execution. A worker spawns a fixed number of threads, each of which execute one task at a time. Signed-off-by: Robert Fratto <robertfratto@gmail.com>	2 months ago