18 Commits (503524e39d2740c61e298c465b5012270ea2e116)

Author	SHA1	Message	Date
Owen Diehl	395030b130	feat(dataobj executor): project node (#17312)	1 month ago
Owen Diehl	920e3b23ed	chore(dataobj,testing,executor): pipeline equality testing (#17311)	1 month ago
Christian Haudum	e828c48702	chore(engine): Implement execution pipeline for the limit operator (#17264) ... Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	1 month ago
Owen Diehl	cff0df63b3	chore(dataobj,executor): CSV arrow batches & pipeline helpers (#17290)	1 month ago
Christian Haudum	64c477b1f7	chore(engine): Add framework for query executor (#17260) ... Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	1 month ago
Christian Haudum	525b41155d	chore(engine): Add expression evaluator to evaluate physical expressions against a given record batch (#17257) ... The evaluator takes a physcial expressions and an arrow.Record as input and return a ColumnVector. This PR only implements the evaluation of the LiteralExpr and ColumnExpr. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	1 month ago
Christian Haudum	1705f98a9f	chore(engine): Wire up new execution engine (#17032) ... This PR connects the already existing components of the new query engine and wires them up in the querier API. The feature flag `-querier.engine.enable-v2-engine` controls whether supported queries should be executed using the new engine. Note that this implementation currently does not execute any queries. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 months ago
Christian Haudum	ba61aa88de	chore(engine): Integrate metastore into physical query planning (#17016) ... This PR integrates the data object lookup using the metastore into the query planning. To achieve that, I extended the `Metastore` interface with an additional function that not only returns the data object paths, but also the stream IDs of streams that match the given label matchers. ```go type Metastore interface { ... // StreamsIDs returns object store paths and stream IDs for all matching objects for the given matchers between [start,end] StreamIDs(ctx context.Context, start, end time.Time, matchers ...*labels.Matcher) ([]string, [][]int64, error) ... } ``` Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 months ago
Christian Haudum	6824f4e575	chore(engine): Add printer for Mermaid diagram format (#17000) ... Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 months ago
Christian Haudum	04d2ba3ca8	chore(engine): Add simple plan optimizer (#16965) ... This PR adds a plan optimizer that pushes down limit and filter predicates to the scan node. ### Example Plan: ``` Limit #0xc0000110f8 offset=0 limit=1000 └── Filter #0xc00039dd10 │ └── Predicate expr=LT(timestamp.builtin, 1742826126000000000) └── Filter #0xc00039ddd0 │ └── Predicate expr=GT(age.metadata, 21) └── SortMerge #0xc00039de60 column=timestamp.builtin order=ASC ├── DataObjScan #0xc00011b420 location=obj1 stream_ids=(1, 2) projections=() direction=0 limit=0 └── DataObjScan #0xc00011b490 location=obj2 stream_ids=(3, 4) projections=() direction=0 limit=0 ``` Optimized plan: ``` Limit #0xc0000110f8 offset=0 limit=1000 └── SortMerge #0xc00039de60 column=timestamp.builtin order=ASC ├── DataObjScan #0xc00011b420 location=obj1 stream_ids=(1, 2) projections=() direction=0 limit=1000 │ ├── Predicate expr=LT(timestamp.builtin, 1742826126000000000) │ └── Predicate expr=GT(age.metadata, 21) └── DataObjScan #0xc00011b490 location=obj2 stream_ids=(3, 4) projections=() direction=0 limit=1000 ├── Predicate expr=LT(timestamp.builtin, 1742826126000000000) └── Predicate expr=GT(age.metadata, 21) ``` Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 months ago
Christian Haudum	40223d7320	chore(engine): Unify `Literal` and `ColumnRef` across logical and physical plan (#16927) ... Literals and column refs are used by both the logical and physical plan. However, the interfaces they need to implement are partly private (`isValue()` and `isExpression()`). Therefore they are wrapped in their own struct that implement these interfaces. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 months ago
Christian Haudum	138093b900	chore(planner): Convert AST into logical plan (#16916) ... Minimal implementation of the logical planner that converts a parsed LogQL statement in AST form into a logical plan in SSA form. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 months ago
Christian Haudum	0a3230f457	chore(ast): Update WalkFn signature to return bool (#16989) ... The boolean return value indicates whether walking/traversing should be continued with child nodes. This PR does not change any existing traversal behaviour. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 months ago
Christian Haudum	efe15486fe	chore(engine): Add first stage of physical query planning (#16891) ... Planning consists of two stages: 1. Conversion stage: Instructions from the logical plan are converted into Nodes in the physical plan. Most instructions can be translated into a single Node. However, `MakeTable` translates into multiple `DataObjScan` nodes. 2. Pushdown stage: a) Push down the limit of the `Limit` node to the `DataObjScan` nodes. b) Push down filter predicates from the `Filter` nodes to the `DataObjScan` nodes. **This PR implements only the first stage.** Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 months ago
Christian Haudum	f79906ca75	chore(engine): Add utility function `canExecuteWithNewEngine` (#16892) ... The function determines whether a query can be executed by the new execution engine. In the first step, the new execution engine will only be able to execute limited/filter queries without any transformation stages. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 months ago
Robert Fratto	cbe97e6c1a	chore(engine): Update logical planner types to be structured as SSA (#16841) ... Signed-off-by: Robert Fratto <robertfratto@gmail.com> Co-authored-by: Christian Haudum <christian.haudum@gmail.com>	2 months ago
Ashwanth	c51fa39684	chore(dataobj): pass down line filters to logs reader (#16802)	2 months ago
Christian Haudum	9691f3e5dc	chore: Move `pkg/dataobj/planner` to `pkg/engine/planner` (#16815) ... The package pkg/dataobj/planner/logical was initially created within the package pkg/dataobj because of the dependency on the internal package pkg/dataobj/internal/metadatamd. However, the package should be located inside a separate package without the dependency on the dataobj package, since the engine should be storage agnostic. --- Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 months ago