9 Commits (383032bccd47df692b25d76095b3ee814e674cf4)

Author	SHA1	Message	Date
Christian Haudum	65de5f95db	chore(engine): Simplify `Pipeline` interface (#19110) ... This PR replaces the two calls to retrieve a value from the pipeline with a single one. To do this, the `Read()` method now returns two arguments: `arrow.Record` and `error`, rather than just the error. If the error is not `nil`, the first argument (arrow.Record) must be ignored by the caller. The `Value()` method is not needed any more and was removed. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	5 months ago
Ashwanth	da4e59f8c6	chore(dataobj): store sort order information in section metadata (#18499)	6 months ago
Christian Haudum	cea56469b9	chore(engine): Pass context to Pipeline's Read() function (#18459) ... This change unifies how the request context is passed through the execution of pipelines. Instead of storing the context on pipelines, pass them as argument to the `Read()` function. This change alters the interface method of `Pipeline` from `Read() bool` to `Read(context.Context) bool`. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	6 months ago
Robert Fratto	4841b2abe2	chore(engine): provide basic implementation of DataObjScan (#17568) ... Co-authored-by: Ashwanth Goli <iamashwanth@gmail.com>	9 months ago
Christian Haudum	1f8e4eebfb	chore(engine): Implement execution pipeline for SortMerge operator (#17406) ... This PR contains an implementation of the k-way merge operation without using a heap, like @rfratto described [here](https://github.com/grafana/loki/pull/17280). The SortMerge is implemented only using slices: * Maintain the following invariant: * For each input pipeline, we store the next record to process. (this already exists as `HeapSortMerge.batches`) * Additionally for each record, track the starting slice offset (which resets to zero whenever a new record is loaded in). * Iteration stops when all input pipelines have been exhausted (no change from how this is now). * To get the next record: * Iterate through each record, looking at the value from their starting slice offset. * Track the top _two_ winners (e.g., the record whose next value is the smallest and the record whose next value is the next smallest). * Find the largest offset in the starting record whose value is still less than the value of the runner-up record from the previous step. * Return the slice of that record using the two offsets, and update the stored offset of the returned record for the next call to `Read`. This approach, like the one with heap, still requires to concatenate (coalesce) the single row records - which is not implemented in this PR yet. On that note, single row records are the worst case scenario with this implementation, not necessarily the regular case. **Update:** After an offline discussion, @owen-d and I agreed on ignoring the worst-case scenario of single-row records for now. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	9 months ago
Owen Diehl	cf54d502fc	feat(dataobj, executor): filter node execution (#17327) ... # Add Filter Pipeline Implementation This PR implements a filter pipeline for Loki's engine executor component. The filter operator enables row-level filtering of Arrow record batches based on boolean expressions. This is a fundamental component for query execution. ## Changes - Implement `NewFilterPipeline` for evaluating filter predicates on record batches - Add `filterBatch` utility to efficiently filter Arrow records - Update `executeFilter` in the executor to use the new implementation - Add comprehensive test suite for the filter pipeline: - Basic literal predicates (true/false) - Column reference predicates - Empty batch handling - Multiple batch processing ## Implementation Notes The filter implementation follows the same pattern as the project pipeline, evaluating predicates against each record and creating a new filtered record based on the results. The implementation supports compound predicates with AND logic. This implementation provides a foundation for more advanced filtering operations as the expression evaluator is enhanced to support more complex expressions.	9 months ago
Owen Diehl	395030b130	feat(dataobj executor): project node (#17312)	9 months ago
Christian Haudum	e828c48702	chore(engine): Implement execution pipeline for the limit operator (#17264) ... Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	9 months ago
Christian Haudum	64c477b1f7	chore(engine): Add framework for query executor (#17260) ... Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	9 months ago