3 Commits (main)

Author	SHA1	Message	Date
Ashwanth	5d961a4cc3	chore(topk): improvements to reduce alloc bytes and alloc space (#19660)	2 months ago
Stas Spiridonov	160dc2c493	chore: removed arrow-go allocators/retain/release (#19569)	2 months ago
Robert Fratto	2e418b13fd	chore(engine): unexport subpackages (#19384) ... This moves packages around to reduce the surface area of the public engine API: * `pkg/engine/planner` moves to `pkg/engine/internal/planner` * `pkg/engine/executor` moves to `pkg/engine/internal/executor` These packages were only used from `pkg/engine` and did not need to be public. We may make them public again in the future if we want to expose subcomponents of the engine. This move means that `pkg/engine/planner/internal/tree` became `pkg/engine/internal/planner/internal/tree`. To reduce the import path, I also moved that package to `pkg/engine/internal/util/tree`. Other than moving files and updating import paths, no code changes are made. Signed-off-by: Robert Fratto <robertfratto@gmail.com>	3 months ago
Robert Fratto	5266865fef	chore(engine): add implementation for performing topk across streams of Arrow records (#18330) ... This commit adds a `engine.topkBatch` type, which can be used by the implementation of execution nodes to perform compute the set of topk rows from a stream of records. This is needed as part of #17976, and more generally for efficiently sorting rows when a limit is also used. Implementing this required quite a few helper utilities in the new `arrowagg` package. The `arrowagg` package provides utilities for aggregating various Arrow types: * `arrowagg.Arrays` to concatenate a stream of `arrow.Array`s. * `arrayagg.Records` to concatenate a stream of `arrow.Record`s, creating a merged schema. * `arrowagg.Mapper` for quickly finding fields in incoming schemas, since arrow-go can return false when comparing two identical schemas/fields. Co-authored-by: Ashwanth <iamashwanth@gmail.com>	6 months ago