2 Commits (21f3d09caab2cc6d64fbb07ceeaea8efbce25814)

Author	SHA1	Message	Date
Robert Fratto	ed1b61d253	chore(engine): improve performance of topk pipeline (#18593)	6 months ago
Robert Fratto	5266865fef	chore(engine): add implementation for performing topk across streams of Arrow records (#18330) ... This commit adds a `engine.topkBatch` type, which can be used by the implementation of execution nodes to perform compute the set of topk rows from a stream of records. This is needed as part of #17976, and more generally for efficiently sorting rows when a limit is also used. Implementing this required quite a few helper utilities in the new `arrowagg` package. The `arrowagg` package provides utilities for aggregating various Arrow types: * `arrowagg.Arrays` to concatenate a stream of `arrow.Array`s. * `arrayagg.Records` to concatenate a stream of `arrow.Record`s, creating a merged schema. * `arrowagg.Mapper` for quickly finding fields in incoming schemas, since arrow-go can return false when comparing two identical schemas/fields. Co-authored-by: Ashwanth <iamashwanth@gmail.com>	7 months ago