Like Prometheus, but for logs.

prometheushacktoberfestgrafanacloudnativeloggingloki

You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

Robert Fratto 06cf52757b chore(dataobj): Add columnar reading APIs to logs and streams sections (#17976) ... This PR introduces columnar reading APIs to both the logs and streams sections. As part of this change, both sections now expose the columns stored in the section, which are then used to define predicates on the readers. The columnar reading APIs emit a sequence of `arrow.Record`s, where each record has no more than the batch size passed to the `Reader.Read` method. Unlike the original row-based readers, the columnar readers: * Allow reading a subset of columns * Have column-based predicates that map literally to the new query engine To be able to represent a column's value for use in predicates, we import [github.com/apache/arrow-go/v18/arrow/scalar](https://pkg.go.dev/github.com/apache/arrow-go/v18@v18.3.0/arrow/scalar). Section Reader implementations perform mapping to and from the internal `dataset.Value`. The implementation of `logs.Reader` is almost an identical copy of the `streams.Reader` implementation. I've opted for duplicating the implementation in the short term, since there's no obvious way for how to deduplicate it in a way keeps the package structure easy to understand. We can change this in the future if a clean and simple approach presents itself. This PR does not yet update `DataObjScan` to make use of these new APIs; that's being left out of scope for another PR to handle.		7 months ago
..
arrowconv.go	chore(dataobj): Add columnar reading APIs to logs and streams sections (#17976)	7 months ago