16 Commits (e40777ea3640f6b4bc577c99bc3bc2e7ae5b3627)

Author	SHA1	Message	Date
Sophie Waldman	7be7c9d594	feat(query engine): Implement line_format and label_format with new engine (#20123) ... Signed-off-by: Sophie Waldman <sophie.waldman@grafana.com>	2 months ago
Stas Spiridonov	0b483ab9c0	fix(thor): Fix for column duplicates in consequent compat stages (#20893)	4 months ago
Ashwanth	73961f18a3	chore(xcap): leaner xcap API (#20771)	4 months ago
Trevor Whitney	5acb96abbd	ci: add drilldown and case insensitive queries to correctness tests (#20603) ... Signed-off-by: Trevor Whitney <trevorjwhitney@gmail.com>	4 months ago
Stas Spiridonov	514a142859	fix(thor): Fix error filtering before range aggregations (#20560)	4 months ago
Stas Spiridonov	bedfb78378	chore: Thor query engine memory improvements, part 2 (#20473)	5 months ago
Ashwanth	11b1dcb2a3	chore(engine): introduce execution capture (#19821)	7 months ago
Stas Spiridonov	7eda6749a4	feat(engine): basic math expressions (#19407)	8 months ago
Stas Spiridonov	160dc2c493	chore: removed arrow-go allocators/retain/release (#19569)	8 months ago
Trevor Whitney	13429aa1ea	refactor: move allocator into execution Context (#19550)	8 months ago
Trevor Whitney	3ce6fa2946	feat: implement `unwrap` as a projection (#19409)	8 months ago
Christian Haudum	290d1b711f	chore(engine): Add support for drop stage (drop projection) (#19533) ... This PR add support for dropping columns from the schema. The drop projection only supports dropping column by name, not by matcher.	8 months ago
Christian Haudum	2214a700b3	chore(engine): Implement new semantic column naming (#19415) ... This PR implements the column naming convention introduced in https://github.com/grafana/loki/pull/19396 Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	8 months ago
Stas Spiridonov	16dab82593	chore: consistent expressionEvaluator.eval result memory ownership (#19438)	8 months ago
Christian Haudum	119275aab7	chore(engine): Column naming conventions (#19396) ... In the new engine, we need fully qualified column names, since columns from different sources can have the same name. Right now, the distinction between columns with the same name is implemented using the `Metadata` field on the `arrow.Field`. However, it is quite cumbersome to parse the column type and data type from this generic map. This PR introduces package with naming conventions for columns, defined by name, data type, and column type. So, this information can be encoded into the `Name` field of the `arrow.Field`. The convention is defined as ``` [DATA_TYPE].[COLUMN_TYPE].[COLUMN_NAME] ``` #### Examples: * `utf8.label.service_name` * `timestamp_ns.builtin.timestamp` The column type can easily be converted into a `Scope`, which is defined by an origin and type. The mapping is as follows: ``` ColumnTypeBuiltin -> Scope{Record, Attribute} ColumnTypeMetadata -> Scope{Record, Builtin} ColumnTypeLabel -> Scope{Resource, Attribute} ColumnTypeParsed -> Scope{Generated, Attribute} ColumnTypeGenerated -> Scope{Generated, Builtin} ColumnTypeAmbiguous -> Scope{Unscoped, Attribute} ``` --- Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	8 months ago
Robert Fratto	2e418b13fd	chore(engine): unexport subpackages (#19384) ... This moves packages around to reduce the surface area of the public engine API: * `pkg/engine/planner` moves to `pkg/engine/internal/planner` * `pkg/engine/executor` moves to `pkg/engine/internal/executor` These packages were only used from `pkg/engine` and did not need to be public. We may make them public again in the future if we want to expose subcomponents of the engine. This move means that `pkg/engine/planner/internal/tree` became `pkg/engine/internal/planner/internal/tree`. To reduce the import path, I also moved that package to `pkg/engine/internal/util/tree`. Other than moving files and updating import paths, no code changes are made. Signed-off-by: Robert Fratto <robertfratto@gmail.com>	8 months ago
Christian Haudum	4305a6aa61	chore(engine): Make literals simple types (#17733) ... Representing data types as simple types (string, int64, float64) makes them more efficient to cast without conversion. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	11 months ago
Christian Haudum	94b1d2d5ce	chore(engine): Improve type system of the new query engine (#17456) ... **What this PR does / why we need it**: The original type system for the new engine was introduced before adding Apache Arrow and with that an additional type system. One aspect of the new type system of this PR is that the logical types of the engine are aligned/interchangeable with the arrow logical types. ```go type Type uint8 const ( NULL = Type(arrow.NULL) BOOL = Type(arrow.BOOL) STRING = Type(arrow.STRING) INT64 = Type(arrow.INT64) FLOAT64 = Type(arrow.FLOAT64) ) ``` The actual data types of Loki (`Boolean`, `String`, `Integer`, `Float`, `Timestamp`, `Duration`, and `Bytes`) are implemented as wrapper around a logical type. Different data types can use the same logical type as representation, such as `Integer`, `Timestamp` and `Duration` both use `INT64`. Since Arrow array types are bound to logical types, and one logical type can have multiple Loki data types, it is required to "annotate" Fields of an arrow.Record using its Metadata, similar to how it is done for distinguishing builtin columns from label/metadata/parsed columns. Additionally, the PR contains a mapping between Loki types and Arrow types. The `Literal` has been updated to use the new types. The old system also used UINT64 as representation of the `Timestamp` type, which is incorrect, since both LogQL and the DataObjects allow timestamps before unix epoch. --- Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	1 year ago
Owen Diehl	395030b130	feat(dataobj executor): project node (#17312)	1 year ago