apitech/loki - loki - Gitea: Git with a cup of tea

Commit Graph

Author	SHA1	Message	Date
Susana Ferreira	10e9695b21	Fix rangemapper with VectorExpr support (#7288 ) What this PR does / why we need it: The new expression type VectorExpr introduced recently in [PR](https://github.com/grafana/loki/pull/7007), is not taken into account in the query mapping in the AST mapper rangemapper. The AST mapper shardmapper was updated in [PR](https://github.com/grafana/loki/pull/7045). Special notes for your reviewer: Checklist - [ ] Reviewed the `CONTRIBUTING.md` guide - [ ] Documentation added - [x] Tests updated - [x] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	3 years ago
Owen Diehl	bac52165d3	highly sharded subqueries are ellided in expr.String (#7203 ) This is mostly a quality of life improvement. Highly sharded subqueries can be burdensome to view and they're logged on every request. This PR will only display the first four subqueries in a `Concat{Log,Sample}Expr`, which is easier on the eyes when viewing logs and is more efficient.	3 years ago
Susana Ferreira	3f3f4eda1d	Fix topk and bottomk operations with int <= 0 (#6937 ) * Fix topk and bottomk operations with int <= 0 * Fix VectorAggregationExpr String method * Update rangemapper_test expression * Add tests * Update CHANGELOG.md	3 years ago
Susana Ferreira	79b2d01970	Add label_replace and literal expressions to instant query split by range (#6515 ) * Add label_replace logic to instant query split by range * Add label_replace and literal expressions support * Support binary operations with literal expressions * Remove Literal expression as splittable * Fix lint * Address comments	4 years ago
Christian Haudum	798677ae7f	Fix panic in instant query splitting when using unwrapped rate (#6348 ) * Fix panic in instant query splitting when using unwrapped rate The range aggregation `rate()` supports both log ranges and unwrapped ranges, e.g. `rate({app="foo"} [$__interval])` and `rate({app="foo"} \| unwrap bar [$__interval])` Since `rate()` was split into multiple `count_over_time()` over total duration, but `count_over_time()` does not support `unwrap`, unwrapped rate queries caused panics. This fix changes the splitting of `rate({app="foo"} \| unwrap bar [$__interval]` into multiple `sum_over_time()` over total duration. Fixes #6344 Signed-off-by: Christian Haudum <christian.haudum@gmail.com> * Add tests Signed-off-by: Christian Haudum <christian.haudum@gmail.com> * Integrate review feedback Co-authored-by: Susana Ferreira <susana.ferreira@grafana.com> Co-authored-by: Susana Ferreira <susana.ferreira@grafana.com>	4 years ago
Callum Styan	97ee1b93f3	[logql] don't try to execute a modified function that is not allowed by the AST (#6346 ) * Panic with an actual error if clone (which calls range aggregation expr's validate) errors, instead of swallowing the error and then attempting to execute a nil function, which panics anyways. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Call the validation function on a modified expression before attempting to execute it. If the validation fails we should return the original expression. Signed-off-by: Callum Styan <callumstyan@gmail.com>	4 years ago
Susana Ferreira	0dfd2cbbcc	Fix label extraction pushdown on instant queries (#6215 ) * Add downstream label extraction tests * Fix label extraction downstream pushdown	4 years ago
Christian Haudum	ee4ab1b394	Fix certain binary expression queries by optimizing through push down (#6132 ) * Add test cases that fail Instant queries of the following type fail and return an `unimplemented` error: ``` sum(count_over_time({foo="bar"} \| logfmt \| duration > 2s [3s])) / sum(count_over_time({foo="bar"} [3s])) ``` Signed-off-by: Christian Haudum <christian.haudum@gmail.com> * Fix certain binary expressions for instant queries If either the left hand side or the right hand side of a binary expression is a noop, we need to return the original expression so the whole expression is a noop as well, and thus not executed using the downstream engine. Otherwise, a binary expression that has a noop on either side, results in an `unimplemented` error when executed using the downstream engine. Signed-off-by: Christian Haudum <christian.haudum@gmail.com> * Optimize instant vector aggregations with log extraction stage This change optimizes queries that use a vector aggregation without grouping around a range aggregation with a label extraction stage such as `json` or `logfmt`. Since the vector aggregation can be pushed down to the downstream query, the downstream query does not create a massive amount of streams, even though it contains a generic label extraction stage. Signed-off-by: Christian Haudum <christian.haudum@gmail.com> * fixup! Add test cases that fail Signed-off-by: Christian Haudum <christian.haudum@gmail.com> * fixup! fixup! Add test cases that fail Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	4 years ago
Susana Ferreira	6b6521f791	Add metrics to range mapper (#6030 ) * Refactor ShardingMetrics to MapperMetrics * Add mapper metrics to range mapper	4 years ago
Susana Ferreira	0bce8d95a2	Split by range of instant queries (#5662 ) * Split by range on Instant queries POC v3 Co-authored-by: Christian Haudum <christian.haudum@gmail.com> * Handle uneven split by duration * Register SplitByRangeMiddleware in roundtripper Signed-off-by: Christian Haudum <christian.haudum@gmail.com> * fixup! Register SplitByRangeMiddleware in roundtripper Signed-off-by: Christian Haudum <christian.haudum@gmail.com> * fixup! fixup! Register SplitByRangeMiddleware in roundtripper Signed-off-by: Christian Haudum <christian.haudum@gmail.com> * Remove rewrite if range aggr has label extraction stage In case a range aggregation has a generic label extraction stage, such as `\| json` or `\| logfmt` and no group by, we cannot split it, because otherwise the downstream queries would result in too many series. Signed-off-by: Christian Haudum <christian.haudum@gmail.com> * Fix linting * Implement range splitting for rate() and bytes_rate() Signed-off-by: Christian Haudum <christian.haudum@gmail.com> * Fix linting * Calculate offset of downstream queries correctly if the outer query range contains an offset as well. Signed-off-by: Christian Haudum <christian.haudum@gmail.com> * Fix linting * Add optimization by moving the outer label grouping downstream * Add label grouping downstream optimization to rate and bytes_rate expressions * Add changelog entry Signed-off-by: Christian Haudum <christian.haudum@gmail.com> * Simplify types in rangemapper Signed-off-by: Christian Haudum <christian.haudum@gmail.com> * fixup! Simplify types in rangemapper Signed-off-by: Christian Haudum <christian.haudum@gmail.com> * Check in Map function if query is splittable by range Since this is the main function of the mapper, we can ensure here that only supported vector/range aggregations are handled. Signed-off-by: Christian Haudum <christian.haudum@gmail.com> * Some code cleanups and variable renaming Signed-off-by: Christian Haudum <christian.haudum@gmail.com> * Extract duplicate code in range aggr mapping into function Signed-off-by: Christian Haudum <christian.haudum@gmail.com> * Add topk to supported splittable vector aggregations Signed-off-by: Christian Haudum <christian.haudum@gmail.com> * Check if query is splittable by range before calling Map() Signed-off-by: Christian Haudum <christian.haudum@gmail.com> * Add more function comments Signed-off-by: Christian Haudum <christian.haudum@gmail.com> * Rename RangeVectorMapper to RangeMapper Signed-off-by: Christian Haudum <christian.haudum@gmail.com> * Fix incorrect import due to rebase Signed-off-by: Christian Haudum <christian.haudum@gmail.com> * Add equivalence test cases with `logfmt` pipeline stage Signed-off-by: Christian Haudum <christian.haudum@gmail.com> * Remove limitation of pushing down vector aggr only if grouping is present Signed-off-by: Christian Haudum <christian.haudum@gmail.com> * Remove TestRangeMappingEquivalenceMockMapper test This test is essentially the same as the test Test_SplitRangeVectorMapping, just using a different representation of the result. Signed-off-by: Christian Haudum <christian.haudum@gmail.com> * fixup! Remove limitation of pushing down vector aggr only if grouping is present Signed-off-by: Christian Haudum <christian.haudum@gmail.com> * fixup! fixup! Remove limitation of pushing down vector aggr only if grouping is present Signed-off-by: Christian Haudum <christian.haudum@gmail.com> * Fix linter errors Signed-off-by: Christian Haudum <christian.haudum@gmail.com> * Better naming of variable Signed-off-by: Christian Haudum <christian.haudum@gmail.com> * Split SplitRangeVectorMapping test into two to have the test for noop queries separated Signed-off-by: Christian Haudum <christian.haudum@gmail.com> Co-authored-by: Christian Haudum <christian.haudum@gmail.com> Co-authored-by: Owen Diehl <ow.diehl@gmail.com>	4 years ago

10 Commits (05d21106817438f523c6161d0a1312812d6db44c)