apitech/loki - loki - Gitea: Git with a cup of tea

Commit Graph

Author	SHA1	Message	Date
Robert Fratto	86c57c8616	chore: consistently use new labels.Labels API (#18490 )	11 months ago
Christian Haudum	1187a73667	chore: LogQL syntax and AST simplification and cosistency (#16028 ) This PR aims for making the grammar file (syntax.y) for the LogQL syntax more readable and the AST and its expressions more consistent in naming. You could say these are cosmetics, and yes they are, but if they can help developers better and faster understand the code it's a win. Personally I found the grammar yacc file intimidating until I understood that it was way more complicated that it needed to be. I hope it enables future additions to the syntax to be easier to implement. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	1 year ago
Karsten Jeschkies	83339cbde0	fix: Adjust with offset in last, first and quantile over time queries. (#15915 ) What this PR does / why we need it: The query sharding for `last_over_time`, `first_over_time` and `quantile_over_time` would not adjust start and end by the passed `offset` for the underlying range query. This change introduces tests and fixes the adjustment.	1 year ago
Christian Haudum	845359d4ab	test: Add range aggregation test for instant queries (#13447 ) This PR adds test coverage for the `streamRangeVectorIterator`, which is used by instant queries to evaluate the single step of the query. Signed-off-by: Christian Haudum <christian.haudum@gmail.com> Co-authored-by: Ashwanth <iamashwanth@gmail.com>	2 years ago
Ed Welch	4c88be0ef2	chore: update loki modules for 3.0 release (#12433 ) Signed-off-by: Edward Welch <edward.welch@grafana.com>	2 years ago
Karsten Jeschkies	f67fff3eb2	Approximate `quantile_over_time` (#10417 ) What this PR does / why we need it: This change shards `quantile_over_time` queries using t-digest or DDSketch approximations. It can be enabled with `querier.shard_aggregations=quantile_over_time`. Outstanding - [x] Replace generic return type of `StepEvaluator` with interface `StepResult`. - [x] Send mapped query with quantile sketch expression from frontend to querier over the wire. - [x] Serialize sketches. See https://github.com/influxdata/tdigest/issues/34 - [x] Add feature flag. Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [x] Documentation added - [x] Tests updated - [x] `CHANGELOG.md` updated - [x] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [x] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) --------- Signed-off-by: Callum Styan <callumstyan@gmail.com> Co-authored-by: Callum Styan <callumstyan@gmail.com>	3 years ago
Angus Dippenaar	258a8b3108	Update nix configuration (#8452 ) Fix nix configuration (mainly failing test), and break up various binaries (ie. logcli, promtail, loki, etc.) into their own pakcages. --------- Co-authored-by: Trevor Whitney <trevorjwhitney@gmail.com>	3 years ago
Karsten Jeschkies	81fae24ce2	Return `StepResult` from `StepEvaluator.Next()`. (#10567 ) What this PR does / why we need it: This change makes the `StepEvaluator` more flexible for changes like #10417 that introduce a different step result. With the interface `StepResult` we can support other vector types in the future. Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [x] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`)	3 years ago
Peter Štibraný	90a1d4593e	Update Prometheus dependency (#9205 )	3 years ago
Christian Haudum	089ec1b05f	Fix various linter errors These changes have been generated using `make lint`. ```console $ golangci-lint --version golangci-lint has version 1.50.0 built from 704109c6 on 2022-10-04T10:25:07Z ``` Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	3 years ago
Travis Patterson	209b281593	Fix min and max aggregations when using instant queries. (#7957 ) Instant queries use the new `streamRangeVectorIterator` which has a new set of `aggregators`. Before, aggregations were based on the first point in the range to be aggregated. With `streamRangeVectorIterator`, points are instead treated one at a time. The aggregations were simply checking whether the value was less than the min or greater than the max without any initialization. This PR fixes these aggregations by initializing `min` or `max` to `math.Nan` so the first point is always considered.	4 years ago
李国忠	6d05ade6a8	[performance] metrics query: range vector support streaming agg when no overlap (#7380 ) metrics query: range vector support streaming agg when no overlap	4 years ago
Cyril Tovena	79f0e349f5	Refactor LogQL syntax into its own package (#5539 ) * Refactor LogQL syntax into its own package Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Merge upstream Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Update pkg/loghttp/series.go Co-authored-by: Susana Ferreira <ssncferreira@gmail.com> * lint Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * lint Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> Co-authored-by: Susana Ferreira <ssncferreira@gmail.com>	4 years ago
Cyril Tovena	e3b74158a5	Fixes log deduplication when mutating Labels using LogQL (#5289 ) * Changes the iterator interface to include labelshash. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Improve unused args and fixes build Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Implement the new hash in the memchunk. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add the hash in logproto Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Insert the hash from ingester-querier Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Rename LabelsHash to StreamHash Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Working through fixing tests. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Refactor HeapIterator into merge and sort Iterator. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * lint Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Order alphabetically when ordering samples. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Do not JSON encode the hash for legacy model. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add more regression tests. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Update CHANGELOG.md Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Review feedback	4 years ago
Cyril Tovena	3af7b7900e	Refactor HeapIterator into Merge and Sort Iterator. (#5281 ) * Refactor HeapIterator into merge and sort Iterator. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * lint Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	4 years ago
Garrett	f70c7ea0bc	Implement offset modifier for range vector aggregation in LogQL (#3455 ) * Implement offset modifier for range vector aggregation in LogQL implementation for https://github.com/grafana/loki/issues/2785, the offset modifier allows changing the time offset for range vectors in a query to support e.g. selective timeshift in Grafana Signed-off-by: garrettlish <garrett.li.sh@gmail.com> * add docs for offset modifier * refine offset expr * add offset unit test in engine.go * remove duplicated offset shift * add offset test in range vector test Co-authored-by: garrettlish <garrettlish@163.com>	5 years ago
Cyril Tovena	c4d3f72191	Fixes tightloop that could happen in the engine with LogQL parser. (#3347 ) I've also added a test to prove the issue was happening and now is fixed. Fixes #3275 Fixes #3264 Fixes #3020 Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago
Cyril Tovena	049c7acc12	Improve logql parser allocations. (#2927 ) * Improve logql parser allocations. Also moving labels parsing into logql package, and I'm not shy to say that promql is faster for this. So we're using promql for parser, it's faster because their lexer is more sophisticated than ours. Result are impressive. ``` ❯ benchcmp before.txt after.txt benchmark old ns/op new ns/op delta Benchmark_ParseLabels-16 24394 8745 -64.15% benchmark old allocs new allocs delta Benchmark_ParseLabels-16 156 20 -87.18% benchmark old bytes new bytes delta Benchmark_ParseLabels-16 24976 2099 -91.60% ``` Combined with #2926, I think we should be able to get rid of `util.ToClientLabels`. But that's for another PR. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes a test. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	6 years ago
Cyril Tovena	0be64fcb34	Improve metric queries by computing samples at the edges. (#2293 ) * First pass breaking the code appart. Wondering how we're going to achieve fast mutation of labels. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Work in progress. I realize I need hash for deduping lines. going to benchmark somes. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Tested some hash and decided which one to use. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Wip Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Starting working on ingester. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Trying to find a better hash function. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * More hash testing we have a winner. xxhash it is. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Settle on xxhash Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Better params interfacing. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add interface for queryparams for things that exist in both type of params. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add storage sample iterator implementations. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixing tests and verifying we don't get collions for the hashing method. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixing ingesters tests and refactoring utility function/tests. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixing and testing that stats are still well computed. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixing more tests. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * More engine tests finished. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes sharding evaluator. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes more engine tests. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fix error tests in the engine. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Finish fixing all tests. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes a bug where extractor was not passed in correctly. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add notes about upgrade. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Renamed and fix a bug. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add memchunk tests and starting test for sampleIterator. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Test heap sample iterator. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * working on test. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Finishing testing all new iterators. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Making sure all store functions are tested. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Benchmark and verify everything is working well. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Make the linter happy. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * use xxhash v2. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fix a flaky test because of map. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * go.mod. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> Co-authored-by: Edward Welch <edward.welch@grafana.com>	6 years ago
Cyril Tovena	18f1b0640d	Bytes aggregations (#2150 ) * Add bytes_rate and bytes_over_time. Still need to works on test. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add more tests. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * lint. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * More lint fixes. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * More comments. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add documentation. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Review feedbacks. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fix bad conflic resolution. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	6 years ago
Peter Štibraný	e4a5360586	Update vendored Cortex to master (1.1+) (#2149 ) * Updated Cortex to master (post 1.1) Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Use new modules system from Cortex. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Treat query errors reported by chunk store as client errors. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Removed obsolete tests. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Fix race for ct variable. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Fix test. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>	6 years ago
Cyril Tovena	9988ce4b9f	Improve protobuf serialization (#2031 ) * Generate custom proto type for Streams. This is the first step to improve protobuf serialization. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Improve protobuf serialization allocations. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Saves allocations by stringifying labels only once. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	6 years ago
Cyril Tovena	fcb4c763dc	Fix a bug in range vector skipping data. (#2058 ) Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	6 years ago
Aditya C S	f9ff2a4b02	Add goimport local-prefixes configuration to .golangci.yml (#1710 )	6 years ago
Cyril Tovena	33f70b70ce	Add duplicates info and remove timing informations. (#1496 ) * Add duplicates info and remove timing informations. Timing information in the chunkenc iterator are too eating too much resources. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * lint Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	6 years ago
Cyril Tovena	625c4f1f53	Move to milliseconds for samples to follow prometheus convention	7 years ago
Cyril Tovena	7f3db9d7f0	LogQL: Vector and Range Vector Aggregation. - adds avg,min,max,top,bottomk,stddev,stdvar,count - updates api documentation - adds tests Improve yacc & go lexer to understand duration Remove support for regexp in all queries Clean up querier and logselector	7 years ago

27 Commits (472f02fddf340ec701e2377e2b07165acbd1f25c)