apitech/loki - loki - Gitea: Git with a cup of tea

Commit Graph

Author	SHA1	Message	Date
Owen Diehl	1530120c2e	Logql comparison ops (#2182 ) * comparison binops * starts comparison operator docs * implements bool modifier for comparison binops * vector-vector binop merging cases, including bool modifier * logql docs formatting * resets protos * linter fixes * goimports local * gofmt	6 years ago
Cyril Tovena	18f1b0640d	Bytes aggregations (#2150 ) * Add bytes_rate and bytes_over_time. Still need to works on test. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add more tests. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * lint. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * More lint fixes. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * More comments. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add documentation. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Review feedbacks. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fix bad conflic resolution. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	6 years ago
Goutham Veeramachaneni	06b50590a7	Fix bug in logql parsing that leads to crash. (#2046 ) * Fix bug in logql parsing that leads to crash. ``` panic: runtime error: invalid memory address or nil pointer dereference [signal SIGSEGV: segmentation violation code=0x1 addr=0x18 pc=0x20ae356] goroutine 39530021 [running]: github.com/grafana/loki/pkg/logql.andFilter.Filter(0x0, 0x0, 0x3beffc0, 0xc01f065bc0, 0xc01aaaec00, 0x13d, 0x200, 0xffffffffffffffff) /src/loki/pkg/logql/filter.go:66 +0x26 github.com/grafana/loki/pkg/chunkenc.(bufferedIterator).Next(0xc00a48fba0, 0x0) /src/loki/pkg/chunkenc/memchunk.go:616 +0x1cb github.com/grafana/loki/pkg/iter.(nonOverlappingIterator).Next(0xc00130ae40, 0x0) /src/loki/pkg/iter/iterator.go:438 +0x135 github.com/grafana/loki/pkg/iter.(timeRangedIterator).Next(0xc00130ae80, 0x0) /src/loki/pkg/iter/iterator.go:497 +0x48 github.com/grafana/loki/pkg/iter.(reverseIterator).load(0xc00072ac40) /src/loki/pkg/iter/iterator.go:555 +0x70 github.com/grafana/loki/pkg/iter.(reverseIterator).Next(0xc00072ac40, 0x0) /src/loki/pkg/iter/iterator.go:563 +0x2f github.com/grafana/loki/pkg/iter.(nonOverlappingIterator).Next(0xc00130af40, 0x10) /src/loki/pkg/iter/iterator.go:438 +0x135 ``` This is caused by people doing `\|= ""` which would result in parsing of TrueFilter but would be returned as nil. Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com> * Add tests for the panic Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com>	6 years ago
Josh Soref	0d9cf4946a	Spelling (#2014 ) * spelling: access * spelling: actual * spelling: asynchronously * spelling: circleci * spelling: circle * spelling: cognitive * spelling: comment * spelling: computation * spelling: correct * spelling: daemon * spelling: elliott * spelling: enabled * spelling: encbuf * spelling: extracted * spelling: explicitly * spelling: independent * spelling: languages * spelling: loose * spelling: message * spelling: metrics * spelling: mismatch * spelling: nonexistent * spelling: output * spelling: pipeline * spelling: pipe line * spelling: position * spelling: prepend * spelling: query * spelling: receive * spelling: received * spelling: reducible * spelling: registry * spelling: response * spelling: retryable * spelling: scanning * spelling: sourced * spelling: supported * spelling: template * spelling: timeout * spelling: unsupported * spelling: adapter * spelling: deferring * spelling: expirable	6 years ago
Owen Diehl	7effeec642	Sharding optimizations I: AST mapping (#1846 ) * [wip] sharding evaluator/ast * [wip] continues experimenting with ast mapping * refactoring in preparation for binops * evaluators can pass state to other evaluators * compiler alignment * Evaluator method renamed to StepEvaluator * chained evaluator impl * tidying up sharding code * handling for ConcatSampleExpr * downstream iterator * structure for downstreaming asts * outlines sharding optimizations * work on sharding mapper * ast sharding optimizations * test for different logrange positions * shard mapper tests * stronger ast sharding & tests * shardmapper tests for string->string * removes sharding evaluator code * removes unused ctx arg * Update pkg/logql/evaluator.go Co-Authored-By: Cyril Tovena <cyril.tovena@gmail.com> Co-authored-by: Cyril Tovena <cyril.tovena@gmail.com>	6 years ago
Cyril Tovena	3829b707c3	Optimize empty filter queries. (#1810 ) * Optimize filter queries such as `{label="foo"} \|~ "" \|~ "." \|= ""` doesn't contains a filter. The query is not really rewritten and the stringer func still returns the correct original query. However the `Filter` func of the `Expr` returns nil which what is being checked when splitting queries in the frontend and also what is being used in the memchunk. see https://github.com/grafana/loki/blob/master/pkg/querier/queryrange/roundtrip.go#L81 and https://github.com/grafana/loki/blob/master/pkg/chunkenc/memchunk.go#L614 Fixes #1792 Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> Add missing tests with combination of empty or regex. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Review feedback. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	6 years ago
Cyril Tovena	d34b0a9d30	Regexp simplification (#1787 ) * Refactor line filter to support custom parsing for perl regexp. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Support alternate, concat regex simplification Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add benchmark between normal and simplified regex. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Working through all possible concat operation. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes concat operations. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Adds more type of capture group Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Improve test robustness Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Support for nexted concat. git push Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes bug with anychar. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * add more not supported cases. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Refactor the concat alternates function. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Improve documentation and refactor code. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes all code using line filter. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Improve tests to check nil filter. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Ensure benchmark is not optimized by the compilation. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Review feedback. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	6 years ago
Cyril Tovena	6363e8da0a	Better logql metric status code. (#1718 ) * Fixes querier logql error to be 400 only for ast parsing error. Improve frontend metrics to use a more granular status_code. Improve prometheus histogram granularity. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add latency type dimensions to bytes throughput metric. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Use IsParseError in panic recovering. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * typo Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	6 years ago
Aditya C S	f9ff2a4b02	Add goimport local-prefixes configuration to .golangci.yml (#1710 )	6 years ago
Owen Diehl	414f95fedc	Literal Expressions in LogQL (#1677 ) * binops in ast * bin op associativity & precedence * binOpEvaluator work * defers close only if constructed without error * tests binary ops * more binops * updates docs * changelog * number literals in ast * [wip] literalExpr parsing * number parsing * signed literals * propagates evaluator close, handles healthchecks * literal evaluator works on non commutative operations * literalExprs cannot be used as legs of logical/set binops * removes comment * single literalExpr tests * reduces binops with literals in ast construction where possible * doc updates * scalar datatypes in logql * scalar serialization type * increases safety and reduces complexity in ast evaluator * recursive literal binop reduction parse test, improves parse errors on literal construction * vector + literal test	6 years ago
Owen Diehl	6bbb61eb2f	Binary operators in LogQL (#1662 ) * binops in ast * bin op associativity & precedence * binOpEvaluator work * defers close only if constructed without error * tests binary ops * more binops * updates docs * changelog * better logql parsing errors for binops Signed-off-by: Owen Diehl <ow.diehl@gmail.com> * adds ^ operator	6 years ago
Owen Diehl	e0a77656fd	removes duplicate logRangeExprExt grammar (#1651 ) * removes duplicate logRangeExprExt grammar * linting errors	6 years ago
Owen Diehl	6930fb2a5c	adds stringer methods for all ast expr types (#1630 ) * adds stringer methods for all ast expr types * Update pkg/logql/ast.go Co-Authored-By: Cyril Tovena <cyril.tovena@gmail.com> Co-authored-by: Cyril Tovena <cyril.tovena@gmail.com>	6 years ago
Owen Diehl	1773900659	compiler guarantees for logql exprs (#1626 )	6 years ago
Owen Diehl	b7d23f41bd	Decouple logql engine/AST from execution context (#1605 ) * logql engine is an interface * [wip] begins agnostic logql evaluator work * decouples logql AST from execution context * healthcheck comments	6 years ago
Ganesh Vernekar	1761db6d71	Support <selector> <range> <filters> for aggregation (#1526 ) Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>	6 years ago
Peter Štibraný	7bb77ecdd8	Convert string to bytes once only when doing string filtering. (#1405 ) This makes it slightly faster: name old time/op new time/op delta ContainsFilter-4 18.2ns ± 2% 9.8ns ± 1% -46.20% (p=0.000 n=10+8) name old alloc/op new alloc/op delta ContainsFilter-4 0.00B 0.00B ~ (all equal) name old allocs/op new allocs/op delta ContainsFilter-4 0.00 0.00 ~ (all equal) Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>	6 years ago
Cyril Tovena	f7ee1c753c	Supports same duration format in LogQL as Prometheus Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	6 years ago
Cyril Tovena	8f9cd97880	Improve API compatibility. - keep supporting regex for old endpoint. - use reserved keyword for removed property in grpc	6 years ago
Cyril Tovena	7d63795350	fix linter & better comments	6 years ago
Cyril Tovena	7f3db9d7f0	LogQL: Vector and Range Vector Aggregation. - adds avg,min,max,top,bottomk,stddev,stdvar,count - updates api documentation - adds tests Improve yacc & go lexer to understand duration Remove support for regexp in all queries Clean up querier and logselector	6 years ago
Cyril Tovena	3346ce1b40	Storage memory improvement (#713 ) * add benchmark for storage queries * improve iterator to load only on next * fix memory retained by lazy chunks * reverse backward lazy iterator	7 years ago
Sandeep Sukhani	2a8a03795a	Improvements in live tailing of logs (#541 ) * Improvements in live tailing of logs Instead of polling for new logs, a grpc stream is opened between ingester and querier to get live logs Querier reconnects to disconnected or newly added ingesters * Added more comments to code for live log tailing * Some code refactoring in live log tailing * handling delayfor in logcli, max delayfor to be 5 seconds * some changes in tail response in live tailing * Fixed issue with stopping ingesters gracefully when live tailing is being used * Added tests for tailer in querier * Live tailing made logql compatible, some code refactoring suggested in PR	7 years ago
Tom Wilkie	80bbd04778	Add an extra filter step if the user specifies a regex using old API. Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	7 years ago
Tom Wilkie	bfb34e4070	Wire up LogQL filters in the ingester. Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	7 years ago

25 Commits (0b5996021fb16bc9404679e2c0028ba9fbfe95e2)