apitech/loki - loki - Gitea: Git with a cup of tea

Commit Graph

Author	SHA1	Message	Date
Max Vorobev	c5911d5342	Loki: implement decolorize filter (#7602 ) Implement `decolorize` filter that get rids of ANSI color codes in the log message. Which issue(s) this PR fixes: Fixes #7601	3 years ago
Christian Haudum	a56a88bd35	Add unwrapped aggregation `rate_counter()` (#6412 ) `rate_counter()` takes an unwrapped range as input, but unlike `rate()` it treats the values extracted from the log lines as "counter metric" like in Prometheus' `rate()` function. This is a replacement for the reverted change of #5013 Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	4 years ago
Cyril Tovena	79f0e349f5	Refactor LogQL syntax into its own package (#5539 ) * Refactor LogQL syntax into its own package Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Merge upstream Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Update pkg/loghttp/series.go Co-authored-by: Susana Ferreira <ssncferreira@gmail.com> * lint Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * lint Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> Co-authored-by: Susana Ferreira <ssncferreira@gmail.com>	4 years ago
Sas Swart	c80244a3e4	Parse duration expressions in accordance with promql (#5275 ) * Add support for Prometheus like duration units in logql * Remove redundant type conversion * deduplicate call to sb.String Add a comment to explain why both promql and time's ParseDuration methods are used * extract duration parsing to separate function. * Add tests to ensure promql duration units are supported * Sort duration rune comment and list of runes in ascending order of length.	4 years ago
Kaviraj	1e0386e9e6	Fix `ip` matcher lexer to differentiate filter from identifier (#4598 ) * Fix `ip` matcher lexer to differentiate filter from identifier Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Revert small fmt fix on expr.y Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * PR remarks from Cyril Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Add ip line filter parser test in mix with ip as label Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com>	4 years ago
Danny Kopping	feb7fb470b	LogQL: Simple JSON expressions (#3280 ) * New approach, still rough Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Adding benchmark Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Adding tests Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Minor refactoring Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Appeasing the linter Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Further appeasing the linter Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Adding more tests Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Adding documentation Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Docs fixup Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Removing unnecessary condition Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Adding extra tests from suggestion in review Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Adding JSONParseErr Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Adding test to cover invalid JSON line Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Adding equivalent benchmarks for JSON and JSONExpression parsing Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Adding suffix if label would be overridden Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Reparenting jsonexpr directory to more appropriate location Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Setting empty label on non-matching expression, to retain parity with label_format Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Adding statement about returned complex JSON types Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Added check for valid label name Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Making json expressions shardable Signed-off-by: Danny Kopping <danny.kopping@grafana.com>	5 years ago
Danny Kopping	fcabfec470	Implementing line comments (#3126 ) * Implementing line comments Adding tests Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Appeasing the linter Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Further appeasement Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Adding tests to ensure that a # within a string is not incorrectly lexed as a comment Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Adding documentation Signed-off-by: Danny Kopping <danny.kopping@grafana.com>	5 years ago
Cyril Tovena	fd5cb96693	LogQL: Labels and Metrics Extraction (#2769 ) * Adds logfmt, regexp and json logql parser Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * hook the ast with parsers. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * hook parser with memchunk. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * hook parser with the storage. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * hook parser with ingesters Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * fixes all tests Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Refactor to pipeline and implement ast parsing. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes the lexer for duration and range Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes all tests and add some for label filters Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add label and line format. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add tests for fmt label and line with validations. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Polishing parsers and add some more test cases Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Finish the unwrap parser, still need to add more tests Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Indent this hell. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Moar tests and it works. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add more tests which lead me to find a bug in the lexer Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add more tests and fix all engine tests Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes match stage in promtail pipelines. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Hook Pipeline into ingester, tailer and storage. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Correctly setup sharding for logqlv2 Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes precedences issue with label filters and add moar tests ✌️ * Adds quantile_over_time, grouping for non associate range aggregation parsing and moar tests * Extract with grouping * Adds parsing duration on unwrap * Improve the lexer to support more common identifier as functions. Also add duration convertion for unwrap. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes the frontend logs to include org_id. The auth middleware was happening after the stats one and so org_id was not set 🤦. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Support byte sizes in label filters. This patch extends the duration label filter with support for byte sizes such as `1kB` and `42MiB`. * Wip on error handling. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes json parser with prometheus label name rules. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * fixup! Support byte sizes in label filters. * Wip error handling, commit before big refactoring. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Refactoring in progress. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Work in progress. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Got something that builds and throw __error__ labels properly now. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add error handling + fixes groupins and post filtering. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * 400 on pipeline errors. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes a races in the log pipeline. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Unsure the key is parsable and valid. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Cleanup and code documentation. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Lint. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Lint. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes frontend handler. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes old test. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fix go1.15 local failing test. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> Co-authored-by: Karsten Jeschkies <k@jeschkies.xyz>	5 years ago
Cyril Tovena	f9a2c436ac	Adds backtick for the quoted string token lexer. (#2095 ) * Fixes unit in stats request log. (#2093) * Fix unit conversion in stats log. 🤦 Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fix the tests because you know ! you know ? Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> wip Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add backtick for the quoted string token lexer. This allows to pass unescaped string to Loki. For example we now support this : `` {name="cassandra"} \|~ `error=\w+` `` `` {name!~`mysql-\d+`} `` As you can see no escape is required when using ` quoted strings. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes newlines. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add back missing test. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	6 years ago
Owen Diehl	414f95fedc	Literal Expressions in LogQL (#1677 ) * binops in ast * bin op associativity & precedence * binOpEvaluator work * defers close only if constructed without error * tests binary ops * more binops * updates docs * changelog * number literals in ast * [wip] literalExpr parsing * number parsing * signed literals * propagates evaluator close, handles healthchecks * literal evaluator works on non commutative operations * literalExprs cannot be used as legs of logical/set binops * removes comment * single literalExpr tests * reduces binops with literals in ast construction where possible * doc updates * scalar datatypes in logql * scalar serialization type * increases safety and reduces complexity in ast evaluator * recursive literal binop reduction parse test, improves parse errors on literal construction * vector + literal test	6 years ago
Ganesh Vernekar	184927ccdc	Split files and make yacc more redable Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>	6 years ago

3 Commits (cbd6ec15ce5616cc7ff31b3d41f54156d0e99ff7)