apitech/loki - loki - Gitea: Git with a cup of tea

Commit Graph

Author	SHA1	Message	Date
Jarrod Funnell	267107b75e	Parser: Allow literal control chars in logfmt decoder (#3932 ) * Allow literal control chars in decoder * Ensure decoding control chars in logfmt works	5 years ago
Owen Diehl	b58e9cdee2	fixes a badly referenced variable name in StepEvaluator code (#3996 )	5 years ago
Cyril Tovena	5d8414fcd6	Intern label keys for LogQL parser. (#3990 ) * Intern label keys for LogQL parser. They often will see the same key over and over and instead of allocating/sanitizing we can re-use the previous value. This mostly speeds up parser that were using a dynamic list of keys like json and logfmt. We could intern values too specially for metrics queries. ``` ❯ benchcmp before.txt after.txt benchmark old ns/op new ns/op delta Benchmark_Parser/json/no_labels_hints-16 3894 2885 -25.91% Benchmark_Parser/json/labels_hints-16 2084 2163 +3.79% Benchmark_Parser/unpack/no_labels_hints-16 915 890 -2.72% Benchmark_Parser/unpack/labels_hints-16 848 846 -0.32% Benchmark_Parser/logfmt/no_labels_hints-16 2229 1146 -48.59% Benchmark_Parser/logfmt/labels_hints-16 1230 1150 -6.50% Benchmark_Parser/regex_greedy/no_labels_hints-16 3881 3754 -3.27% Benchmark_Parser/regex_greedy/labels_hints-16 5443 3737 -31.34% Benchmark_Parser/regex_status_digits/no_labels_hints-16 318 290 -8.89% Benchmark_Parser/regex_status_digits/labels_hints-16 335 289 -13.85% Benchmark_Parser/pattern/no_labels_hints-16 151 151 -0.20% Benchmark_Parser/pattern/labels_hints-16 129 131 +1.24% benchmark old allocs new allocs delta Benchmark_Parser/json/no_labels_hints-16 69 43 -37.68% Benchmark_Parser/json/labels_hints-16 32 36 +12.50% Benchmark_Parser/unpack/no_labels_hints-16 19 17 -10.53% Benchmark_Parser/unpack/labels_hints-16 16 17 +6.25% Benchmark_Parser/logfmt/no_labels_hints-16 46 16 -65.22% Benchmark_Parser/logfmt/labels_hints-16 20 16 -20.00% Benchmark_Parser/regex_greedy/no_labels_hints-16 2 2 +0.00% Benchmark_Parser/regex_greedy/labels_hints-16 2 2 +0.00% Benchmark_Parser/regex_status_digits/no_labels_hints-16 2 2 +0.00% Benchmark_Parser/regex_status_digits/labels_hints-16 2 2 +0.00% Benchmark_Parser/pattern/no_labels_hints-16 2 2 +0.00% Benchmark_Parser/pattern/labels_hints-16 1 1 +0.00% benchmark old bytes new bytes delta Benchmark_Parser/json/no_labels_hints-16 1089 624 -42.70% Benchmark_Parser/json/labels_hints-16 456 480 +5.26% Benchmark_Parser/unpack/no_labels_hints-16 569 537 -5.62% Benchmark_Parser/unpack/labels_hints-16 553 537 -2.89% Benchmark_Parser/logfmt/no_labels_hints-16 576 336 -41.67% Benchmark_Parser/logfmt/labels_hints-16 256 336 +31.25% Benchmark_Parser/regex_greedy/no_labels_hints-16 194 194 +0.00% Benchmark_Parser/regex_greedy/labels_hints-16 194 194 +0.00% Benchmark_Parser/regex_status_digits/no_labels_hints-16 52 52 +0.00% Benchmark_Parser/regex_status_digits/labels_hints-16 52 52 +0.00% Benchmark_Parser/pattern/no_labels_hints-16 35 35 +0.00% Benchmark_Parser/pattern/labels_hints-16 32 32 +0.00% ``` Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Move to labels.go and cap the amount of keys that can be interned to 1024. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago
Cyril Tovena	e9c0687c15	Reject labels with invalid runes when using implicit extraction parser. (#3983 ) `json` and `logfmt` parser will attempt to extract all possible keys and values as labels. But if a values currently contains the rune error (replace when there's an invalid rune) PromQL parser will fail and the request will be fully rejected. This PR removes those labels values when using those two parsers. For other parser you can just decide to not extract thoses values.( At least that's my choice for now). Unfortunatelly this comes with a small cpu overhead prices: ``` ~/go/src/github.com/grafana/loki utf8-parser* ❯ benchcmp before.txt after.txt benchmark old ns/op new ns/op delta Benchmark_Parser/json/no_labels_hints-16 3647 4210 +15.44% Benchmark_Parser/json/labels_hints-16 2210 2247 +1.67% benchmark old allocs new allocs delta Benchmark_Parser/json/no_labels_hints-16 72 72 +0.00% Benchmark_Parser/json/labels_hints-16 35 35 +0.00% benchmark old bytes new bytes delta Benchmark_Parser/json/no_labels_hints-16 1121 1121 +0.00% Benchmark_Parser/json/labels_hints-16 472 472 +0.00% ``` Fixes #2859 Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago
Nick Pillitteri	b830d65bdd	Use the Cortex wrapper for getting tenant ID from a context (#3973 ) This change replaces all uses of the weaveworks/common method for getting tenant/org ID with the Cortex wrapper which performs some extra validation. Signed-off-by: Nick Pillitteri <nick.pillitteri@grafana.com>	5 years ago
Cyril Tovena	bee456737e	Shards Series API. (#3856 ) * Shards Series API. This PR introduces Series Queries Sharding. It does not check the boundaries of ingesters data since I'm assuming https://github.com/grafana/loki/pull/3852 will be merge first. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes tests sorting. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago
Danny Kopping	71889dcb19	Querier/Ingester: Fixing json expression parser bug (#3928 ) * Fixing json expression parser bug Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * UNRELATED CHANGE: pulling main to fix linter issue Signed-off-by: Danny Kopping <danny.kopping@grafana.com>	5 years ago
Cyril Tovena	bdee4a3741	Logs ingester and store queries boundaries. (#3849 ) example: ``` ts=2021-06-14T08:43:30.015895283Z caller=spanlogger.go:87 org_id=4336 traceID=0b6bb74a53efd739 level=debug msg="querying ingester" params="selector={stream=\"stdout\", pod=\"loki-canary-6 64d9d8d8f-99k22\"}, direction=BACKWARD, start=2021-06-14 08:17:03.037241855 +0000 UTC, end=2021-06-14 08:17:23.037241855 +0000 UTC, limit=1000, shards=" ts=2021-06-14T08:43:30.016305592Z caller=spanlogger.go:87 org_id=4336 traceID=0b6bb74a53efd739 level=debug msg="querying store" params="selector={stream=\"stdout\", pod=\"loki-canary-664d 9d8d8f-99k22\"}, direction=BACKWARD, start=2021-06-14 08:17:03.037241855 +0000 UTC, end=2021-06-14 08:17:23.037241855 +0000 UTC, limit=1000, shards=" ``` Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago
Cyril Tovena	59bb6d3fba	LogQL: Pattern Parser (#3837 ) * The beginning of a fun story. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Working on adding ragel. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Adding AST parsing with Yacc and Ragel. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Got a pattern parser working. Reworking ast to works with bytes and not runes. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Setup tests and the matches algorithm. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * moar tests case. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add some validation for the pattern expression. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Hooking to LogQL + performance boost. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Adds documentation Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Improve bound check. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Removes generated files from being linted. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Update docs/sources/logql/_index.md Co-authored-by: Danny Kopping <dannykopping@gmail.com> * Update docs/sources/logql/_index.md Co-authored-by: Danny Kopping <dannykopping@gmail.com> * Review feedback Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Update docs/sources/logql/_index.md Co-authored-by: Danny Kopping <dannykopping@gmail.com> * Update docs/sources/logql/_index.md Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> * Update docs/sources/logql/_index.md Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> * Update docs/sources/logql/_index.md Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> * Update docs/sources/logql/_index.md Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> * Update docs/sources/logql/_index.md Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> * Update docs/sources/logql/_index.md Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> * Update docs/sources/logql/_index.md Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> * Update docs/sources/logql/_index.md Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> * Update docs/sources/logql/_index.md Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> * Update docs/sources/logql/_index.md Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> * Docs suggestions Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> Co-authored-by: Danny Kopping <dannykopping@gmail.com> Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com>	5 years ago
Cyril Tovena	526dc4bbe7	Fixes parser labels hint for grouping. (#3763 ) * Fixes parser labels hint for grouping. This PR include those now. Fixes #3707 Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes array allocations. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago
Cyril Tovena	e5a3afedff	Add fromJson to the template stage. (#3736 ) Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago
Cyril Tovena	806d6a503e	Custom Retention (#3642 ) * Playing around with a POC based on the design doc. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Getting series ID for label matchers is now working. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes tests. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * wip/ Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * We're parsing the index label now. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Starting to extract interfaces to make the code testable. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Work in progress will try to add labels to chunk ref iterator Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Iterator for chunks ref with all labels !!!. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Optimize code away. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * with delete into the mix Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Cleaner but not yet working for v10 and v11. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes series cleaner. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * hooking into the compactor. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Hooking limit retention config. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Finishing off the marker processor. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Working on sweeper and fixing tests. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Adding more tests and founding more bugs along the way. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Bug with path once boltdb is closed. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixing more bug and more robust test. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * More test and cleanup getting close. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes moar bugs with regards to period schema Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fix a flaky tests because of boltdb still open. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add more metrics. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Adding metrics. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Improve benchmark. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes issue. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * lint code. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * more logs. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Save files without using table key Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Improve logging and ability to use more goroutines. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Removes duplicate metrics since histogram contains total too. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add more logs. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes a deadlock bug when too many workers are trying to update the mark file. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes a deadlock when reading and updating db at the same time. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes default config test of boltdb. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * PR Review feedbacks. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Remove the user of assert to not fail a test if it's incorrect on the first shot. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add experimental notice to the flag documentation Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes empty index detection and table deletion. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Do not delete folder it's not necessary with object store. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Better working path cleanup Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * got linted. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago
Ed Welch	6e67d1ea5d	Loki/Promtail: Client Refactor (#3623 ) * move docker driver * move fluent-bit output plugin * move fluentd plugin * move logstash plugin * move promtail * match stage uses simple logql label matcher and filter expressions only * remove remaining logql imports * remove helpers package and move contents to util package * move validation and runtime out of util package * fix imports * move build and cfg packages into util * move ParseRequest function to util package * create a new package for logql models and move marshal/unmarshal code to util folder * move ErrorLabel to logqlmodel * move stats package to logqlmodel * decouple loghttp from logql by moving validation to logql package * move gcplog to tools * move PackedEntryKey constant to logqlmodel * fix stats proto import and rebuild protos * fix windows promtail tests * update drone to new dockerfile locations	5 years ago
Owen Diehl	d9637a6654	locks trailers during iteration (#3595 )	5 years ago
Cyril Tovena	4d1da2edf3	first_over_time and last_over_time (#3050 ) * Wip Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * [POC] last_over_time and first_over_time. This is an attempt to allow users to select the first or last discrete sample values from a range vector. This way we can create gauge, the only drawback being the log need to exists at least in every range. May be we could transfer the last or first value to the next range, but it will still be a problem for small ranges. Still wondering how useful this is compare to min/max_over_time. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago
Cyril Tovena	eff4ff1231	Improve matchers validations. (#3564 ) 1- It currently 500. 2- Happens too late, at ingester level and querier. (better at the frontend) 3- Improve API surface level, we don't need to expose this validation for anything else than ParseLogSelector. see #3216 Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago
Sandeep Sukhani	b152bc2cf3	check for stream selectors to have atleast one equality matcher (#3216 ) * check for stream selectors to have atleast one equality matcher * move equality matchers check to logql	5 years ago
Cyril Tovena	a76a17877b	Add a target to reproduce fuzz testcase (#3553 ) * Add a make target to facilitate fuzz testing locally. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * missing target.... Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago
Cyril Tovena	b11d2effe2	Fixes a bug in MatrixStepper when sharding queries. (#3550 ) * Fixes a bug in MatrixStepper when sharding queries. Since we split correctly metric queries, this bug has show itself. Basically we were not correctly stepping through time. We should always start from start, add the step until the start is after the end. For more read: https://www.robustperception.io/step-and-query_range Fixes #3541 Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes tests. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes tests. for real. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Not easy to get those test working. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago
Lars Lehtonen	b5930d6d9d	single import of jsoniter in logql subpackages (#3544 )	5 years ago
Garrett	f70c7ea0bc	Implement offset modifier for range vector aggregation in LogQL (#3455 ) * Implement offset modifier for range vector aggregation in LogQL implementation for https://github.com/grafana/loki/issues/2785, the offset modifier allows changing the time offset for range vectors in a query to support e.g. selective timeshift in Grafana Signed-off-by: garrettlish <garrett.li.sh@gmail.com> * add docs for offset modifier * refine offset expr * add offset unit test in engine.go * remove duplicated offset shift * add offset test in range vector test Co-authored-by: garrettlish <garrettlish@163.com>	5 years ago
Cyril Tovena	a7e0770acb	Fixes a race introduced by #3434 . (#3517 )	5 years ago
Cyril Tovena	6148794b9b	Fixes a bug where unpack would mutate log line. (#3502 ) * Fixes a bug where unpack would mutate log line. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * add reset. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago
Aditya C S	b1ea8ac050	support math functions in line_format and label_format (#3434 ) * support math functions in line_format and label_format * fix lint and tests * added docs and more tests * update doc * remove toString and int64 * doc adjustement. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> Co-authored-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago
Cyril Tovena	d6ecfd2431	Set the byte slice cap correctly when unsafely converting string. (#3487 ) Fixes #3485 Should also unblock #3481 When unsafely converting string to bytes the cap of the bytes is lost but not when doing bytes to string. It seems that this is only a problem when using go 1.16 or when using delve. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago
Cyril Tovena	64decdbcb6	Set a max size for the logql parser to 5k. (#3471 ) * Set a max size for the logql parser to 5k. The API anyway receives the query via querystring which is normally max out by 2k based on RFC2616. This will improve fuzzing too. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * better error. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago
Cyril Tovena	c49e807a55	Fixes Issue 28593: loki:fuzz_parse_expr: Timeout in fuzz_parse_expr. (#3470 ) This was caused by a lot of allocations when parsing range duration. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago
Cyril Tovena	e29d53631e	Fixes out-of-memory fuzzing issue. (#3469 ) * Fixes out-of-memory fuzzing issue. Fixes Issue 28535 in oss-fuzz: loki:fuzz_parse_expr: Out-of-memory in fuzz_parse_expr Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add test case. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago
Cyril Tovena	a29901d33a	Add limit and line_returned in the query log. (#3423 ) * Add limit and line_returned in the query log. This can be useful to investigate how many wasted queries have been made by the frontend. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * got linted. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago
Cyril Tovena	ce4f78515f	Improve the parser hint tests. (#3429 ) I was able to find out where was the issue with that test yesterday. Now I realized that we can mutate (re-slice) the given log line, and so other test would fail. This also make the parallel back. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago
Ed Welch	6d48fb6942	change ReadStringAsSlice to ReadString so it doesn't parse quotes inside the packed _entry (#3432 )	5 years ago
Cyril Tovena	32d668ceb0	Only unpack entry if the key `_entry` exist. (#3426 ) * Only unpack entry if the key `_entry` exist. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * s/kv/lbs/g Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * fix the test. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago
Cyril Tovena	7886f35b8f	Introduce a unpack parser. (#3420 ) * Introduce a unpack parser. To go with the new pack stage in promtail https://github.com/grafana/loki/pull/3401 this PR introduces a unpack parser that will automatically unpack all labels packed into the log line and set back the original log line. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add documentation. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * tweak promtail docs a little now that we have unpack in Loki Co-authored-by: Edward Welch <edward.welch@grafana.com>	5 years ago
Ed Welch	74a97899dd	Loki: Fix parser hint for extracted labels which collide with stream labels (#3380 ) * add test for issue with hints and extracted labels * one possible solution * instead of replacing the label with the _extracted suffix removed, just add a new label with the suffix removed. This will avoid any cases where a legit json or logfmt key ends with _extracted.	5 years ago
Cyril Tovena	e21d6edc10	Fixes 500 in the querier when returning multiple errors. (#3348 ) This correctly implement the `Is` error interface and uses the `util.MultiError` everywhere we need it. This way we can correcltly access if this was a cancellation which should not translate into 500. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago
Cyril Tovena	f10535f7c1	Fixes a panic with whitespace key. (#3372 ) * Fixes a panic with whitespace key. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Removes empty key. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago
Cyril Tovena	c4d3f72191	Fixes tightloop that could happen in the engine with LogQL parser. (#3347 ) I've also added a test to prove the issue was happening and now is fixed. Fixes #3275 Fixes #3264 Fixes #3020 Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago
Danny Kopping	feb7fb470b	LogQL: Simple JSON expressions (#3280 ) * New approach, still rough Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Adding benchmark Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Adding tests Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Minor refactoring Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Appeasing the linter Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Further appeasing the linter Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Adding more tests Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Adding documentation Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Docs fixup Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Removing unnecessary condition Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Adding extra tests from suggestion in review Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Adding JSONParseErr Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Adding test to cover invalid JSON line Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Adding equivalent benchmarks for JSON and JSONExpression parsing Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Adding suffix if label would be overridden Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Reparenting jsonexpr directory to more appropriate location Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Setting empty label on non-matching expression, to retain parity with label_format Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Adding statement about returned complex JSON types Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Added check for valid label name Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Making json expressions shardable Signed-off-by: Danny Kopping <danny.kopping@grafana.com>	5 years ago
Owen Diehl	5f0c06ac3a	Logql/absent label optimization (#3299 ) * absent_over_time does not extract unnecessary labels * rebase fix	5 years ago
Jacob Lisi	923671a17e	chore: update cortex to latest and fix refs (#3295 ) * chore: update cortex to latest and fix refs Signed-off-by: Jacob Lisi <jacob.t.lisi@gmail.com> * tidy mod file Signed-off-by: Jacob Lisi <jacob.t.lisi@gmail.com>	5 years ago
Cyril Tovena	3bd725e826	Fixes logfmt parser hints. (#3279 ) Recently introduce a bug where label where tested for extraction against non sanitized version. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago
Owen Diehl	4d1c6e0169	adds another test (#3278 )	5 years ago
Cyril Tovena	e8cce09de2	Fix a bug with metric queries and label_format. (#3263 ) * Fix a bug whith metric queries and label_format. In my previous PR where I introduced hints, I forgot to implement label format label required. This would cause label format label to be skipped during parsing. I've also took the opportunity to improve the code. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Refactor and add more tests. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago
Jacob Lisi	c3d1937eab	update vendored cortex and add new replace overrides (#3256 ) * update vendored cortex and add new replace overrides Signed-off-by: Jacob Lisi <jacob.t.lisi@gmail.com> * update upstream function to new package Signed-off-by: Jacob Lisi <jacob.t.lisi@gmail.com> * update all moved functions Signed-off-by: Jacob Lisi <jacob.t.lisi@gmail.com> * update math to util_math Signed-off-by: Jacob Lisi <jacob.t.lisi@gmail.com>	5 years ago
Cyril Tovena	1a36874e1c	Ensure we have parenthetis for bin ops. (#3251 ) This ensure we correctly keep the order of execution when we stringify a bin operation. This has recently introduced a bug because we now use stringify to deep clone a LogQL expression. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago
Cyril Tovena	3e46067cbb	Fixes a bug where slice of Entries where not zeroed before decoding json. (#3249 ) Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago
Cyril Tovena	a5cb38d54c	Fixes unmarshalling of tailing responses. (#3237 ) * Fixes unmarshalling of tailing responses. Some websocket were still using old json package. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes missing update of generated files. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago
Cyril Tovena	fd619b6931	Fixes tail api marshalling for v1. (#3211 ) Introduce a bug by removing the default marshalling (#3163) but the tail api was using the default json. This fixes it by forcing the usage of jsoniter package. Added a missing test so that it doesn't happen again. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago
Cyril Tovena	5eee25b560	Fixes unwrap expressions from last optimization. (#3194 ) Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago
Cyril Tovena	070246c0a4	Uses custom json-iter decoder for log entries. (#3163 ) * Uses custom json-iter decoder for log entries. Previously we were using json.Unmarshal for each line. However json-iter uses a Pool for each calls and I believe this can cause to increase memory usage. For each line we would put in a pool the iterator to re-use it, once put in a pool, the last data is retained, since we handle millions of lines, this can cause problem, using a custom extensions, keep using a pool but at the root object only, not for each line. On top of that we're going to process that json payload 50% faster. ``` ❯ benchcmp before.txt after.txt2 benchmark old ns/op new ns/op delta Benchmark_DecodePushRequest-16 13509236 6677037 -50.57% benchmark old allocs new allocs delta Benchmark_DecodePushRequest-16 106149 38719 -63.52% benchmark old bytes new bytes delta Benchmark_DecodePushRequest-16 10350362 5222989 -49.54% ``` Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes query-tee tests. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago

1 2 3 4 5 ...

275 Commits (6e2b7d25898f321bfd6f94de05d6ce934c785da9)