135 Commits (852becf227ee4c739ce68b77ab19ef7e26ffab60)

Author	SHA1	Message	Date
Sandeep Sukhani	0e5da205a6	structured metadata: track usage of structured metadata in queries (#11505) ... **What this PR does / why we need it**: Adds support for tracking queries referencing structured metadata. It tracks the following kinds of references: * Fields in structured metadata used as label filters, `drop`/`keep` stages etc. in the query. * Fields in structured metadata used in metric queries like `uwrap` and `by` or `without` clauses in aggregations. **Checklist** - [x] Tests updated	1 year ago
Travis Patterson	f668a1b80b	Add context and tenant to pipeline/extractor wrappers (#11528) ... Adding the context here means the wrapping pipeline can know when the query has stopped or been cancelled.	1 year ago
Callum Styan	48c59ba96c	remove mistakenly committed files from 11484 (#11527) ... remove mistakenly committed files (benchmarking related) from #11484 Signed-off-by: Callum Styan <callumstyan@gmail.com>	1 year ago
Karsten Jeschkies	27d21067e1	Clone filterer of label filter (#11514)	1 year ago
Callum Styan	e93243ff5b	reduce cost of pipeline stages by pooling maps we create for labels (#11484) ... If we look at profiling of ingesters in some of our production deployments during which we have high ingester load from querying we see that we spend roughly 15-20% of our CPU time on GC related work. Looking at different `runtime` memory related calls, it looks like ~50% of all CPU time during Pipeline stage processing is the creation of the map for labels that is then passed to text template parsing. This change will make the pipeline processing itself ~15% faster but hopefully will also reduce (very slightly) the impact of GC in heavy query load situations. ``` goos: linux goarch: amd64 pkg: github.com/grafana/loki/pkg/logql/log cpu: AMD Ryzen 9 5950X 16-Core Processor │ base │ mapPool │ │ sec/op │ sec/op vs base │ _Pipeline/pipeline_bytes-32 7.749µ ± ∞ ¹ 6.464µ ± ∞ ¹ ~ (p=1.000 n=1) ² _Pipeline/pipeline_string-32 7.691µ ± ∞ ¹ 6.549µ ± ∞ ¹ ~ (p=1.000 n=1) ² _Pipeline/line_extractor_bytes-32 7.922µ ± ∞ ¹ 6.882µ ± ∞ ¹ ~ (p=1.000 n=1) ² _Pipeline/line_extractor_string-32 7.900µ ± ∞ ¹ 6.878µ ± ∞ ¹ ~ (p=1.000 n=1) ² _Pipeline/label_extractor_bytes-32 7.952µ ± ∞ ¹ 6.976µ ± ∞ ¹ ~ (p=1.000 n=1) ² _Pipeline/label_extractor_string-32 7.940µ ± ∞ ¹ 7.102µ ± ∞ ¹ ~ (p=1.000 n=1) ² geomean 7.858µ 6.805µ -13.41% ¹ need >= 6 samples for confidence interval at level 0.95 ² need >= 4 samples to detect a difference at alpha level 0.05 │ base │ mapPool │ │ B/op │ B/op vs base │ _Pipeline/pipeline_bytes-32 4.963Ki ± ∞ ¹ 1.413Ki ± ∞ ¹ ~ (p=1.000 n=1) ² _Pipeline/pipeline_string-32 5.025Ki ± ∞ ¹ 1.476Ki ± ∞ ¹ ~ (p=1.000 n=1) ² _Pipeline/line_extractor_bytes-32 5.030Ki ± ∞ ¹ 1.478Ki ± ∞ ¹ ~ (p=1.000 n=1) ² _Pipeline/line_extractor_string-32 5.028Ki ± ∞ ¹ 1.478Ki ± ∞ ¹ ~ (p=1.000 n=1) ² _Pipeline/label_extractor_bytes-32 5.027Ki ± ∞ ¹ 1.478Ki ± ∞ ¹ ~ (p=1.000 n=1) ² _Pipeline/label_extractor_string-32 5.027Ki ± ∞ ¹ 1.478Ki ± ∞ ¹ ~ (p=1.000 n=1) ² geomean 5.017Ki 1.466Ki -70.77% ¹ need >= 6 samples for confidence interval at level 0.95 ² need >= 4 samples to detect a difference at alpha level 0.05 │ base │ mapPool │ │ allocs/op │ allocs/op vs base │ _Pipeline/pipeline_bytes-32 37.00 ± ∞ ¹ 35.00 ± ∞ ¹ ~ (p=1.000 n=1) ² _Pipeline/pipeline_string-32 38.00 ± ∞ ¹ 36.00 ± ∞ ¹ ~ (p=1.000 n=1) ² _Pipeline/line_extractor_bytes-32 37.00 ± ∞ ¹ 35.00 ± ∞ ¹ ~ (p=1.000 n=1) ² _Pipeline/line_extractor_string-32 37.00 ± ∞ ¹ 35.00 ± ∞ ¹ ~ (p=1.000 n=1) ² _Pipeline/label_extractor_bytes-32 37.00 ± ∞ ¹ 35.00 ± ∞ ¹ ~ (p=1.000 n=1) ² _Pipeline/label_extractor_string-32 37.00 ± ∞ ¹ 35.00 ± ∞ ¹ ~ (p=1.000 n=1) ² geomean 37.16 35.16 -5.38% ¹ need >= 6 samples for confidence interval at level 0.95 ² need >= 4 samples to detect a difference at alpha level 0.05 ``` --------- Signed-off-by: Callum Styan <callumstyan@gmail.com>	1 year ago
Travis Patterson	be71a80b15	Implement hooks to instrument query pipelines (#11493) ... This PR implements interfaces to inject new pipelines/extractors into the query path of queriers and ingesters.	1 year ago
Karsten Jeschkies	3a7b5d246b	Serialize query AST to JSON. (#11123) ... **What this PR does / why we need it**: This introduces the visitor pattern to serialize the LogQL AST to JSON. We've chose this pattern because it will be more flexible one the AST is encoded into Protobuf and an actual queryplan. **Checklist** - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [ ] Documentation added - [x] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](d10549e3ec) - [ ] If the change is deprecating or removing a configuration option, update the `deprecated-config.yaml` and `deleted-config.yaml` files respectively in the `tools/deprecated-config-checker` directory. [Example PR](0d4416a4b0)	2 years ago
Salva Corts	52a3f16039	Flag categorize labels on streams response (#10419) ... We recently introduced support for ingesting and querying structured metadata in Loki. This adds a new dimension to Loki's labels since now we arguably have three categories of labels: _stream_, _structured metadata_, and _parsed_ labels. Depending on the origin of the labels, they should be used in LogQL expressions differently to achieve optimal performance. _stream_ labels should be added to stream matchers, _structured metadata_ labels should be used in a filter expression before any parsing expression, and _parsed_ labels should be placed after the parser expression extracting them. The Grafana UI has a hard time dealing with this same problem. Before https://github.com/grafana/grafana/pull/73955, the filtering functionality in Grafana was broken since it was not able to distinguish between _stream_ and _structured metadata_ labels. Also, as soon as a parser expression was added to the query, filters added by Grafana would be appended to the end of the query regardless of the label category. The PR above implements a workaround for this problem but needs a better API on Loki's end to mitigate all corner cases. Loki currently returns the following JSON for log queries: ```json ... { "stream": { "cluster": "us-central", "container": "query-frontend", "namespace": "loki", "level": "info", "traceID": "68810cf0c94bfcca" }, "values": [ [ "1693996529000222496", "1693996529000222496 aaaaaaaaa.....\n" ], ... }, { "stream": { "cluster": "us-central", "container": "query-frontend", "namespace": "loki", "level": "debug", "traceID": "a7116cj54c4bjz8s" }, "values": [ [ "1693996529000222497", "1693996529000222497 bbbbbbbbb.....\n" ], ... }, ... ``` As can be seen, there is no way we can distinguish the category of each label. This PR introduces a new flag `X-Loki-Response-Encoding-Flags: categorize-labels` that makes Loki return categorized labels as follows: ```json ... { "stream": { "cluster": "us-central", "container": "query-frontend", "namespace": "loki", }, "values": [ [ "1693996529000222496", "1693996529000222496 aaaaaaaaa.....\n", { "structuredMetadata": { "traceID": "68810cf0c94bfcca" }, "parsed": { "level": "info" } } ], [ "1693996529000222497", "1693996529000222497 bbbbbbbbb.....\n", { "structuredMetadata": { "traceID": "a7116cj54c4bjz8s" }, "parsed": { "level": "debug" } } ], ... }, ... ``` Note that this PR only supports log queries, not metric queries. From a UX perspective, being able to categorize labels in metric queries doesn't have any benefit yet. Having said that, supporting this for metric queries would require some minor refactoring on top of what has been implemented here. If we decide to do that, I think we should do it on a separate PR to avoid making this PR even larger. I also decided to leave out support for Tail queries to avoid making this PR even larger. Once this one gets merged, we can work to support tailing. --- **Note to reviewers** This PR is huge since we need to forward categorized all over the codebase (from parsing logs all the way to marshaling), fortunately, many of the changes come from updating tests and refactoring iterators. Tested out in a dev cell with query `'{stream="stdout"} | label_format new="text"`. - Without the new flag: ``` $ http http://127.0.0.1:3100/loki/api/v1/query_range\?direction\=BACKWARD\&end\=1693996529322486000\&limit\=30\&query\=%7Bstream%3D%22stdout%22%7D+%7C+label_format+new%3D%22text%22\&start\=1693992929322486000 X-Scope-Orgid:REDACTED { "data": { "result": [ { "stream": { "new": "text", "pod": "loki-canary-986bd6f4b-xqmb7", "stream": "stdout" }, "values": [ [ "1693996529000222496", "1693996529000222496 pppppppppppp...\n" ], [ "1693996528499160852", "1693996528499160852 pppppppppppp...\n" ], ... ``` - With the new flag ``` $ http http://127.0.0.1:3100/loki/api/v1/query_range\?direction\=BACKWARD\&end\=1693996529322486000\&limit\=30\&query\=%7Bstream%3D%22stdout%22%7D+%7C+label_format+new%3D%22text%22\&start\=1693992929322486000 X-Scope-Orgid:REDACTED X-Loki-Response-Encoding-Flags:categorize-labels { "data": { "encodingFlags": [ "categorize-labels" ], "result": [ { "stream": { "pod": "loki-canary-986bd6f4b-xqmb7", "stream": "stdout" }, "values": [ [ "1693996529000222496", "1693996529000222496 pppppppppppp...\n", { "parsed": { "new": "text" } } ], [ "1693996528499160852", "1693996528499160852 pppppppppppp...\n", { "parsed": { "new": "text" } } ], ... ```	2 years ago
Travis Patterson	cc1b0dbc24	Don't use unsafe string for label names in the UnpackParser (#10487) ... We've had [some reports](https://github.com/grafana/loki/issues/8936) that `unpack` is mangling label names. We're using `unsafeGetString` to generate these so, when the underlying byte slice changes, the label name is also changed. This change incurs an alloc to make sure the generated string wont change when the underlying bytes do. fixes #8936	2 years ago
Vladyslav Diachenko	00a3c5b026	renamed `non indexed labels` feature to `structured metadata` (#10432) ... Loki team has made a decision to rename the feature because the new name is more precise. --------- Signed-off-by: Vladyslav Diachenko <vlad.diachenko@grafana.com> Co-authored-by: Salva Corts <salva.corts@grafana.com>	2 years ago
Travis Patterson	60edfe0006	Implement "or" statement in line filters (#8962) ... It might be nice to not force a user to use `|~` when they need to express `or` in LogQL. Example: `{app="foo"} |= "foo" or "bar"` Edit: A ton of this is generated code or refactors. I've highlighed the parts most interesting to a reviewer --------- Co-authored-by: Karsten Jeschkies <karsten.jeschkies@grafana.com>	2 years ago
Kaviraj Kanagaraj	c030217e1b	Revert "LogQL: Introduce `distinct` (#8662)" (#10356) ... **What this PR does / why we need it**: This reverts commit 38b298c3fe. We are removing this PR #8662 because [known issue tracked here](https://github.com/grafana/loki/issues/9594) and some issues we facing internally with query splitting and sharding work(hard to parallelize, merging two `distinct` sets can have duplicates, hard to make it work with metric queries, etc). We decided best to remove it before make it to public release. **Which issue(s) this PR fixes**: Fixes #<issue number> **Special notes for your reviewer**: **Checklist** - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [x] Documentation added - [x] Tests updated - [x] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](d10549e3ec) --------- Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com>	2 years ago
Sandeep Sukhani	962e03932d	non-indexed labels: support for deletion of logs with non-indexed labels (#10337) ... **What this PR does / why we need it**: We recently added support for storing and querying non-indexed labels with each log line. This PR wires up the code for supporting deletion using non-indexed labels. **Checklist** - [x] Tests updated	2 years ago
Travis Patterson	12221f43b5	Put unescaped regexes in a raw string literal (#10331) ... [This PR](https://github.com/grafana/loki/pull/10277) fixed an issue where regexes were being escaped multiple times and causing otherwise valid queries to fail our length check. It put the resulting regexes in double quotes which meant that any regexes that _needed_ escaping were now invalid. This PR make is so the unescaped regex is put in a raw string literal so any extra escaping in unnecessary.	2 years ago
Kaviraj Kanagaraj	416320dc5f	Revert "Stopped String() from doubly escaping regex characters" (#10318) ... Reverts grafana/loki#10277 This currently breaks and cause panics for query with regex label matcher. Reverting it till we find the fix.	2 years ago
akhilanarayanan	4013c1f8b5	Stopped String() from doubly escaping regex characters (#10277) ... **What this PR does / why we need it**: When queries have regex, characters that needed escaping were escaped twice. Now they're only singly escaped. This is good because it reduces the character count so users can submit queries closer to the 5120 byte limit. **Which issue(s) this PR fixes**: Sort of addresses #10153 **Special notes for your reviewer**: **Checklist** - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [ ] Documentation added - [x] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](d10549e3ec) Co-authored-by: Akhila Narayanan <akhilanarayanan@Akhilas-MacBook-Air.local>	2 years ago
Salva Corts	cbb272da00	Add `_extracted` to non-indexed labels with same name as stream labels (#10082) ... **What this PR does / why we need it**: In https://github.com/grafana/loki/pull/9702 we added support for returning non-indexed labels in the labels results. The problem is that non-indexed labels may overwrite stream labels if both are named the same way. This PR fixes this by adding an `_extracted` suffix if the non-indexed label is already present in the stream labels. **Special notes for your reviewer**: **Checklist** - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [ ] Documentation added - [x] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](d10549e3ec)	2 years ago
Salva Corts	a0e9f1aeb5	Make naming consistent for non-Indexed labels (#10024) ... **What this PR does / why we need it**: We started using _metadata_ to refer to _non-indexed labels_. This PR replaces all the _metadata_ usages and renames them to _non-indexed labels_. **Checklist** - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](d10549e3ec)	2 years ago
Salva Corts	1d04cd5e31	Metadata to labels result and filtering support (#9702) ... **What this PR does / why we need it**: In #9700, we support encoding and decoding metadata for each entry into the chunks. This PR adds support for returning metadata labels for matching entries in a query to the returned LabelResults. It also supports filtering out logs by metadata labels. **Special notes for your reviewer**: **Checklist** - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [ ] Documentation added - [x] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](d10549e3ec) --------- Co-authored-by: Sandeep Sukhani <sandeep.d.sukhani@gmail.com>	2 years ago
Travis Patterson	23e59fa4a9	reset pipeline caches when tailing rather than making a new pipeline (#9949) ... Recent query optimizations made pipeline creation more expensive with the assumption that it occurred relatively infrequently. To avoid growing pipline caches without bound, integers recreate pipelines on every push. The combination of these things means that tailing high-volume streams can cause ingester failure. This fix adds the ability to clear pipeline caches so we don't need to create a new one each time.	2 years ago
Ashwanth	f9fc1df4c0	logfmt: add --strict flag to enable strict parsing, perform nostrict parsing by default (#9626) ... **What this PR does / why we need it**: Introduces `--strict` flag for logfmt parser to enable strict parsing. With strict parsing enabled, logfmt parser stops scanning the log line and returns early with an error if it encounters any poorly formatted key=value pair. This pr updates the parser to perform non-strict parsing by default where it skips over the invalid tokens. This offers the flexibility to parse semi-structured log lines. Also adds `--keep-empty` flag to retain standalone keys(with empty value) as labels with value set to empty string **Special notes for your reviewer**: **Checklist** - [X] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [X] Documentation added - [X] Tests updated - [x] `CHANGELOG.md` updated - [x] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` --------- Co-authored-by: Michel Hollands <42814411+MichelHollands@users.noreply.github.com>	2 years ago
Salva Corts	61d407b4bb	Add keep stage to LogQL (#9693) ... **What this PR does / why we need it**: This PR adds a new `keep` stage to LogQL. It allows filtering specific labels and helps to avoid hitting the _max streams per query_ limit by keeping only specific labels in the label set. It works the opposite of [drop stage](https://grafana.com/docs/loki/latest/logql/log_queries/#drop-labels-expression). **Special notes for your reviewer**: We do not drop [special error labels](edc6b0bff7/pkg/logqlmodel/error.go (L18-L20)) added by Loki at query time. Users will have to explicitly drop them by using the drop stage. **Checklist** - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [x] Documentation added - [x] Tests updated - [x] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](d10549e3ec) --------- Co-authored-by: Sandeep Sukhani <sandeep.d.sukhani@gmail.com> Co-authored-by: J Stickler <julie.stickler@grafana.com>	2 years ago
Michel Hollands	79485cd30d	Add handling of timezones in unixToTime tests (#9707) ... **What this PR does / why we need it**: The unit tests for unixToTime do not take the current timezone into account. **Checklist** - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [ ] Documentation added - [x] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](d10549e3ec) --------- Signed-off-by: Michel Hollands <michel.hollands@grafana.com>	2 years ago
Trevor Whitney	4a56445686	Upgrade `golangci-lint` and fix linting errors (#9601) ... **What this PR does / why we need it**: Upgrade `golangci-lint` and fixes all the errors. The upgrade includes some stricter linting.	2 years ago
Travis Patterson	065bee7e72	Label Volume Endpoint (#9588) ... For a given set of matchers, returns the top N associated label/value pairs by volume. A query for `{cluster=prod}` will return ``` cluster=prod: size (total logs matching this matcher) . . . nth-label=nth-value ``` This is to service use cases where users want to understand where their log volume has come from by label without making multiple requests to the stats endpoint. Note: This PR is a monster but it's mostly plumbing. I've pointed out the most interesting bits that actually get the volumes from ingesters/indexs	2 years ago
Michel Hollands	22779e13c5	Fix date template function with epoch times (#8886) ... **What this PR does / why we need it**: Adds new toUnixEpoch... functions to convert from a string with a Unix/Epoch time to an integer that can be used in the existing `toDate` function. Note that these are the opposites of some of the functions introduced in https://github.com/grafana/loki/pull/8774. **Which issue(s) this PR fixes**: Fixes https://github.com/grafana/loki/issues/8624. **Special notes for your reviewer**: **Checklist** - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [X] Documentation added - [X] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` --------- Signed-off-by: Michel Hollands <michel.hollands@grafana.com>	2 years ago
Gregor Zeitlinger	b1917a6178	add "alignLeft" and "alignRight" functions (#9672) ... Fixes https://github.com/grafana/loki/issues/9667	2 years ago
Travis Patterson	fd0efe0799	Regex line/label filters return original regex when `String` is called (#9584) ... To optimize regex line and label filters, Loki rewrites them to be non-greedy. This can cause extra characters to appear in the regex when `String()` is called. In addition to confusing logging, this can also cause internal panics when a previously-parsable query becomes too long. This PR stores the original regex in the label/line filter and uses that in calls to `String`. This PR also removes regex optimizations from label selectors. Making the optimization at parse time caused the original regex to be lost.	2 years ago
Bryan Boreham	ee74fbdf3a	logql/log tests: use abstractions over labels (#9532) ... **What this PR does / why we need it**: Call functions like `FromStrings`, so we don't have to know how the data structure is laid out. `FromStrings` sorts labels, so we don't have to call `Sort()`. **Special notes for your reviewer**: This PR only touches tests. Upstream Prometheus is using a different implementation of `labels.Labels`, so I plan to create a series of PRs which make Loki compatible with that. **Checklist** - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - NA Documentation added - [x] Tests updated - NA `CHANGELOG.md` updated - NA Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - NA For Helm chart changes bump the Helm chart version. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2 years ago
Nick Moore	2f87acd284	Fix case insensitive label filter (#9404) ... **What this PR does / why we need it**: When comparing a case insensitive regular expression to a label value, Loki was only matching values that were in upper case. This was caused by compiling a case insensitive regular expression (such as `(?i)hello`), which produced an upper case `prefix` that was then used in `containsLower` as the `substr` value - but that function was written in such a way that only allowed matches of exact case or where the `line` value was upper case and the `substr` value was lower case. This PR added a some additional test cases to demonstrate/understand the issue, and adds two additional checks to the `containsLower` function to enable it to work correctly when either input is in either case. **Which issue(s) this PR fixes**: Fixes #9294 **Special notes for your reviewer**: Given the importance of this code path, I used `BenchmarkLineLabelFilters` to validate that the performance of this code was not affected. Overall, [the differences appeared negligible](https://docs.google.com/spreadsheets/d/1BRKkCzO0i11AObC6ADguDNa7cfG93hB7JLtqYWmnfW0/edit?usp=sharing) on my device (Intel(R) Core(TM) i7-12700H). **Checklist** - [X] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [ ] Documentation added - [X] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	2 years ago
Travis Patterson	48ad068ce1	fix json expression parser (#9410) ... When the value parsed by the `JsonExpressionParser` is an escaped json string, the result still contains escape character and may not be valid json. This causes subsequent json parsing steps to fail. For example: `{app="foo"} | json message | line_format "{{.message}}" | json` fails when `message` remains escaped. This PR unescapes the results of extracted `JsonExpressions` so they can be parsed by subsequent json parsers	2 years ago
李国忠	38b298c3fe	LogQL: Introduce `distinct` (#8662) ... **What this PR does / why we need it**: Introduces `distinct` to LogQL. Usage: `{job="varlogs"} | distinct filename` It is similar to `distinct` in SQL and especially useful for sampling. Similar syntax exists in other log query languages. **Example** For the following log lines: ```json {"event": "access", "id": "1", "time": "2023-02-28 15:12:11"} {"event": "access", "id": "1", "time": "2023-02-28 15:13:11"} {"event": "access", "id": "2", "time": "2023-02-28 15:14:11"} {"event": "access", "id": "2", "time": "2023-02-28 15:15:11"} ``` The query below: ```logql {app="order"} | json | distinct id ``` Will return: ```json {"event": "access", `"id": "1",` "time": "2023-02-28 15:13:11"} {"event": "access", `"id": "2", `"time": "2023-02-28 15:15:11"} ``` **Example with multiple labels** ```logql {app="order"} | json | distinct id,time ``` **Which issue(s) this PR fixes**: Fixes #8649 --------- Co-authored-by: J Stickler <julie.stickler@grafana.com> Co-authored-by: Dylan Guedes <djmgguedes@gmail.com> Co-authored-by: Karsten Jeschkies <karsten.jeschkies@grafana.com>	2 years ago
Peter Štibraný	90a1d4593e	Update Prometheus dependency (#9205)	2 years ago
Travis Patterson	7735b1e98e	Sanitize label keys in the unpack parser (#9121) ... This PR makes the `unpack` parser sanitize label keys like all the other parsers. When invalid label names are included, it causes the user to get `could not write JSON response` errors when the response is serialized to send back to the user.	2 years ago
Ed Welch	edc6b0bff7	Loki: Add a limit for the [range] value on range queries (#8343) ... Signed-off-by: Edward Welch <edward.welch@grafana.com> **What this PR does / why we need it**: Loki does not currently split queries by time to a value smaller than what's in the [range] of a range query. Example ``` sum(rate({job="foo"}[2d])) ``` Imagine now this query being executed over a longer window of a few days with a step of something like 30m. Every step evaluation would query the last [2d] of data. There are use cases where this is desired, specifically if you force the step to match the value in the range, however what is more common is someone accidentally uses `[$__range]` in here instead of `[$__interval]` within Grafana and then sets the query time selector to a large value like 7 days. This PR adds a limit which will fail queries that set the [range] value higher than the configured limit. It's disabled by default. In the future it may be possible for Loki to perform splits within the [range] and remove the need for this limit, but until then this can be an important safeguard in clusters with a lot of data. **Which issue(s) this PR fixes**: Fixes #8746 **Special notes for your reviewer**: **Checklist** - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` --------- Signed-off-by: Edward Welch <edward.welch@grafana.com> Co-authored-by: Karsten Jeschkies <karsten.jeschkies@grafana.com> Co-authored-by: Vladyslav Diachenko <82767850+vlad-diachenko@users.noreply.github.com>	2 years ago
Travis Patterson	163fd9d8af	Short circuit parsing when label matchers are present (#8890) ... This PR makes parsers aware of any downstream label-matcher stages at parse time. As labels are parsed, if one has a matcher, the matcher is checked at parse time. If the label does not match it's matcher, parsing is halted on that log line. **ex 1:** consider the log: `foo=1 bar=2 baz=3` And the query `{} | logfmt | bar=3` When `bar` is parsed it is immediately checked against it's matcher. The match fails so we the parser never spends time parsing the rest of the line. **ex 2:** consider the log: `foo=1 baz=3 bletch=4` And the query `{} | logfmt | bar=3` `bar` is never seen in the log so the whole line is parsed. **Benchmarks:** ``` │ parsers__old_2.txt │ parsers__new_3.txt │ │ sec/op │ sec/op vs base │ _Parser/json/inline_stages-8 3413.5n ± 5% 766.4n ± 4% -77.55% (p=0.000 n=10) _Parser/jsonParser-not_json_line/inline_stages-8 101.5n ± 6% 103.1n ± 8% ~ (p=0.645 n=10) _Parser/unpack/inline_stages-8 383.8n ± 4% 388.0n ± 9% ~ (p=0.954 n=10) _Parser/unpack-not_json_line/inline_stages-8 13.30n ± 2% 13.11n ± 1% ~ (p=0.247 n=10) _Parser/logfmt/inline_stages-8 2105.5n ± 16% 727.7n ± 4% -65.44% (p=0.000 n=10) _Parser/regex_greedy/inline_stages-8 4.220µ ± 4% 4.175µ ± 4% ~ (p=0.739 n=10) _Parser/regex_status_digits/inline_stages-8 319.8n ± 5% 326.4n ± 8% ~ (p=0.481 n=10) _Parser/pattern/inline_stages-8 185.2n ± 7% 154.2n ± 3% -16.74% (p=0.000 n=10) │ parsers__old_2.txt │ parsers__new_3.txt │ │ B/op │ B/op vs base │ _Parser/json/inline_stages-8 280.00 ± 0% 64.00 ± 0% -77.14% (p=0.000 n=10) _Parser/jsonParser-not_json_line/inline_stages-8 16.00 ± 0% 16.00 ± 0% ~ (p=1.000 n=10) _Parser/unpack/inline_stages-8 80.00 ± 0% 80.00 ± 0% ~ (p=1.000 n=10) _Parser/unpack-not_json_line/inline_stages-8 0.000 ± 0% 0.000 ± 0% ~ (p=1.000 n=10) _Parser/logfmt/inline_stages-8 336.00 ± 0% 74.00 ± 0% -77.98% (p=0.000 n=10) _Parser/regex_greedy/inline_stages-8 193.0 ± 1% 192.0 ± 1% ~ (p=0.656 n=10) _Parser/regex_status_digits/inline_stages-8 51.00 ± 0% 51.00 ± 0% ~ (p=1.000 n=10) _Parser/pattern/inline_stages-8 35.000 ± 0% 3.000 ± 0% -91.43% (p=0.000 n=10) │ parsers__old_2.txt │ parsers__new_3.txt │ │ allocs/op │ allocs/op vs base │ _Parser/json/inline_stages-8 18.000 ± 0% 4.000 ± 0% -77.78% (p=0.000 n=10) _Parser/jsonParser-not_json_line/inline_stages-8 1.000 ± 0% 1.000 ± 0% ~ (p=1.000 n=10) _Parser/unpack/inline_stages-8 4.000 ± 0% 4.000 ± 0% ~ (p=1.000 n=10) _Parser/unpack-not_json_line/inline_stages-8 0.000 ± 0% 0.000 ± 0% ~ (p=1.000 n=10) _Parser/logfmt/inline_stages-8 16.000 ± 0% 6.000 ± 0% -62.50% (p=0.000 n=10) _Parser/regex_greedy/inline_stages-8 2.000 ± 0% 2.000 ± 0% ~ (p=1.000 n=10) _Parser/regex_status_digits/inline_stages-8 2.000 ± 0% 2.000 ± 0% ~ (p=1.000 n=10) _Parser/pattern/inline_stages-8 2.000 ± 0% 1.000 ± 0% -50.00% (p=0.000 n=10) ``` --------- Co-authored-by: Owen Diehl <ow.diehl@gmail.com>	2 years ago
Karsten Jeschkies	94725e7908	Define `RequiredLabels` query limit. (#8851) ... **What this PR does / why we need it**: Some end-users can impose great workload on a cluster by selecting too many streams in their queries. We should be able to limit them. Therefore we introduce a new limit `RequiredLabelMatchers` which list label names that must be included in the stream selectors. The implementation follows the same approach as for max query limit. **Which issue(s) this PR fixes**: Fixes #8745 **Checklist** - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [x] Documentation added - [x] Tests updated - [x] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	2 years ago
Travis Patterson	9d20bed26a	Short circuit parsers (#8724) ... In many cases we only need a few labels from a line. Because most of our parsers parse lines incrementally, we can stop parsing a line after we have all the labels we want from it. This pr uses `ParserHints` to keep track of the number of extracted labels. It also provides a way for parsers to know when they should stop parsing. Notes: - parsers had inconsistent ordering between the `ShouldExtract` call and adding the `_extracted` label to duplicates. This PR makes appending `_extracted` always happen before `ShouldExtract` to keep counts of what is extracted compared to expected labels consistent. Next Steps: - When the user specifies a query with a grouping containing the `_extracted` label but there is no duplicate between the passed line and labels, short circuiting will not work. I'll address this in a follow up PR. Benchmarks: To try and show a balanced view of what this buys us, this pr picks a label our of the middle of a line. In a best case this might be much better. In the worst case, we have to parse the whole line. ``` benchstat short_circuit_old.txt short_circuit_new.txt name old time/op new time/op delta KeyExtraction/json-8 456ns ± 3% 256ns ± 2% -43.84% (p=0.000 n=9+10) KeyExtraction/logfmt-8 347ns ± 4% 171ns ± 2% -50.86% (p=0.000 n=10+10) KeyExtraction/logfmt-expression-8 552ns ± 2% 368ns ± 2% -33.23% (p=0.000 n=9+10) name old alloc/op new alloc/op delta KeyExtraction/json-8 5.00B ± 0% 5.00B ± 0% ~ (all equal) KeyExtraction/logfmt-8 5.00B ± 0% 5.00B ± 0% ~ (all equal) KeyExtraction/logfmt-expression-8 16.0B ± 0% 16.0B ± 0% ~ (all equal) name old allocs/op new allocs/op delta KeyExtraction/json-8 1.00 ± 0% 1.00 ± 0% ~ (all equal) KeyExtraction/logfmt-8 1.00 ± 0% 1.00 ± 0% ~ (all equal) KeyExtraction/logfmt-expression-8 2.00 ± 0% 2.00 ± 0% ~ (all equal) ``` --------- Co-authored-by: J Stickler <julie.stickler@grafana.com>	2 years ago
Travis Patterson	5a08a6bcb9	Label selector optimizations (#8763) ... This PR introduces the following label selector optimizations: - any selector that is `=~<literal>` is converted to an `Equals/NotEquals` matcher - Use the greedy->non-greedy regex improvements that exist in line and label filters - Never run `.*` label matchers. Prometheus regex matchers anchor the match to the beginning and end of the line. That means matchers of the form `=~<literal_value>` are just testing equality. Benchmarks -- this is just a quick bench to demonstrate equality matchers are faster than their equivalent regex matchers: ``` BenchmarkMatcherTypes/regex-8 27694834 53.74 ns/op 0 B/op 0 allocs/op BenchmarkMatcherTypes/equals-8 627045850 2.153 ns/op 0 B/op 0 allocs/op ```	2 years ago
Ashwanth	7d2eeddfb8	LogQL: Fix panic when using `unpack` with empty line (#8853) ... **What this PR does / why we need it**: Fixes panic in unpack parser when processing empty lines ``` /src/enterprise-logs/vendor/github.com/grafana/loki/pkg/logql/log/parser.go:583 github.com/grafana/loki/pkg/logql/log.(*UnpackParser).unpack(...) panic: runtime error: index out of range [0] with length 0 ``` Signed-off-by: Ashwanth Goli <iamashwanth@gmail.com>	2 years ago
Ashwanth	d08b87f799	json parser: fix panic when processing empty line (#8835) ... **What this PR does / why we need it**: Fixes panic when an empty line is processed by `JSONExpressionParser` ``` /src/enterprise-logs/vendor/github.com/grafana/loki/pkg/logql/log/parser.go:537 github.com/grafana/loki/pkg/logql/log.isValidJSONStart(...) ... panic: runtime error: index out of range [0] with length 0 ``` **Which issue(s) this PR fixes**: Fixes #<issue number> **Special notes for your reviewer**: **Checklist** - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [ ] Documentation added - [x] Tests updated - [x] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` --------- Signed-off-by: Ashwanth Goli <iamashwanth@gmail.com>	2 years ago
Travis Patterson	37c0b3dec2	Allow math on errors (#8823) ... Currently metric expressions will fail if they contain the parser error label `__error__`. There are cases where you may want to count errors. For example, `sum(count_over_time({foo="bar"} | json | __error__= "JSONParserErr" [$__interval]))` to see how many of your logs are unparsable JSON. This pr uses `ParseHint` to determine whether the `__error__` label was requested in the query and, if it was, do math on it like any other query. Parse hints are only included in VectorAggregations so the label used to signal whether or not to keep errors is never seen by a user. All other operations remain unaffected	2 years ago
Travis Patterson	83ba93c867	fix to-string for noop label filters (#8799) ... Label Filter optimizations omit `.*` label filters from the execution pipeline but intruduced a bug where those stages were also ommitted from calls to `String` on the expression. This results in unparsable queries and panics when queries are cloned via `query.String()` This PR makes `noopLabelFilter` embed matchers and inherit their `String` method just like the `StringLabelFilters` do	2 years ago
Ed Welch	0352b75f0d	Loki: Add new logql template functions `bytes`, `duration`, `unixEpochMillis`, `unixEpochNanos`, `toDateInZone`, `b64Enc`, and `b64Dec` (#8774) ... **What this PR does / why we need it**: This adds several new template functions to the logql to be used in `line_format` and `label_format` - bytes - duration (and duration_seconds) - unixEpochMillis - unixEpochNanos - toDateInZone - b64Enc and b64Dec `bytes` and `duration` adds parity with the `unwrap` pipeline stage in logql, allowing you to convert human readable versions of bytes and times into their respective primitives of bytes and seconds. This makes it possible to do math and comparisons of these extracted values in `line_format` and `label_format` `unixEpochMillis` and `unixEpochNanos` extends the existing `unixEpoch` function which can only generate seconds level precision to allow having millisecond and nanosecond level precision. `toDateInZone` allows for specifying a specific timezone when parsing a date. `b64Enc` and `b64Dec` allow base64 encoding and decoding values. **Which issue(s) this PR fixes**: Fixes #<issue number> **Special notes for your reviewer**: @JStickler I made a lot of changes to the template_functions.md doc page to try and create some more consistency as well as remove some notices that really aren't relevant anymore: * Tried to make sure every function had a `Signature` and `Examples` block * Removed a lot of `NOTE` about functions being introduced in a specific version * Removed documentation for some deprecated string functions (they still exist in code but I removed the docs for them, their exist better replacements for all of them) **Checklist** - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [x] Documentation added - [x] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` --------- Signed-off-by: Edward Welch <edward.welch@grafana.com>	2 years ago
Danny Kopping	e2ac2d50e2	Changes from linter (#8769) ... **What this PR does / why we need it**: We had a few outstanding linter issues, and running `make lint` fixes them all automatically. Signed-off-by: Danny Kopping <danny.kopping@grafana.com>	2 years ago
Travis Patterson	9ac33e216e	Reimplement Unpack in terms of jsonparser rather than jsoniter (#8739) ... This PR reimplements the functionality of `Unpack` using the `jsonparser` library rather than `jsoniter` Benchmarks: ``` name old time/op new time/op delta _Parser/unpack/no_labels_hints-8 900ns ± 4% 309ns ± 3% -65.62% (p=0.000 n=10+10) _Parser/unpack/labels_hints-8 1.06µs ±12% 0.32µs ± 2% -69.72% (p=0.000 n=9+10) _Parser/unpack-not_json_line/no_labels_hints-8 37.5ns ± 8% 10.6ns ± 3% -71.70% (p=0.000 n=9+9) _Parser/unpack-not_json_line/labels_hints-8 39.6ns ±11% 10.3ns ± 2% -74.00% (p=0.000 n=10+9) name old alloc/op new alloc/op delta _Parser/unpack/no_labels_hints-8 536B ± 0% 80B ± 0% -85.07% (p=0.000 n=10+10) _Parser/unpack/labels_hints-8 536B ± 0% 80B ± 0% -85.07% (p=0.000 n=10+10) _Parser/unpack-not_json_line/no_labels_hints-8 0.00B 0.00B ~ (all equal) _Parser/unpack-not_json_line/labels_hints-8 0.00B 0.00B ~ (all equal) name old allocs/op new allocs/op delta _Parser/unpack/no_labels_hints-8 17.0 ± 0% 4.0 ± 0% -76.47% (p=0.000 n=10+10) _Parser/unpack/labels_hints-8 17.0 ± 0% 4.0 ± 0% -76.47% (p=0.000 n=10+10) _Parser/unpack-not_json_line/no_labels_hints-8 0.00 0.00 ~ (all equal) _Parser/unpack-not_json_line/labels_hints-8 0.00 0.00 ~ (all equal) ```	2 years ago
Travis Patterson	9c6e509a6a	Reimplement JsonExpressionParser in terms of jsonparser (#8734) ... This PR reimplements the functionality of `JSONExpressionParser` using the `jsonparser` library rather than `jsoniter` Benchmarks: ``` benchstat json_expression_old.txt json_expression_new.txt name old time/op new time/op delta JsonExpressionParser/json-expression-8 4.15µs ± 4% 0.82µs ± 3% -80.17% (p=0.000 n=10+10) name old alloc/op new alloc/op delta JsonExpressionParser/json-expression-8 1.07kB ± 0% 0.02kB ± 0% -97.76% (p=0.000 n=10+10) name old allocs/op new allocs/op delta JsonExpressionParser/json-expression-8 72.0 ± 0% 2.0 ± 0% -97.22% (p=0.000 n=10+10) ```	2 years ago
Travis Patterson	7f42137584	Use Line Filters in StringLabelFilter (#8659) ... We've done a ton of work to optimize regexes in line filters but nothing in label filters. To avoid duplicating the work or reworking all of our filter types, this PR creates a new label filter that has an optimized line filter. It uses that against the label value to determine if they match. Note on Benchmarks: the input string for this is 1114195 bytes long. The `not` test tells us that our approach is quite a lot slower when the literal pattern isn't present in the input. The size of the input really emphasizes it. Benchmarks: ``` name old time/op new time/op delta LineLabelFilters/foo-8 50.6ns ± 9% 14.3ns ± 1% -71.67% (p=0.000 n=10+8) LineLabelFilters/not-8 15.7ns ± 1% 64165.6ns ± 2% +407766.77% (p=0.000 n=10+10) LineLabelFilters/(foo)-8 58.4ns ± 1% 14.8ns ±10% -74.68% (p=0.000 n=9+9) LineLabelFilters/(foo|ba)-8 64.9ns ± 0% 18.2ns ± 2% -71.90% (p=0.000 n=7+9) LineLabelFilters/(foo|ba|ar)-8 74.8ns ± 7% 21.4ns ± 2% -71.42% (p=0.000 n=10+8) LineLabelFilters/(foo|(ba|ar))-8 65.1ns ± 1% 18.1ns ± 2% -72.22% (p=0.000 n=8+8) LineLabelFilters/foo.*-8 882ns ± 4% 15ns ±11% -98.30% (p=0.000 n=9+10) LineLabelFilters/.*foo-8 11.5ns ± 8% 14.9ns ± 7% +30.39% (p=0.000 n=10+10) LineLabelFilters/.*foo.*-8 5.46µs ± 7% 0.01µs ± 3% -99.74% (p=0.000 n=10+9) LineLabelFilters/(.*)(foo).*-8 6.26µs ± 7% 0.01µs ± 2% -99.77% (p=0.000 n=10+9) LineLabelFilters/(foo.*|.*ba)-8 5.58µs ± 3% 0.02µs ± 8% -99.67% (p=0.000 n=9+10) LineLabelFilters/(foo.*|.*bar.*)-8 7.02µs ± 7% 0.02µs ± 8% -99.73% (p=0.000 n=10+10) LineLabelFilters/.*foo.*|bar-8 5.53µs ± 6% 0.02µs ± 2% -99.67% (p=0.000 n=10+8) LineLabelFilters/.*foo|bar-8 3.73µs ± 6% 0.02µs ± 3% -99.52% (p=0.000 n=10+9) LineLabelFilters/(?:.*foo.*|bar)-8 5.43µs ± 8% 0.02µs ± 8% -99.65% (p=0.000 n=10+10) LineLabelFilters/(?P<foo>.*foo.*|bar)-8 5.71µs ± 3% 0.02µs ±10% -99.68% (p=0.000 n=9+9) LineLabelFilters/.*foo.*|bar|buzz-8 5.34µs ± 2% 0.02µs ± 3% -99.65% (p=0.000 n=10+9) LineLabelFilters/.*foo.*|bar|uzz-8 5.36µs ± 1% 0.02µs ± 3% -99.60% (p=0.000 n=8+8) LineLabelFilters/foo|bar|b|buzz|zz-8 68.8ns ± 8% 22.6ns ± 7% -67.19% (p=0.000 n=10+10) LineLabelFilters/f|foo|foobar-8 77.1ns ± 1% 18.6ns ± 9% -75.90% (p=0.000 n=8+10) LineLabelFilters/f.*|foobar.*|.*buzz-8 5.64µs ± 7% 0.02µs ±10% -99.60% (p=0.000 n=10+10) LineLabelFilters/((f.*)|foobar.*)|.*buzz-8 6.29µs ± 8% 0.02µs ±10% -99.65% (p=0.000 n=10+10) LineLabelFilters/.*-8 963ns ± 8% 2ns ±14% -99.77% (p=0.000 n=10+10) LineLabelFilters/.*|.*-8 4.46µs ± 8% 0.00µs ± 8% -99.95% (p=0.000 n=10+10) LineLabelFilters/.*||||-8 2.99µs ± 1% 0.00µs ±16% -99.93% (p=0.000 n=8+10) LineLabelFilters/#00-8 43.5ns ± 8% 2.1ns ± 1% -95.25% (p=0.000 n=10+8) LineLabelFilters/(?i)foo-8 71.7ns ± 6% 13.1ns ± 1% -81.73% (p=0.000 n=10+8) LineLabelFilters/(?i)界-8 16.5ns ± 8% 150.2ns ± 8% +809.55% (p=0.000 n=10+10) LineLabelFilters/(?i)ïB-8 23.8ns ± 1% 14748979.8ns ± 7% +62074730.81% (p=0.000 n=8+10) LineLabelFilters/(?:)foo|fatal|exception-8 185ns ± 2% 21ns ± 2% -88.38% (p=0.000 n=9+8) LineLabelFilters/(?i)foo|fatal|exception-8 78.4ns ± 3% 20.2ns ± 2% -74.23% (p=0.000 n=9+9) LineLabelFilters/(?i)f|foo|foobar-8 94.3ns ±16% 15.4ns ± 9% -83.66% (p=0.000 n=10+10) LineLabelFilters/(?i)f|fatal|e.*-8 65.7ns ± 5% 18.2ns ± 3% -72.24% (p=0.000 n=9+8) LineLabelFilters/(?i).*foo.*-8 6.04µs ± 8% 0.01µs ± 2% -99.78% (p=0.000 n=10+10) name old alloc/op new alloc/op delta LineLabelFilters/foo-8 0.00B 0.00B ~ (all equal) LineLabelFilters/not-8 0.00B 0.00B ~ (all equal) LineLabelFilters/(foo)-8 0.00B 0.00B ~ (all equal) LineLabelFilters/(foo|ba)-8 0.00B 0.00B ~ (all equal) LineLabelFilters/(foo|ba|ar)-8 0.00B 0.00B ~ (all equal) LineLabelFilters/(foo|(ba|ar))-8 0.00B 0.00B ~ (all equal) LineLabelFilters/foo.*-8 0.00B 0.00B ~ (all equal) LineLabelFilters/.*foo-8 0.00B 0.00B ~ (all equal) LineLabelFilters/.*foo.*-8 0.00B 0.00B ~ (all equal) LineLabelFilters/(.*)(foo).*-8 0.00B 0.00B ~ (all equal) LineLabelFilters/(foo.*|.*ba)-8 0.00B 0.00B ~ (all equal) LineLabelFilters/(foo.*|.*bar.*)-8 0.00B 0.00B ~ (all equal) LineLabelFilters/.*foo.*|bar-8 0.00B 0.00B ~ (all equal) LineLabelFilters/.*foo|bar-8 0.00B 0.00B ~ (all equal) LineLabelFilters/(?:.*foo.*|bar)-8 0.00B 0.00B ~ (all equal) LineLabelFilters/(?P<foo>.*foo.*|bar)-8 0.00B 0.00B ~ (all equal) LineLabelFilters/.*foo.*|bar|buzz-8 0.00B 0.00B ~ (all equal) LineLabelFilters/.*foo.*|bar|uzz-8 0.00B 0.00B ~ (all equal) LineLabelFilters/foo|bar|b|buzz|zz-8 0.00B 0.00B ~ (all equal) LineLabelFilters/f|foo|foobar-8 0.00B 0.00B ~ (all equal) LineLabelFilters/f.*|foobar.*|.*buzz-8 0.00B 0.00B ~ (all equal) LineLabelFilters/((f.*)|foobar.*)|.*buzz-8 0.00B 0.00B ~ (all equal) LineLabelFilters/.*-8 0.00B 0.00B ~ (all equal) LineLabelFilters/.*|.*-8 0.00B 0.00B ~ (all equal) LineLabelFilters/.*||||-8 0.00B 0.00B ~ (all equal) LineLabelFilters/#00-8 0.00B 0.00B ~ (all equal) LineLabelFilters/(?i)foo-8 0.00B 0.00B ~ (all equal) LineLabelFilters/(?i)界-8 0.00B 0.00B ~ (all equal) LineLabelFilters/(?i)ïB-8 0.00B 0.00B ~ (all equal) LineLabelFilters/(?:)foo|fatal|exception-8 0.00B 0.00B ~ (all equal) LineLabelFilters/(?i)foo|fatal|exception-8 0.00B 0.00B ~ (all equal) LineLabelFilters/(?i)f|foo|foobar-8 0.00B 0.00B ~ (all equal) LineLabelFilters/(?i)f|fatal|e.*-8 0.00B 0.00B ~ (all equal) LineLabelFilters/(?i).*foo.*-8 0.00B 0.00B ~ (all equal) name old allocs/op new allocs/op delta LineLabelFilters/foo-8 0.00 0.00 ~ (all equal) LineLabelFilters/not-8 0.00 0.00 ~ (all equal) LineLabelFilters/(foo)-8 0.00 0.00 ~ (all equal) LineLabelFilters/(foo|ba)-8 0.00 0.00 ~ (all equal) LineLabelFilters/(foo|ba|ar)-8 0.00 0.00 ~ (all equal) LineLabelFilters/(foo|(ba|ar))-8 0.00 0.00 ~ (all equal) LineLabelFilters/foo.*-8 0.00 0.00 ~ (all equal) LineLabelFilters/.*foo-8 0.00 0.00 ~ (all equal) LineLabelFilters/.*foo.*-8 0.00 0.00 ~ (all equal) LineLabelFilters/(.*)(foo).*-8 0.00 0.00 ~ (all equal) LineLabelFilters/(foo.*|.*ba)-8 0.00 0.00 ~ (all equal) LineLabelFilters/(foo.*|.*bar.*)-8 0.00 0.00 ~ (all equal) LineLabelFilters/.*foo.*|bar-8 0.00 0.00 ~ (all equal) LineLabelFilters/.*foo|bar-8 0.00 0.00 ~ (all equal) LineLabelFilters/(?:.*foo.*|bar)-8 0.00 0.00 ~ (all equal) LineLabelFilters/(?P<foo>.*foo.*|bar)-8 0.00 0.00 ~ (all equal) LineLabelFilters/.*foo.*|bar|buzz-8 0.00 0.00 ~ (all equal) LineLabelFilters/.*foo.*|bar|uzz-8 0.00 0.00 ~ (all equal) LineLabelFilters/foo|bar|b|buzz|zz-8 0.00 0.00 ~ (all equal) LineLabelFilters/f|foo|foobar-8 0.00 0.00 ~ (all equal) LineLabelFilters/f.*|foobar.*|.*buzz-8 0.00 0.00 ~ (all equal) LineLabelFilters/((f.*)|foobar.*)|.*buzz-8 0.00 0.00 ~ (all equal) LineLabelFilters/.*-8 0.00 0.00 ~ (all equal) LineLabelFilters/.*|.*-8 0.00 0.00 ~ (all equal) LineLabelFilters/.*||||-8 0.00 0.00 ~ (all equal) LineLabelFilters/#00-8 0.00 0.00 ~ (all equal) LineLabelFilters/(?i)foo-8 0.00 0.00 ~ (all equal) LineLabelFilters/(?i)界-8 0.00 0.00 ~ (all equal) LineLabelFilters/(?i)ïB-8 0.00 0.00 ~ (all equal) LineLabelFilters/(?:)foo|fatal|exception-8 0.00 0.00 ~ (all equal) LineLabelFilters/(?i)foo|fatal|exception-8 0.00 0.00 ~ (all equal) LineLabelFilters/(?i)f|foo|foobar-8 0.00 0.00 ~ (all equal) LineLabelFilters/(?i)f|fatal|e.*-8 0.00 0.00 ~ (all equal) LineLabelFilters/(?i).*foo.*-8 0.00 0.00 ~ (all equal) ```	2 years ago
Travis Patterson	c42a0ba868	Optimize .+ regex line filter (#8646) ... Interpolated variables from Grafana dashboards may result in the regex matcher `.+`. Filters with `+` are already being rewritten to be non greedy but there is a further optimization to make replace the regex altogether with a filter that just matchers `len(line) > 0`	2 years ago
Christian Haudum	d4a6b159fa	chore: Cleanup label extraction expression (#8596) ... The aim of this is to remove redundancy from the parser grammar and make the expression name more descriptive. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago