apitech/loki - loki - Gitea: Git with a cup of tea

Commit Graph

Author	SHA1	Message	Date
Susana Ferreira	ef49909430	Refactor sharding engine to downstream engine (#5546 )	3 years ago
Susana Ferreira	16aa9dd55d	Update vectorAggEvaluator to fail for expressions without grouping (#5544 )	3 years ago
Cyril Tovena	a411bb9420	Disable sharding of count/avg when labels are mutated (#5474 )	3 years ago
Cyril Tovena	6dad54fef9	Fixes ingester sample deduping (#5470 ) * Fixes ingester sample deduping * Call get hash only once * Update pkg/iter/entry_iterator.go Co-authored-by: Owen Diehl <ow.diehl@gmail.com> Co-authored-by: Owen Diehl <ow.diehl@gmail.com>	3 years ago
Cyril Tovena	e3db2671f4	Do not insert point when missing when sharding (#5444 )	3 years ago
Sas Swart	c80244a3e4	Parse duration expressions in accordance with promql (#5275 ) * Add support for Prometheus like duration units in logql * Remove redundant type conversion * deduplicate call to sb.String Add a comment to explain why both promql and time's ParseDuration methods are used * extract duration parsing to separate function. * Add tests to ensure promql duration units are supported * Sort duration rune comment and list of runes in ascending order of length.	3 years ago
Cyril Tovena	07e5eb33d3	Do not use WaitGroup context for StepEvaluator (#5425 )	3 years ago
Cyril Tovena	9b6f17f168	Add the total number of subqueries to query statistics (#5397 ) * Add subqueries count Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes tests Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	3 years ago
Sas Swart	64589d0339	Logql: Correct grouping type in pkg/logql/expr.y (#5306 ) * Correct grouping type in pkg/logql/expr.y * Use loki-build-image to build loki instead of golang image * Include goyacc and other required tools to allow Makefile to run * Remove dependencies not relevant to yacc in cmd/loki/Dockerfile * Use go install instead of go get to install goyacc in cmd/loki/Dockerfile * Add goyacc to logcli and loki-canary Dockerfiles * Rerun CI * Rerun CI * Revert "Add goyacc to logcli and loki-canary Dockerfiles" This reverts commit `0ee887c57f`. * Remove goyacc from cmd/loki/Dockerfile * Formatting of cmd/loki/Dockerfile	3 years ago
Ed Welch	b3af2dbeef	Loki: include the component in metrics.go log line (#5379 ) * add a component key to our metrics.go line Signed-off-by: Edward Welch <edward.welch@grafana.com> * new computer... who dis Signed-off-by: Edward Welch <edward.welch@grafana.com> * one more lint Signed-off-by: Edward Welch <edward.welch@grafana.com>	3 years ago
Cyril Tovena	bbaef790db	Add usage report into Loki. (#5361 ) * Adds leader election process Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * fluke Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * fixes the kv typecheck * wire up the http client * Hooking into loki services, hit a bug * Add stats variable. * re-vendor dskit and improve to never fail service * Intrument Loki with the package * Add changelog entry Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes compactor test Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add configuration documentation Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Update pkg/usagestats/reporter.go Co-authored-by: Danny Kopping <dannykopping@gmail.com> * Add boundary check Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add log for success report. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * lint Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Update pkg/usagestats/reporter.go Co-authored-by: Danny Kopping <dannykopping@gmail.com> Co-authored-by: Danny Kopping <dannykopping@gmail.com>	3 years ago
Bryan Boreham	a50cac7674	filters: use faster regexp package (#5315 ) * regexp-filter: improve benchmark Add a few more cases based on real-world usage. Also simplify the loop to just run the number of times requested. Go benchmarks run 'N' times to reach a duration, default 1 second. Previously the benchmark was running N1000000 times, which took 7 seconds minimum for some cases. regexp filter: use modified package with optimisations See https://github.com/grafana/regexp/tree/speedup#readme Includes the following changes proposed upstream: * [regexp: allow patterns with no alternates to be one-pass](https://go-review.googlesource.com/c/go/+/353711) * [regexp: speed up onepass prefix check](https://go-review.googlesource.com/c/go/+/354909) * [regexp: handle prefix string with fold-case](https://go-review.googlesource.com/c/go/+/358756) * [regexp: avoid copying each instruction executed](https://go-review.googlesource.com/c/go/+/355789) * [regexp: allow prefix string anchored at beginning](https://go-review.googlesource.com/c/go/+/377294) * Add grafana/regexp to vendor directory	3 years ago
Cyril Tovena	e3b74158a5	Fixes log deduplication when mutating Labels using LogQL (#5289 ) * Changes the iterator interface to include labelshash. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Improve unused args and fixes build Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Implement the new hash in the memchunk. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add the hash in logproto Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Insert the hash from ingester-querier Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Rename LabelsHash to StreamHash Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Working through fixing tests. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Refactor HeapIterator into merge and sort Iterator. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * lint Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Order alphabetically when ordering samples. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Do not JSON encode the hash for legacy model. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add more regression tests. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Update CHANGELOG.md Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Review feedback	3 years ago
Owen Diehl	2af3ca077e	Logql/parallel binop (#5317 ) * adds justification for keeping Downstreamer parallelism * loads binop legs in parallel * increases downstreamer default concurrency * astmapper spanlogger * always clone expr during mapping to prevent mutability bugs * Revert "astmapper spanlogger" This reverts commit `23f6b55c14`. * cleanup + use errgroup	3 years ago
Vladyslav Diachenko	ea37f4595d	added upgrade notes (#5311 )	3 years ago
Cyril Tovena	3af7b7900e	Refactor HeapIterator into Merge and Sort Iterator. (#5281 ) * Refactor HeapIterator into merge and sort Iterator. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * lint Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	3 years ago
Kaviraj Kanagaraj	f5d570177a	Fork `cortex/pkg/util` into Loki (#5257 ) * Fork `cortex/pkg/util/validation` Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Fork `pkg/cortex/util/limiter` Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * fork util/extract Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Spanlogger add more helpers Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * getting rid of `cortex/pkg/util` Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * With lesser linter errors Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Make linter happy by removing unused method Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Remove missed util_log Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Fix gofmt and goimports Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Do not touch operator Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Do not touch generated go files Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Format Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Make linters happy Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Fix lambda promtail tools Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Upgrade Dockerfile of lambda promtail Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Add tests for `pkg/util` Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com>	3 years ago
Arve Knudsen	a2dd731586	Chore: Add pkg/util/log package (#5187 ) * Chore: Add pkg/util/log package Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Add changelog entry Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix documentation of loki_log_messages_total Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	3 years ago
Cyril Tovena	21e61315df	LogQL: add __line__ function to \| line_format template (#4879 ) * LogQL: add __line__ function to \| line_format template Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * add changelog entry Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Update docs/sources/logql/template_functions.md Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> * Clone the function map to avoid a race between multiple expr Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> Co-authored-by: Owen Diehl <ow.diehl@gmail.com>	3 years ago
Susana Ferreira	191a8a6f9d	Store metrics queueTime in seconds (#5052 ) * Store metrics queueTime in seconds * Address comments * Helper function to convert seconds to nanoseconds	3 years ago
Susana Ferreira	82f8f3c1e5	Log time in queue per request (#4949 ) * Log time in queue per request * Add querierID to log message * Add unescape request url and tenant_ids to log message * Add query enqueue time to metrics.go * Remove "querier request dequeued" log from scheduler * Address comments * Rename enqueue_time to queue_time * Store queue time in nanoseconds (int64) * Use CanonicalMIMEHeaderKey for setting the httpgrpc header	3 years ago
Kaviraj	fe1d28a545	Chore: use `spanlogger` from `dskit` (#5029 ) * Move `cortex/pkg/spanlogger` into `loki`. It uses spanlogger from `dskit/pkg/spanlogger` with small wrapper created on the `pkg/util/spanlogger` in loki repo. Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Remove `cortex/util/spanlogger` -> `loki/util/spanlogger` Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Fix linter (with goimports) and `FromContext` Run goimports with `-local` flag `FromContext` takes util logger to keep the API same as currently used inside loki Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com>	3 years ago
李国忠	a66e148f83	[new feature] logql: extrapolate unwrapped rate function (#5013 ) * [new feature] logql：reimplement metrics rate function * [new feature] logql：reimplement metrics rate function * [new feature] logql：reimplement metrics rate function * [new feature] logql: extrapolate unwrapped rate function #5013	3 years ago
Dylan Guedes	d5783e86a3	Copy `cortex/pkg/querier/astmapper` package dependency into Loki (#4982 ) * Fork cortex `pkg/querier/asmapper` into Loki. * Move away from cortex `pkg/querier/astmapper`. * Fix lint issues. - Use _ in argument of method `pkg/querier/astmapper/subtree_folder.go` since it is not being used - Remove unnecessary cast from `subtree_folder.go`	3 years ago
Kaviraj	38c42a26c6	Move `cortex/pkg/tenant` dependency to Loki. (#4944 )	4 years ago
Marco Pracucci	2b0d020146	Refactor pkg/util/server to avoid cortexpb import in downstream projects vendoring logql (#4833 ) Signed-off-by: Marco Pracucci <marco@pracucci.com>	4 years ago
Cyril Tovena	0a944670ef	Improve LogQL format stages requireLabel (#4769 ) * Improve LogQL format stages. This only includes required labels. ``` ➜ go test -benchmem -run=^$ -bench ^Benchmark_Pipeline ./pkg/logql/log -v -count 5 > after.txt && benchstat before.txt after.txt name old time/op new time/op delta _Pipeline/pipeline_bytes-16 13.8µs ± 6% 11.5µs ± 1% -16.42% (p=0.008 n=5+5) _Pipeline/pipeline_string-16 13.9µs ± 2% 11.5µs ± 2% -17.27% (p=0.008 n=5+5) _Pipeline/line_extractor_bytes-16 14.0µs ± 4% 11.6µs ± 0% -17.45% (p=0.016 n=5+4) _Pipeline/line_extractor_string-16 13.5µs ± 0% 11.5µs ± 0% -14.67% (p=0.008 n=5+5) _Pipeline/label_extractor_bytes-16 13.7µs ± 1% 11.7µs ± 1% -14.69% (p=0.008 n=5+5) _Pipeline/label_extractor_string-16 14.1µs ±11% 11.6µs ± 0% -17.89% (p=0.008 n=5+5) name old alloc/op new alloc/op delta _Pipeline/pipeline_bytes-16 9.51kB ± 0% 7.50kB ± 0% -21.19% (p=0.008 n=5+5) _Pipeline/pipeline_string-16 9.51kB ± 0% 7.50kB ± 0% -21.19% (p=0.008 n=5+5) _Pipeline/line_extractor_bytes-16 9.51kB ± 0% 7.50kB ± 0% -21.19% (p=0.008 n=5+5) _Pipeline/line_extractor_string-16 9.51kB ± 0% 7.50kB ± 0% ~ (p=0.079 n=4+5) _Pipeline/label_extractor_bytes-16 9.51kB ± 0% 7.50kB ± 0% -21.18% (p=0.008 n=5+5) _Pipeline/label_extractor_string-16 9.51kB ± 0% 7.50kB ± 0% -21.18% (p=0.008 n=5+5) name old allocs/op new allocs/op delta _Pipeline/pipeline_bytes-16 46.0 ± 0% 45.0 ± 0% -2.17% (p=0.008 n=5+5) _Pipeline/pipeline_string-16 46.0 ± 0% 45.0 ± 0% -2.17% (p=0.008 n=5+5) _Pipeline/line_extractor_bytes-16 46.0 ± 0% 45.0 ± 0% -2.17% (p=0.008 n=5+5) _Pipeline/line_extractor_string-16 46.0 ± 0% 45.0 ± 0% -2.17% (p=0.008 n=5+5) _Pipeline/label_extractor_bytes-16 46.0 ± 0% 45.0 ± 0% -2.17% (p=0.008 n=5+5) _Pipeline/label_extractor_string-16 46.0 ± 0% 45.0 ± 0% -2.17% (p=0.008 n=5+5) ``` Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * rollback model change Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * remove unused variable Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	4 years ago
Marco Pracucci	c53457feb9	Upgrade Cortex, Prometheus and Thanos (#4830 ) * Upgrade Cortex, Prometheus and Thanos Signed-off-by: Marco Pracucci <marco@pracucci.com> * Fixed imported ordering Signed-off-by: Marco Pracucci <marco@pracucci.com> * Fixed data type used for series ref when recovering from checkpoint/WAL Signed-off-by: Marco Pracucci <marco@pracucci.com> * Upgraded Thanos to fix arm build Signed-off-by: Marco Pracucci <marco@pracucci.com> * Run go mod tidy Signed-off-by: Marco Pracucci <marco@pracucci.com>	4 years ago
Kaviraj	1b63331e3d	Add support for `X-Query-Tags` (#4814 ) * Add support for `X-Query-Tags` 1. Now client can send any metadata about query via `X-Query-Tags` HTTP header 2. Also record `X-Query-Tags` in `metrics.go` for instant and range queries. * PR remarks * Make linter happy * Fix middleware next * Make safe parsing of the header * Add bit more test cases * Minor suggestions * Add middleware chain to querier handlers * Repopulate the HTTP headers before while Encoding	4 years ago
Cyril Tovena	45ed6d560d	Fix up a merge conflict on log filter (#4815 ) * Fix up a merge conflict on log filter Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * use the same parameter order as the stdlib Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	4 years ago
Hiroki Sakamoto	b1b82a6000	[Hot Fix] Merge improving case sensitive search correctly (#4804 ) * Improve case sensitive search to avoid allocations. ``` ❯ benchcmp before.txt after.txt benchmark old ns/op new ns/op delta Benchmark_LineFilter/default_true_(?i)foo-16 2400 2233 -6.96% Benchmark_LineFilter/simplified_true_(?i)foo-16 201 228 +13.13% Benchmark_LineFilter/default_false_(?i)foo-16 2443 2376 -2.74% Benchmark_LineFilter/simplified_false_(?i)foo-16 185 231 +24.96% benchmark old allocs new allocs delta Benchmark_LineFilter/default_true_(?i)foo-16 0 0 +0.00% Benchmark_LineFilter/simplified_true_(?i)foo-16 1 0 -100.00% Benchmark_LineFilter/default_false_(?i)foo-16 0 0 +0.00% Benchmark_LineFilter/simplified_false_(?i)foo-16 1 0 -100.00% benchmark old bytes new bytes delta Benchmark_LineFilter/default_true_(?i)foo-16 0 0 +0.00% Benchmark_LineFilter/simplified_true_(?i)foo-16 128 0 -100.00% Benchmark_LineFilter/default_false_(?i)foo-16 0 0 +0.00% Benchmark_LineFilter/simplified_false_(?i)foo-16 128 0 -100.00% ``` It's not much but for a billions line it makes a big difference. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Simplified version Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Cleanup Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> Co-authored-by: Cyril Tovena <cyril.tovena@gmail.com>	4 years ago
Cyril Tovena	dc222dc98d	Improve case insensitive search to avoid allocations. (#4394 ) * Improve case sensitive search to avoid allocations. ``` ❯ benchcmp before.txt after.txt benchmark old ns/op new ns/op delta Benchmark_LineFilter/default_true_(?i)foo-16 2400 2233 -6.96% Benchmark_LineFilter/simplified_true_(?i)foo-16 201 228 +13.13% Benchmark_LineFilter/default_false_(?i)foo-16 2443 2376 -2.74% Benchmark_LineFilter/simplified_false_(?i)foo-16 185 231 +24.96% benchmark old allocs new allocs delta Benchmark_LineFilter/default_true_(?i)foo-16 0 0 +0.00% Benchmark_LineFilter/simplified_true_(?i)foo-16 1 0 -100.00% Benchmark_LineFilter/default_false_(?i)foo-16 0 0 +0.00% Benchmark_LineFilter/simplified_false_(?i)foo-16 1 0 -100.00% benchmark old bytes new bytes delta Benchmark_LineFilter/default_true_(?i)foo-16 0 0 +0.00% Benchmark_LineFilter/simplified_true_(?i)foo-16 128 0 -100.00% Benchmark_LineFilter/default_false_(?i)foo-16 0 0 +0.00% Benchmark_LineFilter/simplified_false_(?i)foo-16 128 0 -100.00% ``` It's not much but for a billions line it makes a big difference. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * typo Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Simplified version Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Cleanup Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	4 years ago
Karsten Jeschkies	4c9e4f5f06	Make `` and `+` non-greedy to double regex filter speed. (#4775 ) Optimize regular expressions in filters. * Run original RE2 tests. * Run both tests. * Update pkg/logql/log/filter.go Co-authored-by: Bryan Boreham <bjboreham@gmail.com> * Update pkg/logql/log/filter.go Co-authored-by: Bryan Boreham <bjboreham@gmail.com> * Fix tests. * Format code. Co-authored-by: Bryan Boreham <bjboreham@gmail.com>	4 years ago
Karsten Jeschkies	c19a6c5bbc	Join contain filters into one. (#4528 ) * Join contain filters into one. * Update pkg/logql/log/filter.go Co-authored-by: Dylan Guedes <djmgguedes@gmail.com> * Test Filter method. * Test old and new filter consturctors. * Remove comment. * Define more benchmark cases. * Take care of empty contain filters. * Satisfy linter. * USe different method to filter. * Simplify all filter concat. * Format code. * Push regex filters to the end. * Remove unused test. * Add a few more tests. * Make filters private again. * Fix test failure. * Only accumulate successive contain filters. * Reverse accumulation. * Satisfy linter. * Remove old version. Co-authored-by: Dylan Guedes <djmgguedes@gmail.com>	4 years ago
Cyril Tovena	d81f757ecb	Refactor and simplify stats package. (#4712 ) * Refactor and simplify stats package. This PR refactors the stats package and make it smaller and simpler to work with. - It doesn't expose the context which can causes race when mutating it. - Uses concurrency primitive to ensure correctness - Replace trailers with in-batch statistics to propagate ingesters stats. Fixes #3366 Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * fix tests and lint issues. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes moar tests. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * mod-check Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * More fixes. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	4 years ago
Kaviraj	1e0386e9e6	Fix `ip` matcher lexer to differentiate filter from identifier (#4598 ) * Fix `ip` matcher lexer to differentiate filter from identifier Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Revert small fmt fix on expr.y Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * PR remarks from Cyril Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Add ip line filter parser test in mix with ip as label Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com>	4 years ago
Garrett	1c7df620c4	add date time sprig template functions in logql label/line formatter (#4603 ) * add date time sprig template functions in logql label/line formatter With date time sprig template functions, we will be able to filter test jobs which create time is 1 day before as the following: ``` {job="test"} \| label_format nowEpoch=`{{(unixEpoch now)}}`,createDateEpoch=`{{unixEpoch (toDate "2006-01-02" .createDate)}}` \| label_format dateTimeDiff="{{sub .nowEpoch .createDateEpoch}}" \| dateTimeDiff > 86400 ``` * add documents for date time sprig template functions * resolve comments * add golang datetime layout doc reference * update CHANGELOG.md accordingly to include this PR	4 years ago
lingpeng0314	76e2a14029	add group_{left,right} to LogQL (#4510 ) * Add group_left/group_right support * Update docs/sources/logql/_index.md * Minor change for a test case * removes CardManyToMany * removes now-unused IsSetOperator helper Co-authored-by: Owen Diehl <ow.diehl@gmail.com>	4 years ago
Garrett	a046d79261	make LogQL syntax scope from private to public (#4446 ) * make LogQL syntax scope from private to public The follow-up for https://github.com/grafana/loki/pull/3991, it makes using LogQL syntax outside possible, it also makes LogRange implements Expr interface. * make constructor also public * Revert "make constructor also public" This reverts commit `ab9d0a942c`.	4 years ago
Danny Kopping	cff8324f24	Replacing go-kit/kit/log with go-kit/log (#4484 ) * Replacing go-kit/kit/log with go-kit/log Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Appeasing the f&^$ing linter Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Adding depguard Cleaning go mod Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Updating weaveworks after https://github.com/weaveworks/common/pull/226 was merged Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Appeasing the mod checker Signed-off-by: Danny Kopping <danny.kopping@grafana.com>	4 years ago
Mauro Stettler	4b59556342	Update cortex to newer version (#4478 ) * update cortex and thanos to newer version * pin k8s.io/apimachinery * fixes for newer libs * add missing method to mock wal data collector * fixing test by using different gokit level pkg * make check-mod	4 years ago
Garrett	4417340089	add on and ignoring clauses in binOpExpr (#4391 ) * add on and ignoring clauses in binOpExpr * add on/ignoring new operator shardmapper unit test * add doc for on and ignoring keywords * resolve comments * use a scoped error to replace mutate stepEvaluator via type switching * change VectorMatching Include from empty string array to nil * prohibit sharding when we're changing the label groupings such as on or ignoring * add unit test to cover multiple matches for labels error * fix expr.y format issue * resolve comments * reformat on and ignoring keywords doc * add sharding unit test for on and ignoring keywords * resolve comments: add a sentence for each example to help docs readers to understand the larger picture Co-authored-by: garrettlish <garrettlish@163.com>	4 years ago
Cyril Tovena	a6c7ecc2fa	Fixes the pattern parser validation. (#4308 ) We should never have 2 consecutive capture, since when we do we can't figure which data goes where. The previous validation was validating two nodes at the time and moving to the next two nodes so in the end it wasn't validating properly. 🤦 Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	4 years ago
Arve Knudsen	bfb4b00bca	Chore: Reformat Go files (#4304 ) * Chore: Reformat Go files Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Undo change Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	4 years ago
Cyril Tovena	32e55c3043	Fork cortex chunk storage into Loki. (#3842 ) The chunk package has been forked from this commit for future reference and licensing. `09fe17c0e6`	4 years ago
Kaviraj	bfb24e6a4c	Ip matcher for LogQL (#3986 ) * IP matcher and filter via netaddr.IP package Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Adding IP filter into label filter gramer 1. ast tests 2. parse tests Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Label filters all test cases passes Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Rename ipFilter -> ipLabelFilter Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * More tests Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Remove testlogs.txt Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Make linter happy Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Add more cases to the benchmark Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * PR remarks 1. support NEQ 2. reduce allocation. 3. Minor tweaks Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Make IP filter work as line filter 1. Add necessary interface implementation 2. More tests for parser and ast Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Make linter happy Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Bug with filter matcher Its wierd bug. Not handing string() method breaks the parser :( Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Split Line and Label filter to different struct. It got out of hands to handle both in same struct. Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Add basic doc for IP matcher Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Add docs feedback from Karen * Label filter add patter error report * PR remarks 1. Indendation fix 2. Remove string() interface for ip label filte * Linter happy * PR remaks * Fix ip label matcher string() method if pattern is invalid * Fix label filter tests * Have local copy of `pattern` in label filter to use in string() * Rebase conflicts * ast.go fixes	4 years ago
Periklis Tsirakidis	cb337c3b71	Add LogQL AST walker (#3991 ) * Make expressions public scoped * Add walkable implementation * Use walkable in logql/optimize	4 years ago
Jarrod Funnell	267107b75e	Parser: Allow literal control chars in logfmt decoder (#3932 ) * Allow literal control chars in decoder * Ensure decoding control chars in logfmt works	4 years ago
Owen Diehl	b58e9cdee2	fixes a badly referenced variable name in StepEvaluator code (#3996 )	4 years ago
Cyril Tovena	5d8414fcd6	Intern label keys for LogQL parser. (#3990 ) * Intern label keys for LogQL parser. They often will see the same key over and over and instead of allocating/sanitizing we can re-use the previous value. This mostly speeds up parser that were using a dynamic list of keys like json and logfmt. We could intern values too specially for metrics queries. ``` ❯ benchcmp before.txt after.txt benchmark old ns/op new ns/op delta Benchmark_Parser/json/no_labels_hints-16 3894 2885 -25.91% Benchmark_Parser/json/labels_hints-16 2084 2163 +3.79% Benchmark_Parser/unpack/no_labels_hints-16 915 890 -2.72% Benchmark_Parser/unpack/labels_hints-16 848 846 -0.32% Benchmark_Parser/logfmt/no_labels_hints-16 2229 1146 -48.59% Benchmark_Parser/logfmt/labels_hints-16 1230 1150 -6.50% Benchmark_Parser/regex_greedy/no_labels_hints-16 3881 3754 -3.27% Benchmark_Parser/regex_greedy/labels_hints-16 5443 3737 -31.34% Benchmark_Parser/regex_status_digits/no_labels_hints-16 318 290 -8.89% Benchmark_Parser/regex_status_digits/labels_hints-16 335 289 -13.85% Benchmark_Parser/pattern/no_labels_hints-16 151 151 -0.20% Benchmark_Parser/pattern/labels_hints-16 129 131 +1.24% benchmark old allocs new allocs delta Benchmark_Parser/json/no_labels_hints-16 69 43 -37.68% Benchmark_Parser/json/labels_hints-16 32 36 +12.50% Benchmark_Parser/unpack/no_labels_hints-16 19 17 -10.53% Benchmark_Parser/unpack/labels_hints-16 16 17 +6.25% Benchmark_Parser/logfmt/no_labels_hints-16 46 16 -65.22% Benchmark_Parser/logfmt/labels_hints-16 20 16 -20.00% Benchmark_Parser/regex_greedy/no_labels_hints-16 2 2 +0.00% Benchmark_Parser/regex_greedy/labels_hints-16 2 2 +0.00% Benchmark_Parser/regex_status_digits/no_labels_hints-16 2 2 +0.00% Benchmark_Parser/regex_status_digits/labels_hints-16 2 2 +0.00% Benchmark_Parser/pattern/no_labels_hints-16 2 2 +0.00% Benchmark_Parser/pattern/labels_hints-16 1 1 +0.00% benchmark old bytes new bytes delta Benchmark_Parser/json/no_labels_hints-16 1089 624 -42.70% Benchmark_Parser/json/labels_hints-16 456 480 +5.26% Benchmark_Parser/unpack/no_labels_hints-16 569 537 -5.62% Benchmark_Parser/unpack/labels_hints-16 553 537 -2.89% Benchmark_Parser/logfmt/no_labels_hints-16 576 336 -41.67% Benchmark_Parser/logfmt/labels_hints-16 256 336 +31.25% Benchmark_Parser/regex_greedy/no_labels_hints-16 194 194 +0.00% Benchmark_Parser/regex_greedy/labels_hints-16 194 194 +0.00% Benchmark_Parser/regex_status_digits/no_labels_hints-16 52 52 +0.00% Benchmark_Parser/regex_status_digits/labels_hints-16 52 52 +0.00% Benchmark_Parser/pattern/no_labels_hints-16 35 35 +0.00% Benchmark_Parser/pattern/labels_hints-16 32 32 +0.00% ``` Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Move to labels.go and cap the amount of keys that can be interned to 1024. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	4 years ago

... 2 3 4 5 6 ...

372 Commits (6cc581bd26cc00c4912d3ddf50db68839349a378)