2364 Commits (88c25965dad6cae557b63b29c83996a389704a3f)

 

All Branches   

Search

Author	SHA1	Message	Date
Cyril Tovena	88c25965da	Inject the org-id if possible from the start. (#4068) ... This allows the http logger to include the org-id. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	4 years ago
Danny Kopping	a3414a5d9b	Promtail: handling tight loop in journal follow logic (#4066) ... * Handling EBADMSG explicitly, and preventing tight loop Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Appeasing the linter Signed-off-by: Danny Kopping <danny.kopping@grafana.com>	4 years ago
Sandeep Sukhani	19aec6e6a3	fix broken TestCachingStorageClientStoreQueries (#4074)	4 years ago
Danny Kopping	f0542c04e1	Updating drone signature (#4072) ... Signed-off-by: Danny Kopping <danny.kopping@grafana.com>	4 years ago
Sandeep Sukhani	d31f8676fd	Optimize index queries and caching (#4070) ... * Use RangeValuePrefix for building queries for Series IDs * optimize index store queries and caching by not doing broad queries	4 years ago
Cyril Tovena	a74947de18	Restoring drone signature (#4069) ... Signed-off-by: Danny Kopping <danny.kopping@grafana.com> Co-authored-by: Danny Kopping <danny.kopping@grafana.com>	4 years ago
Karen Miller	808c808714	Clarify prose about querier frontend min-sharding-lookback (#4050) ... * Clarify prose about querier frontend min-sharding-lookback Docs edit of prose from https://github.com/grafana/loki/pull/4047. * Update docs/sources/configuration/_index.md Co-authored-by: Danny Kopping <dannykopping@gmail.com> Co-authored-by: Owen Diehl <ow.diehl@gmail.com> Co-authored-by: Danny Kopping <dannykopping@gmail.com>	4 years ago
Cyril Tovena	32e55c3043	Fork cortex chunk storage into Loki. (#3842) ... The chunk package has been forked from this commit for future reference and licensing. 09fe17c0e6	4 years ago
Danny Kopping	14f8faf544	Group memcached queries by container (#4057) ... Signed-off-by: Danny Kopping <danny.kopping@grafana.com>	4 years ago
Owen Diehl	56256bfd5b	Unordered chunks (#4001) ... * merge feature/unordered-replay * interoperable head chunks * memchunk block interop * retain ordered memchunk optimizations when possible * tests+bench for unordered chunk reads * reorder on chunk close * [wip] ingester stream unorderd * unordered writes default in testware config, fixes OOO bug & removes unused lastChunkTimestamp var * validity window is 1/2 of max age & fixes old transfer test * more consistent headblock checking/creation * more cohesive encoding tests * unordered stream test with validity bounds * compat - unordered * reinstates memchunk defaults when rebounding & updates storage test compatibility * lint * reorder across blocks doesnt overflow * respect chunk configs during rebounding when possible * only sync checks on ordered writes	4 years ago
Cyril Tovena	4c2feb32b5	Fixes test helper `newChunk` in the storage pkg. (#4056) ... I wanted to test that overlapping chunks are correctly re-ordered and bump into a rounding issue. Keeping both for future will avoid the same mistake. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	4 years ago
Karen Miller	1e85036409	Revise wording of query frontend service configuration (#4052) ... * Revise wording of query frontend service configuration * Add ClusterIP and publishNotReadyAddresses to Kubernetes YAML	4 years ago
Oleg Zaytsev	b4086dfe33	Upgrade Prometheus and Cortex (#4049) ... This bumps prometheus up to the changes from https://github.com/prometheus/prometheus/pull/9083 and Cortex to https://github.com/cortexproject/cortex/pull/4380 Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>	4 years ago
Danny Kopping	0c6b4dc8c9	Ruler: implementing write relabelling on recording rule samples (#4048) ... * Implementing write relabelling Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Appeasing the linter Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Appeasing the linter Signed-off-by: Danny Kopping <danny.kopping@grafana.com>	4 years ago
Owen Diehl	a8a861b21a	adds query-frontend service hints for k8s, adapted from jsonnet (#4045)	4 years ago
Cyril Tovena	3f3d10afcc	Add min_sharding_lookback limits to the frontends (#4047) ... * Add min_sharding_lookback limits to the frontends Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * improve doc. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * lint Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	4 years ago
Karen Miller	551d68210f	Fix docs typo: IP Matching section "=f" becomes "=" (#4039)	4 years ago
Arve Knudsen	7b11de86c9	loki-build-image: Fix (#4034) ... Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	4 years ago
Karen Miller	826fdfa3b9	Docs: add weights to YAML metadata to order the LogQL subsections in the nav panel (#4031)	4 years ago
Cyril Tovena	6c05970ba1	Improve the sweep lag panel in the retention dashboard. (#4030) ... This uses now a graph panel which shows us lags over time. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	4 years ago
Karsten Jeschkies	17f4a73a24	Filter instant queries and shard them. (#3984) ... * Filter instan queries and shard them. * [WIP] Test instant query sharding * Trace casting error. * WRap LokiInstantRequest for Params interface. * Convert paras to request. * Convert vector. * Use proper query time stamp. * Assert number of calls. * Convert vector to multiple samples. * Format code. * Use type switch. * Format code. * Rename Matrix to proto conversion. * Remvoe comment on caching. * Update pkg/querier/queryrange/roundtrip_test.go Co-authored-by: Cyril Tovena <cyril.tovena@gmail.com> * Remove nil middlware. * Handle LokiInstantRequest correctly. * Assert reponse type. * Return LokiPromResponse. * Verify err in tests. * Set shards. * Correct result type. * Use limits middleware. * Use shars parameter in querier. * Return vector. * Unify params. * Change direction to FORWARD. Co-authored-by: Cyril Tovena <cyril.tovena@gmail.com>	4 years ago
Bearice Ren	5970ab9b1e	Update _index.md (#4029) ... As mentioned in #3259, permission `s3:ListObjects` is needed, updating the document.	4 years ago
Arve Knudsen	3e385464c6	Chore: Upgrade Prometheus and Cortex (#4017) ... * Chore: Upgrade Prometheus Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Upgrade Cortex Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Upgrade Cortex Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix build errors Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix build errors Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix build errors Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Undo unnecessary changes Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	4 years ago
Arve Knudsen	bd30921981	loki-build-image: Upgrade golangci-lint and Go (#4028) ... * loki-build-image: Upgrade golangci-lint Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * loki-build-image: Upgrade Go Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	4 years ago
Jack Baldry	e22e666ca4	fix: Restore /config endpoint and correct handlerFunc for buildinfo (#4027) ... As far as I can tell from reading the "next" documentation, the /loki/api/v1/status/buildinfo endpoint should return the build information. The /version endpoint is also listed further down in the documentation but from the code prior to this change, it looks like it should perform the same function as buildinfo endpoint so I have removed it. I've updated the documentation to reflect my changes. Signed-off-by: Jack Baldry <jack.baldry@grafana.com>	4 years ago
Kaviraj	bfb24e6a4c	Ip matcher for LogQL (#3986) ... * IP matcher and filter via netaddr.IP package Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Adding IP filter into label filter gramer 1. ast tests 2. parse tests Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Label filters all test cases passes Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Rename ipFilter -> ipLabelFilter Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * More tests Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Remove testlogs.txt Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Make linter happy Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Add more cases to the benchmark Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * PR remarks 1. support NEQ 2. reduce allocation. 3. Minor tweaks Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Make IP filter work as line filter 1. Add necessary interface implementation 2. More tests for parser and ast Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Make linter happy Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Bug with filter matcher Its wierd bug. Not handing string() method breaks the parser :( Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Split Line and Label filter to different struct. It got out of hands to handle both in same struct. Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Add basic doc for IP matcher Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Add docs feedback from Karen * Label filter add patter error report * PR remarks 1. Indendation fix 2. Remove string() interface for ip label filte * Linter happy * PR remaks * Fix ip label matcher string() method if pattern is invalid * Fix label filter tests * Have local copy of `pattern` in label filter to use in string() * Rebase conflicts * ast.go fixes	4 years ago
Cyril Tovena	fed9115627	Improve decoding of JSON responses. (#4019) ... * Improves frontend allocations Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Improves also labels Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * refactoring. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add some more tests. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes tests and review feedback. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	4 years ago
Karen Miller	391984dded	Revise portions of the docs LogQL section (#4012) ... * Revise portions of the docs LogQL section * Update query diagram per review request	4 years ago
李国忠	29cc5ad8be	[ui] add '/config' page (#3619) ... * [ui] add '/config' page * config.go * lint	4 years ago
Karen Miller	0e519b46e0	Docs: correct fluentbit config value for DqueSync (#4026)	4 years ago
Sandeep Sukhani	8a9abdfe16	Compactor improvements (#4018) ... * use previously compacted file as seed file and copy index into it * allow configuring compactor to compact multiple tables at a time	4 years ago
Karen Miller	f811dbfc18	Docs: fix bad links (#4024)	4 years ago
Danny Kopping	ac63ea8500	Promtail: adding pipeline stage inspector (#4011) ... * Adding promtail pipeline stage inspector Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Adding library Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Adding docs Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Appeasing the linter Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Further appeasing the linter Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Removing old version of go-cmp to pass check-mod step Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Making screenshot narrower and correcting link Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Apply suggestions from code review Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> * Copy changes Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Copy changes Signed-off-by: Danny Kopping <danny.kopping@grafana.com> Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com>	4 years ago
Christian Simon	2fd633cded	Restrict path segments in TenantIDs (#4020) ... Updates cortex dependency to prevent paths derived from TenantIDs to become vulnerable to path traversal attacks. CVE-2021-36156 CVE-2021-36157 Signed-off-by: Christian Simon <simon@swine.de>	4 years ago
Periklis Tsirakidis	cb337c3b71	Add LogQL AST walker (#3991) ... * Make expressions public scoped * Add walkable implementation * Use walkable in logql/optimize	4 years ago
Lize Cai	8370ed3b04	fix typo in loki docker-compose (#4016)	4 years ago
Owen Diehl	a4db1a4359	Headblock interop (#3995) ... * merge feature/unordered-replay * interoperable head chunks	4 years ago
Owen Diehl	2107148acd	fixes chunk size method in facade (#3981)	4 years ago
Jarrod Funnell	267107b75e	Parser: Allow literal control chars in logfmt decoder (#3932) ... * Allow literal control chars in decoder * Ensure decoding control chars in logfmt works	4 years ago
Sandeep Sukhani	58f9d0c085	add config and documentation about index-gateway (#3956) ... * add config and documentation about index-gateway * Update docs/sources/operations/storage/boltdb-shipper.md Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> * Update docs/sources/operations/storage/boltdb-shipper.md Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com>	4 years ago
Owen Diehl	b58e9cdee2	fixes a badly referenced variable name in StepEvaluator code (#3996)	4 years ago
Owen Diehl	fef99775ea	aligns rule path in docker img with bundled config. closes #3952 (#3997)	4 years ago
Owen Diehl	7289c064b7	Fixes regexReplaceAll docs (#3998) ... closes #3939	4 years ago
Danny Kopping	d6dc43b958	Adding @grafana/loki-team as default CODEOWNERS (#4007) ... In order to notify the team of PRs requiring review Signed-off-by: Danny Kopping <danny.kopping@grafana.com>	4 years ago
Danny Kopping	b72a470d90	Output timestamp with nanosecond precision if available (#4006) ... Signed-off-by: Danny Kopping <danny.kopping@grafana.com>	4 years ago
Sandeep Sukhani	1ed19f7380	add a metric for counting number of failures in opening existing files while loading tables from active index in uploads package (#3977)	4 years ago
Karen Miller	bc566fbbab	Docs: Revise/update the overview section. (#3980) ... * Docs: Revise/update the overview section. * Update docs/sources/overview/_index.md Co-authored-by: Owen Diehl <ow.diehl@gmail.com> * Update docs/sources/overview/_index.md Co-authored-by: Owen Diehl <ow.diehl@gmail.com> Co-authored-by: Owen Diehl <ow.diehl@gmail.com>	4 years ago
Cyril Tovena	5d8414fcd6	Intern label keys for LogQL parser. (#3990) ... * Intern label keys for LogQL parser. They often will see the same key over and over and instead of allocating/sanitizing we can re-use the previous value. This mostly speeds up parser that were using a dynamic list of keys like json and logfmt. We could intern values too specially for metrics queries. ``` ❯ benchcmp before.txt after.txt benchmark old ns/op new ns/op delta Benchmark_Parser/json/no_labels_hints-16 3894 2885 -25.91% Benchmark_Parser/json/labels_hints-16 2084 2163 +3.79% Benchmark_Parser/unpack/no_labels_hints-16 915 890 -2.72% Benchmark_Parser/unpack/labels_hints-16 848 846 -0.32% Benchmark_Parser/logfmt/no_labels_hints-16 2229 1146 -48.59% Benchmark_Parser/logfmt/labels_hints-16 1230 1150 -6.50% Benchmark_Parser/regex_greedy/no_labels_hints-16 3881 3754 -3.27% Benchmark_Parser/regex_greedy/labels_hints-16 5443 3737 -31.34% Benchmark_Parser/regex_status_digits/no_labels_hints-16 318 290 -8.89% Benchmark_Parser/regex_status_digits/labels_hints-16 335 289 -13.85% Benchmark_Parser/pattern/no_labels_hints-16 151 151 -0.20% Benchmark_Parser/pattern/labels_hints-16 129 131 +1.24% benchmark old allocs new allocs delta Benchmark_Parser/json/no_labels_hints-16 69 43 -37.68% Benchmark_Parser/json/labels_hints-16 32 36 +12.50% Benchmark_Parser/unpack/no_labels_hints-16 19 17 -10.53% Benchmark_Parser/unpack/labels_hints-16 16 17 +6.25% Benchmark_Parser/logfmt/no_labels_hints-16 46 16 -65.22% Benchmark_Parser/logfmt/labels_hints-16 20 16 -20.00% Benchmark_Parser/regex_greedy/no_labels_hints-16 2 2 +0.00% Benchmark_Parser/regex_greedy/labels_hints-16 2 2 +0.00% Benchmark_Parser/regex_status_digits/no_labels_hints-16 2 2 +0.00% Benchmark_Parser/regex_status_digits/labels_hints-16 2 2 +0.00% Benchmark_Parser/pattern/no_labels_hints-16 2 2 +0.00% Benchmark_Parser/pattern/labels_hints-16 1 1 +0.00% benchmark old bytes new bytes delta Benchmark_Parser/json/no_labels_hints-16 1089 624 -42.70% Benchmark_Parser/json/labels_hints-16 456 480 +5.26% Benchmark_Parser/unpack/no_labels_hints-16 569 537 -5.62% Benchmark_Parser/unpack/labels_hints-16 553 537 -2.89% Benchmark_Parser/logfmt/no_labels_hints-16 576 336 -41.67% Benchmark_Parser/logfmt/labels_hints-16 256 336 +31.25% Benchmark_Parser/regex_greedy/no_labels_hints-16 194 194 +0.00% Benchmark_Parser/regex_greedy/labels_hints-16 194 194 +0.00% Benchmark_Parser/regex_status_digits/no_labels_hints-16 52 52 +0.00% Benchmark_Parser/regex_status_digits/labels_hints-16 52 52 +0.00% Benchmark_Parser/pattern/no_labels_hints-16 35 35 +0.00% Benchmark_Parser/pattern/labels_hints-16 32 32 +0.00% ``` Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Move to labels.go and cap the amount of keys that can be interned to 1024. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	4 years ago
Michel Hollands	c44c12b930	Update Cortex version (#3992) ... * Update Cortex version Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Changed code due to new Cortex version Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Run go mod tidy Signed-off-by: Michel Hollands <michel.hollands@grafana.com>	4 years ago
Cyril Tovena	4a8f62ba00	Improve head chunk allocations when reading samples. (#3968) ... * Improve head chunk allocations when reading samples. Reading sample is the worst because we are not limited and must return all of them. Re-using memory will helps ingesters read path. ``` ❯ benchcmp before.txt after.txt benchmark old ns/op new ns/op delta BenchmarkHeadBlockSampleIterator/Size_20000-16 2935102 1568140 -46.57% BenchmarkHeadBlockSampleIterator/Size_10000-16 1441980 750856 -47.93% BenchmarkHeadBlockSampleIterator/Size_8000-16 1146807 602796 -47.44% BenchmarkHeadBlockSampleIterator/Size_5000-16 698638 370655 -46.95% benchmark old allocs new allocs delta BenchmarkHeadBlockSampleIterator/Size_20000-16 32 16 -50.00% BenchmarkHeadBlockSampleIterator/Size_10000-16 29 15 -48.28% BenchmarkHeadBlockSampleIterator/Size_8000-16 28 15 -46.43% BenchmarkHeadBlockSampleIterator/Size_5000-16 26 15 -42.31% benchmark old bytes new bytes delta BenchmarkHeadBlockSampleIterator/Size_20000-16 2269664 484068 -78.67% BenchmarkHeadBlockSampleIterator/Size_10000-16 1073582 984 -99.91% BenchmarkHeadBlockSampleIterator/Size_8000-16 827825 819 -99.90% BenchmarkHeadBlockSampleIterator/Size_5000-16 475557 786 -99.83% ``` Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Improve unorderedHeadBlock too. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	4 years ago