2098 Commits (e383472d906f45832c69642376d312ea55f28f24)

 

All Branches   

Search

Author	SHA1	Message	Date
Ward Bekker	e383472d90	Don't auto-include pod labels as loki labels as a sane default (#3390) ... Don't auto-include pod labels as loki labels as a sane default. Folks regularly run into stream limits when they have a bunch of labels on their pods, so better to remove this to get them a nicer first onboarding experience.	4 years ago
Owen Diehl	2d079340c3	uses go_memstats_heap_inuse_bytes on the operational dashboard (#3447)	4 years ago
Caleb Collins-Parks	4f27c758df	add note about regex (#3437) ... to prevent people from running into https://github.com/grafana/loki/issues/3436	4 years ago
Cyril Tovena	a29901d33a	Add limit and line_returned in the query log. (#3423) ... * Add limit and line_returned in the query log. This can be useful to investigate how many wasted queries have been made by the frontend. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * got linted. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	4 years ago
Cyril Tovena	ce4f78515f	Improve the parser hint tests. (#3429) ... I was able to find out where was the issue with that test yesterday. Now I realized that we can mutate (re-slice) the given log line, and so other test would fail. This also make the parallel back. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	4 years ago
Cyril Tovena	7e9ac16dbf	Add fgfprof to Loki and Promtail. (#3424) ... Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	4 years ago
Owen Diehl	18cb7a0050	simplifies jsonnet sharding (#3439)	4 years ago
Ed Welch	6d48fb6942	change ReadStringAsSlice to ReadString so it doesn't parse quotes inside the packed _entry (#3432)	4 years ago
Cyril Tovena	32d668ceb0	Only unpack entry if the key `_entry` exist. (#3426) ... * Only unpack entry if the key `_entry` exist. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * s/kv/lbs/g Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * fix the test. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	4 years ago
Filip Poboril	8c98ee267c	Fix fluentd plugin when kubernetes labels were missing (#3358) ... When kubernetes labels were missing in the received data, plugin failed with error "undefined method `each_key' for nil:NilClass".	4 years ago
Dmitry Ukolov	6ea7122777	Distributor: append several tests for http parser. (#3416)	4 years ago
Alex Martin	99ac2e0c30	docs: promtail: Fix typo w/ windows_events hyperlink. (#3419)	4 years ago
Danny Kopping	6aba7098b3	Adding upgrade documentation related to #3404 (#3418) ... Signed-off-by: Danny Kopping <danny.kopping@grafana.com>	4 years ago
Cyril Tovena	7886f35b8f	Introduce a unpack parser. (#3420) ... * Introduce a unpack parser. To go with the new pack stage in promtail https://github.com/grafana/loki/pull/3401 this PR introduces a unpack parser that will automatically unpack all labels packed into the log line and set back the original log line. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add documentation. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * tweak promtail docs a little now that we have unpack in Loki Co-authored-by: Edward Welch <edward.welch@grafana.com>	4 years ago
Kaviraj	7a9804bce7	doc(gcplog): Advanced log export filter example (#3421) ... * doc(gcplog): Advanced log export filter example * Update docs/sources/clients/promtail/gcplog-cloud.md Co-authored-by: Owen Diehl <ow.diehl@gmail.com> Co-authored-by: Owen Diehl <ow.diehl@gmail.com>	4 years ago
Kaviraj	071fd5be55	Modify script to accept inclusion and exclustion filters as variables (#3422)	4 years ago
Cyril Tovena	20ef66df65	Fixes a race in the scheduling limits. (#3417) ... * Fixes a race in the scheduling limits. Found out the hard way that sharding might not listen correctly to context and so still run queries after the upstream request ends. This can lead to panic when scheduling work which send on a closed channel. I've added a safe guard to wait for all jobs to end before ending the original request but also attemped to fix the sharding downstreamer to properly stop sending work when context is closed. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * typo Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	4 years ago
Robert Fratto	3e4566daf9	remove unneeded prune-ci-tags drone job (#3412)	4 years ago
Ed Welch	2dbdbee5e6	Loki: fix alignment of atomic 64 bit to work with 32 bit OS (#3411) ... * fix alignment of atomic 64 bit to work with 32 bit OS * add comment	4 years ago
gotjosh	144786e67d	Instrumentation: Add histogram for request duration on gRPC client to Ingesters (#3409) ... * Instrumentation: Add histogram for request duration on gRPC client to Ingesters Signed-off-by: gotjosh <josue@grafana.com> * fix lint Co-authored-by: Edward Welch <edward.welch@grafana.com>	4 years ago
Ed Welch	e57af51c37	Promtail: Add pack stage (#3401) ... * Creating a "wrap" stage which can take labels and embed them into a log line by wrapping the log line in JSON * improve tests, deterministic ordering of json * add test for quotes * add test for ingest timestamp * update docs * update docs * update docs * lint * test flake * rename wrap to pack * fix docs * rename files * typo	4 years ago
Danny Kopping	faa1eab620	Remove default docker pipeline stage - this is undocumented and unexpected behaviour (#3404) ... Signed-off-by: Danny Kopping <danny.kopping@grafana.com>	4 years ago
Dr. Jan-Philip Gehrcke	0f881fbf78	distributor: fix snappy pb POST request handling (#3407) (#3408) ... Signed-off-by: Dr. Jan-Philip Gehrcke <jp@opstrace.com>	4 years ago
Ed Welch	877f524c36	Loki: Revert reduce list operations and crash after timeout (#3396) ... * revert: boltdbshipper: fix crash of "timeout" panic when boltdbshipper do init (#3369) This appears to have introduced a different problem, still investigating. * Revert "reduce number of list calls on shared object store when using boltdb-shipper (#3283)" This reverts commit 76e713f5	4 years ago
Ed Welch	cf8be4a55e	remove directories from object store list operation (#3394)	4 years ago
zhanghjster	a3a2c1a079	boltdbshipper: fix crash of "timeout" panic when boltdbshipper do init (#3369) ... * Fix timeout panic during boltdbshipper init. When boltdbshipper failed to load an invalid local index file, it try to reopen with a newly downloaded index, but it will panic with "timeout" error because the process still hold the flock created by the previous db open. This patch do the flock cleanup. * add a few more comments and change function name. Co-authored-by: Edward Welch <edward.welch@grafana.com>	4 years ago
Owen Diehl	7e3610bee8	Distributor overview docs (#3371) ... * distributor overview * edge case edits * Update docs/sources/architecture/distributor.md Co-authored-by: Ed Welch <ed@oqqer.com> * suggestions * further pr feedback * typo Co-authored-by: Ed Welch <ed@oqqer.com>	4 years ago
Sandeep Sukhani	faff8b6632	Boltdb shipper doc fixes (#3265) ... * Remove incorrect value for boltdb_shipper.resync_interval in example doc which anyways has a default value of 5m * Update incorrect default value for resync_interval mentioned in docs * Add boltdb shipper configuration in config reference file	4 years ago
Sandeep Sukhani	ba25b0fc80	increase db retain period in ingesters to cover index cache validity period as well (#3300) ... refactor ingester store query config to simplify the code	4 years ago
Cyril Tovena	70e87256d6	Add finalizer to zstd. (#3306) ... Fixes #3305 Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	4 years ago
Cyril Tovena	afb539fcf3	Fixes some 500 returned by querier when storage cancellation happens. (#3344) ... For unknown reasons, Cortex is transforming all errors to promql.ErrStorage on the storage layer. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	4 years ago
Tomasz Kowalczewski	6eb5cbb7dd	documentation: Add Tjahzi to the list of unofficial clients (#3370) ... Tjahzi is a log4j2 appender for Loki.	4 years ago
Marian Stychuk	b83b1abfa8	Update _index.md (#3373)	4 years ago
Aditya C S	f14ddb8bb3	fix nested captured groups indexing in replace stage (#3381) ... * fixes nested captured groups indexing in replace stage * fix typo in test * use lowercase names * fix lint error * fix lint error	4 years ago
Ed Welch	0d70e340f2	Tooling: Update chunks-inspect to understand the new chunk format as well as new compression algorithms (#3377) ... * Update chunks-inspect to understand the new chunk format as well as new compression algorithms * forgot to add go mod files * Update cmd/chunks-inspect/loki.go Co-authored-by: Kaviraj <kavirajkanagaraj@gmail.com> Co-authored-by: Kaviraj <kavirajkanagaraj@gmail.com>	4 years ago
paaacman	93f1966451	doc - enable environment variable in configuration (#3385) ... Environment variable in configuration works only if promtail is started with `-config.expand-env=true` option. Should we add `Note: This feature is only available in Loki 2.1+.` too?	4 years ago
Cyril Tovena	55b91fc793	Fixes head chunk iterator direction. (#3383) ... * Fixes head chunk iterator direction. For backward queries, since LogQL parser we are using a heapIterator in the headchunk to re-order properly entries. But we also reverse all iterators in the memchunk code, even the headchunk which causes reversal of already reversed entries. This PR skips reversal of the headchunk. Fixes #3345 Fixes #3208 This has for side effects: - when using replication you would not dedupe data properly anymore, since the data is not correctly ordered accross batches. - when using limit it would miss entries. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fix reversal and time filtering of headchunk. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	4 years ago
Owen Diehl	7613197593	prevents duplicate log lines from being replayed. closes #3378 (#3388)	4 years ago
Ed Welch	74a97899dd	Loki: Fix parser hint for extracted labels which collide with stream labels (#3380) ... * add test for issue with hints and extracted labels * one possible solution * instead of replacing the label with the _extracted suffix removed, just add a new label with the suffix removed. This will avoid any cases where a legit json or logfmt key ends with _extracted.	4 years ago
Cyril Tovena	e21d6edc10	Fixes 500 in the querier when returning multiple errors. (#3348) ... This correctly implement the `Is` error interface and uses the `util.MultiError` everywhere we need it. This way we can correcltly access if this was a cancellation which should not translate into 500. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	4 years ago
Owen Diehl	99bfd1cc5e	design doc/ordering constraint removal (#3268) ... * [wip] ordering design doc * moves variance budget to afterthoughts section * chunk sync * followups * edits & chunk diagram * edits * edits * corrects date	4 years ago
Cyril Tovena	f10535f7c1	Fixes a panic with whitespace key. (#3372) ... * Fixes a panic with whitespace key. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Removes empty key. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	4 years ago
Kaviraj	cb59069365	Add some operation details to gcplog doc (#3351) ... * Add some operation details to gcplog doc * Update docs/sources/clients/promtail/gcplog-cloud.md Co-authored-by: Owen Diehl <ow.diehl@gmail.com> * Update docs/sources/clients/promtail/gcplog-cloud.md Co-authored-by: Owen Diehl <ow.diehl@gmail.com> * Update docs/sources/clients/promtail/gcplog-cloud.md Co-authored-by: Owen Diehl <ow.diehl@gmail.com> Co-authored-by: Owen Diehl <ow.diehl@gmail.com>	4 years ago
Owen Diehl	1036f00a5a	Libsonnet/better sharding parallelism defaults (#3357) ... * increases read path parallelism limits to be in line with global parallelism controls. * noop: shard-factor -> 16 when sharding disabled * 2048 sharded queue size	4 years ago
Kaviraj	4c8967c322	Remove extra space between broken link (#3352)	4 years ago
Owen Diehl	5d6d552d1a	removes sharding queue math after global concurrency PR (#3356)	4 years ago
Cyril Tovena	9950046a4b	Bound parallelism frontend (#3342) ... * Bound parallelism at the roundtripper level. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * improve test and correctly forward ctx. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * lint and test. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	4 years ago
Cyril Tovena	f4f1239e55	Fixes ingester stats. (#3350) ... This correclty use the stats context. Introduced by https://github.com/grafana/loki/pull/1837/files#diff-d48cc90b5e10796f4e5256f2c7f4a7009c7c6799b514d7699676f73ae1be09c3L224 Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	4 years ago
Cyril Tovena	c4d3f72191	Fixes tightloop that could happen in the engine with LogQL parser. (#3347) ... I've also added a test to prove the issue was happening and now is fixed. Fixes #3275 Fixes #3264 Fixes #3020 Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	4 years ago
Sandeep Sukhani	3ff088173f	dedupe index on all the queries for a table instead of query batches (#3338) ... * dedupe index on all the queries for a table instead query batches * convert sync.Mutex to sync.RWMutex in deduper to allow concurrent checks for isSeen	4 years ago