2162 Commits (b04af08b1b593a4694947b14873dae16bda23ffb)

 

All Branches   

Search

Author	SHA1	Message	Date
Cyril Tovena	55b91fc793	Fixes head chunk iterator direction. (#3383) ... * Fixes head chunk iterator direction. For backward queries, since LogQL parser we are using a heapIterator in the headchunk to re-order properly entries. But we also reverse all iterators in the memchunk code, even the headchunk which causes reversal of already reversed entries. This PR skips reversal of the headchunk. Fixes #3345 Fixes #3208 This has for side effects: - when using replication you would not dedupe data properly anymore, since the data is not correctly ordered accross batches. - when using limit it would miss entries. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fix reversal and time filtering of headchunk. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	4 years ago
Owen Diehl	7613197593	prevents duplicate log lines from being replayed. closes #3378 (#3388)	4 years ago
Ed Welch	74a97899dd	Loki: Fix parser hint for extracted labels which collide with stream labels (#3380) ... * add test for issue with hints and extracted labels * one possible solution * instead of replacing the label with the _extracted suffix removed, just add a new label with the suffix removed. This will avoid any cases where a legit json or logfmt key ends with _extracted.	4 years ago
Cyril Tovena	e21d6edc10	Fixes 500 in the querier when returning multiple errors. (#3348) ... This correctly implement the `Is` error interface and uses the `util.MultiError` everywhere we need it. This way we can correcltly access if this was a cancellation which should not translate into 500. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	4 years ago
Owen Diehl	99bfd1cc5e	design doc/ordering constraint removal (#3268) ... * [wip] ordering design doc * moves variance budget to afterthoughts section * chunk sync * followups * edits & chunk diagram * edits * edits * corrects date	4 years ago
Cyril Tovena	f10535f7c1	Fixes a panic with whitespace key. (#3372) ... * Fixes a panic with whitespace key. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Removes empty key. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	4 years ago
Kaviraj	cb59069365	Add some operation details to gcplog doc (#3351) ... * Add some operation details to gcplog doc * Update docs/sources/clients/promtail/gcplog-cloud.md Co-authored-by: Owen Diehl <ow.diehl@gmail.com> * Update docs/sources/clients/promtail/gcplog-cloud.md Co-authored-by: Owen Diehl <ow.diehl@gmail.com> * Update docs/sources/clients/promtail/gcplog-cloud.md Co-authored-by: Owen Diehl <ow.diehl@gmail.com> Co-authored-by: Owen Diehl <ow.diehl@gmail.com>	4 years ago
Owen Diehl	1036f00a5a	Libsonnet/better sharding parallelism defaults (#3357) ... * increases read path parallelism limits to be in line with global parallelism controls. * noop: shard-factor -> 16 when sharding disabled * 2048 sharded queue size	4 years ago
Kaviraj	4c8967c322	Remove extra space between broken link (#3352)	4 years ago
Owen Diehl	5d6d552d1a	removes sharding queue math after global concurrency PR (#3356)	4 years ago
Cyril Tovena	9950046a4b	Bound parallelism frontend (#3342) ... * Bound parallelism at the roundtripper level. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * improve test and correctly forward ctx. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * lint and test. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	4 years ago
Cyril Tovena	f4f1239e55	Fixes ingester stats. (#3350) ... This correclty use the stats context. Introduced by https://github.com/grafana/loki/pull/1837/files#diff-d48cc90b5e10796f4e5256f2c7f4a7009c7c6799b514d7699676f73ae1be09c3L224 Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	4 years ago
Cyril Tovena	c4d3f72191	Fixes tightloop that could happen in the engine with LogQL parser. (#3347) ... I've also added a test to prove the issue was happening and now is fixed. Fixes #3275 Fixes #3264 Fixes #3020 Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	4 years ago
Sandeep Sukhani	3ff088173f	dedupe index on all the queries for a table instead of query batches (#3338) ... * dedupe index on all the queries for a table instead query batches * convert sync.Mutex to sync.RWMutex in deduper to allow concurrent checks for isSeen	4 years ago
Cyril Tovena	9e33f8f6b7	Fixes races in multiline stage and promtail. (#3302) ... * Fixes a race in multiline code. Not fully sure how this was happening, but I have a test showing it's possible and a fix for it. I think it should fix #3120. I also remove race within some tests of promtail. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Reset config Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	4 years ago
Cyril Tovena	15b6235cae	Fixes step encoding in logcli. (#3325) ... Fixes #3318 Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	4 years ago
Cyril Tovena	d1b9912602	Add pprof endpoint to docker-driver. (#3331) ... * Add pprof endpoint to docker-driver. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Improve lint. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	4 years ago
Cyril Tovena	bca29c011f	Embed timezone data in Promtail. (#3332) ... Fixes #2995 Fixes #1361 Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	4 years ago
Owen Diehl	47e7ec5ec6	adds some flushing instrumentation/logs (#3340)	4 years ago
Owen Diehl	862cfa4e1f	adds Start() method to WAL interface to delay checkpointing until after replay (#3339)	4 years ago
Sandeep Sukhani	76e713f57d	reduce number of list calls on shared object store when using boltdb-shipper (#3283)	4 years ago
Sandeep Sukhani	1cd37a306c	fix config for statefulset rulers when using boltdb-shipper (#3329)	4 years ago
Owen Diehl	8a9b94a8bc	removes recover flag (#3326)	4 years ago
Owen Diehl	d4fdccbf6f	defaults promtail to 2.1.0 in install script (#3321)	4 years ago
Simon Baier	9a58f83a90	fix fluent-bit output plugin generating invalid JSON (#3240) ... * fix fluent-bit output plugin generating invalid JSON * fix golint error	4 years ago
Sergey	f4dfce4136	Patch 2 - fix link to another documentation files (#3292) ... * Update configuration.md * Update configuration.md	4 years ago
Ed Welch	407afce0e3	Loki: fix validation error and metrics (#3307) ... * Remove the helper functions for creating errors as they cause unintended value manipulation if the value contains any valid golang printf substitute characters, e.g. %s * Improve error message for invalid labels, increment metrics for lines dropped for invalid labels.	4 years ago
marion	1eac0bfc28	Markdown Code highlighting (#3147) ... Co-authored-by: marionxue <xuelong@xingshulin.com>	4 years ago
Kaviraj	527ba6093b	docs(fix): Make best practices docs look better (#3316)	4 years ago
Kaviraj	7cd70aa018	Use project-id from the variables. Remove hardcoding (#3304)	4 years ago
Kaviraj	b15a3414f7	Add step to version Loki docs during public release process. (#3277) ... * Add step to version Loki docs during public release process. * Update docs/sources/maintaining/release.md Co-authored-by: Cyril Tovena <cyril.tovena@gmail.com> Co-authored-by: Cyril Tovena <cyril.tovena@gmail.com>	4 years ago
Danny Kopping	feb7fb470b	LogQL: Simple JSON expressions (#3280) ... * New approach, still rough Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Adding benchmark Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Adding tests Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Minor refactoring Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Appeasing the linter Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Further appeasing the linter Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Adding more tests Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Adding documentation Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Docs fixup Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Removing unnecessary condition Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Adding extra tests from suggestion in review Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Adding JSONParseErr Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Adding test to cover invalid JSON line Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Adding equivalent benchmarks for JSON and JSONExpression parsing Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Adding suffix if label would be overridden Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Reparenting jsonexpr directory to more appropriate location Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Setting empty label on non-matching expression, to retain parity with label_format Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Adding statement about returned complex JSON types Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Added check for valid label name Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Making json expressions shardable Signed-off-by: Danny Kopping <danny.kopping@grafana.com>	4 years ago
Gregory Brzeski	6c8fdd68aa	fix typo in config variable name: BookmarkPath (#3298)	4 years ago
Cyril Tovena	82cead2af7	Increase the windows bookmark buffer. (#3303) ... See this discussion https://github.com/grafana/loki/pull/3246#discussion_r568674040 Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	4 years ago
Owen Diehl	5f0c06ac3a	Logql/absent label optimization (#3299) ... * absent_over_time does not extract unnecessary labels * rebase fix	4 years ago
Owen Diehl	b848ac510b	adds stateful ruler clause for boltdb shipper jsonnet (#3297)	4 years ago
Dmitry Ukolov	2a51fb42bf	Distributor: Loki API can receive gzipped JSON (#3291) ... * Push API recognize HTTP header "Content-Encoding: gzip" and upload gzipped JSON * Fix documentation mistake	4 years ago
Kaviraj	59a34f9867	Make incoming labels from gcp into Loki internal labels. (#3285) ... * Make incoming labels from gcp into Loki internal labels. If these labels are significant, we can always use relabeling. * Add support for rebeling in gcptarget * Test for SnakeCase	4 years ago
Jacob Lisi	923671a17e	chore: update cortex to latest and fix refs (#3295) ... * chore: update cortex to latest and fix refs Signed-off-by: Jacob Lisi <jacob.t.lisi@gmail.com> * tidy mod file Signed-off-by: Jacob Lisi <jacob.t.lisi@gmail.com>	4 years ago
Kaviraj	d667dd20f2	Avoid putting all the GCP labels into loki labels (#3284)	4 years ago
Cyril Tovena	3bd725e826	Fixes logfmt parser hints. (#3279) ... Recently introduce a bug where label where tested for extraction against non sanitized version. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	4 years ago
Chance Zibolski	054b8fcc91	logcli: Refactor logcli Client interface to use time.Time and time.Duration for LiveTailQueryConn method (#3271)	4 years ago
Owen Diehl	4d1c6e0169	adds another test (#3278)	4 years ago
Cyril Tovena	72b3acb925	Windows events (#3246) ... * First commit for windows event targets. I had to add couple of new go modules and hook the windows log in. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add the ability to save where we are. * Finished setup tests and comments. * nits * adding documentation * go tidy * lint windows * nope * add manager for non windows * don't lint forked package * mod check Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Update pkg/promtail/targets/windows/bookmark.go Co-authored-by: Owen Diehl <ow.diehl@gmail.com> * Use passed-in Prometheus registerer. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Apply suggestions from code review Co-authored-by: Owen Diehl <ow.diehl@gmail.com> * testing out windows drone. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * type exec. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * docker image. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * go/go.exe Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * build windows on container. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * trying another image. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * typo in image Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixed local timezone issue. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fix tests with timezone. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes timezone tests. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Update logstash since the last image is not working anymore 🤷 Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> Co-authored-by: Owen Diehl <ow.diehl@gmail.com>	4 years ago
Cyril Tovena	e8cce09de2	Fix a bug with metric queries and label_format. (#3263) ... * Fix a bug whith metric queries and label_format. In my previous PR where I introduced hints, I forgot to implement label format label required. This would cause label format label to be skipped during parsing. I've also took the opportunity to improve the code. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Refactor and add more tests. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	4 years ago
Karsten Jeschkies	287244d110	Update vendored Cortex to 0976147451ee (#3267)	4 years ago
Chance Zibolski	91974e0cfe	logcli: Fix handling of logcli query using --since/--from and --tail (#3270) ... * pkg/logcli/client: Correctly set "start" query param for tail API instead of "from" * pkg/logcli/client: Change Client method variable names to match API query parameters Instead of using from/through, use start/end like the Loki API does.	4 years ago
李国忠	0986d8dafe	[ingester_query] change var "clients" to "reps" (#3274)	4 years ago
Ed Welch	b4eeddc710	Loki migrate-tool (#3151) ... * basics working, needs some more cleanup and more flags for config and a Dockerfile Signed-off-by: Edward Welch <edward.welch@grafana.com> * add label matchers start end time params and ability to remap the chunk to a different user Signed-off-by: Edward Welch <edward.welch@grafana.com> * adding dockerfile and makefile instructions Signed-off-by: Edward Welch <edward.welch@grafana.com> * push out some of the limits show errors from index query * for testing have docker container not run migrate command but instead stay running by tailing /dev/null * big overhaul to add parallelization * adding better stats * adding retry logic * tweaking retry logic * tweaking retry logic * work around changes in latest cortex Signed-off-by: Ed Welch <edward.welch@grafana.com> * fix up imports and changes since the last time this was merged added a readme fixed up the makefile added some comments * fixing some of my garbage * fixing more of my garbage * Update cmd/migrate/README.md Co-authored-by: Owen Diehl <ow.diehl@gmail.com> * Update cmd/migrate/README.md Co-authored-by: Owen Diehl <ow.diehl@gmail.com> * Update cmd/migrate/README.md Co-authored-by: Owen Diehl <ow.diehl@gmail.com> * Update cmd/migrate/README.md Co-authored-by: Owen Diehl <ow.diehl@gmail.com> * Update cmd/migrate/README.md Co-authored-by: Owen Diehl <ow.diehl@gmail.com> * tabs not spaces Co-authored-by: Owen Diehl <ow.diehl@gmail.com>	4 years ago
Dave Henderson	4d9dc8ab96	Remove invalid hostname from default promtail configuration (#3254) ... Signed-off-by: Dave Henderson <dave.henderson@grafana.com>	4 years ago