apitech/loki - loki - Gitea: Git with a cup of tea

Commit Graph

Author	SHA1	Message	Date
Ed Welch	6e67d1ea5d	Loki/Promtail: Client Refactor (#3623 ) * move docker driver * move fluent-bit output plugin * move fluentd plugin * move logstash plugin * move promtail * match stage uses simple logql label matcher and filter expressions only * remove remaining logql imports * remove helpers package and move contents to util package * move validation and runtime out of util package * fix imports * move build and cfg packages into util * move ParseRequest function to util package * create a new package for logql models and move marshal/unmarshal code to util folder * move ErrorLabel to logqlmodel * move stats package to logqlmodel * decouple loghttp from logql by moving validation to logql package * move gcplog to tools * move PackedEntryKey constant to logqlmodel * fix stats proto import and rebuild protos * fix windows promtail tests * update drone to new dockerfile locations	4 years ago
Ed Welch	b04af08b1b	Remove codecov (#3615 ) * remove codecov from CI, also remove circleci tests as we run these somewhere else now already. * leaving an empty test step for now	4 years ago
Cyril Tovena	72dcda165a	Adds chunk filter hook for ingesters. (#3603 ) * Adds chunk filter hook for ingesters. Follow up on https://github.com/grafana/loki/pull/3569 this is the missing piece. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * That interface is not required to change. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	4 years ago
Owen Diehl	7bac502dda	Dashboard/fix operational vars (#3601 ) * loki operational dashboard uses more refined variables * excludes loki canary	4 years ago
Owen Diehl	965d48222b	adds distributor replication factor metric (#3594 )	4 years ago
Michel Hollands	56af725a08	Make the store field public (#3602 ) Signed-off-by: Michel Hollands <michel.hollands@grafana.com>	4 years ago
timazet	9ee6ba7aef	Update _index.md (#3596 ) Typo in example is corrected	4 years ago
Owen Diehl	d9637a6654	locks trailers during iteration (#3595 )	4 years ago
Ward Bekker	46eff3c12c	Added Architecture Diagram (#3526 ) * Added Architecture Diagram * Update _index.md * Rename LokiArchitectureDiagram.svg to loki_architecture_components.svg	4 years ago
Sandeep Sukhani	28d88b3fb8	add loki resource usage dashboard for read and write path (#3584 )	4 years ago
Cyril Tovena	0107a11769	Fixes a race when using specific tenant and multi-client. (#3573 ) * Fixes a race when using specific tenant and multi-client. This was because each client would try to mutate the original set of labels. Instead of cloning the map which can be expensive, I created a little helper that generate the string from the labelset without a set of label name. Fixes #3571 I also added a test to see if this was reproducible and it was indeed: ``` ~/go/src/github.com/grafana/loki master* ❯ go test -timeout 30s -tags dev,gofuzz -race -run ^TestMultiClient_Handle_Race$ github.com/grafana/loki/pkg/promtail/client -v -count=1 -timeout=0s === RUN TestMultiClient_Handle_Race ================== WARNING: DATA RACE Read at 0x00c0000b77d0 by goroutine 22: runtime.mapaccess2_faststr() /usr/local/Cellar/go/1.16.2/libexec/src/runtime/map_faststr.go:107 +0x0 github.com/grafana/loki/pkg/promtail/client.(client).getTenantID() /Users/ctovena/go/src/github.com/grafana/loki/pkg/promtail/client/client.go:376 +0xcb github.com/grafana/loki/pkg/promtail/client.(client).processEntry() /Users/ctovena/go/src/github.com/grafana/loki/pkg/promtail/client/client.go:407 +0x9a github.com/grafana/loki/pkg/promtail/client.(client).run() /Users/ctovena/go/src/github.com/grafana/loki/pkg/promtail/client/client.go:235 +0x36a Previous write at 0x00c0000b77d0 by goroutine 21: runtime.mapdelete_faststr() /usr/local/Cellar/go/1.16.2/libexec/src/runtime/map_faststr.go:297 +0x0 github.com/grafana/loki/pkg/promtail/client.(client).processEntry() /Users/ctovena/go/src/github.com/grafana/loki/pkg/promtail/client/client.go:408 +0x144 github.com/grafana/loki/pkg/promtail/client.(client).run() /Users/ctovena/go/src/github.com/grafana/loki/pkg/promtail/client/client.go:235 +0x36a Goroutine 22 (running) created at: github.com/grafana/loki/pkg/promtail/client.New() /Users/ctovena/go/src/github.com/grafana/loki/pkg/promtail/client/client.go:198 +0x7dc github.com/grafana/loki/pkg/promtail/client.TestMultiClient_Handle_Race() /Users/ctovena/go/src/github.com/grafana/loki/pkg/promtail/client/multi_test.go:126 +0x2c7 testing.tRunner() /usr/local/Cellar/go/1.16.2/libexec/src/testing/testing.go:1194 +0x202 Goroutine 21 (running) created at: github.com/grafana/loki/pkg/promtail/client.New() /Users/ctovena/go/src/github.com/grafana/loki/pkg/promtail/client/client.go:198 +0x7dc github.com/grafana/loki/pkg/promtail/client.TestMultiClient_Handle_Race() /Users/ctovena/go/src/github.com/grafana/loki/pkg/promtail/client/multi_test.go:124 +0x187 testing.tRunner() /usr/local/Cellar/go/1.16.2/libexec/src/testing/testing.go:1194 +0x202 ================== testing.go:1093: race detected during execution of test --- FAIL: TestMultiClient_Handle_Race (0.00s) === CONT testing.go:1093: race detected during execution of test FAIL FAIL github.com/grafana/loki/pkg/promtail/client 0.022s FAIL ``` Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> don't overload variable name Co-authored-by: Edward Welch <edward.welch@grafana.com>	4 years ago
Ed Welch	21b901db9b	Loki: Distributor log message bodySize should always reflect the compressed size (#3572 )	4 years ago
Cyril Tovena	a22bce7286	Add a chunk filter hook in the store. (#3569 ) * Add a chunk filter hook in the store. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * doc. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * fixes the test. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * got linted. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	4 years ago
Michel Hollands	94af4c4f84	Some small docs fixes (#3568 ) Signed-off-by: Michel Hollands <michel.hollands@grafana.com>	4 years ago
Cyril Tovena	4d1da2edf3	first_over_time and last_over_time (#3050 ) * Wip Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * [POC] last_over_time and first_over_time. This is an attempt to allow users to select the first or last discrete sample values from a range vector. This way we can create gauge, the only drawback being the log need to exists at least in every range. May be we could transfer the last or first value to the next range, but it will still be a problem for small ranges. Still wondering how useful this is compare to min/max_over_time. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	4 years ago
Cyril Tovena	ecca5d345c	Properly release the ticker in Loki client. (#3566 ) I think this was only impacting the docker driver who would start/stop for each new followed containers. My assumption is that this would slowly build up and eat CPU over time. Fixes #3319 I arrived to this conclusion since strace was showing a crazy amount of futex syscall and nothing else seems to be the cause. 🙏 Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	4 years ago
Cyril Tovena	eff4ff1231	Improve matchers validations. (#3564 ) 1- It currently 500. 2- Happens too late, at ingester level and querier. (better at the frontend) 3- Improve API surface level, we don't need to expose this validation for anything else than ParseLogSelector. see #3216 Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	4 years ago
Cyril Tovena	d450bbb668	Fixes a test from #3216 . (#3562 ) Minor refactoring missing. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	4 years ago
Sandeep Sukhani	b152bc2cf3	check for stream selectors to have atleast one equality matcher (#3216 ) * check for stream selectors to have atleast one equality matcher * move equality matchers check to logql	4 years ago
Lashan	e0f0ee0585	Update README.md (#3552 ) With Go 1.11 it's not possible to build the Fluent Bit plugin with the following error. ``` build github.com/sercand/kuberesolver: cannot find module for path github.com/sercand/kuberesolver ``` Needed to use Go 1.16.2 to resolve the issue.	4 years ago
Cyril Tovena	a76a17877b	Add a target to reproduce fuzz testcase (#3553 ) * Add a make target to facilitate fuzz testing locally. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * missing target.... Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	4 years ago
Sam Jewell	87ca7a4616	Remove key-value pair missing from logfmt output (#3555 ) Also fix typo	4 years ago
Michel Hollands	ffbabc5715	LBAC changes (#3549 ) * Add querier override and tripperware Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Use new queryable in engine Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Removed unused Queryable Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Only use QueryFrontEndTripperware if set Signed-off-by: Michel Hollands <michel.hollands@grafana.com>	4 years ago
Tom Hu	6ab0e94572	Add Codecov badge to README (#3484 )	4 years ago
Cyril Tovena	b11d2effe2	Fixes a bug in MatrixStepper when sharding queries. (#3550 ) * Fixes a bug in MatrixStepper when sharding queries. Since we split correctly metric queries, this bug has show itself. Basically we were not correctly stepping through time. We should always start from start, add the step until the start is after the end. For more read: https://www.robustperception.io/step-and-query_range Fixes #3541 Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes tests. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes tests. for real. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Not easy to get those test working. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	4 years ago
Robby Milo	37a7189d4e	docs: add title to Lambda Promtail (#3525 )	4 years ago
Stavros Aronis	22edfe34bc	Fix markdown for output stage (#3472 )	4 years ago
Cyril Tovena	71aa8a631a	Add sprig text/template functions to template stage. (#3515 ) * Add sprig text/template function to template stage. Fixes #3129 Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * nit. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * reset vendor file. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	4 years ago
Owen Diehl	24f8f6a4de	compactor docs (#3543 )	4 years ago
Lars Lehtonen	b5930d6d9d	single import of jsoniter in logql subpackages (#3544 )	4 years ago
Charlie N	54bc1468b4	Added docs for GCP internal labels. (#3529 ) Signed-off-by: Whyeasy <Whyeasy@users.noreply.github.com>	4 years ago
Kaviraj	bcbffb1988	Remove memberlist config from ring config. (#3542 )	4 years ago
Cyril Tovena	29a3240e17	Support for single step metric query. (#3540 ) When splitting correctly steps in the frontend, some queries might be of a single step, e.g start == end. This PR add supports for those queries, which are already supported by Cortex and Prometheus, but for some reason not Loki. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	4 years ago
Michel Hollands	f49a510f38	Loki: Update cortex version and fix resulting changes (#3532 ) * Update cortex version and fix resulting changes Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Update changelog Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Fix scrapeconfig test Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Tidy up go.sum Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Fix go.sum Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Update pkg/promtail/scrapeconfig/scrapeconfig_test.go Co-authored-by: Owen Diehl <ow.diehl@gmail.com> * Remove unused fmt import Signed-off-by: Michel Hollands <michel.hollands@grafana.com> Co-authored-by: Owen Diehl <ow.diehl@gmail.com>	4 years ago
Sandeep Sukhani	8694e7303a	split series api queries by day in query-frontend (#3530 )	4 years ago
Garrett	f70c7ea0bc	Implement offset modifier for range vector aggregation in LogQL (#3455 ) * Implement offset modifier for range vector aggregation in LogQL implementation for https://github.com/grafana/loki/issues/2785, the offset modifier allows changing the time offset for range vectors in a query to support e.g. selective timeshift in Grafana Signed-off-by: garrettlish <garrett.li.sh@gmail.com> * add docs for offset modifier * refine offset expr * add offset unit test in engine.go * remove duplicated offset shift * add offset test in range vector test Co-authored-by: garrettlish <garrettlish@163.com>	4 years ago
Owen Diehl	1bec8c301c	fixes broken link (#3513 )	4 years ago
Cyril Tovena	594b596d67	Fixes broken link in the documentation. (#3516 ) Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	4 years ago
Ward Bekker	927767059e	fix spelling in doc (#3518 )	4 years ago
Cyril Tovena	a7e0770acb	Fixes a race introduced by #3434 . (#3517 )	4 years ago
Sandeep Sukhani	b06d9703d5	fix live tailing of logs from Loki (#3509 )	4 years ago
Kaviraj	14b2c093c1	Add unique promtail_instance id to labels for gcptarget (#3501 ) * Add unique promtail_instance id to labels for gcptarget Rationale: To make labelset unique per promtail instance to avoid out-of-order errors * Update docs/sources/clients/promtail/scraping.md Co-authored-by: Danny Kopping <dannykopping@gmail.com> Co-authored-by: Danny Kopping <dannykopping@gmail.com>	4 years ago
Sandeep Sukhani	835f710bc9	make prefix for keys of objects created by boltdb-shipper configurable (#3491 ) * make prefix for keys of objects created by boltdb-shipper configurable * shared store key prefix should have a path separator	4 years ago
Cyril Tovena	6148794b9b	Fixes a bug where unpack would mutate log line. (#3502 ) * Fixes a bug where unpack would mutate log line. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * add reset. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	4 years ago
Neven Miculinic	b67a07e1fb	Added path information to deleted tailed file (#3457 )	4 years ago
Owen Diehl	82035397df	adds deprecation notice for chunk transfers (#3442 )	4 years ago
Chris Miller	6a855dc037	Update boltdb-shipper.md (#3464 )	4 years ago
Ed Welch	36ec92ce5c	Update Go and Alpine versions (#3481 ) * Upgrade go to 1.16 and alpine to 3.13 * update drone to use new build image version * gomod Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> Co-authored-by: Cyril Tovena <cyril.tovena@gmail.com>	4 years ago
Aditya C S	0f5fceef8f	Support labelallow stage in Promtail (#3468 ) * add labelallow stage * added docs * add docs * fix lint * update stage name in _index.md	4 years ago
Owen Diehl	d42e5a2ce9	reduce replay flush threshold (#3496 )	4 years ago

1 2 3 4 5 ...

2163 Commits (6e67d1ea5d128c15e6b4c69895057891c1d1a8f9) All Branches Search

2163 Commits (6e67d1ea5d128c15e6b4c69895057891c1d1a8f9)

All Branches