100 Commits (5ecbbbe5f58fcd4bc931b4acf63276590d769f60)

Author	SHA1	Message	Date
Cyril Tovena	c8f3df33cf	Promtail Kafka target (#4568) ... * Adds a kafka target manager in promtail. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add validations. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Working on tests. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Moar test for the fanout client. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Finishing off tests. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * final adjustement Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Adding topics discovery. Still needs to finish tests. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Finishing off testing it. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Wip Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Revert config changes. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * lint Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add group id as discovered label Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * linter Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add tools for running kafka and testing locally. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * got linted shell Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Update sarama to compile in ARM. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add documentation for kafka target. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Improve code comment. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * add a few s's * Better cancellation support. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * adds `__meta_kafka_` suffix to discovered labels. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> Co-authored-by: Edward Welch <edward.welch@grafana.com>	4 years ago
Roger Steneteg	3b66d94e0f	resolve issue with promtail not scraping target if only path changed in a simpler way that dont need mutex to sync threads (#4599) ... Signed-off-by: Roger Steneteg <rsteneteg@ea.com>	4 years ago
Ed Welch	de4993f921	explicitly define allowed HTTP methods on HTTP endpoints (#4627)	4 years ago
James Callahan	a92541bd84	Promtail: no need for GCP promtail_instance label now that loki supports out-of-order writes (#4556) ... Closes #4474	4 years ago
Owen Diehl	853260cfae	regenerates assets from current vfsgen dependency (#4588)	4 years ago
lingpeng0314	76e2a14029	add group_{left,right} to LogQL (#4510) ... * Add group_left/group_right support * Update docs/sources/logql/_index.md * Minor change for a test case * removes CardManyToMany * removes now-unused IsSetOperator helper Co-authored-by: Owen Diehl <ow.diehl@gmail.com>	4 years ago
Cyril Tovena	93a0b716d4	Refactor client configs in Promtail. (#4567) ... This refactor a big how we mutate the original config for backward compatibility. Also add tests on untested code path. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	4 years ago
Sankalp Rangare	9e2acd656d	add static labels in stages (#4495) ... Signed-off-by: Sankalp Rangare <sankalprangare786@gmail.com>	4 years ago
Lize	69819c2e07	update promtail base image to debian:bullseye-slim (#4516)	4 years ago
Danny Kopping	703d68bc76	Promtail: allow for customisable stream lag labels (#4507) ... * Allow for customisable stream lag labels Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Updating test Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Allowing for multiple labels Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Adding field to docs Signed-off-by: Danny Kopping <danny.kopping@grafana.com>	4 years ago
Danny Kopping	cc1d478bb2	Clarify error message (#4524) ... Signed-off-by: Danny Kopping <danny.kopping@grafana.com>	4 years ago
Roger Steneteg	0ae0d4da12	fix filetarget to not be stuck if no files was detected on startup (#4461) ... Signed-off-by: Roger Steneteg <rsteneteg@ea.com> Co-authored-by: Cyril Tovena <cyril.tovena@gmail.com>	4 years ago
Danny Kopping	cff8324f24	Replacing go-kit/kit/log with go-kit/log (#4484) ... * Replacing go-kit/kit/log with go-kit/log Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Appeasing the f&^$ing linter Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Adding depguard Cleaning go mod Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Updating weaveworks after https://github.com/weaveworks/common/pull/226 was merged Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Appeasing the mod checker Signed-off-by: Danny Kopping <danny.kopping@grafana.com>	4 years ago
Callum Styan	096cf0cfaf	Update golang and loki-build-image image versions. (#4481) ... Signed-off-by: Callum Styan <callumstyan@gmail.com>	4 years ago
Mauro Stettler	4b59556342	Update cortex to newer version (#4478) ... * update cortex and thanos to newer version * pin k8s.io/apimachinery * fixes for newer libs * add missing method to mock wal data collector * fixing test by using different gokit level pkg * make check-mod	4 years ago
MrWong99	00168b341a	Remove unused variable in fluentd output plugin (#4261) ... I couldn't figure out why this variable is there. If I am missing any global variables by fluentd feel free to close this PR without any notice	4 years ago
Ed Welch	11e215d79f	Loki: Common Config (#4347) ... * refactor the pkg/util/cfg package to simplify and make it accessable outside of this package. create a pkg/cfg which handles building the Loki config, this is created separately from pkg/util (which is Apache 2 licensed) to maintain proper separations of AGPL and Apache 2 code Update the main.go for several of the apps with the refactoring * refactor how dynamic/common config is loaded Signed-off-by: Trevor Whitney <trevorjwhitney@gmail.com> * Add common config logic for path prefix Signed-off-by: Trevor Whitney <trevorjwhitney@gmail.com> * Missed a file, update common config with prefix property Signed-off-by: Trevor Whitney <trevorjwhitney@gmail.com> * fix test, remove loki from args Signed-off-by: Trevor Whitney <trevorjwhitney@gmail.com> * Clean up test names Signed-off-by: Trevor Whitney <trevorjwhitney@gmail.com> * fix typo in filename, make linter happy Signed-off-by: Trevor Whitney <trevorjwhitney@gmail.com> * Update pkg/loki/config_wrapper.go Co-authored-by: Owen Diehl <ow.diehl@gmail.com> * Update pkg/loki/config_wrapper.go Co-authored-by: Owen Diehl <ow.diehl@gmail.com> * Update pkg/loki/config_wrapper_test.go Co-authored-by: Owen Diehl <ow.diehl@gmail.com> * remove temp files in test Signed-off-by: Trevor Whitney <trevorjwhitney@gmail.com> Co-authored-by: Trevor Whitney <trevorjwhitney@gmail.com> Co-authored-by: Owen Diehl <ow.diehl@gmail.com>	4 years ago
Owen Diehl	0f10a549da	Removes docker driver empty log line message (#4396) ... Closes https://github.com/grafana/loki/issues/3384	4 years ago
Leo	0a66b897ec	clients/promtail: Add ndjson and plaintext formats to loki_push (#4336) ... * feat(clients/promtail): add ndjson,plaintext formats to loki_push * feat(clients/promtail): add ndjson,plaintext formats to loki_push * feat(clients/promtail): documentation * Address review comments * fmt * fix missing port	4 years ago
Sandeep Sukhani	4784ea2388	add logfmt promtail stage to be able to extract data from logfmt formatted log (#4346)	4 years ago
Johan Fleury	9b0f61d9d3	promtail: add support for TLS/mTLS in syslog receiver (#3907)	4 years ago
Danny Kopping	d5b6d05557	CI: Fixing linter deprecations (#4353) ... * Fixing CI deprecations Replacing deprecated golint with revive (https://github.com/golangci/golangci-lint/pull/1965) Removing interfacer (https://github.com/golangci/golangci-lint/pull/1755) Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Further suggestions from Marco in https://github.com/grafana/loki/issues/1002 Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Clarify instruction Signed-off-by: Danny Kopping <danny.kopping@grafana.com>	4 years ago
Arve Knudsen	bfb4b00bca	Chore: Reformat Go files (#4304) ... * Chore: Reformat Go files Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Undo change Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	4 years ago
Jordan Rushing	0f0cb9fc99	Add missing `Inc()` to correctly increment the `dropStage.dropCount` metric on valid dropped log line; update related docs (#4246)	4 years ago
Kaviraj	b36bc5ab32	Add metrics for gcplog scrape. (#4235) ... * Add metrics for gcplog scrape. Also fix the Ready() method of target * Fix typo with help message	4 years ago
Arve Knudsen	cfb4fc1f55	Use flagext from dskit (#4225) ... Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	4 years ago
Nick Pillitteri	79a7bdbabc	Chore: Use services and modules from grafana/dskit (#4196) ... * Update Cortex vendor to bcf15611345da65e03604a08a0213aaaaac49e54 Signed-off-by: Nick Pillitteri <nick.pillitteri@grafana.com> * Use services and modules packages from grafana/dskit Signed-off-by: Nick Pillitteri <nick.pillitteri@grafana.com> * Update promtail test to set default weaveworks server network type Signed-off-by: Nick Pillitteri <nick.pillitteri@grafana.com> * Punctuation Signed-off-by: Nick Pillitteri <nick.pillitteri@grafana.com>	4 years ago
Arve Knudsen	10c92a901e	Chore: Upgrade to latest Cortex (#4188) ... * Chore: Upgrade to latest Cortex Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix formatting Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	4 years ago
Michał Sochoń	eda7f31356	Bump fluent-bit base image to 1.8 (#4059)	4 years ago
Danny Kopping	a3414a5d9b	Promtail: handling tight loop in journal follow logic (#4066) ... * Handling EBADMSG explicitly, and preventing tight loop Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Appeasing the linter Signed-off-by: Danny Kopping <danny.kopping@grafana.com>	4 years ago
Cyril Tovena	32e55c3043	Fork cortex chunk storage into Loki. (#3842) ... The chunk package has been forked from this commit for future reference and licensing. 09fe17c0e6	4 years ago
李国忠	29cc5ad8be	[ui] add '/config' page (#3619) ... * [ui] add '/config' page * config.go * lint	4 years ago
Danny Kopping	ac63ea8500	Promtail: adding pipeline stage inspector (#4011) ... * Adding promtail pipeline stage inspector Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Adding library Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Adding docs Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Appeasing the linter Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Further appeasing the linter Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Removing old version of go-cmp to pass check-mod step Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Making screenshot narrower and correcting link Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Apply suggestions from code review Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> * Copy changes Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Copy changes Signed-off-by: Danny Kopping <danny.kopping@grafana.com> Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com>	4 years ago
Danny Kopping	b72a470d90	Output timestamp with nanosecond precision if available (#4006) ... Signed-off-by: Danny Kopping <danny.kopping@grafana.com>	4 years ago
Nick Pillitteri	b830d65bdd	Use the Cortex wrapper for getting tenant ID from a context (#3973) ... This change replaces all uses of the weaveworks/common method for getting tenant/org ID with the Cortex wrapper which performs some extra validation. Signed-off-by: Nick Pillitteri <nick.pillitteri@grafana.com>	4 years ago
Cyril Tovena	337aed06a6	Fixes negative gauge in Promtail. (#3971) ... * Fixes negative gauge in Promtail. Fixes #3281 Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * got linted. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	4 years ago
Cyril Tovena	22dcfcde33	Add retention label to loki_distributor_bytes_received_total metrics (#3840) ... * Add retention label on the loki_distributor_bytes_received_total metrics. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Update docs. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Update docs/sources/operations/observability.md Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> * Move ParseRequest to an apache2 package. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add retention label on the loki_distributor_bytes_received_total metrics. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * fixes bad merge. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes bad merge. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com>	4 years ago
Trevor Whitney	de93a1bd54	Promtail: add consul agent service discovery (#3834) ... * working prototype Signed-off-by: Trevor Whitney <trevorjwhitney@gmail.com> * update test for new responses and metadata labels Signed-off-by: Trevor Whitney <trevorjwhitney@gmail.com> * fix linting issues Signed-off-by: Trevor Whitney <trevorjwhitney@gmail.com> * clarify license/copyright/attribution to prometheus code Signed-off-by: Trevor Whitney <trevorjwhitney@gmail.com> * undo changes to go.sum when adding consul dep Signed-off-by: Trevor Whitney <trevorjwhitney@gmail.com> * add documentation around consul and consulagent scrape configs Signed-off-by: Trevor Whitney <trevorjwhitney@gmail.com> * Update docs/sources/clients/promtail/configuration.md Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> * Update docs/sources/clients/promtail/configuration.md Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> * Update docs/sources/clients/promtail/configuration.md Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> * Update docs/sources/clients/promtail/configuration.md Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> * Update docs/sources/clients/promtail/configuration.md Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> * Update docs/sources/clients/promtail/configuration.md Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> * Update docs/sources/clients/promtail/configuration.md Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> * Update docs/sources/clients/promtail/configuration.md Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> * Update docs/sources/clients/promtail/configuration.md Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> * Update docs/sources/clients/promtail/configuration.md Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> Co-authored-by: Ed Welch <edward.welch@grafana.com> Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com>	4 years ago
Sparkle	df06ba088c	upgrade the fluent-bit to latest version (#3793)	4 years ago
Sandeep Sukhani	d765921735	revendor cortex to latest master (#3771)	4 years ago
Cyril Tovena	15d417efe1	Replace satori.uuid with gofrs/uuid (#3743) ... * Replace satori.uuid with gofrs/uuid Fixes #3742 Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fix new import. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	4 years ago
3Xpl0it3r	89bdc9c10a	add debug information for extracted data (#3711) ... * fix the promtail-default-config download link in doc * add debug information for extreacted in stage * add Debug variable * format	4 years ago
3Xpl0it3r	ccb5cc0a9e	remove duplicated code (#3727) ... * fix the promtail-default-config download link in doc * remove duplicated code	4 years ago
Cyril Tovena	7a5fe3dd1c	Fixes docker driver that would panic when closed. (#3709) ... This is happening because it can still received log, but the client might be already closed. Fixes #3705 Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	4 years ago
Kris Budde	33313158f0	added timezone to promtai dry-run logger output #3679" (#3683) ... Signed-off-by: Kris Budde <Kris@budd.ee>	4 years ago
Roger Steneteg	735e8dc17d	Remove targets that dont have any tails, in order for the target to be removed and recreated during next sync (#3587) ... Signed-off-by: Roger Steneteg <rsteneteg@ea.com>	4 years ago
Owen Diehl	d88f3996ea	revendor compatibility: various prom+k8s+cortex (#3664) ... * revendor compatibility: various prom+k8s+cortex * fix logcli client instantiation * go mod tidy	4 years ago
Cyril Tovena	04f4f82b8e	Adds the ability to provide a tripperware to Promtail client. (#3654) ... Fixes #3608 Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	4 years ago
Ed Welch	a9d85de4aa	Re-license to AGPLv3 (#3630) ... For a rationale as to why this was done, please see * https://grafana.com/blog/2021/04/20/grafana-loki-tempo-relicensing-to-agplv3/ * https://grafana.com/blog/2021/04/20/qa-with-our-ceo-on-relicensing/ Signed-off-by: Richard Hartmann <richih@richih.org> Signed-off-by: Edward Welch <edward.welch@grafana.com>	4 years ago
Ed Welch	6e67d1ea5d	Loki/Promtail: Client Refactor (#3623) ... * move docker driver * move fluent-bit output plugin * move fluentd plugin * move logstash plugin * move promtail * match stage uses simple logql label matcher and filter expressions only * remove remaining logql imports * remove helpers package and move contents to util package * move validation and runtime out of util package * fix imports * move build and cfg packages into util * move ParseRequest function to util package * create a new package for logql models and move marshal/unmarshal code to util folder * move ErrorLabel to logqlmodel * move stats package to logqlmodel * decouple loghttp from logql by moving validation to logql package * move gcplog to tools * move PackedEntryKey constant to logqlmodel * fix stats proto import and rebuild protos * fix windows promtail tests * update drone to new dockerfile locations	4 years ago