apitech/loki - loki - Gitea: Git with a cup of tea

Commit Graph

Author	SHA1	Message	Date
Callum Styan	4a21923913	update to alpine:3.6.2 (#6979 ) Updates the alpine image version we use to 3.16.2 and will also build/publish a new loki build image Signed-off-by: Callum Styan <callumstyan@gmail.com> Signed-off-by: Callum Styan <callumstyan@gmail.com>	3 years ago
Christian Simon	88e2705034	Update alpine and golang base images (#6897 ) * Upgrade base images to alpine 3.15.6 everywhere https://alpinelinux.org/posts/Alpine-3.13.12-3.14.8-3.15.6-3.16.2-released.html * Update golang images to 1.18.5	3 years ago
carlospeon	171a09b822	Add support for journal matches (only conjuntions) (#6656 )	3 years ago
Kaviraj Kanagaraj	e8b44172d8	test(promtail): Fix flaky test with promtail bind port. (#6859 ) * test(promtail): Fix flaky test with promtail bind port. You cannot run promtail tests if there is already running promtail in your local. e.g: ``` $ go test -run TestPromtail ./clients/pkg/promtail/ ``` It gives you error ``` --- FAIL: TestPromtail (0.02s) promtail_test.go:112: error creating promtail listen tcp 127.0.0.1:9080: bind: address already in use FAIL FAIL github.com/grafana/loki/clients/pkg/promtail 0.043s ``` Cause for the error is, test uses static default port (the same we use for promtail normally) in tests. This is not too annonying if seen in isolation. But mainly annoying in two cases particularly 1. Hard to run tests parallely in general with these kind of static ports 2. These are causing some real problems in some integeration tests on GEL which starts docker-compose to run some test suites and even stopping containers won't release ports soon enough making CI fail randomly. This PR takes advantage of Go idiom of passing `port=0` to http (and also grpc) servers enforce Go's `net` package to bind some unused random ports. Making each test run choose random ports enabling easy running theses tests in parallel. NOTE: This PR also contains upgrade in `common/server` package that made this fix possible. https://github.com/weaveworks/common/pull/249 Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * go mod tidy Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com>	3 years ago
Alexandre de Sá	6b47aab2de	promtail/client: add bot fields (#6828 ) * chore: add bot fields * Update CHANGELOG.md * chore: update cloudflare configuration section URI * docs: include BotScore and BotScoreSrc to all set of fields * Update CHANGELOG.md	3 years ago
Pablo	c9ef0621d1	Promtail: Add new target for receiving GCP PubSub Push messages (#6777 )	3 years ago
Kaviraj Kanagaraj	0d99d80acd	upgrade `go:1.17.9` to `go:1.18.4` (#6726 )	3 years ago
Pablo	d4cdc37682	promtail: Inject tenant ID when receiving X-Scope-OrgID in heroku target (#6695 ) * inject tenant ID header * fix import order	3 years ago
Pablo	0e28452f1f	Targets: Add Heroku HTTPS drain target (#6448 )	3 years ago
Dylan Guedes	f17d3d768c	Promtail: Add text encoding conversion to file targets (#6395 ) Adds support for text encoding conversion for file targets. To use it, add `encoding: <encoding_name>` into the scrapeconfig.	3 years ago
Johan Fleury	9b980c45ed	Promtail: update certificates used for syslog’s tests (#6534 ) All the certificates now have a 20 year lifetime. This reverts commit `2fc20af9a4`.	3 years ago
Ed Welch	2fc20af9a4	Promtail: disable syslog tests which have expired certs (#6520 ) * disable tests which are failing from hardcoded certs which are now expired. Signed-off-by: Edward Welch <edward.welch@grafana.com> * lint Signed-off-by: Edward Welch <edward.welch@grafana.com>	3 years ago
Tristan Colgate-McFarlane	21f69643b3	promtail: fix infinite loop in file target (fixes #5350 ) (#6135 )	3 years ago
Jesus Vazquez	db9dbe8e1d	Bump prometheus dependency (#6403 ) * Bump prometheus dependency We had to add a replace of prometheus in go mod because thanos is importing an older version and then replacing it. * Undo textparser.New variable mismatch * modtimevfs path update * Fix labels withlabels and withoutlabels usages WithoutLabels and WithLabels methods have dissappeared. Now labels.NewBuilder has to be used. * Fix RuleManager interface for Update() Update now accepts a new param called ruleGroupPostProcessFunc which is an optional function to be executed after rule group processing. In this case I've set it to nil in the usages i've found. * NewQueryRange now accepts QueryOpts The NewQueryRange function now accepts QueryOptions so I had to update all the calls. I've set the opts to nil since we're not using them. * gofmt	3 years ago
Kaviraj Kanagaraj	01481cca0c	Promtail: Support all `cri-o` tags (multiline tags). (#6177 ) * Promtail: Support `cri-o` tags. Hacked something together that works during flight to Grafanafest. * Fix newCri test cases Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Add test for real "panic" stacktrace Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Add bounded size for partial lines slice Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Leave the promtail config alone Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * PR remarks Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * PR remarks: Avoid discarding lines when partial lines reaches upper bound. Instead merge into single full line and print warning. Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com>	3 years ago
Rutger	1794a76613	Promtail: Add metrics for journal target (#6105 ) * Add metrics for journal target There are no metrics for the journal target yet, this adds two basic ones: - Total number of lines processed - Total number of errors processing lines (with a reason) Because of the way the journal works, tailing one file with many possible process writing to it, I chose note to expose counts per unit. * Add changelog entry * Fix compilation of (unsupported) journalmanager target on windows * Extract possible errors to constants & add tests	3 years ago
Trevor Wood	23cc93862d	chore(fluentd): resolve rubocop failures (#6359 ) Signed-off-by: Trevor Wood <Trevor.G.Wood@gmail.com>	3 years ago
Shane St Savage	135f672c91	fix: allow promtail docker_sd_config to handle long log lines (#6256 ) In promtail's docker_sd_config, parsing of a container's logs would fail if any log line was over 64k bytes in length. This is because bufio.Scanner uses a 64k buffer by default. Since there doesn't appear to be a way to elegantly catch `bufio.ErrTooLong` and carry on, the parsing approach was changed to bufio.Reader which can handle lines of arbitrary length.	3 years ago
Jan-Otto Kröpke	a1e0298a57	Support tenant id from value (#6290 ) Signed-off-by: Jan-Otto Kröpke <mail@jkroepke.de>	3 years ago
Trevor Wood	0470e4e7d3	feat(fluentd): allow fluentd_thread label to be configurable (#6240 ) Signed-off-by: Trevor Wood <Trevor.G.Wood@gmail.com>	3 years ago
Callum Styan	b6ac00e1a3	[Promtail] drop lines with malformed json (#6099 ) * Refactor json pipeline stage to have a Run function Signed-off-by: Callum Styan <callumstyan@gmail.com> * Allow for dropping log lines in json pipeline stage if they are not valid json strings Signed-off-by: Callum Styan <callumstyan@gmail.com> * fix linting Signed-off-by: Callum Styan <callumstyan@gmail.com> * Clean up new malformed json test. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Address some review feedback. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Refactor json pipeline stage test to use testify assert instead of t.Errorf Signed-off-by: Callum Styan <callumstyan@gmail.com> * Update changelog and docs with new drop malformed option Signed-off-by: Callum Styan <callumstyan@gmail.com> * Update CHANGELOG.md Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> * Update docs based on Karen's comments. Signed-off-by: Callum Styan <callumstyan@gmail.com> Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com>	3 years ago
Christian Haudum	7fff660144	Add UDP protocol support to Promtail's syslog target (#5790 ) Until now, the syslog target in Promtail did only support TCP. This PR adds support for sending syslog messages to Promtail (via ng-syslog or rsyslog) via UDP. UPD for syslogs can be used if you prefer speed over guaranteed delivery. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	3 years ago
Paschalis Tsilias	179d741c5f	promtail: Add support for exclusion patterns in `static_config` (#5943 ) * PR base Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * Add CHANGELOG entry; updated documentation for new label Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * Follow logic from t.path in exlusion case; always handle matching Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * Make error message unique Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * Only run if t.pathExclude is set Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * Revert "Only run if t.pathExclude is set" This reverts commit 17abb006ce141071c4bca4b7d133bea2aa314594.	3 years ago
Alex Ferm	d7f03be212	Update libsystemd-dev from bullseye-backports for arm32 docker images, fixes (#5938 ) (#6012 )	3 years ago
Vladyslav Diachenko	4a91fdcc7f	migrated gelfTarget to go-gelf/v2 library that contains fix for chunked messages (#5992 )	3 years ago
Callum Styan	8f62334a58	[promtail] Add a log message stating that client level stream lag labels are deprecated (#5850 ) * Add a log message stating that client level stream lag labels are deprecated and will not be used. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Error on startup if a client level stream lag labels value is set. Signed-off-by: Callum Styan <callumstyan@gmail.com> * fix lint error Signed-off-by: Callum Styan <callumstyan@gmail.com>	3 years ago
Christian Simon	9d0fd3a7e3	Upgrade golang version to v1.17.9 (#5899 ) ``` git ls-files '[Dd]ockerfile' \| grep -v ^vendor/ \| xargs sed -i "s/golang:[0-9\\.]*/golang:1.17.9/g" \| ag 'golang:' ```	3 years ago
thepheer	81021c920f	remove excessive underscore from `dockerLabelLogStream` (#5890 )	3 years ago
Karsten Jeschkies	c9f3868287	Update dskit to v0.0.0-20220331160727-49faf69f72ca. (#5881 )	3 years ago
Callum Styan	800b7c6345	[promtail] Mark promtail single client config flags and config file block as deprecated (#5842 ) * Mark promtail single client config flags and config file block as deprecated. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Update some promtail docs examples that still use `client` instead of `clients`. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Fix imports. Signed-off-by: Callum Styan <callumstyan@gmail.com> * properly format url as an array item since we changed to clients from client Signed-off-by: Callum Styan <callumstyan@gmail.com>	3 years ago
Cyril Tovena	dd7200564b	Improve kafka discovery test. (#5860 ) The `Test_TopicDiscovery` expections were wrong, we only need them to be true at some point since this is reconciliation loop. Fixes #5858 Nice catch ! Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	3 years ago
Marc Tudurí	0315102c34	promtail: Add ActiveTargets method (#5839 ) * Add ActiveTargets method to promtail * Update CHANGELOG	3 years ago
Bryan Boreham	1329ea165c	TestPromtail: avoid hanging when test fails midway (#5837 ) * Allow Promtail.Shutdown to be called again I want to do this in a test * TestPromtail: avoid hanging when test fails midway Also call Shutdown via defer so we clean up those resources when the test fails	3 years ago
Christian Simon	3d941ccccb	Update alpine images to 3.15.4 (#5780 )	3 years ago
Christian Haudum	ece1fb5a34	Allow promtail to push RFC5424 formatted syslog messages (#5715 ) * Allow promtail to push RFC5424 formatted syslog messages This PR introduces a new flag `use_rfc5424_message` for syslog scrape targets, which allows to push the full [RFC5424](https://datatracker.ietf.org/doc/html/rfc5424) formatted syslog message received by Promtail to be pushed to Loki, instead of only the message content. Storing the full message allows users to parse the message in Loki instead of relabelling in Promtail. Signed-off-by: Christian Haudum <christian.haudum@gmail.com> * Add changelog entry Signed-off-by: Christian Haudum <christian.haudum@gmail.com> * Add test for pushing RFC5424 messages Signed-off-by: Christian Haudum <christian.haudum@gmail.com> * fixup! Allow promtail to push RFC5424 formatted syslog messages Signed-off-by: Christian Haudum <christian.haudum@gmail.com> Co-authored-by: Cyril Tovena <cyril.tovena@gmail.com>	3 years ago
Karen Miller	c860fc6442	Docs: Document Promtail global rate limiting (#5737 ) * Docs: Document Promtail global rate limiting * Update docs/sources/clients/promtail/configuration.md Co-authored-by: Jennifer Villa <jvilla2013@gmail.com> * Revise strings that explain the new configuration parameters * For rate limiting, limit_config becomes limits_config Co-authored-by: Jennifer Villa <jvilla2013@gmail.com>	3 years ago
Ed Welch	a9ac01bd0f	use an empty registry when setting up the logger for the push server (#5761 ) Signed-off-by: Ed Welch <edward.welch@grafana.com>	3 years ago
李国忠	098b5a2d37	[new] promtail pipeline: Promtail Rate Limit stage #5048 (#5051 ) * [new] promtail pipeline: Promtail Rate Limit stage #5048 * [new] promtail pipeline: Promtail Rate Limit stage #5048 #5051 * [new] promtail pipeline: Promtail Rate Limit stage #5048 #5051 * [new] promtail pipeline: Promtail Rate Limit stage #5048 #5051 * Update clients/pkg/logentry/stages/limit.go Co-authored-by: Karsten Jeschkies <k@jeschkies.xyz> * [new] promtail pipeline: Promtail Rate Limit stage #5048 #5051 * [new] promtail pipeline: Promtail Rate Limit stage #5048 #5051 * [new] promtail pipeline: Promtail Rate Limit stage #5048 #5051 * Update docs/sources/clients/promtail/pipelines.md Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> * Update docs/sources/clients/promtail/stages/limit.md Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> * Update docs/sources/clients/promtail/stages/limit.md Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> * [new] promtail pipeline: Promtail Rate Limit stage #5048 #5051 * [new] promtail pipeline: Promtail Rate Limit stage #5048 #5051 * [new] promtail pipeline: Promtail Rate Limit stage #5048 #5051 * [new] promtail pipeline: Promtail Rate Limit stage #5048 #5051 * Update docs/sources/clients/promtail/stages/limit.md * Update docs/sources/clients/promtail/stages/limit.md * fix import package * Update docs/sources/clients/promtail/stages/limit.md Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> * Update docs/sources/clients/promtail/stages/limit.md Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> * Update docs/sources/clients/promtail/pipelines.md Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> Co-authored-by: Karsten Jeschkies <k@jeschkies.xyz> Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> Co-authored-by: Cyril Tovena <cyril.tovena@gmail.com>	3 years ago
Karsten Jeschkies	a5921f791a	Revert changes to generated assets_vfsdata.go. (#5728 )	3 years ago
Karsten Jeschkies	46956d0bf6	Use dskit/tenant intead of loki/pkg/tenant. (#5724 )	3 years ago
Paul Ryan	4e3f3b7f7e	Promtail: Fix retry/stop when erroring for out of cloudflare retention range (e.g. over 168 hours old) (#5698 ) * fix: 5696 don't retry when out of range for cloudflare	3 years ago
Franz Wong	d67306b657	Promtail: Rename config name limit_config to limits_config (#5707 ) * Rename promtail config name 'limit_config' to 'limits_config' * Add changelog entry * Fix formatting	3 years ago
Susana Ferreira	1d1461824d	Move promtail StreamLagLabels config to upper level config.Config (#5686 ) * Move promtail StreamLagLabels config to upper level config.Config * Fix promtail config load test * Fix promtail configuration reference * Add options promtail config section * Kick GitHub CI Signed-off-by: Danny Kopping <danny.kopping@grafana.com> Co-authored-by: Danny Kopping <danny.kopping@grafana.com>	3 years ago
Callum Styan	2ebbecfcbd	Require unique client configs when there are multiple clients. (#5688 ) * Require unique client configs when there are multiple clients. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Fixing linter error Signed-off-by: Danny Kopping <danny.kopping@grafana.com> Co-authored-by: Danny Kopping <danny.kopping@grafana.com>	3 years ago
Callum Styan	2c2578e84c	Fix bug with new promtail stream lag metric where labels might not be (#5675 ) applied if they don't exist on the source stream Signed-off-by: Callum Styan <callumstyan@gmail.com>	3 years ago
Callum Styan	01d63c8ae8	[promtail] Refactor promtail client metrics so that we can't have duplicate metrics collected for the lag metric. (#5521 ) * Refactor promtail client metrics so that we can't have duplicate metrics collected for the lag metric. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Set up new metrics passing for other client types. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Fix lint issues. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Update changelog and docs, also rename nested client configs struct to configs. Signed-off-by: Callum Styan <callumstyan@gmail.com> Co-authored-by: Owen Diehl <ow.diehl@gmail.com>	3 years ago
Cyril Tovena	8f693e77b0	Update Prometheus to latest (#5545 ) * Update Prometheus to latest Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Update promql.QueryTracker Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes querier test to use new grafana/regexp Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes test with new textparse signature Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * lint Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	3 years ago
Cyril Tovena	c8bdb37ef3	update loki go version to 1.17.8 (#5543 ) Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	3 years ago
Travis Patterson	c654910f70	Fix orphaned metrics in the file tailer (#5497 )	3 years ago
Holger Hees	f359a8280f	Update out_loki.rb (#5512 ) this will fix "UndefinedConversionError error="\"\\xC2\" from ASCII-8BIT to UTF-8"" for messages of systemd journal	3 years ago

1 2 3

141 Commits (766d4b7b5d2475d9424ca03e2a746e4c0e46660f)