apitech/loki - loki - Gitea: Git with a cup of tea

Commit Graph

Author	SHA1	Message	Date
loanshark	43e542ee64	Storage: Remove `SignatureVersionv2` from the AWS S3 API request authentication. (#10295 ) AWS S3 stopped accepting requests signed using SigV2 in all regions on June 24, 2019, any requests signed using SigV2 made after this time will fail. * https://docs.aws.amazon.com/AmazonS3/latest/userguide/UsingAWSSDK.html#UsingAWSSDK-sig2-deprecation * https://aws.amazon.com/blogs/aws/amazon-s3-update-sigv2-deprecation-period-extended-modified/	3 years ago
Danny Kopping	beed298549	Dynamic client-side throttling to avoid object storage rate-limits (GCS only) (#10140 ) What this PR does / why we need it: Across the various cloud providers' object storage services, there are different rate-limits implemented. Rate-limits can be imposed under multiple conditions, such as server-side scale up (ramping up from low volume to high, "local" limit), reaching some defined upper limit ("absolute" limit), etc. We cannot know apriori when these rate-limits will be imposed, so we can't set up a client-side limiter to only allow a certain number of requests through per second. Additionally, that would require global coordination between queriers - which is difficult. With the above constraints, I have instead taken inspiration from TCP's [congestion control algorithms](https://en.wikipedia.org/wiki/TCP_congestion_control). This PR implements [AIMD](https://en.wikipedia.org/wiki/Additive_increase/multiplicative_decrease) (Additive Increase, Multiplicative Decrease), which is used in the congestion _avoidance_ phase of congestion control. The default window size (object store requests per second) is 2000; in other words, we skip the "slow start" phase. The controller uses the Go [`rate.Limiter`](https://pkg.go.dev/golang.org/x/time/rate), which implements the token-bucket algorithm. To put it simply: - every successful request widens the window (per second client rate-limit) - every rate-limited response reduces the window size by a backoff factor (0.5 by default, so it will halve) - when the limit has been reached, the querier will be delayed from making further requests until tokens are available	3 years ago
J Stickler	d0545bff2d	[docs] Create top level Send data section, part 2 (#10247 ) Part of the database information architecture Epic # 8710 Which issue(s) this PR fixes: Second half of work for issue # 8741 that was started in PR #10192 When closed, fixes #8741 Move the following files under Send data: ./sources/clients/promtail/_index.md ./sources/clients/promtail/configuration.md ./sources/clients/promtail/gcplog-cloud.md ./sources/clients/promtail/installation.md ./sources/clients/promtail/logrotation/_index.md ./sources/clients/promtail/pipelines.md ./sources/clients/promtail/scraping.md ./sources/clients/promtail/stages/_index.md ./sources/clients/promtail/stages/cri.md ./sources/clients/promtail/stages/decolorize.md ./sources/clients/promtail/stages/docker.md ./sources/clients/promtail/stages/drop.md ./sources/clients/promtail/stages/json.md ./sources/clients/promtail/stages/labelallow.md ./sources/clients/promtail/stages/labeldrop.md ./sources/clients/promtail/stages/labels.md ./sources/clients/promtail/stages/limit.md ./sources/clients/promtail/stages/logfmt.md ./sources/clients/promtail/stages/match.md ./sources/clients/promtail/stages/metrics.md ./sources/clients/promtail/stages/multiline.md ./sources/clients/promtail/stages/output.md ./sources/clients/promtail/stages/pack.md ./sources/clients/promtail/stages/regex.md ./sources/clients/promtail/stages/replace.md ./sources/clients/promtail/stages/static_labels.md ./sources/clients/promtail/stages/template.md ./sources/clients/promtail/stages/tenant.md ./sources/clients/promtail/stages/timestamp.md ./sources/clients/promtail/troubleshooting/_index.md This PR also - Revises the Clients landing page to clarify supported clients. - Updates the metadata (descriptions, weights) - Adds aliases to redirect from the old URLs. - Updates cross-references broken by the move/renaming. - A few other small fixes (headings, typos, etc.) Special notes for your reviewer: Please review the updates to the Clients landing page (now called Send Data) as I've made some extensive edits to try to clarify recommended/supported clients. The file is docs/sources/send-data/_index.md --------- Co-authored-by: Michel Hollands <42814411+MichelHollands@users.noreply.github.com>	3 years ago
Charles Korn	611481b6cf	Upgrade dskit (#10249 )	3 years ago
Charles Korn	8e6a5433ac	Upgrade dskit and remove dependency on weaveworks/common (#10220 ) What this PR does / why we need it: This PR upgrades `github.com/grafana/dskit` removes Loki's dependency on `github.com/weaveworks/common`. The changes in dskit, apart from the migration (https://github.com/grafana/dskit/pull/356), are: * https://github.com/grafana/dskit/pull/347 * https://github.com/grafana/dskit/pull/349, which required small changes in `pkg/canary/reader/reader.go` and `pkg/canary/writer/push.go` * https://github.com/grafana/dskit/pull/352 * https://github.com/grafana/dskit/pull/354 Which issue(s) this PR fixes: (none) Special notes for your reviewer: Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [n/a] Documentation added - [n/a] Tests updated - [n/a] `CHANGELOG.md` updated - [n/a] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [n/a] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [n/a] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) --------- Co-authored-by: Michel Hollands <42814411+MichelHollands@users.noreply.github.com>	3 years ago
Dylan Guedes	1221658fc5	Introduces cache to TSDB postings (#9621 ) What this PR does / why we need it: Introduces a new interface for fetching TSDB postings and implements a cached TSDB postings. The new postings interface is named `PostingsReader` and contains two implementations: a simple one, which is identical to the former Postings behavior, and a new one, that caches postings using the existing `index_queries_cache_config`. By default, the simple/former implementation is used. The new cached implementation can be configured in the following way: ```yaml storage_config: tsdb_shipper: cache_postings: true ``` In the future, we'll remove the `cache_postings` flag and change the behavior to cache postings by default.	3 years ago
Mohamed-Amine Bouqsimi	2fcde18207	Fix query_timeout warning messages (#10083 ) What this PR does / why we need it: This PR fixes the warning messages that are logged in relation to the timeouts migration. The messages reference the per-tenant config, but the config in question in the `AdjustForTimeoutsMigration` function is the global `limits_config`. This is a source of confusion for users. Which issue(s) this PR fixes: Fixes #9801	3 years ago
Danny Kopping	8fa4134ef9	Update dskit (#10091 )	3 years ago
Christian Haudum	ab3d58a265	Add initial delay before starting flush loop (#9995 ) What this PR does / why we need it: Delay the first flush operation by up to 0.8x the flush time period. This will ensure that multiple ingesters started at the same time do not flush at the same time. Flushing at the same time can cause concurrently writing the same chunk to object storage, which in AWS S3 leads to being rate limited. Additionally, add a +/- 1% jitter to each flush check interval to prevent flushing at the same time for reasons mentioned above. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	3 years ago
J Stickler	cc2aeac860	Break up and move Best Practices topic. (#9912 ) I realized that I should have moved the Best Practices topic as part of #8711, since most of the content is best practices for using labels. This PR: * Splits the file into two parts (Best practices for labels and best practices for configuration) * Moves the two files to Get Started and Configure, respectively. * Updates page weights for both sections as needed.	3 years ago
Vladyslav Diachenko	be1a9c4ec5	Revert "updated to the latest version of dskit" (#9953 ) Reverts grafana/loki#9920	3 years ago
Vladyslav Diachenko	3243676f4c	updated to the latest version of dskit (#9920 ) What this PR does / why we need it: Updated Loki with the latest version of dskit Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) --------- Signed-off-by: Vladyslav Diachenko <vlad.diachenko@grafana.com>	3 years ago
Travis Patterson	cb2b3ca9ea	Add limit to the series volume endpoint (#9833 ) To protect index gateways, this PR adds a per-tenant limit to the `series_volume` endpoint so the accumulation of series can't grow without bound.	3 years ago
J Stickler	7797a98074	TOC Sync: Move configuration files under Configure (#9879 ) Which issue(s) this PR fixes: Fixes #8713 Special notes for your reviewer: Replaces PR [#8817](https://github.com/grafana/loki/pull/8817), which was stale and badly out of date. Part of the Database products table of contents alignment project. Moves configuration files from Configuration to new Configure heading in TOC. Updates front matter for the topic. Fixes links broken by the move. Updates the MakeFile to point to the new Configure DOCS_SOURCES_PATH.	3 years ago

14 Commits (ca19e8d7941bfac86355b2853ef2ea110f32075e)