apitech/loki - loki - Gitea: Git with a cup of tea

Commit Graph

Author	SHA1	Message	Date
renovate[bot]	246a1dfbe2	fix(deps): update module github.com/ncw/swift to v2 (#13951 ) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> Co-authored-by: Paul Rogers <paul.rogers@grafana.com>	1 year ago
renovate[bot]	d43b2de1b4	fix(deps): update module github.com/fsouza/fake-gcs-server to v1.47.7 (#13935 ) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> Co-authored-by: Paul Rogers <paul.rogers@grafana.com>	1 year ago
Dylan Guedes	3c9c647922	fix: Read "404" as object not exist (#13901 ) What this PR does / why we need it: Modify the s3 client to recognize "NotFound" errors when checking if an object error is "NotFound". I did some tests locally and `Head` calls to objects that doesn't exist gives a "NotFound" instead of "NoSuchKey".	1 year ago
Vladyslav Diachenko	bb257f54b3	feat(loki): add ability to disable AWS S3 dualstack endpoints usage (#13785 ) Signed-off-by: Vladyslav Diachenko <vlad.diachenko@grafana.com>	1 year ago
Marco Pracucci	b88583da7d	feat: upgrade prometheus (#13671 ) Signed-off-by: Marco Pracucci <marco@pracucci.com> Signed-off-by: Vladyslav Diachenko <vlad.diachenko@grafana.com> Co-authored-by: Vladyslav Diachenko <vlad.diachenko@grafana.com>	1 year ago
Trevor Whitney	5a87ccb648	feat: instrument failed chunk encoding/decoding (#13684 )	2 years ago
Cyril Tovena	d42476aa58	fix: Handle block offset exceeding chunk length in memchunk.go (#13661 )	2 years ago
benclive	d9c441efcc	feat: Implement GetObjectRange for all storage providers (#13650 )	2 years ago
Cyril Tovena	0076bbdb42	chore: Refactor storage interface for rf1 (#13415 )	2 years ago
Christian Haudum	69b805d8ff	test: Fix flaky congestion test (#13410 ) Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
benclive	973aa2d7ba	feat: Use prefixed WAL storage path in Object Storage (#13377 )	2 years ago
Vladyslav Diachenko	33bd390f19	chore: adjusted SegmentWriter to the reader conversion (#13342 ) Signed-off-by: Vladyslav Diachenko <vlad.diachenko@grafana.com> Co-authored-by: George Robinson <george.robinson@grafana.com> Co-authored-by: Ben Clive <ben.clive@grafana.com>	2 years ago
George Robinson	7f35179cd3	feat: Ingester RF-1 (#13365 )	2 years ago
Paul Rogers	40ee766724	feat: Collect duplicate log line metrics (#13084 )	2 years ago
Dylan Guedes	c1fada9af0	refactor: Remove unnecessary spanlogger usage (#13255 ) Historically, we've been using `spanlogger` to enrich our spans. That isn't the right usage for it (as of now), as they'll be emitting log lines. Our current usage is causing: - Noisy/important log lines less visible - Unnecessary logging volume - Spread of the bad practice. With this PR I'm moving away from `spanlogger` all places where the log message is too bad or nonexistent. That's because these cases are indicating we don't care about that log line at all, and only about the data injected in the span.	2 years ago
Owen Diehl	1086783a1d	fix: separates directory creation from permission checks (#13248 )	2 years ago
Paul Rogers	d16a3bf139	test: First pass at unregistering metrics so that we can run multiple tests (#12927 )	2 years ago
MarkDaveny	3a46d3717d	style: fix function names (#12817 ) Signed-off-by: MarkDaveny <peicuiping@aliyun.com>	2 years ago
Trevor Whitney	af09f53a33	ci: release workflows get build image from makefile (#12778 )	2 years ago
Ed Welch	4c88be0ef2	chore: update loki modules for 3.0 release (#12433 ) Signed-off-by: Edward Welch <edward.welch@grafana.com>	2 years ago
Paul Rogers	aca2a38ad6	test: gcs object client test data race (#12324 )	2 years ago
BiKangNing	75e031d8e8	chore: fix function names in doc strings (#12313 ) Signed-off-by: depthlending <bikangning@outlook.com>	2 years ago
Paul Rogers	0d1a3694f5	test: Don't start server until all configured (#12226 )	2 years ago
Danny Kopping	b5f488caec	fix(query/query-frontend): correct congestion control stats aggregation (#12072 ) Signed-off-by: Danny Kopping <danny.kopping@grafana.com>	2 years ago
Danny Kopping	609fdb3214	feat(querier/query-frontend): track and log congestion control latency (#12058 ) Signed-off-by: Danny Kopping <danny.kopping@grafana.com>	2 years ago
Christian Haudum	b9ce005ec1	fix: Ensure working dir for bloomstore exists (#12019 ) Fail startup if directory does not exist or there are not enough permissions. This prevents the bloomstore to fail later in the process once it tries to download and extract blocks. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
Christian Haudum	73edf7a943	(chore) Bloomshipper: Separate store and client (#11865 ) What this PR does / why we need it: This PR removes the `StoreAndClient` interface that was accepted by the `BloomShipper`. Since the `BloomStore` had to not only implement the `Store` interface, but also the `Client` interface, it caused re-implementation of the same methods in different ways. Now the shipper solely relies on the `Store` interface. See individual commit messages for more context. Tests have been rewritten from scratch and placed in their own respective test files for store and client. --------- Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
Michel Hollands	2a63d24729	Fix the ObjectExists method on FSObjectClient (#11747 ) What this PR does / why we need it: This method wasn't working. A test was added as well. Special notes for your reviewer: Checklist - [X] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [X] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) - [ ] If the change is deprecating or removing a configuration option, update the `deprecated-config.yaml` and `deleted-config.yaml` files respectively in the `tools/deprecated-config-checker` directory. [Example PR](`0d4416a4b0`) Signed-off-by: Michel Hollands <michel.hollands@gmail.com>	2 years ago
akevdmeer	18778cd548	De-duplicate common prefixes as returned for individual buckets (#11317 ) With multiple buckets configured on a single AWS/S3 store, each object is stored on one of the buckets by hashing its object key in `bucketFromKey()`. That is sufficient for `GetObject()` and `PutObject()`. `List()` needs to list each bucket and combine the results. It appends all object keys into a `storageObjects` slice. That works because each key uniquely exists in only one of the buckets. But that is not the case for the common prefixes; a common prefix may exist in multiple buckets. This PR removes duplicates from the common prefixes as gathered from the multiple buckets in `List()`. Without this fix, we find that a repeated common prefix leads to a table being compacted multiple times concurrently which is not safe. E.g. this error results when the compactor concurrently tries to compact a table and a 'losing' execution finds that its clean-up has already been done: `level=error caller=compactor.go:128 table-name=loki_index_tsdb_19683 msg="failed to remove downloaded index file" path=/var/loki/compactor/loki_index_tsdb_19683/1700667008-loki-write-0-1700660468255353690.tsdb err="remove /var/loki/compactor/loki_index_tsdb_19683/1700667008-loki-write-0-1700660468255353690.tsdb: no such file or directory" `	2 years ago
Angus Dippenaar	258a8b3108	Update nix configuration (#8452 ) Fix nix configuration (mainly failing test), and break up various binaries (ie. logcli, promtail, loki, etc.) into their own pakcages. --------- Co-authored-by: Trevor Whitney <trevorjwhitney@gmail.com>	2 years ago
Michel Hollands	e93f5bfd93	Add dashboard changes for migrating from cortex_ metrics to loki_ metrics (#11082 ) What this PR does / why we need it: - add missing namespace param for ruler manager metrics - update the loki mixin to use the renamed metrics so both are supported - created the compiled mixin - make the prefix of the cortex_distributor_ingester_clients metrics configurable Checklist - [X] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) - [ ] If the change is deprecating or removing a configuration option, update the `deprecated-config.yaml` and `deleted-config.yaml` files respectively in the `tools/deprecated-config-checker` directory. [Example PR](`0d4416a4b0`) --------- Signed-off-by: Michel Hollands <michel.hollands@gmail.com>	2 years ago
Karsten Jeschkies	4ed3b117d2	Use newest build image 0.31.2 with golangci-lint update. (#11118 ) What this PR does / why we need it: This just updates the Loki build imaage changed in #11114. Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) - [ ] If the change is deprecating or removing a configuration option, update the `deprecated-config.yaml` and `deleted-config.yaml` files respectively in the `tools/deprecated-config-checker` directory. [Example PR](`0d4416a4b0`)	2 years ago
Shantanu Alshi	fb6f6d2e2f	Stop using global logger in modules (#11051 ) What this PR does / why we need it: Refactors Loki to inject logger from the constructors instead of using global logger variable. It is generally considered a good practice. Which issue(s) this PR fixes: [Fixes #<issue number>](https://github.com/grafana/loki-private/issues/810) Special notes for your reviewer: Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [x] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) - [ ] If the change is deprecating or removing a configuration option, update the `deprecated-config.yaml` and `deleted-config.yaml` files respectively in the `tools/deprecated-config-checker` directory. <!-- TODO(salvacorts): Add example PR -->	2 years ago
April Schleck	4b750cc0cc	Storage: Allow setting a constant prefix for all created keys (#10096 ) What this PR does / why we need it: Adds a new option under the aws stanza named key_prefix. This is useful if you have many Loki installations and do not want to create a million buckets. This is different than `compactor.shared_store_key_prefix` because it also affects eg the `loki_cluster_seed.json` file. Which issue(s) this PR fixes: Fixes #5889 Special notes for your reviewer: Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [x] Documentation added - [x] Tests updated - [x] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [x] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) --------- Signed-off-by: Edward Welch <edward.welch@grafana.com> Co-authored-by: Ed Welch <edward.welch@grafana.com>	2 years ago
Michel Hollands	8628b15624	Use metrics namespace for more metrics (#11025 ) What this PR does / why we need it: Use the metrics namespace setting instead of hardcoding to `cortex`. This is a follow up to (and based on) https://github.com/grafana/loki/pull/11014. Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [X] Tests updated - [x] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) - [ ] If the change is deprecating or removing a configuration option, update the `deprecated-config.yaml` and `deleted-config.yaml` files respectively in the `tools/deprecated-config-checker` directory. <!-- TODO(salvacorts): Add example PR --> --------- Signed-off-by: Michel Hollands <michel.hollands@gmail.com> Co-authored-by: Ashwanth <iamashwanth@gmail.com>	2 years ago
Ashwanth	a118e994d8	Remove shared_store and shared_store_key_prefix from shipper and compactor (#10840 ) What this PR does / why we need it: #### Removes `shared_store` and `shared_store_key_prefix` from index shipper and compactor configs and their corresponding CLI flags. - `-tsdb.shipper.shared-store` - `-boltdb.shipper.shared-store` - `-tsdb.shipper.shared-store.key-prefix` - `-boltdb.shipper.shared-store.key-prefix` - `-boltdb.shipper.compactor.shared-store` - `-boltdb.shipper.compactor.shared-store.key-prefix` `shared_store` has been a confusing option allowing users to easily misconfigure Loki. Going forward `object_store` setting in the [period_config](https://grafana.com/docs/loki/latest/configure/#period_config) (which already configured the store for chunks) will be used to configure store for the index. And the newly added `path_prefix` option under the `index` key in `period_config` will configure the path under which index tables are stored. This change enforces chunks and index files for a given period reside together in the same storage bucket. More details in the upgrade guide. --- `-compactor.delete-request-store` has to be explicitly configured going forward. Without setting this, loki wouldn't know which object store to use for storing delete requests. Path prefix for storing deletes is decided by `-compactor.delete-request-store.key-prefix` which defaults to `index/`. Checklist - [X] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [X] Documentation added - [X] Tests updated - [x] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [X] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`)	2 years ago
Karsten Jeschkies	9fcc42dc48	Support protobuf `QueryRequest` in querier. (#10858 ) What this PR does / why we need it: This is one step to support pure protobuf encoding without `httpgrpc`. It is only on the scheduler and is fully backwards compatible. Which issue(s) this PR fixes: This is a sub change of https://github.com/grafana/loki/pull/10688 Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [x] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) --------- Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> Co-authored-by: Kaviraj <kavirajkanagaraj@gmail.com> Co-authored-by: Danny Kopping <dannykopping@gmail.com>	2 years ago
Roel Arents	5fb1cdb5dd	azure: add connection-string (#10187 ) What this PR does / why we need it: Adds a connection-string option to the azure blob storage config. So the [Azurite](https://learn.microsoft.com/en-us/azure/storage/common/storage-use-azurite) emulator can be used in local test environments (my use case). But also to support the use of a SAS token. See the (future) addition of this option to the Thanos object storage client: https://github.com/thanos-io/objstore/pull/51 Which issue(s) this PR fixes: n/a Special notes for your reviewer: https://github.com/thanos-io/objstore/pull/51 isn't merged yet. But I'm already creating this PR to see what others think. Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [x] Documentation added - [x] Tests updated - [x] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [x] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`)	2 years ago
Christian Haudum	769979c9f6	chore(storage): Refactor store factory (#10518 ) What this PR does / why we need it: This PR makes a clearer distinction between supported and deprecated index/chunk storage types. There are individual commits for the different factory functions: * Refactor table client factory * Refactor index client factory * Refactor chunk client factory * Split initStore function --------- Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
Vladyslav Diachenko	00a3c5b026	renamed `non indexed labels` feature to `structured metadata` (#10432 ) Loki team has made a decision to rename the feature because the new name is more precise. --------- Signed-off-by: Vladyslav Diachenko <vlad.diachenko@grafana.com> Co-authored-by: Salva Corts <salva.corts@grafana.com>	2 years ago
Dylan Guedes	6ce510dc55	Storage: Introduce new "ObjectExists" API (#10360 ) What this PR does / why we need it: Introduce to our `Store` interface a new function named `ObjectExists`, which can be used by external tools to validate if a given file is present on an object store without downloading the file. Which issue(s) this PR fixes: N/A	2 years ago
Danny Kopping	e3ab2c22de	Fix CLI flags for congestion control (#10394 ) What this PR does / why we need it: Adds `congestion-control` prefixes to new flags. Currently to enable congestion control via a flag, one must use `-store.enabled` which is obviously misleading. --------- Signed-off-by: Danny Kopping <danny.kopping@grafana.com>	2 years ago
Christian Haudum	31981bd4af	Remove dependency on Bigchunk, which is a leftover of Cortex and is only used in tests (#10354 ) What this PR does / why we need it: This PR removes the dependency on "Bigchunk", which is a "default" implementation of the `chunk.Data` interface and used only in tests. Special notes for your reviewer: The PR replaces the usage of this chunk type with the Loki implementation (`chunkenc.MemChunk` wrapped into `chunkenc.Facade`) where possible. In case where it was not possible (tests in `pkg/storage/chunk` package) due to otherwise circular import, it was replaces by the `chunk.dummyChunk` implementation, which is just a no-op that implements `chunk.Data` interface) The PR also removes the protos for the ingester client (`pkg/ingester/client/ingester.proto`) which were only referenced in the removed `pkg/util/chunkcompat/compat.go` file. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
Shantanu Alshi	325c332f3e	Remove deprecated s3 sse toggle (#10377 ) This PR removes a legacy and already deprecated boolean setting `-s3.sse-encryption` in favor of the more fine-grain `-s3.sse.*` settings.	2 years ago
Ashwanth	dff3622eaa	deprecate old index types and non-object stores (#10341 ) What this PR does / why we need it: Deprecates old index types: `aws, aws-dynamo, gcp, gcp-columnkey, bigtable, bigtable-hashed, cassandra, grpc`. Only `tsdb` and `boltdb-shipper` are supported. Deprecates non-object stores. The next major release might not allow the usage of these as `object_store` in `schema_config`: `aws-dynamo, gcp, gcp-columnkey, bigtable, bigtable-hashed, cassandra, grpc.` Deprecates table manager which is not needed once when using `boltdb-shipper` or `tsdb` index type Which issue(s) this PR fixes: Fixes #<issue number> Special notes for your reviewer: Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [x] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`)	2 years ago
akhilanarayanan	3303844cb0	Refactored forGivenIngestors to use DoUntilQuorum instead of Do (#9984 ) This changes the forGivenIngestors function in pkg/querier/ingester_querier.go to use `DoUntilQuorum` (from [PR #293](https://github.com/grafana/dskit/pull/293)) instead of `Do`. `DoUntilQuorum` is much more efficient than `Do`. The exact differences between them can be seen [here](https://github.com/grafana/dskit/pull/293). --------- Co-authored-by: Trevor Whitney <trevorjwhitney@gmail.com>	2 years ago
Danny Kopping	9734c4b832	Minor fixes to congestion control mechanism (#10319 ) What this PR does / why we need it: Minor logging / metric improvements to #10140	2 years ago
loanshark	43e542ee64	Storage: Remove `SignatureVersionv2` from the AWS S3 API request authentication. (#10295 ) AWS S3 stopped accepting requests signed using SigV2 in all regions on June 24, 2019, any requests signed using SigV2 made after this time will fail. * https://docs.aws.amazon.com/AmazonS3/latest/userguide/UsingAWSSDK.html#UsingAWSSDK-sig2-deprecation * https://aws.amazon.com/blogs/aws/amazon-s3-update-sigv2-deprecation-period-extended-modified/	2 years ago
Danny Kopping	a30629e36e	Azure Blob Storage is no longer experimental (#10299 ) What this PR does / why we need it: Azure Blob Storage is being used in production by many organisations, including Grafana Labs. It should no longer be marked experimental. Which issue(s) this PR fixes: Fixes #10298 Signed-off-by: Danny Kopping <danny.kopping@grafana.com>	2 years ago
Danny Kopping	beed298549	Dynamic client-side throttling to avoid object storage rate-limits (GCS only) (#10140 ) What this PR does / why we need it: Across the various cloud providers' object storage services, there are different rate-limits implemented. Rate-limits can be imposed under multiple conditions, such as server-side scale up (ramping up from low volume to high, "local" limit), reaching some defined upper limit ("absolute" limit), etc. We cannot know apriori when these rate-limits will be imposed, so we can't set up a client-side limiter to only allow a certain number of requests through per second. Additionally, that would require global coordination between queriers - which is difficult. With the above constraints, I have instead taken inspiration from TCP's [congestion control algorithms](https://en.wikipedia.org/wiki/TCP_congestion_control). This PR implements [AIMD](https://en.wikipedia.org/wiki/Additive_increase/multiplicative_decrease) (Additive Increase, Multiplicative Decrease), which is used in the congestion _avoidance_ phase of congestion control. The default window size (object store requests per second) is 2000; in other words, we skip the "slow start" phase. The controller uses the Go [`rate.Limiter`](https://pkg.go.dev/golang.org/x/time/rate), which implements the token-bucket algorithm. To put it simply: - every successful request widens the window (per second client rate-limit) - every rate-limited response reduces the window size by a backoff factor (0.5 by default, so it will halve) - when the limit has been reached, the querier will be delayed from making further requests until tokens are available	2 years ago

1 2 3

102 Commits (bbbd82bc73322d662ba81efeda3884efcdc09708)