apitech/loki - loki - Gitea: Git with a cup of tea

Commit Graph

Author	SHA1	Message	Date
Shantanu Alshi	f1bbdc51dc	Remove split_queries_by_interval and forward_headers_list (#10395 ) …range config Removed split_queries_by_interval and forward_headers_list from query_range config The fields were deprecated and hence going to be removed in the next major version What this PR does / why we need it: Which issue(s) this PR fixes: Fixes #<issue number> Special notes for your reviewer: Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [x] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [x] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`)	2 years ago
Dylan Guedes	6ce510dc55	Storage: Introduce new "ObjectExists" API (#10360 ) What this PR does / why we need it: Introduce to our `Store` interface a new function named `ObjectExists`, which can be used by external tools to validate if a given file is present on an object store without downloading the file. Which issue(s) this PR fixes: N/A	2 years ago
Karsten Jeschkies	3a4c0e2d47	Rename evaluator and its method to emphasize it's a factory. (#10381 )	2 years ago
Danny Kopping	e3ab2c22de	Fix CLI flags for congestion control (#10394 ) What this PR does / why we need it: Adds `congestion-control` prefixes to new flags. Currently to enable congestion control via a flag, one must use `-store.enabled` which is obviously misleading. --------- Signed-off-by: Danny Kopping <danny.kopping@grafana.com>	2 years ago
Travis Patterson	60edfe0006	Implement "or" statement in line filters (#8962 ) It might be nice to not force a user to use `\|~` when they need to express `or` in LogQL. Example: `{app="foo"} \|= "foo" or "bar"` Edit: A ton of this is generated code or refactors. I've highlighed the parts most interesting to a reviewer --------- Co-authored-by: Karsten Jeschkies <karsten.jeschkies@grafana.com>	2 years ago
Bryan Boreham	a32cbeb37a	indexshipper/storage: fix race conditions (#10314 ) What this PR does / why we need it: There were many race conditions reported if you run `go test -race` on this package. Google's 'singleflight' package is much neater and makes all the race warnings go away. Which issue(s) this PR fixes: Relates to #8586 Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - NA Documentation added - NA Tests updated - [x] `CHANGELOG.md` updated - NA If the change is worth mentioning in the release notes, add `add-to-release-notes` label - NA Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - NA For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`)	2 years ago
Karsten Jeschkies	f58ef7008b	Remove `t2.4xlarger` from sizing tool. (#10347 ) What this PR does / why we need it: The sizing tool was listing an AWS EC2 instance that does not exsit. Which issue(s) this PR fixes: Fixes #10315 Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`)	2 years ago
Trevor Whitney	85e2e52279	fix: respect query matcher in ingester when getting label values (#10375 ) There is a bug when passing a query parameter to the `/label/:label/values`. Currently the ingester is not respecting this query string, and so is always returning all label values. This PR fixes that bug.	2 years ago
Shantanu Alshi	145319e5f5	Remove experimental enable ruler api toggle (#10380 ) Remove `-experimental.ruler.enable-api` in favour of `-ruler.enable-api`. The former was already deprecated for some time.	2 years ago
Christian Haudum	31981bd4af	Remove dependency on Bigchunk, which is a leftover of Cortex and is only used in tests (#10354 ) What this PR does / why we need it: This PR removes the dependency on "Bigchunk", which is a "default" implementation of the `chunk.Data` interface and used only in tests. Special notes for your reviewer: The PR replaces the usage of this chunk type with the Loki implementation (`chunkenc.MemChunk` wrapped into `chunkenc.Facade`) where possible. In case where it was not possible (tests in `pkg/storage/chunk` package) due to otherwise circular import, it was replaces by the `chunk.dummyChunk` implementation, which is just a no-op that implements `chunk.Data` interface) The PR also removes the protos for the ingester client (`pkg/ingester/client/ingester.proto`) which were only referenced in the removed `pkg/util/chunkcompat/compat.go` file. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
Shantanu Alshi	92ba055ae5	Remove ruler.wal-cleaer.period (#10378 ) The config with typo is now replaced by `-ruler.wal-cleaner.period`	2 years ago
Shantanu Alshi	325c332f3e	Remove deprecated s3 sse toggle (#10377 ) This PR removes a legacy and already deprecated boolean setting `-s3.sse-encryption` in favor of the more fine-grain `-s3.sse.*` settings.	2 years ago
Karsten Jeschkies	f8658fdc38	Enable sharding for avg_over_time. (#10373 ) What this PR does / why we need it: The range aggregation `avg_over_time` is simply shardable if there's no label reduction. If there is one it can be express as ``` sum by (method) ( sum_over_time( {container="app"} \| json \| unwrap bytes [$__interval]) ) / sum by (method) ( count_over_time( {container="app"} \| json [$__interval] ) ) ``` Note that `sum by`, `sum_over_time` and `count_over_time` are shardable once more. Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [x] Tests updated - [x] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) --------- Co-authored-by: Owen Diehl <ow.diehl@gmail.com>	2 years ago
Danny Kopping	c725de6035	Querier: add chunk refs fetch time to metrics (#10362 ) What this PR does / why we need it: Adds an entry to the `metrics.go` log line to track how long a query waits for its chunk references to be returned by the index. This is useful when trying to understand where processing time is being spent.	2 years ago
Ashwanth	dff3622eaa	deprecate old index types and non-object stores (#10341 ) What this PR does / why we need it: Deprecates old index types: `aws, aws-dynamo, gcp, gcp-columnkey, bigtable, bigtable-hashed, cassandra, grpc`. Only `tsdb` and `boltdb-shipper` are supported. Deprecates non-object stores. The next major release might not allow the usage of these as `object_store` in `schema_config`: `aws-dynamo, gcp, gcp-columnkey, bigtable, bigtable-hashed, cassandra, grpc.` Deprecates table manager which is not needed once when using `boltdb-shipper` or `tsdb` index type Which issue(s) this PR fixes: Fixes #<issue number> Special notes for your reviewer: Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [x] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`)	2 years ago
Shantanu Alsi	586b9042c6	Upgrade Thanos objstore, dskit and other modules (#10366 ) What this PR does / why we need it: Upgrades various common modules including dskit, thanos objstore, prometheus, etc. Fixes errors/ changes resulting due to module upgrades. This is required in order to eventually upgrade BEL dependency in GEL to point to master. Which issue(s) this PR fixes: None Special notes for your reviewer: Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [x] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`)	2 years ago
Ashwanth	18482c67d6	compactor: deprecate -boltdb.shipper.compactor. prefix in favor of -compactor. (#10344 ) What this PR does / why we need it: Compactor config use `-boltdb.shipper.compactor.` as prefix which is misleading as the config options apply to tsdb index as well. This pr introduces `-compactor.` prefix which can now be used to configure compactor configuration options and deprecates `-boltdb.shipper.compactor.` Special notes for your reviewer: Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [x] Documentation added - [ ] Tests updated - [x] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`)	2 years ago
Kaviraj Kanagaraj	c030217e1b	Revert "LogQL: Introduce `distinct` (#8662 )" (#10356 ) What this PR does / why we need it: This reverts commit `38b298c3fe`. We are removing this PR #8662 because [known issue tracked here](https://github.com/grafana/loki/issues/9594) and some issues we facing internally with query splitting and sharding work(hard to parallelize, merging two `distinct` sets can have duplicates, hard to make it work with metric queries, etc). We decided best to remove it before make it to public release. Which issue(s) this PR fixes: Fixes #<issue number> Special notes for your reviewer: Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [x] Documentation added - [x] Tests updated - [x] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) --------- Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com>	2 years ago
Kaviraj Kanagaraj	760c255431	doc(schema): Start recommending `v12` schema instead of `v11` (#10355 ) What this PR does / why we need it: `v12` is stable schema with more efficient/stable chunks and index formats. Start recommending it instead of old `v11`. Which issue(s) this PR fixes: Fixes #<issue number> Special notes for your reviewer: Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [x] Documentation added --------- Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com>	2 years ago
Ashwanth	cc6634972f	config: deprecates unordered-writes and few unused configs (#10324 ) What this PR does / why we need it: Deprecates the following configs: out of order writes will be the default and the only format with the next major release - ingester.unordered-writes these need not be configurable, were added for testing - log.use-buffered - log.use-sync unused - frontend.forward-headers-list Which issue(s) this PR fixes: Fixes #<issue number> Special notes for your reviewer: Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [x] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`)	2 years ago
Ashwanth	470cbbf118	querier: remove query_timeout and engine timeout from querier conf (#10302 ) What this PR does / why we need it: Removes `query_timeout` and `engine:timeout` from querier's config section Both these configs have been deprecated in 2.7.0 release in favor of `query_timeout` in `limits_config` which also allows applying per-tenant overrides at runtime. CLI flag `-querier.engine.timeout` is also removed Checklist - [X] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [X] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [X] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) --------- Co-authored-by: Christian Haudum <christian.haudum@gmail.com> Co-authored-by: J Stickler <julie.stickler@grafana.com>	2 years ago
Sandeep Sukhani	962e03932d	non-indexed labels: support for deletion of logs with non-indexed labels (#10337 ) What this PR does / why we need it: We recently added support for storing and querying non-indexed labels with each log line. This PR wires up the code for supporting deletion using non-indexed labels. Checklist - [x] Tests updated	2 years ago
Christian Haudum	210c937c41	chore(schema-v13): Fix series store schema creation for v13 (#10348 ) The creation of a series store schema failed when using v13 schema version. Special notes for your reviewer: When adding the `v13` to the series store test cases (`pkg/storage/stores/series/series_store_test.go`) they fail. Example error: ``` === RUN TestSeriesStore_LabelValuesForMetricName/foo_/_toms_/_v13_/_store series_store_test.go:700: ========= Running labelValues with metricName foo with labelName toms with schema v13 series_store_test.go:93: Error Trace: /home/christian/sandbox/grafana/loki/pkg/storage/stores/series/series_store_test.go:93 /home/christian/sandbox/grafana/loki/pkg/storage/stores/series/series_store_test.go:77 /home/christian/sandbox/grafana/loki/pkg/storage/stores/series/series_store_test.go:702 Error: Received unexpected error: invalid schema version Test: TestSeriesStore_LabelValuesForMetricName/foo_/_toms_/_v13_/_store === RUN TestSeriesStore_LabelValuesForMetricName/foo_/_toms_/_v13_/_cached_store series_store_test.go:700: ========= Running labelValues with metricName foo with labelName toms with schema v13 series_store_test.go:93: Error Trace: /home/christian/sandbox/grafana/loki/pkg/storage/stores/series/series_store_test.go:93 /home/christian/sandbox/grafana/loki/pkg/storage/stores/series/series_store_test.go:77 /home/christian/sandbox/grafana/loki/pkg/storage/stores/series/series_store_test.go:702 Error: Received unexpected error: invalid schema version Test: TestSeriesStore_LabelValuesForMetricName/foo_/_toms_/_v13_/_cached_store ``` Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
Travis Patterson	12221f43b5	Put unescaped regexes in a raw string literal (#10331 ) [This PR](https://github.com/grafana/loki/pull/10277) fixed an issue where regexes were being escaped multiple times and causing otherwise valid queries to fail our length check. It put the resulting regexes in double quotes which meant that any regexes that _needed_ escaping were now invalid. This PR make is so the unescaped regex is put in a raw string literal so any extra escaping in unnecessary.	2 years ago
Kaviraj Kanagaraj	4bd5e617c3	chore(schema-v13): Fix `periodConfig.Validate` for `v13` schema (#10336 ) What this PR does / why we need it: Fix periodConfigs `Validate()` for new schema `v13`. Which issue(s) this PR fixes: Fixes #<issue number> Special notes for your reviewer: Found when testing archiver with latest loki Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com>	2 years ago
Karsten Jeschkies	3789f1717c	Set buckets for querier_wait_seconds histogram. (#10334 ) What this PR does / why we need it: We should set buckets for the `querier_wait_seconds` histogram. Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`)	2 years ago
Kaviraj Kanagaraj	bbfb13c3e4	Pin `chunk` and `index` format to `schema` version. (#10213 ) We pin all three `Chunk`, `HeadBlock` and `TSDB` Version to `schema` version in period config. This is the following mapping (after being discussed with @owen-d and @sandeepsukhani ) `v12` (current existing schema) - ChunkFormatV3 (UnorderedHeadBlock) + TSDBv2 `v13` (introducing new schema) - ChunkFormatV4 (UnorderedWithNonIndexedLabelsHeadBlockFmt) + TSDBv3 Note the new schema `v13` supports the latest chunk and index format. NOTES for Reviewer 1. General approach is we removed the idea of `index.LiveFormat`, `chunkenc.DefaultChunkFormat` and `chunkenc.DefaultHeadBlockFmt` and made following two changes. These variables were used before to tie chunk and tsdb formats specific to Loki versions. This PR remove that coupling and pin these formats to `schema` version instead. 1. These variables were replaced with explicit chunk and index formats within those packages (and it's tests) 2. If these variables were used outside it's own packages say by ingester, compactor, etc. Then we extract correct chunk and index versions from the `schema` config. 2. Add two methods to `periodConfig`. (1) `ChunkFormat()` returning chunk and head format tied to schema (2) `TSDBFormat()` returning tsdb format tied to schema. 2. Other ideas I thought of doing but didn't end up doing is make `ChunkFormat` and `IndexFormat` as separate type (rather than `byte` and `int` currently. Similar to `HeadBlockFmt` type). But didnt' do it eventually to keep the PR small and don't want to complicate with lots of changes. 4. Moved couple of test cases from `chunkenc` to `config` package, because the test case was actually testing methods on `schemaconfig` and it was creating cycling dependencies. --------- Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com>	2 years ago
akhilanarayanan	3303844cb0	Refactored forGivenIngestors to use DoUntilQuorum instead of Do (#9984 ) This changes the forGivenIngestors function in pkg/querier/ingester_querier.go to use `DoUntilQuorum` (from [PR #293](https://github.com/grafana/dskit/pull/293)) instead of `Do`. `DoUntilQuorum` is much more efficient than `Do`. The exact differences between them can be seen [here](https://github.com/grafana/dskit/pull/293). --------- Co-authored-by: Trevor Whitney <trevorjwhitney@gmail.com>	2 years ago
Christian Haudum	e3b25fb36c	Deprecate ruler setting with misleading name (#10322 ) The setting defaults to `0s` and is not supposed to be changed, therefore it should be removed in a future version (e.g. 3.0) of Loki. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
Bryan Boreham	cf353bb9a0	Series Index Store: fix race in GetSeries (#10310 )	2 years ago
Danny Kopping	9734c4b832	Minor fixes to congestion control mechanism (#10319 ) What this PR does / why we need it: Minor logging / metric improvements to #10140	2 years ago
Kaviraj Kanagaraj	416320dc5f	Revert "Stopped String() from doubly escaping regex characters" (#10318 ) Reverts grafana/loki#10277 This currently breaks and cause panics for query with regex label matcher. Reverting it till we find the fix.	2 years ago
Bryan Boreham	f89a51fde8	Tracing: elide small traces for Stats call (#10308 ) What this PR does / why we need it: At work I see a lot of traces getting data dropped due to being tens of megabytes in size. Looking at what was captured, I see many many repeats of these four spans: <img width="1070" alt="image" src="https://github.com/grafana/loki/assets/8125524/4268e664-6910-49bb-8f68-69b139a5a66b"> Events on the parent trace are cheaper and, for small operations like this, just as useful. Where there was an event line I added a `function` field matching the previous span name. For `storeEntry.Stats()` I didn't add an event line as it doesn't seem to do very much. But please correct me; I'm not hugely familiar with the Loki codebase. Special notes for your reviewer: Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - NA Documentation added - NA Tests updated - [x] `CHANGELOG.md` updated - NA If the change is worth mentioning in the release notes, add `add-to-release-notes` label - NA Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - NA For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`)	2 years ago
Bryan Boreham	ecd371db87	indexshipper/compactor: fix two race conditions (#10313 ) What this PR does / why we need it: We must call wg.Add before starting the goroutine. And protect the data in mockCompactorClient. And another fix in shipper/index/compactor. Which issue(s) this PR fixes: Part of #8586 Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - NA Documentation added - NA Tests updated - NA `CHANGELOG.md` updated - NA If the change is worth mentioning in the release notes, add `add-to-release-notes` label - NA Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - NA For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`)	2 years ago
loanshark	43e542ee64	Storage: Remove `SignatureVersionv2` from the AWS S3 API request authentication. (#10295 ) AWS S3 stopped accepting requests signed using SigV2 in all regions on June 24, 2019, any requests signed using SigV2 made after this time will fail. * https://docs.aws.amazon.com/AmazonS3/latest/userguide/UsingAWSSDK.html#UsingAWSSDK-sig2-deprecation * https://aws.amazon.com/blogs/aws/amazon-s3-update-sigv2-deprecation-period-extended-modified/	2 years ago
Owen Diehl	f53498c4aa	allows filter-only matchers in series endpoint (#10269 ) We already allow empty matchers `{}` as a special case in series queries to return all series. This PR also allows specifying only filter-style matchers. This is specifically helpful when a tenant is trying to understand all streams which do _not_ have a corresponding label. ``` $ logcli series '{does_not_exist=""}' --analyze-labels --org-id="fake" 2023/08/16 10:13:32 http://localhost:3100/loki/api/v1/series?end=1692206012889997000&match=%7Bdoes_not_exist%3D%22%22%7D&start=1692202412889997000 Total Streams: 6 Unique Labels: 2 ``` Co-authored-by: Michel Hollands <42814411+MichelHollands@users.noreply.github.com>	2 years ago
Arve Knudsen	a60e0167a8	Upgrade to latest dskit main (#10300 ) What this PR does / why we need it: Upgrade to latest dskit main branch, which mainly has changes to its `log` package, which requires corresponding fixing up in Loki.	2 years ago
Karsten Jeschkies	b1a948e28b	Record waiting time for queriers to receive requests. (#10253 ) What this PR does / why we need it: We want to investigate whether work stealing among queriers would be useful. So we are recording the wait time of queriers for new requests in their queue. Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`)	2 years ago
Danny Kopping	a30629e36e	Azure Blob Storage is no longer experimental (#10299 ) What this PR does / why we need it: Azure Blob Storage is being used in production by many organisations, including Grafana Labs. It should no longer be marked experimental. Which issue(s) this PR fixes: Fixes #10298 Signed-off-by: Danny Kopping <danny.kopping@grafana.com>	2 years ago
akhilanarayanan	4013c1f8b5	Stopped String() from doubly escaping regex characters (#10277 ) What this PR does / why we need it: When queries have regex, characters that needed escaping were escaped twice. Now they're only singly escaped. This is good because it reduces the character count so users can submit queries closer to the 5120 byte limit. Which issue(s) this PR fixes: Sort of addresses #10153 Special notes for your reviewer: Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [x] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) Co-authored-by: Akhila Narayanan <akhilanarayanan@Akhilas-MacBook-Air.local>	2 years ago
Danny Kopping	beed298549	Dynamic client-side throttling to avoid object storage rate-limits (GCS only) (#10140 ) What this PR does / why we need it: Across the various cloud providers' object storage services, there are different rate-limits implemented. Rate-limits can be imposed under multiple conditions, such as server-side scale up (ramping up from low volume to high, "local" limit), reaching some defined upper limit ("absolute" limit), etc. We cannot know apriori when these rate-limits will be imposed, so we can't set up a client-side limiter to only allow a certain number of requests through per second. Additionally, that would require global coordination between queriers - which is difficult. With the above constraints, I have instead taken inspiration from TCP's [congestion control algorithms](https://en.wikipedia.org/wiki/TCP_congestion_control). This PR implements [AIMD](https://en.wikipedia.org/wiki/Additive_increase/multiplicative_decrease) (Additive Increase, Multiplicative Decrease), which is used in the congestion _avoidance_ phase of congestion control. The default window size (object store requests per second) is 2000; in other words, we skip the "slow start" phase. The controller uses the Go [`rate.Limiter`](https://pkg.go.dev/golang.org/x/time/rate), which implements the token-bucket algorithm. To put it simply: - every successful request widens the window (per second client rate-limit) - every rate-limited response reduces the window size by a backoff factor (0.5 by default, so it will halve) - when the limit has been reached, the querier will be delayed from making further requests until tokens are available	2 years ago
Mohamed-Amine Bouqsimi	176e9b7292	Add filter on rules' labels to API query (#9128 )	2 years ago
Danny Kopping	a18ca92375	Track effectiveness of hedged requests (#10281 ) What this PR does / why we need it: We currently have request hedging implemented in many object store clients. Right now we track how many hedged requests we make, and how many are rate-limited, but nothing actually tells us _how effective_ the hedged requests are. In other words, we don't know how many hedged requests were issued and _won_ (completed before the original).	2 years ago
Owen Diehl	9097f1ff42	Sharding optimizations (#10101 ) A few bugfixes and more sharding optimizations * fix bug on `<aggr> by\|without ()` groupings which removed the grouping while downstreaming * shardable implementations for `max+min`, operation specific merge strategies which enable many more types of sharded requests, even when label-reduction is performed at edge.	2 years ago
Trevor Whitney	4d2a94ebd3	Return intersecting labels when aggregating volume by label (#10248 ) This PR changes the behavior the `index/volume` endpoint to return intersecting labels when requesting volume aggregating by label. Previously it only returned labels that matched selectors in the query. The old behavior is still possible using the `targetLabels` argument.	2 years ago
Charles Korn	611481b6cf	Upgrade dskit (#10249 )	2 years ago
Salva Corts	287f29bc46	Update chunk and head fmt to v4 (non-indexed labels) (#10242 ) What this PR does / why we need it: This PR updates the default chunk and heads format to v4 to support writing and reading non-indexed labels. --------- Co-authored-by: Sandeep Sukhani <sandeep.d.sukhani@gmail.com>	2 years ago
Charles Korn	8e6a5433ac	Upgrade dskit and remove dependency on weaveworks/common (#10220 ) What this PR does / why we need it: This PR upgrades `github.com/grafana/dskit` removes Loki's dependency on `github.com/weaveworks/common`. The changes in dskit, apart from the migration (https://github.com/grafana/dskit/pull/356), are: * https://github.com/grafana/dskit/pull/347 * https://github.com/grafana/dskit/pull/349, which required small changes in `pkg/canary/reader/reader.go` and `pkg/canary/writer/push.go` * https://github.com/grafana/dskit/pull/352 * https://github.com/grafana/dskit/pull/354 Which issue(s) this PR fixes: (none) Special notes for your reviewer: Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [n/a] Documentation added - [n/a] Tests updated - [n/a] `CHANGELOG.md` updated - [n/a] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [n/a] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [n/a] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) --------- Co-authored-by: Michel Hollands <42814411+MichelHollands@users.noreply.github.com>	2 years ago
Dylan Guedes	0e729e032c	Configuration: Allow customization of default runtime values (#10193 ) What this PR does / why we need it: Adds a new section to our runtime configuration that will allow customization of default runtime values. For instance, our default `log_push_request` is `false`. Previously, there was no way of setting it to `true`, so you'd have to set it for every existing tenant. Now you can by making usage of the new `default` section.	2 years ago
Karsten Jeschkies	a3741d1068	Include response type in merging error message and merge already merged views. (#10218 )	2 years ago

1 2 3 4 5 ...

2271 Commits (bcf917f65ad70568467d4af02c385607c7032ac9)