apitech/loki - loki - Gitea: Git with a cup of tea

Commit Graph

Author	SHA1	Message	Date
Sandeep Sukhani	162bbb11be	do not run retention loop when retention is not enabled (#11280 ) What this PR does / why we need it: Do not run retention loop when retention is not enabled	2 years ago
Owen Diehl	6e93d150d8	utilities for reducing ngram allocations in token iterator construction (#11276 ) Previously, the `calculatePrefix` function was used to generate the token buffer used, but this wouldn't necessarily have enough space to append the ngram afterwards without reallocating+copying the token buffer. This PR ensures we allocate enough space for both prefix and ngram in the token buffer.	2 years ago
Quentin Bisson	4455cd9d7d	[helm] Fix tracing configuration (#11186 ) What this PR does / why we need it: This PR allows user to enable tracing in the new SSD setup and fixes incorrect documentation because it is currently impossible to enable tracing in this chart (cf. `766f27645d/cmd/loki/main.go (L81)`) Which issue(s) this PR fixes: Fixes #<issue number> Special notes for your reviewer: Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [x] Documentation added - [x] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) - [ ] If the change is deprecating or removing a configuration option, update the `deprecated-config.yaml` and `deleted-config.yaml` files respectively in the `tools/deprecated-config-checker` directory. [Example PR](`0d4416a4b0`) --------- Signed-off-by: QuentinBisson <quentin@giantswarm.io>	2 years ago
Sandeep Sukhani	529084935c	use nanosecond precision for timestamp in compacted boltdb-shipper index file names (#11277 ) What this PR does / why we need it: In PR #9884, we split the compaction and retention loop to run them concurrently. Although we make sure we do not work on the same index from compaction and retention loop, there is a chance that one could run immediately after the other and finish quickly enough to build the index with the same name as the previous one because in `boltdb-shipper` index, we use epoch with `Seconds` precision while building the name of the compacted index file. Since compaction uploads the new file first and then deletes the old file, if the index is built with the same name, we end up uploading the file and deleting it afterwards. This PR fixes the issue by using ns precision for the timestamp in the filenames. Special notes for your reviewer: This is not a problem for TSDB since we also add a checksum to the filenames of the index during compaction.	2 years ago
Pavaningithub	658f4f1f26	Fix Typo in the _index.md (#11242 ) What this PR does / why we need it: Fixing the small typo in the provided example [here](https://grafana.com/docs/loki/latest/setup/install/helm/configure-storage/#configure-storage). This PR just updates the doc. Which issue(s) this PR fixes: Fixes #<issue number> Special notes for your reviewer: Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) - [ ] If the change is deprecating or removing a configuration option, update the `deprecated-config.yaml` and `deleted-config.yaml` files respectively in the `tools/deprecated-config-checker` directory. [Example PR](`0d4416a4b0`) Co-authored-by: Michel Hollands <42814411+MichelHollands@users.noreply.github.com>	2 years ago
Bilal Khan	49d5761d90	Fixed the grammatical mistake in an _index.md file (#11226 ) What this PR does / why we need it: In this PR, I fixed the grammatical mistake in an _index.md file. Which issue(s) this PR fixes: Fixes #<issue number> Special notes for your reviewer: Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [x] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) - [ ] If the change is deprecating or removing a configuration option, update the `deprecated-config.yaml` and `deleted-config.yaml` files respectively in the `tools/deprecated-config-checker` directory. [Example PR](`0d4416a4b0`) Signed-off-by: Bilal Khan <bilalkhanrecovered@gmail.com> Co-authored-by: J Stickler <julie.stickler@grafana.com>	2 years ago
Paul Rogers	5f4a719bb6	tokenizer v1 cleanup (#11272 ) What this PR does / why we need it: Removes all usage of v1 tokenizers, renames v2 to v1 since we never released this in a production way. Which issue(s) this PR fixes: Fixes #<issue number> Special notes for your reviewer: Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) - [ ] If the change is deprecating or removing a configuration option, update the `deprecated-config.yaml` and `deleted-config.yaml` files respectively in the `tools/deprecated-config-checker` directory. [Example PR](`0d4416a4b0`)	2 years ago
Karsten Jeschkies	8d34f857bc	Propagate trace ID with HTTP gRPC request. (#11251 ) The changes in https://github.com/grafana/loki/pull/10688 did not propage the trace ID from the context. `Frontend.RoundTripGRPC` would inject the trace ID into the request. That's not done in `Frontend.Do`. This changes extends the `codec.EncodeRequest` to inject the trace ID there. This is more inline with other metadata.	2 years ago
Sandeep Sukhani	6a62b8cf42	compaction: Separate metrics for tracking retention and compaction (#11263 ) What this PR does / why we need it: In PR #9884, we separated the retention loop from compaction to avoid blocking compaction for too long due to some intensive delete requests. Currently, we track retention and compaction using the same metrics. This PR adds separate metrics for monitoring retention operation. I have also updated the Retention dashboard to use the new metrics.	2 years ago
Joao Marcal	5535267f2c	operator: Remove outdated BoltDB dashboards (#11022 ) Co-authored-by: Periklis Tsirakidis <periklis@redhat.com>	2 years ago
Joao Marcal	0c10f28a46	operator: Adds new value v13 to schema (#10932 ) Co-authored-by: Periklis Tsirakidis <periklis@redhat.com>	2 years ago
Kaviraj Kanagaraj	30d0030bf9	inflight-logging: Add extra metadata to inflight requests logging (#11243 ) What this PR does / why we need it: logging: Add extra metadata to inflight requests This adds extra metadata (similar to what we have in `metrics.go`) but for queries in in-flight (both started and retrying) Changes: Adds following data 1. Query Hash 2. Start and End time 3. Start and End delta 4. Length of the query 5. Moved the helper util to `queryutil` package because of cyclic dependencies with `logql` package. Which issue(s) this PR fixes: Fixes #<issue number> Special notes for your reviewer: Find the screenshots of log entries looks like (both in `retry.go` and `roundtrip.go`) ![Screenshot 2023-11-16 at 13 01 32](https://github.com/grafana/loki/assets/3735252/177e97ed-6ee8-41dd-b088-2e4f49562ba0) ![Screenshot 2023-11-16 at 13 02 15](https://github.com/grafana/loki/assets/3735252/fb328a37-dbe3-483e-b083-f21327858029) Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [x] Tests updated - [x] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) - [ ] If the change is deprecating or removing a configuration option, update the `deprecated-config.yaml` and `deleted-config.yaml` files respectively in the `tools/deprecated-config-checker` directory. [Example PR](`0d4416a4b0`) --------- Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com>	2 years ago
Sandeep Sukhani	c716e498ad	compactor: do not block compation when retention is taking too long (#9884 ) What this PR does / why we need it: Currently, we perform compaction and apply retention in the same loop. Although we have a flag for not applying retention every time we perform compaction, we still see compaction getting blocked when processing some intensive delete requests(processed while applying retention). This PR separates out the compaction and retention to run in a separate loop. I have added a table-locking feature to avoid compaction and retention from processing the same tables at a time. However, compaction and retention would treat locked tables differently, as explained below: * When compaction sees a table is locked: It would skip the table and move on to the following table. However, before skipping, it would check if the table has any uncompacted files and increment the newly added counter called `loki_compactor_skipped_compacting_locked_tables_total` to track how often we are skipping compacting tables which have uncompacted files. * When retention sees a table is locked: It would wait for the lock to be released since we can't skip any tables while processing delete requests. Special notes for your reviewer: * The check for tables with uncompacted files looks for count > 1 because initially, we did not support per tenant index in `boltdb-shipper`, so a table can have a single compacted multi-tenant index file. In a rare case where we have a single file which was supposed to be compacted away, it is okay to have a single uncompacted file for a while. The aim here is to not block compaction for too long in a large cell with too many uncompacted files. * Retention only works on the compacted index, so we first compact down any uncompacted files while applying retention. Checklist - [x] Tests updated --------- Co-authored-by: Ashwanth <iamashwanth@gmail.com>	2 years ago
Owen Diehl	58eaad9947	boundscheck & partitioning fingerprints between blocks (#11237 ) Adds a few utilities for comparing fingerprints to blocks that cover a specific fingerprint range. Will likely need to be refactored with more comprehensive types, but the logic still applies.	2 years ago
Owen Diehl	6db9d778a1	inverts the logic when testing a block against a list of chunks (#11248 ) returns the list of chunks in the bloom which failed the test so we can merge these results across blocks	2 years ago
Owen Diehl	15d98fa7ca	iterable tokenizer w/ comparison benching (#11255 ) Rebuilds tokenizers based on an iterable approach. This ends up being significantly faster -- benchmarks below. This PR doesn't remove the old tokenizers, just adds the new ones. Revamping tokenizer consumers & removing the old variants can be done in a separate PR. Roughly, raw tokenizers complete in 66% of the time and chunkID prefixed tokenizers in ~55-60% of the time. ``` pkg: github.com/grafana/loki/pkg/storage/bloom/v1 BenchmarkTokens/three/v1-10 134811 8648 ns/op 0 B/op 0 allocs/op BenchmarkTokens/three/v2-10 179797 5728 ns/op 0 B/op 0 allocs/op BenchmarkTokens/threeSkip1/v1-10 215822 5260 ns/op 0 B/op 0 allocs/op BenchmarkTokens/threeSkip1/v2-10 343784 3514 ns/op 0 B/op 0 allocs/op BenchmarkTokens/threeChunk/v1-10 82394 14559 ns/op 0 B/op 0 allocs/op BenchmarkTokens/threeChunk/v2-10 146973 8138 ns/op 128 B/op 1 allocs/op BenchmarkTokens/threeSkip1Chunk/v1-10 152475 7878 ns/op 0 B/op 0 allocs/op BenchmarkTokens/threeSkip1Chunk/v2-10 251373 4776 ns/op 128 B/op 1 allocs/op PASS ok github.com/grafana/loki/pkg/storage/bloom/v1 10.206s ```	2 years ago
Chris Knutson	5395c36314	ksonnet: generate tsdb_shipper storage_config even if using_boltdb_shipper is false (#11195 ) What this PR does / why we need it: When configuring tsdb shipper without using bolt db shipper, the ksonnet library fails to generate the tsdb_shipper storage config.	2 years ago
Periklis Tsirakidis	8d0fc171cc	operator: Update dependencies and dev tools (#11232 )	2 years ago
Dylan Guedes	4309add1b2	Fix per-pod panel unit (from 'ms' to 's') (#11245 ) What this PR does / why we need it: Change per-pod latency panel unit from 'ms' to 's' (the metric is in the 's' unit)	2 years ago
Trevor Whitney	5c5948367a	Do not run snyk pr comment workflow on forks (#11240 ) Only run the snyk pr comment workflow on PRs from branches, not on forks. We can't run this `on: pull_request_target` because in needs access to the `SNYK_TOKEN` secret, and when run `on: pull_request`, forks don't have permissions to comment on the PR (because they don't get the `GITHUB_TOKEN` secret.	2 years ago
Angus Dippenaar	258a8b3108	Update nix configuration (#8452 ) Fix nix configuration (mainly failing test), and break up various binaries (ie. logcli, promtail, loki, etc.) into their own pakcages. --------- Co-authored-by: Trevor Whitney <trevorjwhitney@gmail.com>	2 years ago
keyolk	8328345a8b	lambda-promtail: fix IAM policy for clouddwatch log stream (#10909 )	2 years ago
Poyzan	d22c1fd39d	[bloom-compactor] Add configs to enable compactor per tenant (#11235 ) What this PR does / why we need it: We want to control bloom compaction per tenant basis. Adding configs to enable/disable bloom compactor. Which issue(s) this PR fixes: Fixes #<issue number> Special notes for your reviewer: Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [x] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) - [ ] If the change is deprecating or removing a configuration option, update the `deprecated-config.yaml` and `deleted-config.yaml` files respectively in the `tools/deprecated-config-checker` directory. [Example PR](`0d4416a4b0`)	2 years ago
Poyzan	3f0f8fa5f5	[bloom-compactor] Move meta.json creation at the end of compaction cycle (#11234 ) What this PR does / why we need it: This is a follow up from https://github.com/grafana/loki/pull/11115 Instead of a creating a meta file per bloom creation, create a meta file per compaction cycle. Which issue(s) this PR fixes: Fixes #<issue number> Special notes for your reviewer:	2 years ago
Karsten Jeschkies	c6f421233b	Increase test coverage for AST serialization to >94%. (#11230 ) What this PR does / why we need it: This is a follow up to https://github.com/grafana/loki/pull/11123 and fixes a few bugs discovered by increasing the test coverage. Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [x] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) - [ ] If the change is deprecating or removing a configuration option, update the `deprecated-config.yaml` and `deleted-config.yaml` files respectively in the `tools/deprecated-config-checker` directory. [Example PR](`0d4416a4b0`)	2 years ago
Karsten Jeschkies	3a7b5d246b	Serialize query AST to JSON. (#11123 ) What this PR does / why we need it: This introduces the visitor pattern to serialize the LogQL AST to JSON. We've chose this pattern because it will be more flexible one the AST is encoded into Protobuf and an actual queryplan. Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [x] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) - [ ] If the change is deprecating or removing a configuration option, update the `deprecated-config.yaml` and `deleted-config.yaml` files respectively in the `tools/deprecated-config-checker` directory. [Example PR](`0d4416a4b0`)	2 years ago
Ashwanth	32e9ee8209	chore: follow-up to #11151 and #11025 (#11225 ) What this PR does / why we need it: - remove usage of `enforce_metric_name` in ksonnet - https://github.com/grafana/loki/pull/11151 - make `cortex_ruler_client_request_duration_seconds` prefix configurable - updates upgrade guide to remove reference to metrics that loki does not export Which issue(s) this PR fixes: Fixes #<issue number> Special notes for your reviewer: Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) - [ ] If the change is deprecating or removing a configuration option, update the `deprecated-config.yaml` and `deleted-config.yaml` files respectively in the `tools/deprecated-config-checker` directory. [Example PR](`0d4416a4b0`)	2 years ago
Karsten Jeschkies	117f108642	Return error if span context was not found (#11207 ) What this PR does / why we need it: This is a follow up to https://github.com/grafana/loki/pull/10956. `lokigrpc.GetParentSpanForRequest` would ignore `opentracing.ErrSpanContextNotFound`. However, that should be up to the caller to decide whether this error should be ignored or not. Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) - [ ] If the change is deprecating or removing a configuration option, update the `deprecated-config.yaml` and `deleted-config.yaml` files respectively in the `tools/deprecated-config-checker` directory. [Example PR](`0d4416a4b0`)	2 years ago
Owen Diehl	265018b729	populate empty resonse for IndexStatsResponse and VolumeResponse (#11209 ) Fixes a panic when the underlying response struct is accessed but not populated after an out-of-bounds request is short-circuited via `NewEmptyResponse` [here](https://github.com/grafana/loki/blob/main/pkg/querier/queryrange/limits.go#L155-L164). This PR ensures the embedded types are not nil in accordance with the other variants in this function.	2 years ago
Christian Haudum	e5b006ce0a	Enable/disable bloom gateway per tenant (#11203 ) This commits adds a per-tenant setting to enable/disable the use of the bloom gateway component for filtering chunks. As an operator, you want to control what tenant can use the filtering of chunks based on search string, because the creation of the bloom filters requires a significant amount of resources. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
Karsten Jeschkies	05c4b77ed0	Define QueryResponse and QueryRequest protobufs. (#10956 ) What this PR does / why we need it: This will be a follow up to https://github.com/grafana/loki/pull/10688 finally switching over to protos. Hang on tight! Everything is behind the feature flag `-frontend.encoding=protobuf` which will disable the transcoding to HTTP and HTTPgRPC. Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [x] Tests updated - [x] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) --------- Signed-off-by: Callum Styan <callumstyan@gmail.com> Co-authored-by: Callum Styan <callumstyan@gmail.com> Co-authored-by: Danny Kopping <dannykopping@gmail.com>	2 years ago
Salva Corts	54edb2143f	Followups #11154 (#11204 ) Addressing comments from #11154 after it was merged	2 years ago
Salva Corts	4248825ad2	Bloom-compactor Sharding (#11154 ) What this PR does / why we need it: This PR adds tenant and fingerprint (FP) sharding to bloom compactors. Note that the bloom-compactor doesn't yet perform any compaction, but iterates through all tables, tenants, and series checking if the compactor owns the tenant and the series (by the series FP). Actual compaction will be implemented with https://github.com/grafana/loki/pull/11115. A new structure `Job` is added which will carry around all the context for a compaction job such as the tenant ID, the table name, and the series FP. The sharding strategy has two methods: - `OwnsTenant(tenant string)`: Checks if the compactor shard owns the tenant. - `OwnsJob(job Job)`: Checks (again) if the compactor owns the job's tenant. Then, it checks if the compactor owns the job's fingerprint by looking inside the tenant subring. We add a new per-tenant limit: `bloom_compactor_shard_size`. If it's 0, the tenant can use all compactors (i.e. `OwnsTenant` will always return `true`), otherwise, only `bloom_compactor_shard_size` out of the total number of compactors will own the tenant. A given job's FP will be owned by exactly one compactor within the tenant shard. Special notes for your reviewer: - Added a bunch of metrics in `metrics.go` - Added a test for the sharding strategy	2 years ago
Jack Baldry	0bc38e5775	Update to latest version of doc-validator (#11200 ) https://github.com/grafana/technical-documentation/releases/tag/doc-validator%2Fv4.0.0 Signed-off-by: Jack Baldry <jack.baldry@grafana.com>	2 years ago
Kaviraj Kanagaraj	4b20f95098	chore(jsonnet): Upgrade memcached-exporter to `0.11.3` (#11187 ) What this PR does / why we need it: Rationale being 1. Unify with extstore based cache we use on cloud 2. Looks like some extra useful metrics are exported on `0.11.3` Which issue(s) this PR fixes: Fixes #<issue number> Special notes for your reviewer: Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) - [ ] If the change is deprecating or removing a configuration option, update the `deprecated-config.yaml` and `deleted-config.yaml` files respectively in the `tools/deprecated-config-checker` directory. [Example PR](`0d4416a4b0`) Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com>	2 years ago
fostec82	32f1ec2fda	feat(logstash): clients logstash output structured metadata support (#10899 ) What this PR does / why we need it: This PR adds support for Loki Structured Metadata to the logstash output plugin. Special notes for your reviewer: Given that Structure Metadata is enabled as an experimental feature plugin changes were done in a way that is backward compatible for users not wanted structured metadata. Co-authored-by: Jared King <kingjs@gmail.com> Co-authored-by: J Stickler <julie.stickler@grafana.com>	2 years ago
Tobias McNulty	42e81487e1	Fix S3 Helm example (#11180 ) What this PR does / why we need it: S3 Helm example uses the wrong keys. See: `cc6f0eca98/production/helm/loki/values.yaml (L280-L281)` Which issue(s) this PR fixes: Fixes #<issue number> Special notes for your reviewer: Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) - [ ] If the change is deprecating or removing a configuration option, update the `deprecated-config.yaml` and `deleted-config.yaml` files respectively in the `tools/deprecated-config-checker` directory. [Example PR](`0d4416a4b0`) --------- Co-authored-by: J Stickler <julie.stickler@grafana.com>	2 years ago
Ashwanth	766f27645d	config: remove flags & YAML settings deprecated in 2.9.x (#11151 ) What this PR does / why we need it: - remove `-ruler.evaluation-delay-duration` - remove `boltdb.shipper.compactor.deletion-mode` - remove compactor flags with prefix`-boltdb.shipper.compactor.` - remove `validation.enforce-metric-name` - remove `log.use-buffered`, `log.use-sync` Special notes for your reviewer: `log.use-buffered`, `log.use-sync` are not mentioned in the upgrade guide since they were hidden fields Checklist* - [X] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [X] Tests updated - [x] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [X] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) - [X] If the change is deprecating or removing a configuration option, update the `deprecated-config.yaml` and `deleted-config.yaml` files respectively in the `tools/deprecated-config-checker` directory. [Example PR](`0d4416a4b0`) --------- Co-authored-by: J Stickler <julie.stickler@grafana.com>	2 years ago
Paul Rogers	c4ed0d0b7f	Tokenizer tests and TokenizeLine updates (#11133 ) What this PR does / why we need it: The thrust of this PR is to ensure we have tests for each major function of the Bloom Tokenizer. In addition, there was some cleanup, in that constants are used to set some common parameters. Lastly, the TokenizeLine() call was updated to correctly tokenize a line when a "skip tokenizer" is utilized. Which issue(s) this PR fixes: Fixes #<issue number> Special notes for your reviewer: Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) - [ ] If the change is deprecating or removing a configuration option, update the `deprecated-config.yaml` and `deleted-config.yaml` files respectively in the `tools/deprecated-config-checker` directory. [Example PR](`0d4416a4b0`)	2 years ago
Poyzan	cc6f0eca98	Return checksum from BuildFrom function (#11178 ) What this PR does / why we need it: Follow up from https://github.com/grafana/loki/pull/11115 We want to know the checksum of the bloom blocks before uploading them to storage. Instead of opening the file and reading it, this PR returns the checksum from where it's created.	2 years ago
Karsten Jeschkies	979530b6b9	Propagate query metrics and cache num information. (#11176 ) What this PR does / why we need it: https://github.com/grafana/loki/pull/10858 removed the extraction of the query time header on the querier side and the generation of the cache number. This change adds them back and uses the headers of the response format. Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) - [ ] If the change is deprecating or removing a configuration option, update the `deprecated-config.yaml` and `deleted-config.yaml` files respectively in the `tools/deprecated-config-checker` directory. [Example PR](`0d4416a4b0`) --------- Signed-off-by: Edward Welch <edward.welch@grafana.com> Co-authored-by: Edward Welch <edward.welch@grafana.com>	2 years ago
Trevor Whitney	e4f8c1916f	Default build-image target arch to amd64 (#11148 ) What this PR does / why we need it: The changes in https://github.com/grafana/loki/pull/11130 broke `make build-image` and `make build-image-push`. This defaults the target arch added in #11130 so those make targets still work.	2 years ago
Trevor Whitney	b1f9be5171	always align volume range timestamp to end of step (#11136 ) What this PR does / why we need it: We had originally assumed that it would be weird for a user to get back a timeseries response from `volume_range` that didn't include a datapoint with a timestamp at the start of their requested range. This was a bad assumption. The `volume_range` endpoint is inherently doing aggregations. For each step, a datapoint is calculated representing the percent of chunk volumes that the selector appeared in. It therefore stands to reason that the steps volume was not that value until the end of the step time range, as we are to assume volume is monotonically increasing. The fact that we were using the start of the range made it difficult to turn this data into a per-second rate, which this PR aims to fix. Which issue(s) this PR fixes: Fixes #11134	2 years ago
Poyzan	fa378ac0ec	Add first step for creating a single bloom from series (#11115 ) What this PR does / why we need it: This is the first step to create a bloom from a series. The logic to get tenants and tables is a temporary hack and will be replaced by the ring logic in https://github.com/grafana/loki/pull/11154 Which issue(s) this PR fixes: Fixes #<issue number> Special notes for your reviewer: It does not handle appending multiple blooms into blocks. It does not handle creating and uploading of meta files. Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) - [ ] If the change is deprecating or removing a configuration option, update the `deprecated-config.yaml` and `deleted-config.yaml` files respectively in the `tools/deprecated-config-checker` directory. [Example PR](`0d4416a4b0`)	2 years ago
Dylan Guedes	30fbe38341	Revert "Introduces cache to TSDB postings (#9621 )" (#11157 ) This reverts commit `1221658fc5`. What this PR does / why we need it: We decided to discontinue the postings cache a few releases ago for not finding any evidences that postings was the culprit for the bad igw performance. Plus, we noticed a major issue with it: it doesn't bubble down shards correctly, so every query end up processing every shard.	2 years ago
Ed Welch	9418fb7125	Loki: fix mapping of non-retryable error codes (#11164 ) What this PR does / why we need it: Fallout of a recent change to reduce complexity in our tripperware/middleware in #10688 lead to non-retryable errors being retried by the retry middleware in the frontend. This PR is a solution to that problem, I'm not sure it's the best solution, but it's a solution.... Which issue(s) this PR fixes: Fixes #<issue number> Special notes for your reviewer: Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) - [ ] If the change is deprecating or removing a configuration option, update the `deprecated-config.yaml` and `deleted-config.yaml` files respectively in the `tools/deprecated-config-checker` directory. [Example PR](`0d4416a4b0`) Signed-off-by: Edward Welch <edward.welch@grafana.com>	2 years ago
Karsten Jeschkies	6d95a86ae2	Extract query tags from HTTP requests. (#11147 ) What this PR does / why we need it: https://github.com/grafana/loki/pull/10858 removed the code that would inject query tags into the context of the querier. This change adds an extraction in the decode method.	2 years ago
Periklis Tsirakidis	fecdd85210	operator: Update deps to secure webhooks for CVE-2023-44487 (#11129 )	2 years ago
Robert Jacob	cea2e0ed58	operator: Update skipRange for next OpenShift release (#11106 )	2 years ago
Eli Lowinger	dfc0013163	Use correct plugin name for fluentbit ecs task (#11127 ) What this PR does / why we need it: Currently the docs suggest using the `loki` output plugin, however the options provided align with the `grafana-loki` plugin Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) - [ ] If the change is deprecating or removing a configuration option, update the `deprecated-config.yaml` and `deleted-config.yaml` files respectively in the `tools/deprecated-config-checker` directory. [Example PR](`0d4416a4b0`) Co-authored-by: J Stickler <julie.stickler@grafana.com>	2 years ago

1 2 3 4 5 ...

5893 Commits (162bbb11be9c527d14dd3d01816d0091eb8691fc) All Branches Search

5893 Commits (162bbb11be9c527d14dd3d01816d0091eb8691fc)

All Branches