apitech/loki - loki - Gitea: Git with a cup of tea

Commit Graph

Author	SHA1	Message	Date
Periklis Tsirakidis	c573defcbb	operator: Fix storing authentication credentials in the Loki ConfigMap (#11357 ) Co-authored-by: Robert Jacob <xperimental@solidproject.de> Co-authored-by: Robert Jacob <rojacob@redhat.com>	1 year ago
Karsten Jeschkies	8dde7b9d49	Support tail requests with protobuf encoding. (#11426 ) What this PR does / why we need it: The `TailRequest` would use `bytes plan` instead of `Plan plan`. The former is not well supported in gogoprotobuf. Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [x] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) - [ ] If the change is deprecating or removing a configuration option, update the `deprecated-config.yaml` and `deleted-config.yaml` files respectively in the `tools/deprecated-config-checker` directory. [Example PR](`0d4416a4b0`)	1 year ago
Karsten Jeschkies	f67fff3eb2	Approximate `quantile_over_time` (#10417 ) What this PR does / why we need it: This change shards `quantile_over_time` queries using t-digest or DDSketch approximations. It can be enabled with `querier.shard_aggregations=quantile_over_time`. Outstanding - [x] Replace generic return type of `StepEvaluator` with interface `StepResult`. - [x] Send mapped query with quantile sketch expression from frontend to querier over the wire. - [x] Serialize sketches. See https://github.com/influxdata/tdigest/issues/34 - [x] Add feature flag. Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [x] Documentation added - [x] Tests updated - [x] `CHANGELOG.md` updated - [x] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [x] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) --------- Signed-off-by: Callum Styan <callumstyan@gmail.com> Co-authored-by: Callum Styan <callumstyan@gmail.com>	1 year ago
J Stickler	5e3496739c	Update notes to use shortcode (#11408 ) What this PR does / why we need it: Updates Notes to use the Hugo admonition shortcode. Which issue(s) this PR fixes: Replaces PR #9409 which has gone stale.	1 year ago
Zirko	ee008cff93	Helm: remove ruler.enabled condition in networkpolicies which lead to linting errors (#11305 ) What this PR does / why we need it: This PR removes the `if .Values.ruler.enabled` condition from the `loki-egress-alertmanager` networkpolicy as this was causing linting issues (i.e nil pointer) whenever enabling the networkpolicies since the `ruler` field is not present in the values. From what I know, the ruler is now part of the backend, thus I also replaced the `rulerSelectorLabels` by the `backendSelectorLabels`. Special notes for your reviewer: Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [x] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [x] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) - [ ] If the change is deprecating or removing a configuration option, update the `deprecated-config.yaml` and `deleted-config.yaml` files respectively in the `tools/deprecated-config-checker` directory. [Example PR](`0d4416a4b0`) --------- Signed-off-by: QuantumEnigmaa <thibaud@giantswarm.io> Co-authored-by: Michel Hollands <42814411+MichelHollands@users.noreply.github.com>	1 year ago
Sandeep Sukhani	62a2217d83	compactor: fix metric name for a compactor (#11412 ) What this PR does / why we need it: I added a new metric but prefixed it by mistake with `loki_boltdb_shipper` instead of `loki_compactor`. This PR fixes it.	1 year ago
Karsten Jeschkies	51567addc9	Wrap VolumeResponse. (#11402 ) What this PR does / why we need it: This fixes a regression in `frontend.encoding=protobuf`: ``` body rpc error: code = Internal desc = invalid response format, got (queryrange.VolumeResponse) ``` Checklist* - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) - [ ] If the change is deprecating or removing a configuration option, update the `deprecated-config.yaml` and `deleted-config.yaml` files respectively in the `tools/deprecated-config-checker` directory. [Example PR](`0d4416a4b0`)	1 year ago
J Stickler	f6b5cde1ea	Add alt text to images (#11400 ) Adding alt text to images that were flagged in this week's 404 and build error report.	1 year ago
J Stickler	bc9a00695e	Update structured metadata note (#11399 ) Update the note on the Structured Metadata page now that it is available on Cloud.	1 year ago
na--	e14b3474c1	tests: Disable some `clients/cmd/fluent-bit` tests when `CGO_ENABLED=0` (#11406 ) What this PR does / why we need it: `CGO_ENABLED=0 go test ./...` fails with compile-time errors because the tests are assuming a `logger` variable is defined. However, that variable is defined (and initialized in an `init()`) here, in a file that imports `"C"` and thus requires CGO to be enabled: `5b8d0e666d/clients/cmd/fluent-bit/out_grafana_loki.go (L3-L34)`	1 year ago
Karsten Jeschkies	5b8d0e666d	Remove querier wait time metric. (#11233 ) What this PR does / why we need it: We would like to know how long a querier worker is idle to understand if workstealing would have an impact. The original metric was too noisy and its cardinality was too high. Instead, we are going to log the wait time. Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) - [ ] If the change is deprecating or removing a configuration option, update the `deprecated-config.yaml` and `deleted-config.yaml` files respectively in the `tools/deprecated-config-checker` directory. [Example PR](`0d4416a4b0`) --------- Co-authored-by: Danny Kopping <dannykopping@gmail.com>	1 year ago
Elliot Dobson	0945b186ff	lambda-promtail: cloudwatch: add '__aws_log_type' label (#11335 ) What this PR does / why we need it: Adds the `__aws_log_type` label to AWS CloudWatch logs in `lambda-promtail`. Which issue(s) this PR fixes: N/A Special notes for your reviewer: AWS S3 & Kinesis log types already have this label. The `lambda-promtail` documentation [here](https://github.com/grafana/loki/blob/main/docs/sources/send-data/lambda-promtail/_index.md?plain=1#L154) suggests that CloudWatch logs has this label added as well, but in practice it does not AFAICT. Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [x] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) - [ ] If the change is deprecating or removing a configuration option, update the `deprecated-config.yaml` and `deleted-config.yaml` files respectively in the `tools/deprecated-config-checker` directory. [Example PR](`0d4416a4b0`)	1 year ago
Periklis Tsirakidis	e590963471	operator: Add infra annotations for OpenShift based deployments (#11393 )	1 year ago
Periklis Tsirakidis	d683aedb1b	operator: Add tools to test on Azure (#11340 ) )	1 year ago
Salva Corts	a0b462d366	Bloom-Gateway cache (#11380 ) What this PR does / why we need it: This PR adds caching to the bloom-gateway client. It uses the result cache from https://github.com/grafana/loki/pull/11343. Here's how we: - Merge responses: group all chunks by FP and remove duplicated chunk checksums. - Extract responses based on time span: For all chunks in each FP, add to the extracted response only the chunks that overlaps with the desired start and end time.	1 year ago
Christian Haudum	79693d79ef	Fix `getFromThrough()` function for ShortRefs (#11386 ) What this PR does / why we need it: Since chunks don't necessarily have the same duration, we cannot assume that the last item of a slice sorted by `From` time also has the greatest `Through` time. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	1 year ago
Gregor Zeitlinger	abb63b21b9	Update template_functions.md (#9683 ) My previous fix (part of #9672) was wrong - reverting the change Co-authored-by: J Stickler <julie.stickler@grafana.com>	2 years ago
Pangidoan Butar	7f79db908b	Update log configuration (#11211 ) What this PR does / why we need it: Which issue(s) this PR fixes: Fixes #<issue number> Special notes for your reviewer: Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) - [ ] If the change is deprecating or removing a configuration option, update the `deprecated-config.yaml` and `deleted-config.yaml` files respectively in the `tools/deprecated-config-checker` directory. [Example PR](`0d4416a4b0`) --------- Co-authored-by: J Stickler <julie.stickler@grafana.com>	2 years ago
akevdmeer	18778cd548	De-duplicate common prefixes as returned for individual buckets (#11317 ) With multiple buckets configured on a single AWS/S3 store, each object is stored on one of the buckets by hashing its object key in `bucketFromKey()`. That is sufficient for `GetObject()` and `PutObject()`. `List()` needs to list each bucket and combine the results. It appends all object keys into a `storageObjects` slice. That works because each key uniquely exists in only one of the buckets. But that is not the case for the common prefixes; a common prefix may exist in multiple buckets. This PR removes duplicates from the common prefixes as gathered from the multiple buckets in `List()`. Without this fix, we find that a repeated common prefix leads to a table being compacted multiple times concurrently which is not safe. E.g. this error results when the compactor concurrently tries to compact a table and a 'losing' execution finds that its clean-up has already been done: `level=error caller=compactor.go:128 table-name=loki_index_tsdb_19683 msg="failed to remove downloaded index file" path=/var/loki/compactor/loki_index_tsdb_19683/1700667008-loki-write-0-1700660468255353690.tsdb err="remove /var/loki/compactor/loki_index_tsdb_19683/1700667008-loki-write-0-1700660468255353690.tsdb: no such file or directory" `	2 years ago
Christian Haudum	856b57336c	Pass `--build-arg` to docker-driver build command (#11379 ) This fixes an issue with the build system when the Go version is updated in the build image and go.mod relies on the new features of the Go version. See failed CI run https://drone.grafana.net/grafana/loki/31777/28/2 Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
Christian Haudum	19b0160821	Bloom gateway client: Use binary search to determin membership of fingerprints to server instances (#11336 ) Instead of iterating over all fingerprints and determine their respective instance from the ring individually, this new approach takes calculates the token ranges from the instances from the ring and matches and partitions the fingerprints based on that using a binary search to determine membership to a token range. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
Salva Corts	489ac8d529	Extract results cache into new pkg (#11343 ) What this PR does / why we need it: This extracts the results cache from `queryrangebase` into its own pkg so we can reuse it in other components such as the bloom-gateway without having to import `queryrangebase`. - Most of the logic inside `pkg/querier/queryrange/queryrangebase/results_cache.go` now lives in `pkg/storage/chunk/cache/results_cache/cache.go`. - Some of the tests in `pkg/querier/queryrange/queryrangebase/results_cache.go` are moved into pkg/storage/chunk/cache/results_cache/cache_test.go. - Note that here we don't have access to the types we use in `queryrangebase` so we created a new set of mock request/response types to test with.	2 years ago
Christian Haudum	0e433f304e	Bloom Gateway: process blocks immediately when they are available (#11360 ) At current point in time, the worker calls `w.store.GetBlockQueriersForBlockRefs()` to obtain a list of block queriers sorted by fingerprint in ascending order. However, the shipper used by the store to fetch blocks by its refs, calls a callback function when a requested block is available. Instead of waiting for all callbacks to be called and then returning a single list of block queriers, the store now exposes a new function `ForEach()` that also accepts a callback that is passed to the underlying shipper. This allows to process blocks in the worker as soon as they become available. Some block may be served from cache, some may need to be downloaded. That way, we reduce waiting time. --------- Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
Paul Rogers	9538499e60	Add error return to PopulateSeriesWithBloom (#11359 ) What this PR does / why we need it: Adds an error return value to PopulateSeriesWithBloom Which issue(s) this PR fixes: Fixes #<issue number> Special notes for your reviewer: Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) - [ ] If the change is deprecating or removing a configuration option, update the `deprecated-config.yaml` and `deleted-config.yaml` files respectively in the `tools/deprecated-config-checker` directory. [Example PR](`0d4416a4b0`) --------- Co-authored-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
Ashwanth	d9f3bf30b1	config: adds `frontend.max-query-capacity` to tune per-tenant query capacity (#11284 ) What this PR does / why we need it: Adds a new config `frontend.max-query-capacity` that allows users to configure what portion of the the available querier replicas can be used by a tenant. `max_query_capacity` is the corresponding YAML option that can be configured in limits or runtime overrides. For example, setting this to 0.5 would allow a tenant to use half of the available queriers. This complements the existing `frontend.max-queriers-per-tenant`. When both are configured, the smaller value of the resulting querier replica count is considered: ``` min(frontend.max-queriers-per-tenant, ceil(querier_replicas * frontend.max-query-capacity)) ``` All queriers will handle requests for a tenant if neither limits are applied. Which issue(s) this PR fixes: Fixes #<issue number> Special notes for your reviewer: noticed that we don't pass down the shuffle sharding limits for frontend (only using it with schedulers) `26f097162a/pkg/loki/modules.go (L895)` but the [docs](`26f097162a/pkg/validation/limits.go (L276)`) mention that`frontend.max-queriers-per-tenant` applies to frontend as well. ``` This option only works with queriers connecting to the query-frontend / query-scheduler, not when using downstream URL. ``` Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [x] Documentation added - [x] Tests updated - [x] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) - [ ] If the change is deprecating or removing a configuration option, update the `deprecated-config.yaml` and `deleted-config.yaml` files respectively in the `tools/deprecated-config-checker` directory. [Example PR](`0d4416a4b0`) --------- Co-authored-by: J Stickler <julie.stickler@grafana.com> Co-authored-by: Danny Kopping <danny.kopping@grafana.com>	2 years ago
Paul Rogers	ceda046229	Bloom tokenization performance tweaks (#11338 ) What this PR does / why we need it: The main work of this PR is to improve the performance of the reassemble() function. By changing the assignment of the 'cur' variable to not do addition, and incrementing the 'pos' variable separately, the benchmarks improved as follows: BenchmarkTokens/threeSkip1Chunk/v2-10 88864 13487 ns/op BenchmarkTokens/threeSkip1ChunkAt2/v2At2-10 98607 12256 ns/op (This was validated with multiple runs, on multiple size inputs) The remaining changes in the PR are just to add some benchmarking for PopulateSeriesWithBloom, in case we need to do other comparisons for future improvements.	2 years ago
Poyzan	11ef833ed3	Compact multiple series into a bloom block (#11295 ) What this PR does / why we need it: The existing `Job` abstraction has 1 stream fingerprint and compactor was making a 1bloom and 1 block at the end of compaction. This PR changes the `Job` struct to have a list of stream fingerprints. At the end of compaction n streams are iterated over to create n blooms that are appended to 1 block. Stream fingerprints are added to the Job after checking if they are in owned by that compactor's ring, so it is still benefiting from sharding strategy. Resulting block will only contain series blooms that belong to 1 tsdb table. relates to https://github.com/grafana/loki-private/issues/856 Which issue(s) this PR fixes: Fixes #<issue number> Special notes for your reviewer: Logic to control block size will be added in a follow up PR.	2 years ago
someStrangerFromTheAbyss	0b9e573140	Helm: Add extraContainers to write pods (#11319 ) Adds the extraContainers to the write pod: Which issue(s) this PR fixes: This fix will allow my team to add a pods that makes modification to the incoming logs before sending them to loki with minimal network transfer. Special notes for your reviewer: NONE Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [x] Documentation added - [x] Tests updated - [x] `CHANGELOG.md` updated - [x] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [x] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [x] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) - [x] If the change is deprecating or removing a configuration option, update the `deprecated-config.yaml` and `deleted-config.yaml` files respectively in the `tools/deprecated-config-checker` directory. [Example PR](`0d4416a4b0`) --------- Co-authored-by: Michel Hollands <42814411+MichelHollands@users.noreply.github.com>	2 years ago
Dmitriy Okladin	a106b061cb	jsonnet: add memberlist_ring_enabled check condition for consul_config_map (#11312 ) The jsonnet lib should not generate a ConfigMap for consul if you are using memberlist. Fixes #11311	2 years ago
Rens Groothuijsen	cf84b2e9b8	fluent-bit: Redact basic auth credentials from fluent-bit logging (#11308 ) This redacts the basic auth credentials from the configured URL that is displayed when fluent-bit is started, if these are included. Go's built-in redaction only hides the password, turning a url like `http://user:pass@example.com` into `http://user:xxxxx@example.com`. Fixes #10514	2 years ago
Christian Haudum	fe4d9bea68	Remove worker callback function that is only used in tests (#11356 ) The queue already exposes the functionality to track connected consumers (workers), so there is no need to have callback that is only used in tests. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
Robert Jacob	cd3a04d6d7	operator: Run unit tests of API module as well (#11328 )	2 years ago
Christian Haudum	2411c575fb	Wait for all workers to stop when stopping blockdownloader (#11353 ) The `blockDownloader` does not wait until all of its worker goroutines are finished. This leads to a flaky test, see #11347 To properly fix the behaviour, the `stop()` function should wait until all workers are done. Closes #11347 Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
Salva Corts	71ddec7550	Bloom-Gateway: Use pools to reduce allocations (#11324 ) What this PR does / why we need it: This PR addresses some TODOs in the Bloom-Gateway component to reduce allocations by pooling. Note for reviewers: - I'm not really sure about the max size for the preallocated pools. Used 16K for all of them but that's probably too much for pools such as the `responsesPool`, `addressesPool`... Feel free to leave a comment in the PR with an estimate on the size you think it would make sense to set as the max.	2 years ago
Ashwanth	6d011bedbc	config: fix index periodic config unmarshal (#11344 ) What this PR does / why we need it: Unmarshalling for `IndexPeriodicTableConfig` is not working as expected since it contains an inline field that implements `yaml.Unmarshaler` interface. Configuring `path_prefix` is resulting in a error because it tries to Unmarshal to the parent inline field directly. Fix this by implementing `yaml.Unmarshaler` interface for `IndexPeriodicTableConfig` related to: https://github.com/go-yaml/yaml/issues/125 ``` field path_prefix not found in type struct { Prefix string "yaml:\"prefix\""; Period model.Duration "yaml:\"period\""; Tags config.Tags "yaml:\"tags\"" } ``` Which issue(s) this PR fixes: Fixes #<issue number> Special notes for your reviewer: Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [x] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) - [ ] If the change is deprecating or removing a configuration option, update the `deprecated-config.yaml` and `deleted-config.yaml` files respectively in the `tools/deprecated-config-checker` directory. [Example PR](`0d4416a4b0`)	2 years ago
Karsten Jeschkies	b05544a939	Remove obsolete empty test in middleware. (#11304 ) What this PR does / why we need it: A check for `len(queryRangeMiddleware) > 0` is always true so we can remove it. Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) - [ ] If the change is deprecating or removing a configuration option, update the `deprecated-config.yaml` and `deleted-config.yaml` files respectively in the `tools/deprecated-config-checker` directory. [Example PR](`0d4416a4b0`)	2 years ago
Sven Grossmann	1496eb4752	Fix `or` filter not being stringified correctly (#11337 ) What this PR does / why we need it: This PR adds a missing `String()` call for the `or` statement in line filters. It also fixes a bug where `or` statements were not applied in conjunction with an IPLineFilter expression. Which issue(s) this PR fixes: Fixes #<issue number> Special notes for your reviewer: Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) - [ ] If the change is deprecating or removing a configuration option, update the `deprecated-config.yaml` and `deleted-config.yaml` files respectively in the `tools/deprecated-config-checker` directory. [Example PR](`0d4416a4b0`) --------- Co-authored-by: Travis Patterson <travis.patterson@grafana.com>	2 years ago
Robert Jacob	5270d10a7e	Fix minimum Go version for module (#11145 ) What this PR does / why we need it: The main module of Loki already uses features of Go 1.21 (`cmp` module, `context.WithTimeoutCause`), so it can't be built with Go versions lower than 1.21 anymore. The module file should reflect this. The `go.sum` file was updated automatically by the Go tool `go mod tidy` after updating the version. Special notes for your reviewer: Go 1.21 module files introduce a new `toolchain` attribute. I have set this to the currently available latest version. --------- Co-authored-by: Periklis Tsirakidis <periklis@redhat.com>	2 years ago
William Travis Holton	1d232b824c	Helm: Add swift credentials and container name to common config (#11256 ) What this PR does / why we need it: Add swift to common config via helm. Which issue(s) this PR fixes: No issue Special notes for your reviewer: Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [x] Documentation added - [ ] Tests updated - [x] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [x] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) - [ ] If the change is deprecating or removing a configuration option, update the `deprecated-config.yaml` and `deleted-config.yaml` files respectively in the `tools/deprecated-config-checker` directory. [Example PR](`0d4416a4b0`)	2 years ago
Paul Rogers	df1a3ce79a	Initial pass at metrics for bloom tokenizer (#11329 ) What this PR does / why we need it: Removes the stubbed out metrics from the Bloom Tokenizer, and puts in metrics that were being captured and deemed useful during initial POC testing. Which issue(s) this PR fixes: Fixes #<issue number> Special notes for your reviewer: Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) - [ ] If the change is deprecating or removing a configuration option, update the `deprecated-config.yaml` and `deleted-config.yaml` files respectively in the `tools/deprecated-config-checker` directory. [Example PR](`0d4416a4b0`)	2 years ago
Periklis Tsirakidis	21a7f96b63	operator: Add support for blocking queries per tenant (#11094 ) Co-authored-by: Robert Jacob <xperimental@solidproject.de>	2 years ago
Salva Corts	10210b884c	Categorized Labels: Return empty JSON object if no parsed nor structured metadata labels (#11325 ) This is a followup PR for https://github.com/grafana/loki/pull/10419 wrt this comment https://github.com/grafana/loki/pull/10419#discussion_r1406285386. If there is no parsed nor structured-metadata labels in a given entry, and the categorize-labels header is set, add an empty object to the entry array. For example, we currently return: ```json { "data": { "encodingFlags": [ "categorize-labels" ], "result": [ { "stream": { "agent": "promtail", "filename": "/var/log/nginx/json_access.log", "host": "appfelstrudel", "job": "nginx_access_log" }, "values": [ [ "1701096036028751750", "example line" ] ] } ] } } ``` But we want: ```json { "data": { "encodingFlags": [ "categorize-labels" ], "result": [ { "stream": { "agent": "promtail", "filename": "/var/log/nginx/json_access.log", "host": "appfelstrudel", "job": "nginx_access_log" }, "values": [ [ "1701096036028751750", "example line", {} <<<--- WE WANT THIS ] ] } ] } } ```	2 years ago
Joao Marcal	bd505f8e2d	mixins: Adds structured metadata panels (#11087 ) Co-authored-by: Periklis Tsirakidis <periklis@redhat.com>	2 years ago
Salva Corts	67ecf5007b	Make Blooms-Gateway queue settings configurable (#11321 ) What this PR does / why we need it: This PR adds three new settings to the Bloom-Gateway config: - `worker_concurrency`: replaces the `numWorkers` constant. Named after the scheduler's `scheduler_worker_concurrency` config. - `max_outstanding_per_tenant`: replaces the `maxTasksPerTenant` constant. Named after the frontend's `max_outstanding_per_tenant` config. - `pending_tasks_initial_capacity`: replaces the pendingTasksInitialCap constant. Note to the reviewers: - I think in the future `max_outstanding_per_tenant` should be configurable per-tenant. This would require refactoring the queue pkg and the frontend/scheduler, so I think it doesn't make sense it in this PR. - I moved the `Limits` struct definition to `config.go` so have all the adjustable settings in the same place.	2 years ago
Kaviraj Kanagaraj	2a83cf01c0	chore(jsonnet): Fix the storage config on shipper.libsonnet (#11323 ) What this PR does / why we need it: We found this is wrongly configured which removes some config(`active_index_directory` and `cache_location`) in our Loki clusters Introduced via this PR https://github.com/grafana/loki/pull/11195/files Which issue(s) this PR fixes: Fixes #<issue number> Special notes for your reviewer: Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com>	2 years ago
Joao Marcal	b29b4565a5	operator: Add tools to test on GCP (#11303 ) Co-authored-by: Periklis Tsirakidis <periklis@redhat.com>	2 years ago
Christian Haudum	d62d4e37d1	Bloom Gateway: Implement chunk filtering using workers that multiplex requests (#11181 ) This change adds an internal request queue to the bloom gateway. Instead of executing every single request individually, which involves resolving bloom blocks, downloading them if needed and executing the chunk filtering, requests are now enqueued to the internal, per-tenant queue. The queue implements the same shuffle sharding mechanism as the queue in the query scheduler component. Workers then dequeue a batch of requests for a single tenant and multiplex them into a single processing task for each day. This has the big advantage that the chunks of multiple requests can be processed in a single sequential scan through a set a bloom blocks, without needing to skip back and forth within the binary stream of the block. --------- Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
renovate[bot]	09cb9ae76f	fix(deps): update github.com/grafana/loki/pkg/push digest to `e523809` (main) (#11107 ) [![Mend Renovate logo banner](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com) This PR contains the following updates: \| Package \| Type \| Update \| Change \| \|---\|---\|---\|---\| \| [github.com/grafana/loki/pkg/push](https://togithub.com/grafana/loki) \| require \| digest \| `0a7737e` -> `e523809` \| --- ### Configuration 📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied. ♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 Ignore: Close this PR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR has been generated by [Mend Renovate](https://www.mend.io/free-developer-tools/renovate/). View repository job log [here](https://developer.mend.io/github/grafana/loki). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy4zMS41IiwidXBkYXRlZEluVmVyIjoiMzcuNTkuOCIsInRhcmdldEJyYW5jaCI6Im1haW4ifQ==--> Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>	2 years ago
Danny Kopping	e523809216	bug(volume): return 400 error for invalid volume request (#11313 ) What this PR does / why we need it: Making a request to the `/volume{,_range}` endpoints results in a 500 is a LogQL expression is given instead of the expected label matchers. The reason for this is that the error is not wrapped in such a way as to be interpreted by `ClientHTTPStatusAndError` correctly. 5xx responses from the querier are automatically retried, so any invalid request currently results in 5 retries which are unnecessary. Signed-off-by: Danny Kopping <danny.kopping@grafana.com>	2 years ago
Vladyslav Diachenko	75cfe59596	bloom blocks downloading queue (#11201 ) implemented bloom blocks downloading queue to control the concurrency of downloading the blocks from the storage Signed-off-by: Vladyslav Diachenko <vlad.diachenko@grafana.com>	2 years ago

1 2 3 4 5 ...

5960 Commits (c573defcbbd0928cc0f718e51ef1d15373a3252c) All Branches Search

5960 Commits (c573defcbbd0928cc0f718e51ef1d15373a3252c)

All Branches