apitech/loki - loki - Gitea: Git with a cup of tea

Commit Graph

Author	SHA1	Message	Date
Yogev Levy	5675baecbe	Adding annotations support for statefulset-read.yaml (#11690 ) Adding fixing lake of annotations support for `read` StatefulSet What this PR does / why we need it: Currently there is no use for `read.annotations` at the `statefulset-read.yaml` file. The key exists on `values.yaml` file, with no use. Which issue(s) this PR fixes: Fixes #11688 Special notes for your reviewer: Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [x] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) - [ ] If the change is deprecating or removing a configuration option, update the `deprecated-config.yaml` and `deleted-config.yaml` files respectively in the `tools/deprecated-config-checker` directory. [Example PR](`0d4416a4b0`) --------- Co-authored-by: Michel Hollands <42814411+MichelHollands@users.noreply.github.com>	2 years ago
Paul Rogers	d4e359dea8	Change distributor latency to have a regular expression equals (#11827 ) Currently, the distributor latency panel uses a "=" in the code as opposed to a "=~", causing the panel to render as empty. This is to fix that issue. What this PR does / why we need it: Which issue(s) this PR fixes: Fixes #<issue number> Special notes for your reviewer: Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) - [ ] If the change is deprecating or removing a configuration option, update the `deprecated-config.yaml` and `deleted-config.yaml` files respectively in the `tools/deprecated-config-checker` directory. [Example PR](`0d4416a4b0`)	2 years ago
Dylan Guedes	3b7ac359bc	Spans: Stop emitting `GetChunks` and `IndexClient.GetChunkRefs` (#11825 ) What this PR does / why we need it: Modify our code to not emit the `GetChunks` and the `IndexClient.GetChunkRefs` spans. We don't worse our observability by getting rid of them because their data is redundant with parent spans. Example: - If an specific index-gateway or ingester struggles, the parent span `/logproto.Querier/GetChunkIDs` will report it just fine - If an specific query causes trouble the `query.Exec` span will show the query name, the `start` and `end`, etc. Also, its children will show the slower parts of the query (in case we have a weak link) Which issue(s) this PR fixes: N/A	2 years ago
Robert Jacob	38d205b967	operator: Improve messages for errors in storage secret (#11824 )	2 years ago
Meng Ye	9d4d101169	docs: fix runtime_config example (#11798 ) What this PR does / why we need it: Which issue(s) this PR fixes: Fixes #<issue number> Special notes for your reviewer: Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) - [ ] If the change is deprecating or removing a configuration option, update the `deprecated-config.yaml` and `deleted-config.yaml` files respectively in the `tools/deprecated-config-checker` directory. [Example PR](`0d4416a4b0`) --------- Co-authored-by: J Stickler <julie.stickler@grafana.com>	2 years ago
Owen Diehl	a56b56c8b0	[Bloom] Keyspace aware bloom creation, library utilities (#11810 ) This PR continues work refactoring the bloom compactor pkg and adds a bunch of utilities for keyspace manipulation/mgmt. A lot of the bloom block logic is contained here as a skeleton. This code is not complete, but is fine to merge as we don't use this yet and should be easier to review before it gets too large.	2 years ago
Vladyslav Diachenko	821c0b1ae0	DequeueMany fix (#11797 ) What this PR does / why we need it: Previously we relied only on the index of the tenant's queue to read the requests from the same tenant's queue. However, as long as the queue is aggressively used by the consumers in parallel, there are a few edge cases when knowing the index of last used tenant's queue is not enough to guarantee that we dequeue items from exactly the same tenant's queue. Special notes for your reviewer: Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [x] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) - [ ] If the change is deprecating or removing a configuration option, update the `deprecated-config.yaml` and `deleted-config.yaml` files respectively in the `tools/deprecated-config-checker` directory. [Example PR](`0d4416a4b0`) --------- Signed-off-by: Vladyslav Diachenko <vlad.diachenko@grafana.com>	2 years ago
Christian Haudum	ecab33a932	Bloom Gateway: Extract processing logic from bloom worker into separate struct (#11812 ) What this PR does / why we need it: The processor executes a set of tasks. It cleanly separates I/O from logic and therefore is more testable than the current worker implementation. `9602214abf/pkg/bloomgateway/processor.go (L33-L36)` Note: The processor is not used yet in the worker. Certain parts must be refactored when #11810 is merged. --------- Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
Christian Haudum	e0e143a88f	Bloom Gateway: Make tasks cancelable (#11792 ) This PR refactors the bloom gateway workers so that tasks that have been enqueued by requests do not end up locking the results channel and therefore the worker, in case the request was cancelled (`context cancelled`) or timed out (`context deadline exceeded`). It also handles errors from the shipper in a way that they are returned to the waiting request asap so it can return and does not need to wait for all tasks to finish. This PR also fixes the worker shutdown in a way that it now gracefully stops and continues to work off the remaining tasks from the queue. --------- Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
Christian Haudum	c01a823f1f	(chore) Bloom Gateway: Avoid unnecessary merging of tasks prior to passing them to the FusedQuerier (#11809 ) What this PR does / why we need it: The `requestIterator` (`v1.Iterator[v1.Request]`) can be built directly from the task and does not require any unnecessary merging of tasks, since this is done by the `v1.FusedQuerier` already. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
Poyzan	de83ae0fc7	Update Helm chart docs and yaml (#11800 ) What this PR does / why we need it: Following recent loki `v2.9.4` and GEL `v1.8.6` patch releases, updates Helm chart docs and yaml. Loki release drone creates is the [PR as expected](https://github.com/grafana/loki/pull/11769/files), but there seems to be a problem with the GEL drone generating the docs -which is looked at separately. Instead I manually updated them in one go.	2 years ago
Sheikh-Abubaker	012cf92dd2	area/grafana-agent-operator: Bumped grafana-agent-operator version (#11576 ) What this PR does / why we need it: Which issue(s) this PR fixes: Fixes #11572 Special notes for your reviewer: Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [x] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) - [ ] If the change is deprecating or removing a configuration option, update the `deprecated-config.yaml` and `deleted-config.yaml` files respectively in the `tools/deprecated-config-checker` directory. [Example PR](`0d4416a4b0`) --------- Signed-off-by: Sheikh-Abubaker <sheikhabubaker761@gmail.com> Co-authored-by: Michel Hollands <42814411+MichelHollands@users.noreply.github.com>	2 years ago
Vladislav	541b109c3d	Docs: Fixed docker_sd_configs top level key name (#11803 ) What this PR does / why we need it: Misspelling in top level key name, promtail accept docker_sd_configs but not docker_sd_config Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) - [ ] If the change is deprecating or removing a configuration option, update the `deprecated-config.yaml` and `deleted-config.yaml` files respectively in the `tools/deprecated-config-checker` directory. [Example PR](`0d4416a4b0`) Co-authored-by: J Stickler <julie.stickler@grafana.com>	2 years ago
Joao Marcal	fe4ba0c3ba	operator: Add OpenShift CloudCredentials support for AWS STS (#11524 ) Co-authored-by: Periklis Tsirakidis <periklis@redhat.com>	2 years ago
Owen Diehl	de4f56e42d	[Blooms] Refactoring bloom compactor to isolate state from logic (#11793 ) This PR refactors a bunch of code to separate state+I/O related complexity from logic so we can test and extend it more easily. It also adds more logic for the bloom generator to use.	2 years ago
William Dumont	215b5fd2fd	<Promtail>: Fix flaky docker test (#11777 ) What this PR does / why we need it: We ported this fix a few days ago to the Agent via this [PR](https://github.com/grafana/agent/pull/6201) but the added test is often failing in our drone pipeline. I believe that there are two reasons why this test is flaky: - the test expects that the 5 last lines of logs are the expected ones but it seems that other lines might be logged as well (https://drone.grafana.net/grafana/agent/16045/4/2) - we saw that the simulated container did not always have enough time to stop before calling the tgt.startIfNotRunning() to restart it. (https://drone.grafana.net/grafana/agent/16077/4/2) Fix: - the test now uses "assert.EventuallyWithT" and checks that within 5 seconds the expected log lines will be amongst the logs without duplicate. - the test now stops the simulated container before restarting it Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) --------- Co-authored-by: Michel Hollands <42814411+MichelHollands@users.noreply.github.com>	2 years ago
Salva Corts	3df253fdb7	Add test-target-branch stage to drone again (#11799 ) What this PR does / why we need it: Adding the stage back after removing it in https://github.com/grafana/loki/pull/11785 to fix the test failing in main.	2 years ago
Salva Corts	d97ac48681	Add timing metrics for compaction (#11785 ) What this PR does / why we need it: This PR modifies the metrics of the bloom compactor to: - Add new histogram metrics to track the time spent to process compaction cycles, tenants and jobs. - Replace `_succeeded` and `_failed` metrics with a new `_completed` metric with a status label. Which issue(s) this PR fixes: Fixes #<issue number> Special notes for your reviewer: Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) - [ ] If the change is deprecating or removing a configuration option, update the `deprecated-config.yaml` and `deleted-config.yaml` files respectively in the `tools/deprecated-config-checker` directory. [Example PR](`0d4416a4b0`)	2 years ago
Periklis Tsirakidis	88df82295d	operator: Update Loki operand to v2.9.4 (#11778 ) What this PR does / why we need it: Update Loki operand to v2.9.4 Which issue(s) this PR fixes: Fixes #<issue number> Special notes for your reviewer: Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [x] Documentation added - [ ] Tests updated - [x] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) - [ ] If the change is deprecating or removing a configuration option, update the `deprecated-config.yaml` and `deleted-config.yaml` files respectively in the `tools/deprecated-config-checker` directory. [Example PR](`0d4416a4b0`)	2 years ago
George Wilson	1002ba00df	Note the helm chart and docs do not agree (#11782 ) What this PR does / why we need it: Docs do not match the code in the helm chart - where the read deployment is made as a statefulset with ReadWriteOnce disks (https://github.com/grafana/loki/blob/main/production/helm/loki/templates/read/statefulset-read.yaml). I suspect the docs are correct that it's possible - but given it's the official helm chart - the deviation should be noted. Which issue(s) this PR fixes: N/A Special notes for your reviewer: Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [x] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) - [ ] If the change is deprecating or removing a configuration option, update the `deprecated-config.yaml` and `deleted-config.yaml` files respectively in the `tools/deprecated-config-checker` directory. [Example PR](`0d4416a4b0`) --------- Co-authored-by: J Stickler <julie.stickler@grafana.com>	2 years ago
Trevor Whitney	406b10c675	accidentally merged this from ##11333 (#11788 ) What this PR does / why we need it: I did not intend to merge this change, reverting to what it was before merging #11333	2 years ago
Trevor Whitney	a064f5d670	feat: add release please config (#11333 ) What this PR does / why we need it: The new release process is being developed in another repo for now as it enables much faster feedback loops. We are at the stage where we are pulling the Loki repo and building images, so it's time to move the `release-please` config into Loki, and configure it for the various packages we will be releasing. This PR adds the two basic configs that `release-please` requires. This PR does not change anything about the release process, it just enables us to keep iterating out of band until we're ready to move over. Which issue(s) this PR fixes: Re #11332	2 years ago
Dylan Guedes	9964a30b62	Jsonnet: Change by-pod source code to reuse the `latency` util function (#11783 ) What this PR does / why we need it: Modify our implementation of `p99LatencyByPod` to reuse the `latencyPanel` util function.	2 years ago
Owen Diehl	191466523e	[Blooms] Fix panic when accessing page outside of page bounds (#11787 ) Move variable declaration to only access the desired page when it's valid. Also removes an unused (duplicate) test fn.	2 years ago
Christian Haudum	9f3fd24958	Bloom Shipper: Fix bug in block boundary check (#11784 ) What this PR does / why we need it: The check `isOutsideRange()` returned incorrect results, because the end timestamp of the search interval is not inclusive. --------- Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
Michel Hollands	2a63d24729	Fix the ObjectExists method on FSObjectClient (#11747 ) What this PR does / why we need it: This method wasn't working. A test was added as well. Special notes for your reviewer: Checklist - [X] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [X] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) - [ ] If the change is deprecating or removing a configuration option, update the `deprecated-config.yaml` and `deleted-config.yaml` files respectively in the `tools/deprecated-config-checker` directory. [Example PR](`0d4416a4b0`) Signed-off-by: Michel Hollands <michel.hollands@gmail.com>	2 years ago
Christian Haudum	6871f3c555	Bloom Gateway: Partition requests into multiple tasks before enqueuing (#11768 ) What this PR does / why we need it: This PR changes at which point a request is split into multiple tasks in case the chunks boundaries of the request overlap multiple days. This solves the problem that the request handler needs to know for how many responses from the block querier needs to wait. Previously, a single task that spanned across multiple days was executed multiple times (once for each matching day). --------- Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
Ashwanth	5c8fd520d3	fix(background cache): increment queue size exactly once (#11776 ) What this PR does / why we need it: While looking at the metrics from one of our cells, we noticed that the queue size is set to a very high value even when the length is pretty much 0. Background cache is incrementing the queue size twice for each enqueued key. This pr removes the additional increment call. `TestBackgroundSizeLimit` might be flaky because of the [writeBackLoop](`8eb09c78c8/pkg/storage/chunk/cache/background.go (L193)`) which dequeues from the channel and reduces the queue size concurrently. To make the test predictable, I have set the `WriteBackGoroutines` to 0. Which issue(s) this PR fixes: Fixes #<issue number> Special notes for your reviewer: Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [x] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) - [ ] If the change is deprecating or removing a configuration option, update the `deprecated-config.yaml` and `deleted-config.yaml` files respectively in the `tools/deprecated-config-checker` directory. [Example PR](`0d4416a4b0`)	2 years ago
Christian Haudum	600e390b16	Extend `TestPartitionFingerprintRange` test case (#11750 ) What this PR does / why we need it: The test cases now also cover partially out of bounds blocks as well as inconsecutive block ranges. --------- Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
Christian Haudum	5acc29f2f6	Bloom Gateway: Fix panic in chunk removal after receiving query results (#11770 ) What this PR does / why we need it: While removing not matching chunks from a fingerprint, the for loop over the chunks removes items, and therefore the index of the current iteration needs to be adjusted to accommodate for the removal, otherwise it may panic with `out of bounds`. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
Travis Patterson	2c9482d8a2	reset the basemap when the labelsbuilder is reset (#11771 ) This PR fixes an issue where the `label_format` stage sometimes doesn't populate labels when it should. The label `baz` exists on all log lines but is only populated on the first line. ![image](https://github.com/grafana/loki/assets/1413241/86585a94-937f-4e07-9657-b94699ce8586) The issue is this call to [IntoMap](`06bb20914b/pkg/logql/log/fmt.go (L403)`). It tries to avoid copying the map multiple times [here](`06bb20914b/pkg/logql/log/labels.go (L482)`) but the `baseMap` variable isn't reset when the labels builder is reset. As a result, calls to to `IntoMap` after the builder has been reset mean that `!b.hasDel() && !b.hasAdd() && !b.HasErr()` evaluates to `true` but `b.baseMap == nil` also evaluates to `true` so no items are copied. This PR makes it so that `b.baseMap` is reset along with the rest of the builder's state in `Reset`	2 years ago
Owen Diehl	06bb20914b	[Blooms] Specs out BloomGenerator (#11755 ) This is my attempt at making more testable implementations of the work done by the `bloomcompactor` pkg. It's largely inspired by what already exists there, but without depending on things like object storage, chunk clients, etc, it's much easier to locally prove behavior. I'm planning to build on top of this, continuing to integrate work already done in this pkg.	2 years ago
Poyzan	6d66ea382c	[Fix] OR statements not being evaluated as part of nested line filters (#11735 ) What this PR does / why we need it: When nester line filters were being evaluated, the `OR` statements in right expressions were omitted and only `LineFilters` were returned. Resulting in only first value being returned. It can be reproduced in stringer unit test failing with, ``` Error Trace: /Users/poyzannur/workspace/loki/pkg/logql/syntax/ast_test.go:535 Error: Not equal: expected: "{app=\"foo\"} \|= \"foo\" or \"bar\" \|= \"baz\" or \"bal\"" actual : "{app=\"foo\"} \|= \"foo\" or \"bar\" \|= \"baz\"" ``` We now return the `OR` expression as part of nested line filters. Thanks a million to @ashwanthgoli for help with debugging, and extra unit test. Which issue(s) this PR fixes: Fixes https://github.com/grafana/support-escalations/issues/9042 Special notes for your reviewer: Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [x] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) - [ ] If the change is deprecating or removing a configuration option, update the `deprecated-config.yaml` and `deleted-config.yaml` files respectively in the `tools/deprecated-config-checker` directory. [Example PR](`0d4416a4b0`)	2 years ago
Sandeep Sukhani	e625d7f1ca	otel: add support for per tenant configuration for mapping otlp data to loki format (#11143 ) What this PR does / why we need it: In OTEL, we pick select Resource Attributes to identify the streams and store all the other attributes as Structured Metadata, as explained here. The problem however is that the list of Resource Attributes that are picked as Stream labels are hardcoded, and there is no way to drop unwanted data. This PR adds support for configuring how data is mapped from OTEL to Loki format per tenant. It also adds support for dropping unwanted data. We decided to make the config look similar to Prometheus's relabling config to make it familiar. Special notes for your reviewer: Opening a draft PR to get some initial feedback. I will add documentation in a separate PR. Checklist - [x] Tests updated - [x] `CHANGELOG.md` updated	2 years ago
dependabot[bot]	ca50e9e4a3	Bump golang.org/x/crypto from 0.14.0 to 0.17.0 in /tools/lambda-promtail (#11521 ) Bumps [golang.org/x/crypto](https://github.com/golang/crypto) from 0.14.0 to 0.17.0. <details> <summary>Commits</summary> <ul> <li><a href="`9d2ee975ef`"><code>9d2ee97</code></a> ssh: implement strict KEX protocol changes</li> <li><a href="`4e5a26183e`"><code>4e5a261</code></a> ssh: close net.Conn on all NewServerConn errors</li> <li><a href="`152cdb1503`"><code>152cdb1</code></a> x509roots/fallback: update bundle</li> <li><a href="`fdfe1f8531`"><code>fdfe1f8</code></a> ssh: defer channel window adjustment</li> <li><a href="`b8ffc16e10`"><code>b8ffc16</code></a> blake2b: drop Go 1.6, Go 1.8 compatibility</li> <li><a href="`7e6fbd82c8`"><code>7e6fbd8</code></a> ssh: wrap errors from client handshake</li> <li><a href="`bda2f3f5cf`"><code>bda2f3f</code></a> argon2: avoid clobbering BP</li> <li><a href="`325b735346`"><code>325b735</code></a> ssh/test: skip TestSSHCLIAuth on Windows</li> <li><a href="`1eadac50a5`"><code>1eadac5</code></a> go.mod: update golang.org/x dependencies</li> <li><a href="`b2d7c26edb`"><code>b2d7c26</code></a> ssh: add (Client).DialContext method</li> <li>Additional commits viewable in <a href="https://github.com/golang/crypto/compare/v0.14.0...v0.17.0">compare view</a></li> </ul> </details> <br /> [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=golang.org/x/crypto&package-manager=go_modules&previous-version=0.14.0&new-version=0.17.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) You can trigger a rebase of this PR by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/grafana/loki/network/alerts). </details> > Note* > Automatic rebases have been disabled on this pull request as it has been open for over 30 days. Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	2 years ago
renovate[bot]	a627fb6aa1	chore(deps): update module golang.org/x/crypto to v0.17.0 [security] (main) (#11522 ) [![Mend Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com) This PR contains the following updates: \| Package \| Change \| Age \| Adoption \| Passing \| Confidence \| \|---\|---\|---\|---\|---\|---\| \| golang.org/x/crypto \| `v0.14.0` -> `v0.17.0` \| [![age](https://developer.mend.io/api/mc/badges/age/go/golang.org%2fx%2fcrypto/v0.17.0?slim=true)](https://docs.renovatebot.com/merge-confidence/) \| [![adoption](https://developer.mend.io/api/mc/badges/adoption/go/golang.org%2fx%2fcrypto/v0.17.0?slim=true)](https://docs.renovatebot.com/merge-confidence/) \| [![passing](https://developer.mend.io/api/mc/badges/compatibility/go/golang.org%2fx%2fcrypto/v0.14.0/v0.17.0?slim=true)](https://docs.renovatebot.com/merge-confidence/) \| [![confidence](https://developer.mend.io/api/mc/badges/confidence/go/golang.org%2fx%2fcrypto/v0.14.0/v0.17.0?slim=true)](https://docs.renovatebot.com/merge-confidence/) \| ### GitHub Vulnerability Alerts #### [CVE-2023-48795](https://togithub.com/warp-tech/russh/security/advisories/GHSA-45x7-px36-x8w8) ### Summary Terrapin is a prefix truncation attack targeting the SSH protocol. More precisely, Terrapin breaks the integrity of SSH's secure channel. By carefully adjusting the sequence numbers during the handshake, an attacker can remove an arbitrary amount of messages sent by the client or server at the beginning of the secure channel without the client or server noticing it. ### Mitigations To mitigate this protocol vulnerability, OpenSSH suggested a so-called "strict kex" which alters the SSH handshake to ensure a Man-in-the-Middle attacker cannot introduce unauthenticated messages as well as convey sequence number manipulation across handshakes. Warning: To take effect, both the client and server must support this countermeasure. As a stop-gap measure, peers may also (temporarily) disable the affected algorithms and use unaffected alternatives like AES-GCM instead until patches are available. ### Details The SSH specifications of ChaCha20-Poly1305 (chacha20-poly1305@openssh.com) and Encrypt-then-MAC (-etm@openssh.com MACs) are vulnerable against an arbitrary prefix truncation attack (a.k.a. Terrapin attack). This allows for an extension negotiation downgrade by stripping the SSH_MSG_EXT_INFO sent after the first message after SSH_MSG_NEWKEYS, downgrading security, and disabling attack countermeasures in some versions of OpenSSH. When targeting Encrypt-then-MAC, this attack requires the use of a CBC cipher to be practically exploitable due to the internal workings of the cipher mode. Additionally, this novel attack technique can be used to exploit previously unexploitable implementation flaws in a Man-in-the-Middle scenario. The attack works by an attacker injecting an arbitrary number of SSH_MSG_IGNORE messages during the initial key exchange and consequently removing the same number of messages just after the initial key exchange has concluded. This is possible due to missing authentication of the excess SSH_MSG_IGNORE messages and the fact that the implicit sequence numbers used within the SSH protocol are only checked after the initial key exchange. In the case of ChaCha20-Poly1305, the attack is guaranteed to work on every connection as this cipher does not maintain an internal state other than the message's sequence number. In the case of Encrypt-Then-MAC, practical exploitation requires the use of a CBC cipher; while theoretical integrity is broken for all ciphers when using this mode, message processing will fail at the application layer for CTR and stream ciphers. For more details see [https://terrapin-attack.com](https://terrapin-attack.com). ### Impact This attack targets the specification of ChaCha20-Poly1305 (chacha20-poly1305@openssh.com) and Encrypt-then-MAC (-etm@openssh.com), which are widely adopted by well-known SSH implementations and can be considered de-facto standard. These algorithms can be practically exploited; however, in the case of Encrypt-Then-MAC, we additionally require the use of a CBC cipher. As a consequence, this attack works against all well-behaving SSH implementations supporting either of those algorithms and can be used to downgrade (but not fully strip) connection security in case SSH extension negotiation (RFC8308) is supported. The attack may also enable attackers to exploit certain implementation flaws in a man-in-the-middle (MitM) scenario. --- ### Configuration 📅 Schedule: Branch creation - "" (UTC), Automerge - At any time (no schedule defined). 🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied. ♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 Ignore: Close this PR and you won't be reminded about these updates again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR has been generated by [Mend Renovate](https://www.mend.io/free-developer-tools/renovate/). View repository job log [here](https://developer.mend.io/github/grafana/loki). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy45My4xIiwidXBkYXRlZEluVmVyIjoiMzcuMTM1LjAiLCJ0YXJnZXRCcmFuY2giOiJtYWluIn0=--> Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>	2 years ago
Christian Haudum	de251c3fc2	Fix flaky bloom filter e2e test (#11726 ) What this PR does / why we need it: The test looks at metrics to verify how many chunks have been filtered out on the bloom gateway. However, with the default split-by interval, it can happen that the query frontend splits the single query into multiple sub-queries and therefore resulting in chunks being filtered out multiple times. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
Christian Haudum	6d1b6ff20c	Fix jsonnet config for bloom gateway blocks cache (#11741 ) What this PR does / why we need it: The value is supposed to be a time.Duration value, rather than an int value for seconds. This changes fixes the rather small cache TTL. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
Christian Haudum	de3f1786a1	Bloom Gateway: Remove unnecessary wrapper around v1.Request (#11739 ) What this PR does / why we need it: The wrapper `FilterRequest` wraps `v1.Request` but the additional field with the error channel is never used. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
Pieter	b32d99308d	gateway: fix crash on ipv4 only systems (#11722 ) What this PR does / why we need it: Loki Gateway pod crashes on systems that only have IPv4 enabled since PR https://github.com/grafana/loki/pull/9510 Which issue(s) this PR fixes: Fixes #9681 Special notes for your reviewer: Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [x] Documentation added - [ ] Tests updated - [x] `CHANGELOG.md` updated - [x] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [x] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [x] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) - [x] If the change is deprecating or removing a configuration option, update the `deprecated-config.yaml` and `deleted-config.yaml` files respectively in the `tools/deprecated-config-checker` directory. [Example PR](`0d4416a4b0`) --------- Signed-off-by: Pieter van der Giessen <pieter@pionative.com> Co-authored-by: Michel Hollands <42814411+MichelHollands@users.noreply.github.com>	2 years ago
Owen Diehl	fda991cdc7	[blooms][testware] Adds testing for the merge builder, sourcing multiple blocks (#11725 ) Adds a few tests to aid in debugging behavior & add some more coverage. Notably tests the `MergeBuilder` sourcing blocks with varying degrees of duplicate data.	2 years ago
Bayan Taani	837cfc010c	operator: Add a custom metric that collects Lokistacks requiring a schema upgrade (#11513 ) Co-authored-by: Robert Jacob <rojacob@redhat.com> Co-authored-by: Periklis Tsirakidis <periklis@redhat.com>	2 years ago
Periklis Tsirakidis	71d3ab8c54	operator: Upgrade k8s.io, sigs.k8s.io and openshift deps (#11718 )	2 years ago
Juraj Michálek	0ed536cd7a	Add query-frontend option to select request headers in query logs (#11499 ) What this PR does / why we need it: Adding feature present in mimir, specifically https://github.com/grafana/mimir/pull/5030. Adds a config option to the query-frontend to specify a list of request headers to include in query logs. For example, setting -frontend.log-query-request-headers="X-Grafana-Org-Id" and sending a query with X-Grafana-Org-Id:1 results in query log lines that include header_x_grafana_org_id=1. Which issue(s) this PR fixes: Fixes #11422	2 years ago
Sandeep Sukhani	b581db0415	jsonnet: fix unicode quotes in jsonnet to avoid tools failing to parse it properly (#11716 ) What this PR does / why we need it: We have a linter in place in our internal deployment tooling to catch the usage of Unicode quotes because it has caused some issues due to shells/tooling not recognizing them. The linter has caught one such instance in the jsonnet in the Loki repo, which is fixed in this PR.	2 years ago
Christian Haudum	390f04d3b2	Fix flaky `TestMultiTenantQuery` integration test (#11733 ) What this PR does / why we need it: Fixes the flaky `TestMultiTenantQuery` integration test. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
Christian Haudum	7c60390364	Remove unnecessary wrapper around bloom shipper (#11724 ) What this PR does / why we need it: The bloom store was initially thought as a translation layer between the bloom gateway and the bloom shipper to be able to simplify the API of the shipper. However, it turned out that the store calls were only a 1-to-1 mapping of the shipper calls and therefore adding unnecessary complexity and call stack. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
Christian Haudum	522efb9d9a	Fix flaky block downloader test (#11723 ) The flaky test is caused by a race condition caused by concurrent writes of a counter that is used to track how often a method was called. Fixed by converting into an atomic integer. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
ptQa	b4b0bd7dca	Fix duplicate logs from docker containers (#11563 ) This fixes various issues with docker_sd on promtail. Mostly related to duplicate logs being send to loki from promtail. Root case of issue is that positions file is updated after process function, but in memory field `since` of `Target` struct is not updated.	2 years ago
Owen Diehl	a199662ff2	fixes seek in bloom blocks to correctly return the first fp >= target_fp (#11712 ) While debugging some unrelated code, I found that `Seek` was improperly implemented. It should return the first value greater or equal than the target. Previously it would cause iterators to return different values when reset.	2 years ago

1 2 3 4 5 ...

6134 Commits (5675baecbee2affb2bf05bb44f1c8c0516bc002c) All Branches Search

6134 Commits (5675baecbee2affb2bf05bb44f1c8c0516bc002c)

All Branches