apitech/loki - loki - Gitea: Git with a cup of tea

Commit Graph

Author	SHA1	Message	Date
dependabot[bot]	9aadb348ed	Bump golang.org/x/crypto from 0.9.0 to 0.10.0 (#9865 ) Bumps [golang.org/x/crypto](https://github.com/golang/crypto) from 0.9.0 to 0.10.0. <details> <summary>Commits</summary> <ul> <li><a href="`8e447d8cc5`"><code>8e447d8</code></a> go.mod: update golang.org/x dependencies</li> <li><a href="`22c9997952`"><code>22c9997</code></a> go.mod: tell x repo tagging to ignore dep on net</li> <li><a href="`d0b316056b`"><code>d0b3160</code></a> x509roots: add new module</li> <li><a href="`16222386f4`"><code>1622238</code></a> Add support for "<a href="mailto:hmac-sha2-512-etm@openssh.com">hmac-sha2-512-etm@openssh.com</a>"</li> <li>See full diff in <a href="https://github.com/golang/crypto/compare/v0.9.0...v0.10.0">compare view</a></li> </ul> </details> <br /> [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=golang.org/x/crypto&package-manager=go_modules&previous-version=0.9.0&new-version=0.10.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	2 years ago
wanddynosios	8037d96558	helm: fix dashboard multiple levels (#9010 ) What this PR does / why we need it: Which issue(s) this PR fixes: Fixes #9009 Special notes for your reviewer: Dashboard from the helm chart had a `=` instead of a `=~` operator for the level filter Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [x] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` --------- Co-authored-by: Michel Hollands <42814411+MichelHollands@users.noreply.github.com>	2 years ago
Ashwanth	de937e72cd	promtail: apply defaults to HTTP client config (#9863 ) What this PR does / why we need it: Apply defaults to HTTPClientConfig and validate it. similar to how it's done [here](`e9644d8186/vendor/github.com/prometheus/prometheus/config/config.go (L524)`) in prometheus. promtail [configuration page](https://grafana.com/docs/loki/latest/clients/promtail/configuration/) incorrectly claims that `follow_redirects` is true, but it is being set to false given that the defaults are not being applied currently Which issue(s) this PR fixes: Fixes #<issue number> Special notes for your reviewer: Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [x] Tests updated - [x] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) --------- Signed-off-by: Ashwanth Goli <iamashwanth@gmail.com> Co-authored-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
Robert Jacob	8d74b8ccae	operator: Regenerate bundles after docs change (#9859 )	2 years ago
dependabot[bot]	0d9a124e69	Bump github.com/alicebob/miniredis/v2 from 2.30.2 to 2.30.4 (#9850 ) Bumps [github.com/alicebob/miniredis/v2](https://github.com/alicebob/miniredis) from 2.30.2 to 2.30.4. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/alicebob/miniredis/releases">github.com/alicebob/miniredis/v2's releases</a>.</em></p> <blockquote> <h2>fix ZADD, fix COPY, speedy SPOP</h2> <ul> <li>fix ZADD LT/LG (thanks <a href="https://github.com/sejin-P"><code>@sejin-P</code></a>)</li> <li>fix COPY (thanks <a href="https://github.com/jerargus"><code>@jerargus</code></a>)</li> <li>quicker SPOP</li> </ul> <h2>ZMSCORE, lua error_reply, lua cache</h2> <ul> <li>fix lua error_reply (thanks <a href="https://github.com/pkierski"><code>@pkierski</code></a>)</li> <li>fix use of blocking functions in lua</li> <li>support for ZMSCORE (thanks <a href="https://github.com/lsgndln"><code>@lsgndln</code></a>)</li> <li>lua cache (thanks <a href="https://github.com/tonyhb"><code>@tonyhb</code></a>)</li> </ul> </blockquote> </details> <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/alicebob/miniredis/blob/master/CHANGELOG.md">github.com/alicebob/miniredis/v2's changelog</a>.</em></p> <blockquote> <h3>v2.30.4</h3> <ul> <li>fix ZADD LT/LG (thanks <a href="https://github.com/sejin-P"><code>@sejin-P</code></a>)</li> <li>fix COPY (thanks <a href="https://github.com/jerargus"><code>@jerargus</code></a>)</li> <li>quicker SPOP</li> </ul> <h3>v2.30.3</h3> <ul> <li>fix lua error_reply (thanks <a href="https://github.com/pkierski"><code>@pkierski</code></a>)</li> <li>fix use of blocking functions in lua</li> <li>support for ZMSCORE (thanks <a href="https://github.com/lsgndln"><code>@lsgndln</code></a>)</li> <li>lua cache (thanks <a href="https://github.com/tonyhb"><code>@tonyhb</code></a>)</li> </ul> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href="`082add4914`"><code>082add4</code></a> changelog</li> <li><a href="`1dfc2776b5`"><code>1dfc277</code></a> quicker SPOP</li> <li><a href="`fb75409cf5`"><code>fb75409</code></a> do proper list copy</li> <li><a href="`f507fa8d65`"><code>f507fa8</code></a> Merge pull request <a href="https://redirect.github.com/alicebob/miniredis/issues/330">#330</a> from sejin-P/ZADD-LT-default-value</li> <li><a href="`b0d3aeee12`"><code>b0d3aee</code></a> fix; check value existence when applying LT GT option to ZADD</li> <li><a href="`df419454ee`"><code>df41945</code></a> gofmt error</li> <li><a href="`ae47488375`"><code>ae47488</code></a> typo</li> <li><a href="`d6eb001ed5`"><code>d6eb001</code></a> update changelog</li> <li><a href="`a946a99f2c`"><code>a946a99</code></a> use the cleaner variable to get the result length</li> <li><a href="`a3211dedea`"><code>a3211de</code></a> Merge pull request <a href="https://redirect.github.com/alicebob/miniredis/issues/328">#328</a> from tonyhb/perf/precompile-lua-functions</li> <li>Additional commits viewable in <a href="https://github.com/alicebob/miniredis/compare/v2.30.2...v2.30.4">compare view</a></li> </ul> </details> <br /> [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=github.com/alicebob/miniredis/v2&package-manager=go_modules&previous-version=2.30.2&new-version=2.30.4)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	2 years ago
Sandeep Sukhani	7cbfdb5d6d	cache: fix a mistake in returning cached error in the index list cache (#9853 ) What this PR does / why we need it: I did a mistake in returning a cached error in the code pushed in PR #9798. Fixing it in the PR with some tests. Checklist - [x] Tests updated	2 years ago
Ashwanth	26f36d07f4	ksonnet: refer to tsdb_shipper_shared_store only if using_tsdb_shipper is true (#9852 ) What this PR does / why we need it: https://github.com/grafana/loki/pull/9790 introduced a new config `tsdb_shipper_shared_store`, but it's being referred to even if tsdb index is not enabled. This fixes it by accessing the value only if using_tsdb_shipper is set to true Which issue(s) this PR fixes: Fixes #<issue number> Special notes for your reviewer: Checklist - [X] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`)	2 years ago
Sandeep Sukhani	e089b4b60d	address feedback on pr 9798 (#9842 ) What this PR does / why we need it: PR #9798 got merged by mistake without me addressing the feedback. There is some critical feedback which needs to be addressed, particularly https://github.com/grafana/loki/pull/9798#discussion_r1243526743.	2 years ago
Jonathan Duncan	1d3f0b1001	Documentation: limits_config.enforce_metric_name removed (#9393 ) What this PR does / why we need it: ["The config still exists though, it's just not used anywhere."](https://github.com/grafana/loki/issues/2096) Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	2 years ago
Hamish Forbes	d0baa40df3	Only modify identical timestamps when increment-duplicate-timestamps enabled (#9437 ) When `-validation.increment-duplicate-timestamps` is enabled out of order log entries within the same push request can have their timestamps incorrectly mutated. The code is currently just looking for 2 consecutive entries where the timestamp is not increasing. It should only affect entries where the timestamp is a duplicate Test push request payload: ```json { "streams": [ { "stream": { "increment_duplicate_timestamp": "fixed\|true\|false" }, "values": [ [ "1683665111628000000", "1a: 1683665111628000000" ], [ "1683665111628000000", "1b: 1683665111628000000" ], [ "1683665111628000003", "3: 1683665111628000003" ], [ "1683665111628000002", "2: 1683665111628000002" ] ] } ] } ```` As only the first 2 of these timestamps are the same I would expect all but `1b` to have the timestamps recorded as-is and the logs returned by a query in the order: 1a,1b,2,3. Actual Results: ``` > logcli query --forward --limit 100 -z UTC -q --no-labels --since 60m -o jsonl '{increment_duplicate_timestamp="fixed"}' \| jq { "line": "1a: 1683665111628000000", "timestamp": "2023-05-09T20:45:11.628Z" } { "line": "1b: 1683665111628000000", "timestamp": "2023-05-09T20:45:11.628000001Z" } { "line": "2: 1683665111628000002", "timestamp": "2023-05-09T20:45:11.628000002Z" } { "line": "3: 1683665111628000003", "timestamp": "2023-05-09T20:45:11.628000003Z" } > logcli query --forward --limit 100 -z UTC -q --no-labels --since 60m -o jsonl '{increment_duplicate_timestamp="true"}' \| jq { "line": "1a: 1683665111628000000", "timestamp": "2023-05-09T20:45:11.628Z" } { "line": "1b: 1683665111628000000", "timestamp": "2023-05-09T20:45:11.628000001Z" } { "line": "3: 1683665111628000003", "timestamp": "2023-05-09T20:45:11.628000003Z" } { "line": "2: 1683665111628000002", "timestamp": "2023-05-09T20:45:11.628000004Z" } > logcli query --forward --limit 100 -z UTC -q --no-labels --since 60m -o jsonl '{increment_duplicate_timestamp="false"}' \| jq { "line": "1a: 1683665111628000000", "timestamp": "2023-05-09T20:45:11.628Z" } { "line": "1b: 1683665111628000000", "timestamp": "2023-05-09T20:45:11.628Z" } { "line": "2: 1683665111628000002", "timestamp": "2023-05-09T20:45:11.628000002Z" } { "line": "3: 1683665111628000003", "timestamp": "2023-05-09T20:45:11.628000003Z" } ``` `true` and `false` were written to Loki 2.8.2 with the `-validation.increment-duplicate-timestamps` enabled or disabled `fixed` with this patch applied and the flag enabled Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [x] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	2 years ago
aryalireza	dc4277fd86	Update tanka.md (#9820 ) Add delete loki's environment and resources from k8s cluster. What this PR does / why we need it: Which issue(s) this PR fixes: Fixes #<issue number> Special notes for your reviewer: Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) --------- Co-authored-by: Travis Patterson <masslessparticle@gmail.com> Co-authored-by: J Stickler <julie.stickler@grafana.com> Co-authored-by: Travis Patterson <travis.patterson@grafana.com>	2 years ago
Alvin Townsend	8e25567e9f	Loki Promtail: Don't Create home directory for `promtail` user in promtail-postinstall.sh (#9800 ) What this PR does / why we need it: When the `promtail-postinstall` script creates a `promtail` user, I do not believe that user requires a home directory to function correctly, and there are cases where the directory cannot be created causing the installation to fail. Which issue(s) this PR fixes: n/a - trivial fix Special notes for your reviewer: Checklist - [X] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required)	2 years ago
Sandeep Sukhani	2ed3c15c22	index: allow refreshing index file names cache for specific tables (#9798 ) What this PR does / why we need it: We have a cache for index files available in the object storage, which helps reduce the number of list calls we make to the object storage. Currently, we only support refreshing the whole index cache, which includes all the files for all tables and users. However, in a large cell with many tables and tenants, it is less likely that all the tables are being referenced. Also, most iterative operations would only need a fresh listing of just a few tables at a time so it is wasteful to always refresh the whole index cache. This PR adds support for refreshing the index cache for only specific tables. I am doing it as part of some compactor improvements to run compaction and retention in separate goroutines to avoid blocking compaction when some expensive delete requests are being processed. To make those changes, I needed a way to cost-effectively refresh the index files in the table being picked up for processing. Checklist - [x] Tests updated	2 years ago
Bryan Boreham	5c3e2a5cf8	Fix link to blog in readme (#9838 ) I noticed this link was 404-ing.	2 years ago
Karsten Jeschkies	b35bbd80d6	Support content negotiation between query frontend and querier. (#9813 ) What this PR does / why we need it: Currently, the querier sends results to the query frontend in JSON which is then decoded to Protobuf. It is more efficient to send the results as Protobuf. This will also allow to extend the results with custom data structures. The change is backwards compatible through content negotiation. Special notes for your reviewer: Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [x] Documentation added - [x] Tests updated - [x] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`)	2 years ago
ZhenYi Liang	0711580154	helm: support endpoint_suffix for azure storage (#9791 ) What this PR does / why we need it: According the [official document](https://grafana.com/docs/loki/latest/storage/#using-account-name-and-key), support storage_config.azure.endpoint_suffix, example blob.core.chinacloudapi.cn Which issue(s) this PR fixes: Fixes #<issue number> Special notes for your reviewer: Checklist - [X] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [x] Documentation added - [ ] Tests updated - [x] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [x] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) --------- Co-authored-by: Michel Hollands <42814411+MichelHollands@users.noreply.github.com> Co-authored-by: J Stickler <julie.stickler@grafana.com>	2 years ago
cui fliter	4b53909ccf	all: remove repetitive the (#9823 ) What this PR does / why we need it: Which issue(s) this PR fixes: Fixes #<issue number> Special notes for your reviewer: Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [x] Documentation added - [x] Tests updated - [x] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [x] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [x] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) Signed-off-by: cui fliter <imcusg@gmail.com>	2 years ago
Owen Diehl	b251a10234	uses lowercase standard for logging and adds total bytes to shard res… (#9828 ) Minor logging cleanup + helpful to have total bytes on this log line to correlate without needing to derive total bytes from factor * bytes_per_shard	2 years ago
Periklis Tsirakidis	bcdc807dd3	operator: Expose limits config setting cardinality_limit (#9830 )	2 years ago
Karsten Jeschkies	125e168a91	Replace flaky server for log level handler test. (#9822 ) What this PR does / why we need it: I've noticed that the `TestLevelHandler` were flaky because the code wasn't waiting to the server to be running. Since the test is for a handler it's much simpler to call it directly with a response recorder. Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [x] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`)	2 years ago
Lukas Grossar	6a1f34f4f1	Helm: remove PVC retention policy in single-binary mode when persistence is disabled (#9805 ) When there are no volumeClaimTemplates configured in the StatefulSet then the Kubernetes API doesn't store the configuration for persistentVolumeClaimRetentionPolicy in turn leading to the application being constantly out of sync in e.g. Argo CD What this PR does / why we need it: Fixes an issue where the chart is out constantly out of sync in e.g. Argo CD Which issue(s) this PR fixes: Special notes for your reviewer: Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [x] Documentation added - [ ] Tests updated - [x] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [x] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [x] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) Co-authored-by: Michel Hollands <42814411+MichelHollands@users.noreply.github.com>	2 years ago
Dylan Guedes	1d20fdbf57	LogPushFailures: Log the tenant id (#9819 ) What this PR does / why we need it: Modify the `writefailures.Manager` to - Log the tenant id. Otherwise, it is hard to identify which tenant is responsible for a problematic push - Log stream parsing failures Which issue(s) this PR fixes: N/A	2 years ago
Trevor Whitney	d4cfbaac4f	Implement series volume range queries (#9812 ) This PR adds a `series_volume_range` endpoint which allows for series volume queries over time with a specified step, returning timeseries data in the form of a Prometheus matrix response. The existing `series_volume` endpoint still returns Prometheus vector responses, and hardcodes the step to 0.	2 years ago
Salva Corts	6cc581bd26	Add back cache stats for index stats requests (#9816 ) What this PR does / why we need it: In https://github.com/grafana/loki/pull/9536, we added cache stats for index stats requests. That PR had a bug that inflated the query stats due to reusing the stats context in the query engine. Therefore, we had to revert the PR at https://github.com/grafana/loki/pull/9721. This PR brings back the changes from https://github.com/grafana/loki/pull/9536 but fixes the inflated starts by no longer reusing the same context in the query engine, but rather creating a new one for the shard resolver. I tested it on a dev cluster and seems to be working fine. here's the output for the same query: Stats with the bug from #9536: ``` ... Cache.StatsResult.Requests 980 Cache.StatsResult.EntriesRequested 490 Cache.StatsResult.EntriesFound 0 Cache.StatsResult.EntriesStored 490 Cache.StatsResult.BytesSent 0 B Cache.StatsResult.BytesReceived 0 B ... Summary.BytesProcessedPerSecond 43 GB Summary.LinesProcessedPerSecond 93305142 Summary.TotalBytesProcessed 945 GB Summary.TotalLinesProcessed 2059694183 ``` Stats from _main_ ``` ... Summary.BytesProcessedPerSecond 1.6 GB Summary.LinesProcessedPerSecond 3403718 Summary.TotalBytesProcessed 95 GB Summary.TotalLinesProcessed 207971404 ``` Stats with fix in this PR ``` .. Cache.StatsResult.Requests 132 Cache.StatsResult.EntriesRequested 66 Cache.StatsResult.EntriesFound 0 Cache.StatsResult.EntriesStored 66 Cache.StatsResult.BytesSent 0 B Cache.StatsResult.BytesReceived 0 B ... Summary.BytesProcessedPerSecond 4.3 GB Summary.LinesProcessedPerSecond 9468900 Summary.TotalBytesProcessed 95 GB Summary.TotalLinesProcessed 207793816 ``` As can be seen, with the changes in this PR, the summary stats are no longer inflated. Which issue(s) this PR fixes: Fixes https://github.com/grafana/loki/pull/9536 Special notes for your reviewer: I think it's ok to skip reviewing the changes from the commit cherry-picking the changes from https://github.com/grafana/loki/pull/9536 Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`)	2 years ago
Tiago Posse	a24ef7b206	Helm: support metricRelabelings for loki serviceMonitors (#9784 ) What this PR does / why we need it: Add support for `metricRelabelings` in the `ServiceMonitor` resources when monitoring Loki. Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [x] Documentation added - [ ] Tests updated - [x] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [x] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) --------- Signed-off-by: Tiago Posse <tiagoposse@gmail.com> Co-authored-by: Tiago Posse <tiago.posse@hi.health>	2 years ago
Victor Login	15538ac4fc	loki: upgrade version grafana-agent-operator (#9041 ) What this PR does / why we need it: Which issue(s) this PR fixes: Fixes: - https://github.com/prometheus-community/helm-charts/issues/3122 - https://github.com/grafana/helm-charts/issues/2236 Special notes for your reviewer: update dependencies for fix problem with CRD Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [x] Documentation added - [x] Tests updated - [x] `CHANGELOG.md` updated - [x] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` Signed-off-by: Victor Login <batazor@evrone.com>	2 years ago
manohar-koukuntla	999259a4d3	jsonnet: Add TSDB shipper configuration flag (#9790 ) This commit adds the TSDB equivalent of `use_boltdb_shipper` flag to be able to configure `tsdb_shipper` section. Fixes #9789	2 years ago
dependabot[bot]	579e54228e	Bump github.com/prometheus/client_golang from 1.15.1 to 1.16.0 (#9794 ) Bumps [github.com/prometheus/client_golang](https://github.com/prometheus/client_golang) from 1.15.1 to 1.16.0. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/prometheus/client_golang/releases">github.com/prometheus/client_golang's releases</a>.</em></p> <blockquote> <h2>v1.16.0</h2> <h2>What's Changed</h2> <ul> <li>[BUGFIX] api: Switch to POST for LabelNames, Series, and QueryExemplars. <a href="https://redirect.github.com/prometheus/client_golang/issues/1252">#1252</a></li> <li>[BUGFIX] api: Fix undefined execution order in return statements. <a href="https://redirect.github.com/prometheus/client_golang/issues/1260">#1260</a></li> <li>[BUGFIX] native histograms: Fix bug in bucket key calculation. <a href="https://redirect.github.com/prometheus/client_golang/issues/1279">#1279</a></li> <li>[ENHANCEMENT] Reduce constrainLabels allocations for all metrics. <a href="https://redirect.github.com/prometheus/client_golang/issues/1272">#1272</a></li> <li>[ENHANCEMENT] promhttp: Add process start time header for scrape efficiency. <a href="https://redirect.github.com/prometheus/client_golang/issues/1278">#1278</a></li> <li>[ENHANCEMENT] promlint: Improve metricUnits runtime. <a href="https://redirect.github.com/prometheus/client_golang/issues/1286">#1286</a></li> </ul> <!-- raw HTML omitted --> <ul> <li>Merge v1.15 to main by <a href="https://github.com/bwplotka"><code>@bwplotka</code></a> in <a href="https://redirect.github.com/prometheus/client_golang/pull/1250">prometheus/client_golang#1250</a></li> <li>Switch to POST for LabelNames, Series, and QueryExemplars to DoGetFallback by <a href="https://github.com/jacksontj"><code>@jacksontj</code></a> in <a href="https://redirect.github.com/prometheus/client_golang/pull/1252">prometheus/client_golang#1252</a></li> <li>✏️ [collectors]: fix typo in test assertion by <a href="https://github.com/vegerot"><code>@vegerot</code></a> in <a href="https://redirect.github.com/prometheus/client_golang/pull/1153">prometheus/client_golang#1153</a></li> <li>Added interactive tutorial [kubeCon] by <a href="https://github.com/bwplotka"><code>@bwplotka</code></a> in <a href="https://redirect.github.com/prometheus/client_golang/pull/1255">prometheus/client_golang#1255</a></li> <li>Fixed tutorial. by <a href="https://github.com/bwplotka"><code>@bwplotka</code></a> in <a href="https://redirect.github.com/prometheus/client_golang/pull/1256">prometheus/client_golang#1256</a></li> <li>Bump golang.org/x/sys from 0.6.0 to 0.7.0 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/prometheus/client_golang/pull/1265">prometheus/client_golang#1265</a></li> <li>Cleanup proto use in tests by <a href="https://github.com/SuperQ"><code>@SuperQ</code></a> in <a href="https://redirect.github.com/prometheus/client_golang/pull/1264">prometheus/client_golang#1264</a></li> <li>Fix tutorial on WSL-based systems by <a href="https://github.com/marevers"><code>@marevers</code></a> in <a href="https://redirect.github.com/prometheus/client_golang/pull/1257">prometheus/client_golang#1257</a></li> <li>Fix undefined execution order in return statements by <a href="https://github.com/PiotrLewandowski323"><code>@PiotrLewandowski323</code></a> in <a href="https://redirect.github.com/prometheus/client_golang/pull/1260">prometheus/client_golang#1260</a></li> <li>Merge release 1.15.1 to main by <a href="https://github.com/bwplotka"><code>@bwplotka</code></a> in <a href="https://redirect.github.com/prometheus/client_golang/pull/1267">prometheus/client_golang#1267</a></li> <li>GitHub Workflows security hardening by <a href="https://github.com/sashashura"><code>@sashashura</code></a> in <a href="https://redirect.github.com/prometheus/client_golang/pull/1180">prometheus/client_golang#1180</a></li> <li>add process start time header to client_golang prometheus by <a href="https://github.com/logicalhan"><code>@logicalhan</code></a> in <a href="https://redirect.github.com/prometheus/client_golang/pull/1278">prometheus/client_golang#1278</a></li> <li>Fix bug in bucket key calculation by <a href="https://github.com/beorn7"><code>@beorn7</code></a> in <a href="https://redirect.github.com/prometheus/client_golang/pull/1279">prometheus/client_golang#1279</a></li> <li>Bump github.com/prometheus/procfs from 0.9.0 to 0.10.1 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/prometheus/client_golang/pull/1283">prometheus/client_golang#1283</a></li> <li>Reduce constrainLabels allocations by <a href="https://github.com/khasanovbi"><code>@khasanovbi</code></a> in <a href="https://redirect.github.com/prometheus/client_golang/pull/1272">prometheus/client_golang#1272</a></li> <li>added circleci as gh action YAML by <a href="https://github.com/krishnaduttPanchagnula"><code>@krishnaduttPanchagnula</code></a> in <a href="https://redirect.github.com/prometheus/client_golang/pull/1281">prometheus/client_golang#1281</a></li> <li>Improve metricUnits runtime by <a href="https://github.com/avlitman"><code>@avlitman</code></a> in <a href="https://redirect.github.com/prometheus/client_golang/pull/1286">prometheus/client_golang#1286</a></li> <li>Moving fully to GH actions. by <a href="https://github.com/bwplotka"><code>@bwplotka</code></a> in <a href="https://redirect.github.com/prometheus/client_golang/pull/1288">prometheus/client_golang#1288</a></li> <li>Fix docstring references to renamed native histogram fields / functions. by <a href="https://github.com/juliusv"><code>@juliusv</code></a> in <a href="https://redirect.github.com/prometheus/client_golang/pull/1290">prometheus/client_golang#1290</a></li> <li>Fixed README & CHANGELOG; Added fmt makefile command (+bingo) for easier contributions. by <a href="https://github.com/bwplotka"><code>@bwplotka</code></a> in <a href="https://redirect.github.com/prometheus/client_golang/pull/1289">prometheus/client_golang#1289</a></li> <li></li> </ul> <!-- raw HTML omitted --> <h2>New Contributors</h2> <ul> <li><a href="https://github.com/vegerot"><code>@vegerot</code></a> made their first contribution in <a href="https://redirect.github.com/prometheus/client_golang/pull/1153">prometheus/client_golang#1153</a></li> <li><a href="https://github.com/marevers"><code>@marevers</code></a> made their first contribution in <a href="https://redirect.github.com/prometheus/client_golang/pull/1257">prometheus/client_golang#1257</a></li> <li><a href="https://github.com/PiotrLewandowski323"><code>@PiotrLewandowski323</code></a> made their first contribution in <a href="https://redirect.github.com/prometheus/client_golang/pull/1260">prometheus/client_golang#1260</a></li> <li><a href="https://github.com/sashashura"><code>@sashashura</code></a> made their first contribution in <a href="https://redirect.github.com/prometheus/client_golang/pull/1180">prometheus/client_golang#1180</a></li> <li><a href="https://github.com/logicalhan"><code>@logicalhan</code></a> made their first contribution in <a href="https://redirect.github.com/prometheus/client_golang/pull/1278">prometheus/client_golang#1278</a></li> <li><a href="https://github.com/khasanovbi"><code>@khasanovbi</code></a> made their first contribution in <a href="https://redirect.github.com/prometheus/client_golang/pull/1272">prometheus/client_golang#1272</a></li> <li><a href="https://github.com/krishnaduttPanchagnula"><code>@krishnaduttPanchagnula</code></a> made their first contribution in <a href="https://redirect.github.com/prometheus/client_golang/pull/1281">prometheus/client_golang#1281</a></li> <li><a href="https://github.com/avlitman"><code>@avlitman</code></a> made their first contribution in <a href="https://redirect.github.com/prometheus/client_golang/pull/1286">prometheus/client_golang#1286</a></li> </ul> <p><strong>Full Changelog</strong>: <a href="https://github.com/prometheus/client_golang/compare/v1.15.1...v1.16.0">https://github.com/prometheus/client_golang/compare/v1.15.1...v1.16.0</a></p> </blockquote> </details> <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/prometheus/client_golang/blob/main/CHANGELOG.md">github.com/prometheus/client_golang's changelog</a>.</em></p> <blockquote> <h2>1.16.0 / 2023-06-15</h2> <ul> <li>[BUGFIX] api: Switch to POST for LabelNames, Series, and QueryExemplars. <a href="https://redirect.github.com/prometheus/client_golang/issues/1252">#1252</a></li> <li>[BUGFIX] api: Fix undefined execution order in return statements. <a href="https://redirect.github.com/prometheus/client_golang/issues/1260">#1260</a></li> <li>[BUGFIX] native histograms: Fix bug in bucket key calculation. <a href="https://redirect.github.com/prometheus/client_golang/issues/1279">#1279</a></li> <li>[ENHANCEMENT] Reduce constrainLabels allocations for all metrics. <a href="https://redirect.github.com/prometheus/client_golang/issues/1272">#1272</a></li> <li>[ENHANCEMENT] promhttp: Add process start time header for scrape efficiency. <a href="https://redirect.github.com/prometheus/client_golang/issues/1278">#1278</a></li> <li>[ENHANCEMENT] promlint: Improve metricUnits runtime. <a href="https://redirect.github.com/prometheus/client_golang/issues/1286">#1286</a></li> </ul> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href="`3583c1e1d0`"><code>3583c1e</code></a> Cut v1.16.0 (<a href="https://redirect.github.com/prometheus/client_golang/issues/1292">#1292</a>)</li> <li><a href="`2feda42e44`"><code>2feda42</code></a> Fixed README & CHANGELOG; Added fmt makefile command (+bingo) for easier cont...</li> <li><a href="`5b9cf9c6a8`"><code>5b9cf9c</code></a> Merge pull request <a href="https://redirect.github.com/prometheus/client_golang/issues/1290">#1290</a> from prometheus/fix-nh-docstring-refs</li> <li><a href="`1b19d5f458`"><code>1b19d5f</code></a> Fix docstring references to renamed native histogram fields / functions.</li> <li><a href="`7352ab7f80`"><code>7352ab7</code></a> Moving fully to GH actions. (<a href="https://redirect.github.com/prometheus/client_golang/issues/1288">#1288</a>)</li> <li><a href="`e4ff34d23e`"><code>e4ff34d</code></a> Improve metricUnits runtime (<a href="https://redirect.github.com/prometheus/client_golang/issues/1286">#1286</a>)</li> <li><a href="`781ea28024`"><code>781ea28</code></a> added circleci as gh action YAML (<a href="https://redirect.github.com/prometheus/client_golang/issues/1281">#1281</a>)</li> <li><a href="`a09a1d34cb`"><code>a09a1d3</code></a> Reduce constrainLabels allocations (<a href="https://redirect.github.com/prometheus/client_golang/issues/1272">#1272</a>)</li> <li><a href="`8840afcfc2`"><code>8840afc</code></a> Bump github.com/prometheus/procfs from 0.9.0 to 0.10.1 (<a href="https://redirect.github.com/prometheus/client_golang/issues/1283">#1283</a>)</li> <li><a href="`5e78d5f66b`"><code>5e78d5f</code></a> Merge pull request <a href="https://redirect.github.com/prometheus/client_golang/issues/1279">#1279</a> from prometheus/beorn7/histogram</li> <li>Additional commits viewable in <a href="https://github.com/prometheus/client_golang/compare/v1.15.1...v1.16.0">compare view</a></li> </ul> </details> <br /> [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=github.com/prometheus/client_golang&package-manager=go_modules&previous-version=1.15.1&new-version=1.16.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	2 years ago
Ashwanth	f9fc1df4c0	logfmt: add --strict flag to enable strict parsing, perform nostrict parsing by default (#9626 ) What this PR does / why we need it: Introduces `--strict` flag for logfmt parser to enable strict parsing. With strict parsing enabled, logfmt parser stops scanning the log line and returns early with an error if it encounters any poorly formatted key=value pair. This pr updates the parser to perform non-strict parsing by default where it skips over the invalid tokens. This offers the flexibility to parse semi-structured log lines. Also adds `--keep-empty` flag to retain standalone keys(with empty value) as labels with value set to empty string Special notes for your reviewer: Checklist - [X] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [X] Documentation added - [X] Tests updated - [x] `CHANGELOG.md` updated - [x] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` --------- Co-authored-by: Michel Hollands <42814411+MichelHollands@users.noreply.github.com>	2 years ago
Dylan Guedes	dc5d458c14	Failures logging: avoid logging generic errors (#9796 ) What this PR does / why we need it: Modify our write path to not log generic messages with the `writefailures.Manager`. With generic messages I mean: contextCancel errors and contextDeadlineExceeded errors. Which issue(s) this PR fixes: N/A	2 years ago
Salva Corts	08e505dd50	Return all errors for a failed push request (#9807 ) What this PR does / why we need it: Distributors enforce limits and validate each push payload. before this PR, if a single push request yields multiple errors, only the last error is forwarded to the client. This PR modifies the distributor push handler so we forward all errors from a single push request to the client. Which issue(s) this PR fixes: Fixes #<issue number> Special notes for your reviewer: - I wonder if the use of [MultiError][1] is well-suited for this. [This test case][2] will return 100 errors with the same content. I wonder if we can leverage the MultiError to group errors that have the same content. Printing for example: `1 error like: 'Error parsing labels...'; 100 errors like: 'Max entry size...'`. This would require extra processing but If we do this processing in the Error() method I think it should be ok-ish? - Edit: I ended up implementing this in [`4b0816f`](`4b0816f67f`) Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [x] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) [1]: `d4a4728933/pkg/util/errors.go (L32)` [2]: `f544b93c20/pkg/distributor/distributor_test.go (L75)`	2 years ago
Owen Diehl	e2a63e582c	adds tsdb-max-bytes-per-shard limit (#9811 ) Adds the per tenant limit `tsdb-max-bytes-per-shard` which is used in configuring the shard size for tsdb subqueries. This effectively gives control over how big subqueries should be (assuming they're shardable). The default is no different (`600MB`) than the previously hardcoded initial value. This should help us iterate to find optimal shard sizes to improve operations in the long term.	2 years ago
Vladyslav Diachenko	00cf3133e3	disabled indexGateway client for boltdbShipper in migration tool. (#9809 ) What this PR does / why we need it: Without this fix, the migration tool tries to initialize the ingexGateway client, which is unreachable. This leads to the error: `Failed to create source store: error creating index client: index gateway grpc dial: failed to build resolver: passthrough: received empty target in Build()`	2 years ago
Christian Haudum	a65c99d9bf	Add per-tenant request counter metric to index gateway server (#9797 ) This commit add a counter metric `loki_index_gateway_requests_total` with labels `operation`, `tenant`, `status` for gRPC requests that are served by the index gateway. What for? The per-tenant RPS on the index gateway is used to derive the per-tenant shard factor. Why tracking on the server? Unlike tracking index gateway RPS on the client side, tracking on the server side does not yield that many series, even in multi-tenant installations with a lot of tenants, because the amount of index gateway instances is relatively small compared to the amount of queriers and frontends. Special notes for your reviewer: The previous approach of tracking requests on the client https://github.com/grafana/loki/pull/9781 has been abandoned. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
Travis Patterson	8ca035ffbf	Log Volume: Do the prometheus-format conversion in dedicated middleware (#9776 ) This PR moves the Prometheus-format conversion to it's own middleware. This keeps it all in one place and consolidates logic that was in many places. It also sets us up to make range queries as a next step.	2 years ago
dependabot[bot]	af71030e30	Bump anchore/sbom-action from 0.14.2 to 0.14.3 (#9793 ) Bumps [anchore/sbom-action](https://github.com/anchore/sbom-action) from 0.14.2 to 0.14.3. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/anchore/sbom-action/releases">anchore/sbom-action's releases</a>.</em></p> <blockquote> <h2>v0.14.3</h2> <h2>Changes in v0.14.3</h2> <ul> <li>update Syft to v0.84.0 (<a href="https://redirect.github.com/anchore/sbom-action/issues/418">#418</a>) [<a href="https://github.com/anchore-actions-token-generator">anchore-actions-token-generator</a>]</li> </ul> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href="`78fc58e266`"><code>78fc58e</code></a> chore(deps): update Syft to v0.84.0 (<a href="https://redirect.github.com/anchore/sbom-action/issues/418">#418</a>)</li> <li>See full diff in <a href="https://github.com/anchore/sbom-action/compare/v0.14.2...v0.14.3">compare view</a></li> </ul> </details> <br /> [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=anchore/sbom-action&package-manager=github_actions&previous-version=0.14.2&new-version=0.14.3)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	2 years ago
Victor Login	30fff3cd65	upgrade version nginxinc/nginx-unprivileged to 1.23 (#8984 ) What this PR does / why we need it: This PR upgrade nginxinc/nginx-unprivileged to 1.23 Which issue(s) this PR fixes: Fixes #<issue number> Special notes for your reviewer: Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` Co-authored-by: Travis Patterson <travis.patterson@grafana.com>	2 years ago
Ward Loos	c88a3688e0	Helm - Loki: Fix empty starting line in runtimeConfig (#8657 ) What this PR does / why we need it: The `runtime` ConfigMap in the Loki Helm chart (introduced in `7aa596752d`) starts with an empty line because it improperly removes the current line. ```yaml --- # Source: loki/templates/runtime-configmap.yaml apiVersion: v1 kind: ConfigMap metadata: name: loki-runtime data: runtime-config.yaml: \| {} ``` While this generates technically valid YAML, it looks weird and interacts badly with some tools. E.g.: - Python ruamel.yaml library: https://stackoverflow.com/questions/75584262/ruamel-yaml-adds-incorrect-indentation-indicator - VS Code which is set to trim whitespace on save (yellow highlighting is for objects, not strings): <img width="213" alt="Screenshot 2023-02-28 at 21 13 10" src="https://user-images.githubusercontent.com/4059732/221967899-719622c8-7393-4336-8c07-934e85a94a30.png"> This PR removes this empty start line: ```yaml --- # Source: loki/templates/runtime-configmap.yaml apiVersion: v1 kind: ConfigMap metadata: name: loki-runtime data: runtime-config.yaml: \| {} ``` Which issue(s) this PR fixes: Fixes #<issue number> Special notes for your reviewer: Checklist - [X] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	2 years ago
pixiono	7a11a1642b	loki chart: fix service/ingress mapping (#8708 ) What this PR does / why we need it: If you currently deploy the Loki chart under a release name other than "loki" errors occur which is why loki is not accessible via Ingress. The problem looks like this: > $ k describe ingress xyz-loki-app > Name: xyz-loki-app > Labels: app.kubernetes.io/instance=xyz-loki-app > app.kubernetes.io/managed-by=Helm > app.kubernetes.io/name=loki > [...] > Rules: > Host Path Backends > ---- ---- -------- > loki.example.com > /api/prom/tail xyz-loki-app-read:3100 (<error: endpoints "xyz-loki-app-read" not found>) > /loki/api/v1/tail xyz-loki-app-read:3100 (<error: endpoints "xyz-loki-app-read" not found>) > /loki/api xyz-loki-app-read:3100 (<error: endpoints "xyz-loki-app-read" not found>) > /api/prom/rules xyz-loki-app-read:3100 (<error: endpoints "xyz-loki-app-read" not found>) > /loki/api/v1/rules xyz-loki-app-read:3100 (<error: endpoints "xyz-loki-app-read" not found>) > /prometheus/api/v1/rules xyz-loki-app-read:3100 (<error: endpoints "xyz-loki-app-read" not found>) > /prometheus/api/v1/alerts xyz-loki-app-read:3100 (<error: endpoints "xyz-loki-app-read" not found>) > /api/prom/push xyz-loki-app-write:3100 (<error: endpoints "xyz-loki-app-write" not found>) > /loki/api/v1/push xyz-loki-app-write:3100 (<error: endpoints "xyz-loki-app-write" not found>) The name of the services is: loki-read & loki-write. I found [another PR with the same problem](https://github.com/grafana/loki/pull/8234) but with a different approach, so I would suggest here to solve the name problem without the question if .name or .fullname should be used. Which issue(s) this PR fixes: Fixes # Special notes for your reviewer: Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [x] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	2 years ago
René Scheibe	954df433e9	Promtail: Fix file descriptor leak (#8987 ) This reduces the amount of open file descriptors per followed journal file from 3 to 2. It also releases file descriptors for removed files (journal vacuuming). The `defaultJournalEntryFunc` is only called once upon startup. It creates a `Journal` instance to only read a single entry to determine the age of the entry referenced in the positions file. Right after, this instance is not required anymore. The actually used `Journal` instance used to follow the journal is created inside `journalTargetWithReader`.	2 years ago
Alexandru Gologan	f785faa820	Helm: add additionalRuleLabels to PrometheusRule alerts (#9020 ) What this PR does / why we need it: Adds the ability to specify other labels on the Prometheus alerts. Specifying: ```yaml monitoring: rules: additionalRuleLabels: custom_label: custom_value ``` will add it to all alerts: ```yaml - alert: LokiRequestErrors annotations: ... expr: ... for: 15m labels: severity: critical custom_label: custom_value ``` This approach is inspired by: [kube-prometheus-stack/rules](https://github.com/prometheus-community/helm-charts/blob/main/charts/kube-prometheus-stack/templates/prometheus/rules-1.14/general.rules.yaml) Though in their case they run [sync_prometheus_rules.py](https://github.com/prometheus-community/helm-charts/blob/main/charts/kube-prometheus-stack/hack/sync_prometheus_rules.py) to create the `tpl`. I can understand if this is not the approach (manual) you want to take as editing the tpl adding the labels everywhere and escaping go tpl `{{ }}` is quite painful. Which issue(s) this PR fixes: N/A Special notes for your reviewer: Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [x] Documentation added - [x] Tests updated - [ ] `CHANGELOG.md` updated - [x] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	2 years ago
Jim Gustavsson	9eca39601d	Helm: Add PDB when gateway autoscaling is enabled (#8439 ) What this PR does / why we need it: The logic for adding PDB (PodDisruptionBudget) for gateway is not taking autoscaling into account. It is only added when `replicas` > 1. Instead, use the following logic: - When autoscaling is disabled, PDB shall be added when `replicas` > 1 - When autoscaling is enabled, PDB shall be adedd when `autoscaling.minReplicas` > 1 Which issue(s) this PR fixes: Fixes #<issue number> Special notes for your reviewer: Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` Co-authored-by: Travis Patterson <travis.patterson@grafana.com>	2 years ago
Nyaundi Brian	5add792487	Update _index.md (#9028 ) What this PR does / why we need it: Typo fix Which issue(s) this PR fixes: None	2 years ago
Bryan Boreham	6928aac776	loser tree: allow sequences to contain maxVal (#9057 ) What this PR does / why we need it: The tree stores the loser in each node, or maxVal if the sequence has ended. If sequences can contain maxVal, we need to take care not to let an ended sequence overtake an active one. Some test cases from Charles who pointed out the issue in #9053. Note this situation cannot happen for real in Loki, since we don't expect log lines with a timestamp that far into the future, but this is supposed to be a re-usable library. Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - NA Documentation added - [x] Tests updated - NA `CHANGELOG.md` updated - NA Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` Co-authored-by: Charles Korn <charles.korn@grafana.com>	2 years ago
Ryo Shindo	ed0d15a6bf	loki: enable PathPrefix for queryHandler endpoints (#8406 ) What this PR does / why we need it: Hi, loki developers. I would like to use the option `http_path_prefix` as described in the official documentation. This setting is valid for endpoints defined at `4d55a24ed3/pkg/loki/loki.go (L477-L494)` but not for the endpoints defined at `4d55a24ed3/pkg/loki/modules.go (L366-L392)` . The current latest version with `http_path_prefix` will return the expected response for the former endpoint, but will return a 404 not found error when accessing the latter endpoint. For example, if you define `http_path_prefix = /xxx`, requests for `/xxx/ready` will work as expected, but requests for `/xxx/loki/api/v1/query` will result in 404 errors. I've created a pull request for this issue because I've been experiencing some inconvenience when using loki. Please check it out. Which issue(s) this PR fixes: Fixes https://github.com/grafana/loki/issues/4756 Special notes for your reviewer: Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [x] Tests updated --------- Co-authored-by: Michel Hollands <42814411+MichelHollands@users.noreply.github.com> Co-authored-by: Travis Patterson <travis.patterson@grafana.com>	2 years ago
Sandeep Sukhani	3e1f2fc273	caching: do not try to fill the gap in log results cache when the new query interval does not overlap the cached query interval (#9757 ) What this PR does / why we need it: Currently, when we find a relevant cached negative response for a logs query, we do the following: * If the cached query completely covers the new query: * return back an empty response. * else: * fill the gaps on either/both sides of the cached query. The problem with filling the gaps is that when the cached query does not overlap at all with the new query, we have to extend the query beyond what the query requests for. However, with the logs query, we have a limit on the number of lines we can send back in the response. So, this could result in the query response having logs which were not requested by the query, which then get filtered out by the [response extractor](`b78d3f0552/pkg/querier/queryrange/log_result_cache.go (L299)`), unexpectedly resulting in an empty response. For example, if the query was cached for start=15, end=20 and we get a `backwards` query for start=5, end=10. To fill the gap, the query would be executed for start=5, end=15. Now, if we have logs more than the query `limit` in the range 10-15, we would filter out all the data in the response extractor and send back an empty response to the user. This PR fixes the issue by doing the following changes when handling cache hit: * If the cached query completely covers the new query: * return back an empty response[_existing_]. * else if the cached query does not overlap with the new query: * do the new query as requested. * If the new query results in an empty response and has a higher interval than the cached query: * update the cache * else: * query the data for missing intervals on both/either side[_existing_] * update the cache with extended intervals if the new queries resulted in an empty response[_existing_] Special notes for your reviewer: We could do further improvements in the handling of queries not overlapping with cached query by selectively extending the queries based on query direction and cached query lying before/after the new query. For example, if the new query is doing `backwards` query and the `cachedQuery.End` < `newQuery.Start`, it should be okay to extend the query and do `cachedQuery.End` to `newQuery.End` to fill the cache since query would first fill the most relevant data before hitting the limits. I did not want to complicate the fix so went without implementing this approach. We can revisit later if we feel we need to improve our caching. Checklist - [x] Tests updated - [x] `CHANGELOG.md` updated --------- Co-authored-by: Travis Patterson <travis.patterson@grafana.com>	2 years ago
Paul Rogers	0da462558f	Migrate individual chunks as opposed to all chunks. (#9780 ) This way, if an error occurs, we get as much as we can. What this PR does / why we need it: In some situations, not all blob data may be accessible. For instance, in the case where we have started migration between object storage buckets, some data may be in the old bucket. If we wanted to migrate data from the new bucket, those chunks will error with a 404, as they are in the old bucket. This PR is so that we continue on and get all of the chunks that we can, rather than stopping with whatever error may arise. Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`)	2 years ago
Travis Patterson	cc33749b73	fix integration test for multiple periods (#9782 ) The integration test is writing logs across two index periods but there's a chance that the log `lineB` is written after the beginning of the second period after it's added. This means we never get `lineB` in the results. This change pushes back the write time of the first logs so they're always stay in the first index period.	2 years ago
Ashwanth	10e55a6359	store: fix incorrect usage of period config in table range (#9754 ) What this PR does / why we need it: We are incorrectly taking reference to a loop variable in store init, I overlooked this part as the reference was being taken in the call to `GetIndexTableNumberRange` As a result of this, `TableRange` of each period (which is used by index-shipper) points to the latest `periodConfig`. This in itself is not a problem if the index prefix doesn't change as we use the `periodConfig` only to generate the `tableName`. But if the index prefix changes across periods, we end up losing access to the indexes using the old prefix. Special notes for your reviewer: Checklist - [X] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [x] Tests updated - [x] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) --------- Signed-off-by: Ashwanth Goli <iamashwanth@gmail.com>	2 years ago
Susana Ferreira	35465d0297	Fix instant query summary split stats (#9773 ) What this PR does / why we need it: Fix instant query summary statistic's `splits` corresponding to the number of subqueries a query is split into based on `split_queries_by_interval`. * Update rangemapper with a statistics structure to include the number of split queries a query is mapped into. * In the `split_by_range` middleware once the mapped query is returned update the middleware statistics with the number of split queries. This value will then be merged with the statistics of the Loki response. Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [x] Tests updated - [x] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`)	2 years ago

1 2 3 4 5 ...

5268 Commits (9aadb348ed895aea2069c7d3f4fdf9c052e5f18b) All Branches Search

5268 Commits (9aadb348ed895aea2069c7d3f4fdf9c052e5f18b)

All Branches