apitech/loki - loki - Gitea: Git with a cup of tea

Commit Graph

Author	SHA1	Message	Date
Grot (@grafanabot)	05d771fee3	[CI/CD] Update yaml file `./production/helm/loki/Chart.yaml` (+1 other) (#10015 ) Here is a summary of the updates contained in this PR: * Update attribute `$.appVersion` in yaml file `./production/helm/loki/Chart.yaml` to the following value: `2.8.3` * Bump version of Helm Chart Add changelog entry to `./production/helm/loki/CHANGELOG.md` Re-generate docs Co-authored-by: Michel Hollands <42814411+MichelHollands@users.noreply.github.com>	2 years ago
Vladyslav Diachenko	416e01ca8e	[promtail] non_indexed_labels stage (#9986 ) What this PR does / why we need it: added non_indexed_labels stage to copy values from extracted data tonon-indexed-labels field of the Entry. Special notes for your reviewer: Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) --------- Signed-off-by: Vladyslav Diachenko <vlad.diachenko@grafana.com>	2 years ago
Salva Corts	7e248e1e90	Support non-indexed labels in WAL (#10011 ) What this PR does / why we need it: In https://github.com/grafana/loki/pull/9700, we support encoding and decoding non-indexed labels for each entry into the chunks. This PR adds support for writing/reading the non-indexed labels to/from the WAL. Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [x] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`)	2 years ago
parham zardoshti	0eb435c49c	helm: add `/config` proxy_pass to nginx gateway (#10000 ) ### issue: According to the issue https://github.com/grafana/loki/issues/9917 Adding Grafana Loki Enterprise to Grafana Apps Enterprise needs access to the GEL `/config` API path for cluster configuration if GEL Apps in Grafana doesn't have an access to this path, client won't be able to see `/Cluster Configuration` and also won't be able to create a data source in the Apps with the related Grafana Enterprise logs plugin. ### more information: `HOME/Apps/Grfana Enterprise Logs/Cluster Configuration` ``` A note about this cluster configuration page The parameter values shown below are those returned by http://example.com:30519/config. If you have configured other components in your cluster with different parameter values, those will not be reflected below. For documentation on the function of each configuration parameter, see [Configuration reference](https://grafana.com/docs/enterprise-logs/latest/config/reference/). ``` <img width="712" alt="Screenshot 1402-04-30 at 03 17 34" src="https://github.com/grafana/loki/assets/53074289/48170dce-345d-42d8-bbaa-1bdf4d7bd2f3"> This commit will add `/config` to nginx gateway on SSD mode loki helm chart deployment --------- Signed-off-by: Michel Hollands <michel.hollands@gmail.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: Michel Hollands <42814411+MichelHollands@users.noreply.github.com> Co-authored-by: Joao Marcal <jmarcal@redhat.com> Co-authored-by: Periklis Tsirakidis <periklis@redhat.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Salva Corts <salva.corts@grafana.com>	2 years ago
MOZGIII	08e741b7c6	helm: Allow altering the canary label name (#9945 ) What this PR does / why we need it: Allows changing the label name at the canary in the helm chart. Which issue(s) this PR fixes: Idk, none? Special notes for your reviewer: This is a very trivial change. Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [x] Documentation added - [x] Tests updated - [x] `CHANGELOG.md` updated - [x] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [x] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [x] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) --------- Co-authored-by: Michel Hollands <42814411+MichelHollands@users.noreply.github.com> Co-authored-by: Travis Patterson <travis.patterson@grafana.com>	2 years ago
Salva Corts	3f161f5c1a	Improve observability for non-indexed labels usage (#9993 ) What this PR does / why we need it: In https://github.com/grafana/loki/pull/9700, we support encoding and decoding metadata for each entry into the chunks. In this PR we: - Update the bytes processed stats to account for the bytes from those non-indexed labels - Add new stats for bytes processed for those non-indexed labels - Add new ingestion metrics to track ingested non-indexed bytes Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [x] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`)	2 years ago
dependabot[bot]	85d23c2452	Bump go.uber.org/atomic from 1.10.0 to 1.11.0 (#9940 ) Bumps [go.uber.org/atomic](https://github.com/uber-go/atomic) from 1.10.0 to 1.11.0. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/uber-go/atomic/releases">go.uber.org/atomic's releases</a>.</em></p> <blockquote> <h2>v1.11.0</h2> <h3>Fixed</h3> <ul> <li>Fix <code>Swap</code> and <code>CompareAndSwap</code> for <code>Value</code> wrappers without initialization.</li> </ul> <h3>Added</h3> <ul> <li>Add <code>String</code> method to <code>atomic.Pointer[T]</code> type allowing users to safely print underlying values of pointers.</li> </ul> </blockquote> </details> <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/uber-go/atomic/blob/master/CHANGELOG.md">go.uber.org/atomic's changelog</a>.</em></p> <blockquote> <h2><a href="https://github.com/uber-go/atomic/compare/v1.10.0...v1.11.0">1.11.0</a> - 2023-05-02</h2> <h3>Fixed</h3> <ul> <li>Fix <code>Swap</code> and <code>CompareAndSwap</code> for <code>Value</code> wrappers without initialization.</li> </ul> <h3>Added</h3> <ul> <li>Add <code>String</code> method to <code>atomic.Pointer[T]</code> type allowing users to safely print underlying values of pointers.</li> </ul> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href="`76f817c8b7`"><code>76f817c</code></a> Update CHANGELOG.md</li> <li><a href="`2a750c4499`"><code>2a750c4</code></a> Preparing release v1.11.0</li> <li><a href="`544d6aa5c2`"><code>544d6aa</code></a> Minimize permissions to CI workflows (<a href="https://redirect.github.com/uber-go/atomic/issues/133">#133</a>)</li> <li><a href="`1505d282ac`"><code>1505d28</code></a> Fix Swap and CompareAndSwap for Value wrappers (<a href="https://redirect.github.com/uber-go/atomic/issues/130">#130</a>)</li> <li><a href="`78a3b8ec6c`"><code>78a3b8e</code></a> Implement String method for pointers (<a href="https://redirect.github.com/uber-go/atomic/issues/122">#122</a>)</li> <li><a href="`159e329dd8`"><code>159e329</code></a> chore(gen-*): Move templates into external files (<a href="https://redirect.github.com/uber-go/atomic/issues/121">#121</a>)</li> <li>See full diff in <a href="https://github.com/uber-go/atomic/compare/v1.10.0...v1.11.0">compare view</a></li> </ul> </details> <br /> [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=go.uber.org/atomic&package-manager=go_modules&previous-version=1.10.0&new-version=1.11.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	2 years ago
Joao Marcal	0fd9331044	operator: Fix OIDC.IssuerCAPath by updating it to type CASpec (#9972 ) Co-authored-by: Periklis Tsirakidis <periklis@redhat.com>	2 years ago
Michel Hollands	85c6a436a7	New build image for release branches (#9997 ) What this PR does / why we need it: Loki build image with go 1.20.6 and previous version of golangci. Which issue(s) this PR fixes: See the Included README.md --------- Signed-off-by: Michel Hollands <michel.hollands@gmail.com>	2 years ago
J Stickler	cc2aeac860	Break up and move Best Practices topic. (#9912 ) I realized that I should have moved the Best Practices topic as part of #8711, since most of the content is best practices for using labels. This PR: * Splits the file into two parts (Best practices for labels and best practices for configuration) * Moves the two files to Get Started and Configure, respectively. * Updates page weights for both sections as needed.	2 years ago
Karsten Jeschkies	efac690328	Use series hash to identify uniques in merge. (#9985 ) What this PR does / why we need it: A run of the protobuf encoding for the querier showed a spike in memory usage in the `codec.MergeResponse` method. The routine was allocating a lot of memory for the `series.String()` method which is then hashed. Instead, we can use a `uint64` hash. ``` › go test -v -run=^$ -bench "Benchmark_MergeResponses$" -count=10 ./pkg/querier/queryrange ... │ before.txt │ after.txt │ │ sec/op │ sec/op vs base │ _MergeResponses-16 4.563 ± 1% 1.197 ± 1% -73.77% (p=0.000 n=10) │ before.txt │ after.txt │ │ B/op │ B/op vs base │ _MergeResponses-16 6578.715Mi ± 0% 2.362Mi ± 0% -99.96% (p=0.000 n=10) │ before.txt │ after.txt │ │ allocs/op │ allocs/op vs base │ _MergeResponses-16 40500.1k ± 0% 100.1k ± 0% -99.75% (p=0.000 n=10) ``` Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [x] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`)	2 years ago
Mohamed-Amine Bouqsimi	9b62fe7d93	operator: Custom configuration for LokiStack admin groups (#9931 )	2 years ago
Trevor Whitney	ae4cd27d83	add sharded to querystats bytes processed (#9934 ) Add a sharded label to the `loki_logql_querystats_bytes_processed_per_second` metric that we use for determining our `LokiQueryRangeReadPerformanceBudgetBurn` metric so we can exclude slow queries with <= 1 shard from burning our SLO budget.	2 years ago
Ed Welch	fa6f9c638a	Loki: add 'post_filter_lines' stat for tracking how many lines are match a queries filter expression(s) (#9983 ) What this PR does / why we need it: To better understand query behavior and labeling strategies of users, logging the number of log lines "post filtering" can be very useful, we already log the total_lines processed in a query so this will allow us to see how many lines in the label selector matched what the query was looking for. Which issue(s) this PR fixes: Fixes #<issue number> Special notes for your reviewer: Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) Signed-off-by: Edward Welch <edward.welch@grafana.com>	2 years ago
Kaviraj Kanagaraj	4d18a5b006	loki: Add `CODEOWNERS` file (#9961 ) What this PR does / why we need it: This is to have CODEOWNERS file consistent across all projects under github.com/grafana Which issue(s) this PR fixes: Fixes #NA Special notes for your reviewer: Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com>	2 years ago
Vladyslav Diachenko	28e9e15639	[jsonnet] removed deprecated policy.v1beta1 (#9978 ) What this PR does / why we need it: in k8s version 1.25 some k8s APIs are deprecated, such as policy.v1beta1 which is moved to policy.v1 Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [x] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) --------- Signed-off-by: Vladyslav Diachenko <vlad.diachenko@grafana.com>	2 years ago
Salva Corts	78aad0b805	Fix buffered iterator (#9976 ) What this PR does / why we need it: In https://github.com/grafana/loki/pull/9700, we added support for writing non-indexed labels from entries into chunks. This PR introduced two bugs: - When the buffered iterator is closed, the buffer to read metadata labels is put back into the pool but not set to nil. Subsequent uses of the iterator may write on top of the buffer that was put back on the pool. This may lead to inconsistent/incorrect results. - Inside the buffered iterator, we were not correctly handling EOFs while reading the number of labels and each label length. This was due to the `lastAttemp` variable not being reset. This PR adds a new test for these two bugs and also fixes them. Which issue(s) this PR fixes: Fixes https://github.com/grafana/loki/pull/9700 Special notes for your reviewer: Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`)	2 years ago
Mohamed-Amine Bouqsimi	d4f364b4e1	operator: Add namespace and tenantId labels to RecordingRules (#9971 ) Co-authored-by: Periklis Tsirakidis <periklis@redhat.com>	2 years ago
Ed Welch	6d2b3e8b76	Loki: Fix a merge error when getting rules from sharded rulers using the `/prometheus/api/v1/rules` endpoint (#9973 ) What this PR does / why we need it: The `/prometheus/api/v1/rules` api endpoint will return all the rules currently loaded and being evaluated by the ruler. Previously when we merged the rule groups from all rulers in a sharded ruler setup, we only used the rule group name as the key in the map used for merging the results. This introduced a bug when multiple rule groups have the same name but in different namespaces, leading to some rules being reported in the incorrect namespace. This is just a visualization bug, the rules are in the correct space and operating as defined, however Grafana uses this API in conjunction with `/loki/api/v1/rules` to track the state of a rule being added or removed. The `/loki/api/v1/rules` endpoint returns rules as they exist in storage. When someone has multiple rule groups with the same name in different namespaces this can lead to the Grafana UI showing rules in a perpetual state of creating and deleting as it's possible for rules to be returned from `/prometheus/api/v1/rules` in the wrong group. Adding the namespace to the key used in the map for merging rule results prevents this issue. Which issue(s) this PR fixes: Fixes #<issue number> Special notes for your reviewer: Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) Signed-off-by: Edward Welch <edward.welch@grafana.com>	2 years ago
Joao Marcal	50c4216d00	operator: add mTLS authentication to tenants (#9906 )	2 years ago
Salva Corts	ce91076dd4	Add non-indexed metadata to chunks (#9700 ) What this PR does / why we need it: In https://github.com/grafana/loki/pull/9694 we support adding metadata labels for each entry in the push payload. In this PR we take that metadata and add it to the entries in the chunk. Supporting serialization and deserialization of those metadata labels. --------- Signed-off-by: Vladyslav Diachenko <vlad.diachenko@grafana.com> Co-authored-by: Vladyslav Diachenko <vlad.diachenko@grafana.com>	2 years ago
Adrien	c3da1ca04b	Docs : fix promtal typo (#9951 ) What this PR does / why we need it: Fix a typo in installation documentation, Promtal instead of Promtail Which issue(s) this PR fixes: Fixes #<issue number> Special notes for your reviewer: Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [x] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) Co-authored-by: J Stickler <julie.stickler@grafana.com>	2 years ago
Robert Jacob	cd962322df	operator: Fix application tenant alertmanager configuration (#9963 ) Co-authored-by: Periklis Tsirakidis <periklis@redhat.com>	2 years ago
Joao Marcal	c1a04f4bef	operator: add initContainer to pods of zone aware components (#9795 ) Co-authored-by: Periklis Tsirakidis <periklis@redhat.com>	2 years ago
dependabot[bot]	a7536ef03b	Bump helm/kind-action from 1.7.0 to 1.8.0 (#9959 ) Bumps [helm/kind-action](https://github.com/helm/kind-action) from 1.7.0 to 1.8.0. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/helm/kind-action/releases">helm/kind-action's releases</a>.</em></p> <blockquote> <h2>v1.8.0</h2> <h2>What's Changed</h2> <ul> <li><a href="https://redirect.github.com/helm/kind-action/issues/54">#54</a> ignore the occasional post delete cluster by <a href="https://github.com/jerry153fish"><code>@jerry153fish</code></a> in <a href="https://redirect.github.com/helm/kind-action/pull/79">helm/kind-action#79</a></li> <li>bump kind to v0.20.0 and script cleanup by <a href="https://github.com/cpanato"><code>@cpanato</code></a> in <a href="https://redirect.github.com/helm/kind-action/pull/88">helm/kind-action#88</a></li> </ul> <h2>New Contributors</h2> <ul> <li><a href="https://github.com/jerry153fish"><code>@jerry153fish</code></a> made their first contribution in <a href="https://redirect.github.com/helm/kind-action/pull/79">helm/kind-action#79</a></li> </ul> <p><strong>Full Changelog</strong>: <a href="https://github.com/helm/kind-action/compare/v1.7.0...v1.8.0">https://github.com/helm/kind-action/compare/v1.7.0...v1.8.0</a></p> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href="`dda0770415`"><code>dda0770</code></a> bump kind to v0.20.0 and script cleanup (<a href="https://redirect.github.com/helm/kind-action/issues/88">#88</a>)</li> <li><a href="`a2c862bdb6`"><code>a2c862b</code></a> <a href="https://redirect.github.com/helm/kind-action/issues/54">#54</a> ignore the occasional post delete cluster (<a href="https://redirect.github.com/helm/kind-action/issues/79">#79</a>)</li> <li><a href="`1307bb2fde`"><code>1307bb2</code></a> Bump actions/checkout from 3.5.2 to 3.5.3 (<a href="https://redirect.github.com/helm/kind-action/issues/87">#87</a>)</li> <li>See full diff in <a href="https://github.com/helm/kind-action/compare/v1.7.0...v1.8.0">compare view</a></li> </ul> </details> <br /> [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=helm/kind-action&package-manager=github_actions&previous-version=1.7.0&new-version=1.8.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	2 years ago
Trevor Whitney	443693fb72	Add 3rd backend target to docker-compose stack (#9899 ) * Adds 3rd backend target to `production/docker` stack * Uses a anchor for the image to make local overrides easier * Updates the config to use TSDB @dannykopping you seem to be the SME here, mind taking a look?	2 years ago
Salva Corts	88c17ed7ab	Include entry metadata bytes in metrics tracking ingestion stats (#9808 ) What this PR does / why we need it: In https://github.com/grafana/loki/pull/9694 we added support for adding metadata labels to each entry in a push payload. In this PR we update the following metrics to take into account the bytes needed to store those labels: - `loki_distributor_bytes_received_total` - `distributor_bytes_received` Which issue(s) this PR fixes: Fixes #<issue number> Special notes for your reviewer: Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) --------- Co-authored-by: Sandeep Sukhani <sandeep.d.sukhani@gmail.com>	2 years ago
Shweta Padubidri	97d34ea5af	operator: Gateway pods do not need to have INSTANCE_AVAILABILITY_ZONE env set in their containers (#9950 )	2 years ago
Salva Corts	15af77ba83	Remove Prometheus dependency for push pkg (#9937 ) What this PR does / why we need it: In https://github.com/grafana/loki/pull/9694, we modified the `push` pkg to import Prometheus as a dependency to use the `labels.Labels` type for the entries' non-indexed labels. Having Prometheus as a dependency is problematic since any project importing the `push` pkg will need to import Prometheus as a result. In fact, this is one of the reasons why the `push` pkg was extracted from Loki in the first place (https://github.com/grafana/loki/pull/8259). This PR removes the dependency of Prometheus from the `push` pkg by copying some bits of the implementation for Prometheus' `labels.Labels`. We copy: - The Labels struct definition - The JSON Marshaling and Unmarshaling methods for the labels. We need this so labels are encoded as maps instead of an array of objects. --- Notes for reviewers: - To implement the JSON Marshaling and Unmarshaling methods the `push` pkg now depends on `golang.org/x/exp`. I think it should be fine for projects importing the `push` pkg to also depend on `golang.org/x/exp`, right?	2 years ago
Vladyslav Diachenko	be1a9c4ec5	Revert "updated to the latest version of dskit" (#9953 ) Reverts grafana/loki#9920	2 years ago
Travis Patterson	23e59fa4a9	reset pipeline caches when tailing rather than making a new pipeline (#9949 ) Recent query optimizations made pipeline creation more expensive with the assumption that it occurred relatively infrequently. To avoid growing pipline caches without bound, integers recreate pipelines on every push. The combination of these things means that tailing high-volume streams can cause ingester failure. This fix adds the ability to clear pipeline caches so we don't need to create a new one each time.	2 years ago
Travis Patterson	6dc722bc9f	correctly handle unpack when reordering stages (#9936 ) The optimization to reorder stages doesn't take into account the `unpack` stage. When line filters come after unpack stages, they shouldn't be moved.	2 years ago
Trevor Whitney	9e22a89bcb	Call FetchChunks in parallel from series index store (#9935 ) This PR changes the invocation of `FetchChunks` in the series index store to happen in parallel. It seems like this was the original intention due to the `// bound concurrency` comment when allocating the `groups` slice. This came out of an investigation of some slow queries, and the traces showed many serial calls to `FetchChunks`.	2 years ago
Shweta Padubidri	5bd8c7dd57	operator: Add Pod annotations with node topology labels to support zone aware scheduling (#9503 ) Signed-off-by: Shweta Padubidri <spadubid@redhat.com> Co-authored-by: Robert Jacob <rojacob@redhat.com> Co-authored-by: Periklis Tsirakidis <periklis@redhat.com>	2 years ago
Dylan Guedes	9ed7fa9df4	WriteFailures: Add metrics to increase monitoring (#9904 ) What this PR does / why we need it: Add metrics for our write failures manager. This way we can easily monitor how much logging we're doing and how much we're discarding. Which issue(s) this PR fixes: N/A	2 years ago
Karsten Jeschkies	a09cb07e98	Define protobufs for topk and cms. (#9933 ) What this PR does / why we need it: This change introduces Protobuf models for the sketch data structures which will be used to shard topk queries. Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [x] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) --------- Signed-off-by: Callum Styan <callumstyan@gmail.com> Co-authored-by: Callum Styan <callumstyan@gmail.com>	2 years ago
Christian Haudum	db38b7bccc	Remove unused fallback in index gateway client store (#9916 ) The fallback was once used by boltdb-shipper, but since boltdb-shipper changed to not use the index gateway, the fallback was also never used. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
dependabot[bot]	89d00fbccd	Bump golang.org/x/oauth2 from 0.7.0 to 0.10.0 (#9903 ) Bumps [golang.org/x/oauth2](https://github.com/golang/oauth2) from 0.7.0 to 0.10.0. <details> <summary>Commits</summary> <ul> <li><a href="`ec5679f607`"><code>ec5679f</code></a> go.mod: update golang.org/x dependencies</li> <li><a href="`989acb1bfe`"><code>989acb1</code></a> all: update dependencies to their latest versions</li> <li><a href="`2323c81c8d`"><code>2323c81</code></a> go.mod: update golang.org/x dependencies</li> <li><a href="`839de2255f`"><code>839de22</code></a> google: don't check for IsNotExist for well-known file</li> <li><a href="`0690208dba`"><code>0690208</code></a> go.mod: update golang.org/x dependencies</li> <li><a href="`451d5d662f`"><code>451d5d6</code></a> internal: remove repeated definite articles</li> <li><a href="`cfe200d5bb`"><code>cfe200d</code></a> oauth2: parse RFC 6749 error response</li> <li>See full diff in <a href="https://github.com/golang/oauth2/compare/v0.7.0...v0.10.0">compare view</a></li> </ul> </details> <br />	2 years ago
dependabot[bot]	8b5dcb0fb9	Bump golang.org/x/net from 0.10.0 to 0.12.0 (#9902 ) Bumps [golang.org/x/net](https://github.com/golang/net) from 0.10.0 to 0.12.0. <details> <summary>Commits</summary> <ul> <li><a href="`c73c09c390`"><code>c73c09c</code></a> go.mod: update golang.org/x dependencies</li> <li><a href="`4fc2eb9973`"><code>4fc2eb9</code></a> http2: revert Transport change from CL 486156</li> <li><a href="`63727cc582`"><code>63727cc</code></a> http2: validate Host header before sending</li> <li><a href="`1bb09e6b1e`"><code>1bb09e6</code></a> quic: pass the connection ID length into 1-RTT packet parsing</li> <li><a href="`952fc9c67e`"><code>952fc9c</code></a> quic: move ack_delay_exponent handling out of frame parsing</li> <li><a href="`02fe9a5973`"><code>02fe9a5</code></a> quic: loss detection</li> <li><a href="`c8a2c5a181`"><code>c8a2c5a</code></a> quic: remove stray debugging print</li> <li><a href="`5d50b4094a`"><code>5d50b40</code></a> quic: add packetFate enum</li> <li><a href="`ee81e8c8e8`"><code>ee81e8c</code></a> quic: correct rttvar updates</li> <li><a href="`ab184e6fa8`"><code>ab184e6</code></a> quic: rename side type to connSide</li> <li>Additional commits viewable in <a href="https://github.com/golang/net/compare/v0.10.0...v0.12.0">compare view</a></li> </ul> </details> <br /> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	2 years ago
Periklis Tsirakidis	9617c06a70	operator: Use PodAntiAffinity for all components (#9930 ) Co-authored-by: Robert Jacob <xperimental@solidproject.de> Co-authored-by: Robert Jacob <rojacob@redhat.com>	2 years ago
Callum Styan	4d3b15ae89	Implement probabilistic topk via count min sketch and min heap (#9882 ) This PR currently implements the following structures: - Count Min Sketch: http://dimacs.rutgers.edu/~graham/pubs/papers/cmencyc.pdf - sketch-bf: https://ietresearch.onlinelibrary.wiley.com/doi/epdf/10.1049/ell2.12482 improvements to the heap usage for Count Min Sketch - HeavyKeeper: https://www.usenix.org/system/files/conference/atc18/atc18-gong.pdf I also read and evaluated the following: - topkapi: https://www.cs.rice.edu/~as143/Papers/topkapi.pdf - space saving algorithm: https://www.cse.ust.hk/~raywong/comp5331/References/EfficientComputationOfFrequentAndTop-kElementsInDataStreams.pdf Description: As part of our work to ensure every query type is parallelizable we started exploring probabilistic data structures as a way to reduce memory consumption and allow merging of partial results after sharding of queries. In this PR I have implemented a few sketch structures as part of investigating structures that are applicable to topk queries. Unfortunately while many papers made claims of improved performance over the original count min sketch paper, topkapi and heavy keeper either didn't behave as expected or were not mergeable while keeping the same degree of accuracy. Count min sketch when using conservative updates (which vastly reduces error over counting, especially for low flows), also can't keep the same degree of accuracy, but it is still more accurate after merging than heavy keeper. I've also applied some of the principles outlined in the sketchbf paper related to reducing the # of heap operations required when observing events into the sketch, which sped up processing in my benchmark of 100k unique streams each with 1-2k randomized # of occurrences by nearly 3x. --------- Signed-off-by: Callum Styan <callumstyan@gmail.com> Co-authored-by: Karsten Jeschkies <karsten.jeschkies@grafana.com>	2 years ago
J Stickler	8e59fcac91	Update redirects to use new alias format. (part 2 of 2) (#9890 ) Breaking the alias updates into two PRs to make for smaller reviews. Special notes for your reviewer: @jdbaldry commented on #9717 that the alias format had changed, but we [agreed](https://github.com/grafana/loki/pull/9717#issuecomment-1594960563) that I could update it in a follow up PR. This is that follow up. Also updating previously merged file moves that used the old alias format. <strike>I could not get relative redirects to work for the LIDS or Maintaining files, so had to use the older format, which did work.</strike> Updated weights for some pages. Updated a couple of `menuTitles` that I missed updating on previous PRs. --------- Co-authored-by: Jack Baldry <jack.baldry@grafana.com>	2 years ago
Antoon Huiskens	22aae2e107	Replacing sed with awk for determining systemd-version provides more … (#9925 ) See https://github.com/grafana/loki/issues/9924 What this PR does / why we need it: This PR addresses a parsing bug for package post install scripts. Which issue(s) this PR fixes: Fixes #9924 Special notes for your reviewer: Checklist - [X] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [X] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`)	2 years ago
Karsten Jeschkies	c2c5249676	Remove goroutines from limiter middleware. (#9923 ) What this PR does / why we need it: The `limitedRoundTripper` would spawn thousands if not millions of goroutines. However, they were mostly idle waiting for downstream calls. This would congest the Go scheduler. A semaphore is more suitable to control concurrency as the [example](https://pkg.go.dev/golang.org/x/sync/semaphore#example-package-WorkerPool) in the documentation illustrates.	2 years ago
Robert Jacob	25c26ec27d	operator: Fix update of labels and annotations of PodTemplates (#9860 )	2 years ago
Karsten Jeschkies	2dfac72835	Clarify that label filter expressions are not supported in promtail. (#9894 ) What this PR does / why we need it: Some users of Promtail seem to be confused that certain types of filter expressions are not supported in Promtail's `match` stage. This change clarifies what kind of filter expressions are supported. Which issue(s) this PR fixes: Fixes #5209 Special notes for your reviewer: Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [x] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) --------- Co-authored-by: J Stickler <julie.stickler@grafana.com>	2 years ago
Salva Corts	aae13c376d	Add metadata to push payload (#9694 ) What this PR does / why we need it: We are adding support for attaching labels to each log line. This is one of the series of the PRs broken up to make it easier to review changes. This PR updates the push payload to send labels with each log entry optionally. The log labels are supposed to be in the same format as the stream labels. Just to put it out, here is how it would look for proto and json push payload with same data: proto(`endpoint`: `(/loki/api/v1/push\|/api/prom/push)`, `Content-Type`: `application/x-protobuf`)(payload built using [push.Stream](`4cd1246b88/pkg/push/types.go (L12)`)): ``` push.Stream{ Entries: []logproto.Entry{ { Timestamp: time.Unix(0, 1688515200000000000), Line: "log line", Labels: `{foo="bar"}`, }, }, Labels: `{app="test"}`, } ``` v1(`endpoint`: `/loki/api/v1/push`, `Content-Type`: `application/json`): ```json { "streams": [{ "stream": { "app": "test" }, "values": [ ["1688515200000000000", "log line", { "foo": "bar" }] ] }] } ``` legacy-json(`/api/prom/push`, `Content-Type`: `application/json`): ```json { "streams": [{ "labels": "{app=\"test\"}", "entries": [{ "ts": "2023-07-05T00:00:00.000000000Z", "line": "log line", "labels": "{foo=\"bar\"}" }] }] } ``` Which issue(s) this PR fixes: Special notes for your reviewer: We may need to add more thoughtful tests. Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [x] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) --------- Co-authored-by: Sandeep Sukhani <sandeep.d.sukhani@gmail.com>	2 years ago
Christian Haudum	90fc5109dd	Remove unused package pkg/prom1 (#9927 ) Removes old, unused code. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
J Stickler	b3b4317861	Fix image links in moved files. (#9910 ) Noticed the other day that images in files that have been moved as part of #8710 were not displaying in `/next/`. This PR updates the image references.	2 years ago
J Stickler	1c9160bc4b	Remove aliases to current location. (#9911 ) Re: [this comment ](https://github.com/grafana/loki/pull/9890#pullrequestreview-1521562982)from @jdbaldry on #9890 that "Relative aliases that refer to the current page location won't be effective and should be removed because they can cause confusion and might result in mistaken redirects in the future if pages are moved again." Removing aliases that refer to the current location of the pages.	2 years ago

... 4 5 6 7 8 ...

5593 Commits (407ab22d9d7c30b3daba6ae4783ae02801fd1780) All Branches Search

5593 Commits (407ab22d9d7c30b3daba6ae4783ae02801fd1780)

All Branches