5304 Commits (9617c06a70a5388d2d93b7a2902fa2f8d44d549e)

 

All Branches   

Search

Author	SHA1	Message	Date
Dylan Guedes	c6e18a2efb	Loki: Humanize downloaded file size (bytes format) (#9746) ... **What this PR does / why we need it**: Humanize reported size of downloaded file. **Which issue(s) this PR fixes**: N/A	2 years ago
Marcin Dufrat	ba8388b8fd	Fix config as a secret mount for single binary statefulset (#9733) ... **What this PR does / why we need it**: With https://github.com/grafana/loki/pull/8526 came support for config stored as a secret, but `single-binary` statefulset was forgotten. **Checklist** - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [ ] Documentation added - [ ] Tests updated - [x] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [x] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](d10549e3ec) --------- Co-authored-by: Michel Hollands <42814411+MichelHollands@users.noreply.github.com>	2 years ago
Joao Marcal	636f3843b7	operator: adjusts 1x.extra-small resources according to findings (#9735)	2 years ago
Danny Kopping	ccf44f8840	Adding 3 existing maintainers, and our newest one: Ashwanth! (#9741) ... Signed-off-by: Danny Kopping <danny.kopping@grafana.com> Co-authored-by: Susana Ferreira <ssncferreira@gmail.com>	2 years ago
Dylan Guedes	c0d9a2b4ba	Log push failures per-entry instead of in batches (#9720) ... **What this PR does / why we need it**: Modify how we log push errors to do it per-entry instead of accumulating all errors and then logging them all in a single pass. **Which issue(s) this PR fixes**: N/A	2 years ago
dependabot[bot]	62bb5b27ae	Bump cachix/install-nix-action from 18 to 22 (#9731) ... Bumps [cachix/install-nix-action](https://github.com/cachix/install-nix-action) from 18 to 22. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/cachix/install-nix-action/releases">cachix/install-nix-action's releases</a>.</em></p> <blockquote> <h2>install-nix-action-v22</h2> <ul> <li>Nix 2.16.1</li> <li>Fix issues with System Integrity Protection when using macos-12</li> </ul> <h2>install-nix-action-v21</h2> <ul> <li>pin Nix to 2.15.1 (recent releases broke too many things)</li> <li>fix the action to work on custom containers</li> </ul> <h2>install-nix-action-v20</h2> <p><a href="https://discourse.nixos.org/t/nix-2-14-0-released/25900/4Fix">https://discourse.nixos.org/t/nix-2-14-0-released/25900/4Fix</a> support for Nix 2.14 new profile location: <a href="https://redirect.github.com/NixOS/nix/pull/5226">NixOS/nix#5226</a></p> <h2>install-nix-action-v19</h2> <ul> <li>expose github_access_token option and use the default github token by default to prevent rate limiting</li> <li>support <code>--no-daemon</code> installation</li> </ul> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href="6ed004b9cc"><code>6ed004b</code></a> Merge pull request <a href="https://redirect.github.com/cachix/install-nix-action/issues/184">#184</a> from cachix/macos-bump</li> <li><a href="e27879448e"><code>e278794</code></a> Nix: 2.15.1 -&gt; 2.16.1</li> <li><a href="8ab3881720"><code>8ab3881</code></a> use system certs</li> <li><a href="16b951426e"><code>16b9514</code></a> Merge pull request <a href="https://redirect.github.com/cachix/install-nix-action/issues/182">#182</a> from l0b0/feat/configure-editors</li> <li><a href="2c203fd87b"><code>2c203fd</code></a> feat: Configure editors</li> <li><a href="4b933aa7eb"><code>4b933aa</code></a> Nix: 2.15.1</li> <li><a href="35806937f1"><code>3580693</code></a> Merge pull request <a href="https://redirect.github.com/cachix/install-nix-action/issues/179">#179</a> from joergdw/fix-action-path</li> <li><a href="3eb7a24508"><code>3eb7a24</code></a> Merge pull request <a href="https://redirect.github.com/cachix/install-nix-action/issues/178">#178</a> from cachix/docs/149</li> <li><a href="840ed7ce9a"><code>840ed7c</code></a> Document how to pass env vars to modern nix commands</li> <li><a href="b2f4229533"><code>b2f4229</code></a> Fix action to make it work on custom containers;</li> <li>Additional commits viewable in <a href="https://github.com/cachix/install-nix-action/compare/v18...v22">compare view</a></li> </ul> </details> <br /> [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	2 years ago
dependabot[bot]	4c32ff83fc	bump github.com/heroku/x from 0.0.55 to 0.0.59 (#9732) ... [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	2 years ago
Kaviraj Kanagaraj	74a4dca54e	fix(async_store): Fix the `storageconfig.EnableAsyncStore` config (#9725) ... **What this PR does / why we need it**: During reverting this [PR](https://github.com/grafana/loki/pull/9715). I missed one important flag switch. Which this PR fixes it. **Which issue(s) this PR fixes**: Fixes NA **Special notes for your reviewer**: We found a bug on querier due to async_store and we tracked down to[ this PR](https://github.com/grafana/loki/pull/9476). So I reverted the PR in [this PR ](https://github.com/grafana/loki/pull/9715)(manually. Auto revert was not possible as things changed a lot meanwhile) But during my revert PR I missed switching this async store config. **Checklist** - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com>	2 years ago
Bryan Boreham	695ea32226	stores/series tests: use abstractions over Labels (#9613) ... **What this PR does / why we need it**: Stop assuming `labels.Labels` is a slice. **Special notes for your reviewer:** This PR only touches tests. Upstream Prometheus is using a different implementation of `labels.Labels`, so I plan to create a series of PRs which make Loki compatible with that. #9532 was the first. **Checklist** - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - NA Documentation added - [x] Tests updated - NA `CHANGELOG.md` updated - NA Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - NA For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](d10549e3ec)	2 years ago
Bryan Boreham	904ea0586a	Remove some unused code (#9611) ... **What this PR does / why we need it**: Remove some functions I came across while making Loki compatible with different `Labels` structures. Move `MetricToLabels` to the only place it is now used, and make it call abstractions instead of assuming `Labels` is a slice. **Checklist** - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - NA Documentation added - NA Tests updated - NA `CHANGELOG.md` updated - NA Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - NA For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](d10549e3ec)	2 years ago
sarah-spang	6157e28998	Loki ECS Client: Update Name in Options config (#9616) ... Configuration options show "Name": "grafana-loki", however, fluentbit specifies this should just be "loki" A few users have reported receiving an error when using the documented "Name" option, but it works as expected when using the Name specified in the fluentbit docs: Reference: https://docs.fluentbit.io/manual/pipeline/outputs/loki#fluent-bit-+-grafana-cloud	2 years ago
zirain	8bf4e12edf	helm: fix resource not respect namespace (#9678) ... **What this PR does / why we need it**: this PR make all resources have some behavior when running `helm template loki grafana/loki --name istio-system` istio use it here: https://github.com/istio/istio/blob/master/manifests/addons/gen.sh#L89 **Which issue(s) this PR fixes**: Fixes #<issue number> **Special notes for your reviewer**: **Checklist** - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](d10549e3ec) Signed-off-by: hejianpeng <hejianpeng2@huawei.com>	2 years ago
Salva Corts	61d407b4bb	Add keep stage to LogQL (#9693) ... **What this PR does / why we need it**: This PR adds a new `keep` stage to LogQL. It allows filtering specific labels and helps to avoid hitting the _max streams per query_ limit by keeping only specific labels in the label set. It works the opposite of [drop stage](https://grafana.com/docs/loki/latest/logql/log_queries/#drop-labels-expression). **Special notes for your reviewer**: We do not drop [special error labels](edc6b0bff7/pkg/logqlmodel/error.go (L18-L20)) added by Loki at query time. Users will have to explicitly drop them by using the drop stage. **Checklist** - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [x] Documentation added - [x] Tests updated - [x] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](d10549e3ec) --------- Co-authored-by: Sandeep Sukhani <sandeep.d.sukhani@gmail.com> Co-authored-by: J Stickler <julie.stickler@grafana.com>	2 years ago
Justin Roberson	262e9d02e1	helm: make ServiceMonitor cluster label respect clusterLabelOverride (#9690) ... **What this PR does / why we need it**: As of #9218 `clusterLabelOverride` can be used to set the `cluster` label, however the ServiceMonitor still uses `loki.fullname`. This fixes that so it will use `loki.clusterLabel` to be consistent with other components. **Which issue(s) this PR fixes**: Fixes #<issue number> **Special notes for your reviewer**: **Checklist** - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](d10549e3ec)	2 years ago
Trevor Whitney	7696938928	Update PR template to include add to release notes label (#9691) ... **What this PR does / why we need it**: Includes a checklist option to the PR template about the `add-to-release-notes` label. **Special notes for your reviewer**: **Checklist** - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](d10549e3ec)	2 years ago
Ed Welch	e4654c9773	Jsonnet: add dns+ to join_members in memberlist config (#9723) ... **What this PR does / why we need it**: We use a headless service called `gossip-ring` which contains all the memberlist members, previously we passed this headless service into our memberlist code as ``` join_members: - gossip-ring.loki-ops.svc.cluster.local:7946 ``` What this resulted in was the dskit wrapper around the memberlist library thinking there was only 1 join_node, so it passed this directly to the join method in memberlist. The memberlist lib would then do a DNS resolution on this and it would find **all** the members behind this service. This is generally not noticeable until you get to very large clusters with thousands of memberlist members and it results in new joiners attempting to join to ALL the existing members instead of just a few. This PR prefixes the url with `dns+` which changes the behavior inside dskit in an important way. ``` join_members: - dns+gossip-ring.loki-ops.svc.cluster.local:7946 ``` Now dskit will resolve the address to find all the memberlist members, first, then apply code which selects a subset of all the nodes to be used for first join (based on some logarithmic math) This subset is then passed to the memberlist library which will join just this set instead of all members. **Which issue(s) this PR fixes**: Fixes #<issue number> **Special notes for your reviewer**: **Checklist** - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](d10549e3ec) Signed-off-by: Edward Welch <edward.welch@grafana.com>	2 years ago
J Stickler	32320736db	TOC Sync: Move files under Get Started. (#9717) ... **Which issue(s) this PR fixes**: Fixes #8711 **Special notes for your reviewer**: Part of the Database products table of contents alignment project. Moves Getting Started and Fundamentals files under Get Started in TOC. Updates front matter for the topic (headings, page weights, descriptions, aliases to redirect from old URL to new URL) Fixes links broken by the move.	2 years ago
Salva Corts	b7359c5d53	Revert "Add summary stats and metrics for stats cache (#9536)" (#9721) ... This reverts commit af287ac3eb. There is a bug in this PR that inflates the stats returned for the query since we reuse the stats ctx in the query execution engine.	2 years ago
Kaviraj Kanagaraj	4c0e01b93f	revert(9476): Bringback async store like before for the queriers (#9715) ... This seems to cause some issue with `querier`, not getting chunks from ingesters correctly via `async_store`. I narrowed down to this PR. But hard `revert` seems not possible anymore. So created this manually. https://github.com/grafana/loki/pull/9476 Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com>	2 years ago
J Stickler	11e2375e6d	Removing 'please' from the docs. (#9658) ... **Which issue(s) this PR fixes**: Fixes #8922 **Special notes for your reviewer**: This is to conform with our Style Guide. https://developers.google.com/style/tone#politeness-and-use-of-please	2 years ago
J Stickler	c2f96221dc	Update that Loki accepts out-of-order rewrites by default. (#9670) ... **Which issue(s) this PR fixes**: Fixes #7477	2 years ago
Travis Patterson	db97058a84	Series volume endpoint (#9704) ... This changes the `label_volume` endpoint to the `series_volume` endpoint. The new endpoint still returns volumes but now it does it for the requested streams defined by the selector names passed rather than individual labels. All relevant non-requested labels are aggregated into the returned results ex: Assume we have the following streams: ``` {cluster="prod", team="A", component="foo"} {cluster="prod", team="B", component="foo"} {cluster="dev", team="A", component="foo"} {cluster="dev", team="B", component="foo"} ``` - requesting `{cluster="prod"}` returns one result for all streams containing `{cluster="prod"}` - requesting `{cluster=~".+"}` returns two results for the streams containing `{cluster="prod"}` and `{cluster="dev"}` - requesting `{cluster=~".+", team=".+"}` returns four results for the streams containing: ``` {cluster="prod", team="A"} {cluster="prod", team="B"} {cluster="dev", team="A"} {cluster="dev", team="B"} ``` --------- Co-authored-by: Trevor Whitney <trevorjwhitney@gmail.com>	2 years ago
Dan Poltawski	093ec20b43	packaging: fix postinst on debian oses (#9706) ... Previously the systemd version extract pulled in debian string breaking the detection on debian variants **What this PR does / why we need it**: Fixes postinst error on Debian variants **Which issue(s) this PR fixes**: Fixes #9701 #9302 **Special notes for your reviewer**: Tests on various OS variants: Debian/12 ``` $ systemctl --version systemd 252 (252.6-1) +PAM +AUDIT +SELINUX +APPARMOR +IMA +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 -IDN +IPTC +KMOD +LIBCRYPTSETUP +LIBFDISK +PCRE2 -PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD -BPF_FRAMEWORK -XKBCOMMON +UTMP +SYSVINIT default-hierarchy=unified $ systemctl --version | head -1 | sed 's/systemd \([0-9]\+\).*/\1/' 252 ``` CentOS/7 ``` $ systemctl --version systemd 219 +PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 -SECCOMP +BLKID +ELFUTILS +KMOD +IDN $ systemctl --version | head -1 | sed 's/systemd \([0-9]\+\).*/\1/' 219 ``` Debian/11 ``` $ systemctl --version systemd 247 (247.3-7+deb11u2) +PAM +AUDIT +SELINUX +IMA +APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +ZSTD +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=unified $ systemctl --version | head -1 | sed 's/systemd \([0-9]\+\).*/\1/' 247 ``` Ubuntu/20.04 ``` $ systemctl --version systemd 245 (245.4-4ubuntu3.20) +PAM +AUDIT +SELINUX +IMA +APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=hybrid $ systemctl --version | head -1 | sed 's/systemd \([0-9]\+\).*/\1/' 245 ``` **Checklist** - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](d10549e3ec) Co-authored-by: Michel Hollands <42814411+MichelHollands@users.noreply.github.com>	2 years ago
Michel Hollands	79485cd30d	Add handling of timezones in unixToTime tests (#9707) ... **What this PR does / why we need it**: The unit tests for unixToTime do not take the current timezone into account. **Checklist** - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [ ] Documentation added - [x] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](d10549e3ec) --------- Signed-off-by: Michel Hollands <michel.hollands@grafana.com>	2 years ago
Jan Jansen	8abe080fd6	Promtail: improve behavior of partial lines (#9508) ... **What this PR does / why we need it**: I started to see OOM after upgrading to a promtail version with #8497. So played around to make partial line behavior more stable. You can now truncate partial lines and configure max partial lines buffer. **Which issue(s) this PR fixes**: Fixes #<issue number> **Special notes for your reviewer**: **Checklist** - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [x] Documentation added - [x] Tests updated - [x] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` Signed-off-by: Jan Jansen <jan.jansen@gdata.de> Co-authored-by: Michel Hollands <42814411+MichelHollands@users.noreply.github.com>	2 years ago
Jan-Otto Kröpke	14f96c0c7b	Add loki.memberlistConfig to fully control the memberlist configuration (#9646) ... **What this PR does / why we need it**: Add loki.memberlistConfig to fully control the memberlist configuration On singleBinary with replica 1, memberlist isn't need and its counter-productive since it delays the start of the loki binary. At the moment, it isn't possible to remove the memberlist defaults. **Which issue(s) this PR fixes**: Fixes #<issue number> **Special notes for your reviewer**: **Checklist** - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [x] Documentation added - [ ] Tests updated - [x] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [x] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](d10549e3ec)	2 years ago
Jack Baldry	8391e557b1	Update doc-validator to v3.0.0 (#9666) ... ## 3.0.0 ### Added - Structured output for use with [`reviewdog`](https://github.com/reviewdog/reviewdog). You can achieve the original error output by piping the output to the following `jq` expression: `jq -r '"ERROR: \(.location.path):\(.location.range.start.line):\(.location.range.start.column): \(.message)"'`. - Suggestions for simple link fixes. In GitHub, [`reviewdog`](https://github.com/reviewdog/reviewdog) comments these suggestions for convenient replacement. - Support for anchors referring to repeated headings. In the case that a page has multiple headings that share the same text, the renderer appends a zero indexed, numbered suffix to the identifier. For example, when there are two headings that are both "Heading text", the first anchor identifier is `heading-text` and the second is `heading-text-1`. - Error when running `doc-validator` on no files. **Notes for reviewer:** This also updates the `make-docs` procedure. Please ensure you can run `make docs` from the `docs/` directory before approving and merging this PR. --------- Signed-off-by: Jack Baldry <jack.baldry@grafana.com>	2 years ago
Terje Sannum	986d4ca756	Helm: add support for additional labels on loki-canary pods (#9673) ... Add support for setting additional labels on the loki-canary pods --------- Signed-off-by: Terje Sannum <terje.sannum@nav.no>	2 years ago
Deepansh Mathur	50126731f8	Update README.md (#9698) ... Backticks were missing for a code block **What this PR does / why we need it**: **Which issue(s) this PR fixes**: Fixes #<issue number> **Special notes for your reviewer**: **Checklist** - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](d10549e3ec)	2 years ago
Mohamed-Amine Bouqsimi	7069b897e5	Fix Ruler remote write panic (#9662) ... **What this PR does / why we need it**: This PR fixes the Ruler remote write panic, caused by the no-op scrape manager not being correctly configured.	2 years ago
Robert Jacob	74b16fb71d	operator: Fix availability of demo LokiStack size (#9689)	2 years ago
Trevor Whitney	4a56445686	Upgrade `golangci-lint` and fix linting errors (#9601) ... **What this PR does / why we need it**: Upgrade `golangci-lint` and fixes all the errors. The upgrade includes some stricter linting.	2 years ago
Travis Patterson	065bee7e72	Label Volume Endpoint (#9588) ... For a given set of matchers, returns the top N associated label/value pairs by volume. A query for `{cluster=prod}` will return ``` cluster=prod: size (total logs matching this matcher) . . . nth-label=nth-value ``` This is to service use cases where users want to understand where their log volume has come from by label without making multiple requests to the stats endpoint. Note: This PR is a monster but it's mostly plumbing. I've pointed out the most interesting bits that actually get the volumes from ingesters/indexs	2 years ago
Piotr	4d997a5a10	Fix promtail cluster template not finding all clusters. (#9684) ... **What this PR does / why we need it**: In promtail-mixin, the dropdown template for clusters would only include clusters that run loki. So if a cluster only run promtail and not loki, it doesn't appear.	2 years ago
Kaviraj Kanagaraj	57f9452231	Revert 9217 chaudum/tsdb chunkrefs pool (#9685) ... Revert #9217 (potential bug in query result) --------- Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com>	2 years ago
Salva Corts	73ac208981	Improve docs for empty value in cache compression config (#9649) ... **What this PR does / why we need it**: Follow up PR for https://github.com/grafana/loki/pull/9535#discussion_r1218167670 **Checklist** - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [x] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](d10549e3ec)	2 years ago
Christophe Collot	f2394355d8	feat(lambda-promtail): add cloudfront log file ingestion support (#9573) ... **What this PR does / why we need it**: This PR enables ingesting logs from Cloudfront log files stored in s3 (batch). The current setup only supports streaming Cloudfront logs through AWS Kinesis, whereas this PR implements the same flow as for VPC Flow logs, Load Balancer logs, and Cloudtrail logs (s3 --> SQS (optional) --> Lambda Promtail --> Loki) **Special notes for your reviewer**: + The Cloudfront log file format is different from the already implemented services, meaning we had to build yet another regex. AWS never bothered making all services follow the same log file naming convention but the "good" thing is that it's now very unlikely they will change it in the future. + The Cloudfront file name does not have any mention of the AWS account or the time of log it contains, it means we have to infer the log type from the filename format instead of finding the exact string "cloudfront" in the filename. This is why in `getLabels`, if no `type` parameter is found in the regex, we use the key corresponding to the name of the matching parser. + I introduced a new `parser` struct to group together several parameters specific to a type of log (and avoid relying too much on map key string matching and / or if statements for specific use cases) + I've been successfully running this code in several AWS environments for days. + I corrected a typo from my previous PR #9497 (wrong PR number in Changelog.md) **Checklist** - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [x] Documentation added - [x] Tests updated - [x] `CHANGELOG.md` updated - [x] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [x] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](d10549e3ec) --------- Co-authored-by: Michel Hollands <42814411+MichelHollands@users.noreply.github.com>	2 years ago
Salva Corts	c6fbff26e1	Add config to avoid caching stats for recent data (#9537) ... **What this PR does / why we need it**: When we query the stats for recent data, we query both the ingesters and the index gateways for the stats. ebdb2b1800/pkg/storage/async_store.go (L112-L114) ebdb2b1800/pkg/storage/async_store.go (L126-L127) Then we merge all the responses, which means summing up all the stats ebdb2b1800/pkg/storage/async_store.go (L157-L158) ebdb2b1800/pkg/storage/stores/index/stats/stats.go (L23-L26) Because we have a replication factor of 3, this means that we will get the stats from the ingesters repeated up to 3 times, hence inflating the stats. In the stats cache, we store the stats for a given matcher set for the whole day, then we extract the stats from the cache by the factor of time from the request that is stored in the cache: 336283acad/pkg/querier/queryrange/index_stats_cache.go (L33) 336283acad/pkg/querier/queryrange/index_stats_cache.go (L40) Inflated stats for recent data will be cached, so subsequent stats extracted from the cache will be inflated regardless of the time. This PR adds a new per-tenant limit `max_stats_cache_freshness` to not cache requests with an end time that falls within Now minus this duration. Here's a scenario illustrating this. The graphs below show the bytes stats queried in the sharding middleware. We are running a log filter query that won't match any log, every 5 seconds with a length of 3h.  As can be seen, after enabling the stats cache and configuring`do_not_cache_request_within` to not cache stats for requests within 30m, the bytes stats used in the sharding middleware stopped increasing. In both cases the stats cache hit ration was 100%.  **Special notes for your reviewer**: - Blocked by https://github.com/grafana/loki/pull/9535 - Note that this PR doesn't fix the root issue of inflated stats form the ingesters, but rather buys us some time to work on that. **Checklist** - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [x] Documentation added - [x] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](d10549e3ec)	2 years ago
Michel Hollands	22779e13c5	Fix date template function with epoch times (#8886) ... **What this PR does / why we need it**: Adds new toUnixEpoch... functions to convert from a string with a Unix/Epoch time to an integer that can be used in the existing `toDate` function. Note that these are the opposites of some of the functions introduced in https://github.com/grafana/loki/pull/8774. **Which issue(s) this PR fixes**: Fixes https://github.com/grafana/loki/issues/8624. **Special notes for your reviewer**: **Checklist** - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [X] Documentation added - [X] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` --------- Signed-off-by: Michel Hollands <michel.hollands@grafana.com>	2 years ago
Bruno FERNANDO	1b410dbc01	feat(promtail): add CF ClientRequestSource field (#9669) ... **What this PR does / why we need it**: Hey folks 👋 Little contribution here to add a useful log field for cloudflare users. Indeed I add the [ClientRequestSource field](https://developers.cloudflare.com/logs/reference/clientrequestsource/ ) which is pretty useful when debugging some specific traffic handled by cloudflare Extra: Since I was on the documentation I fixed an indentation issue that I spotted Don't hesitate to reach me if you have any questions Cheers 😉 **Which issue(s) this PR fixes**: Fixes #<issue number> **Special notes for your reviewer**: Loki rocks 🚀 **Checklist** - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [x] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](d10549e3ec)	2 years ago
Gregor Zeitlinger	b1917a6178	add "alignLeft" and "alignRight" functions (#9672) ... Fixes https://github.com/grafana/loki/issues/9667	2 years ago
Ashwanth	98d1307c7d	config: ensure storage config defaults apply to named stores (#9650) ... **What this PR does / why we need it**: Since named store config does not register any flags, storage configs defined under it do not get the defaults. For example [aws_storage_config](https://grafana.com/docs/loki/latest/configuration/#aws_storage_config) sets the default `storage_class` to `STANDARD`, but the same doesn't get applied by default when using named stores. This PR ensures that named storage configs are always assigned default values when they are unmarshalled by implementing `yaml.Unmarshaler` interface	2 years ago
Pepe Cano	4cebc2d85c	Docs: replace `k6 Cloud` mention (#9599) ... k6 is now available as a managed service on Grafana Cloud. This is a small doc changes to remove the mention of `k6 Cloud`. --------- Co-authored-by: J Stickler <julie.stickler@grafana.com>	2 years ago
Danny Kopping	1db560fdc1	Adding background cache (en|de)queue counters (#9665) ... **What this PR does / why we need it**: The background writeback cache exposes gauge metric currently for the current queue size. Gauges can be useful, but they are susceptible to sample errors because they only represent the point in time as the time of the scrape. Exposing counters for the bytes (en|de)queued to/from the cache will be more useful because they can be aggregated. Signed-off-by: Danny Kopping <danny.kopping@grafana.com>	2 years ago
Dylan Guedes	609bc22933	Distributor: Make key configurable when logging failures (#9659) ... **What this PR does / why we need it**: Make appending `insight=true` key-value pair to log failures configurable. **Which issue(s) this PR fixes**: N/A	2 years ago
Nils Griebner	d58125885b	Make table manager retention options configurable in helm chart values (#9647) ... **What this PR does / why we need it**: Configuration options for table manager retention are hard-coded in the helm chart values at the moment so that it's not possible to enable retention deletes. **Which issue(s) this PR fixes**: Fixes #8676 **Special notes for your reviewer**: - **Checklist** - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [x] Documentation added - [x] Tests updated - [x] `CHANGELOG.md` updated - [x] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [x] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](d10549e3ec) Co-authored-by: Michel Hollands <42814411+MichelHollands@users.noreply.github.com>	2 years ago
Ed Welch	79c5fdd9c0	Canary: Refactor how the direct Canary push works (#9569) ... **What this PR does / why we need it**: When doing some testing a while back I ran into several cases were we didn't handle errors/retries properly in the code we added for the canaries to send directly to Loki. The changes here effectively duplicate the way promtail's client does sends to Loki which is code we've been using/relying on for years now. I also had a hard time following the previous implementation which used the io.Writer interface for swapping out writing to stdout vs pushing to Loki and refactored this to use a new interface `EntryWriter` which has implementations that write to stdout and to Loki Unfortunately the changes in the diff don't read super well with this refactoring so hopefully the explanation helps. **Which issue(s) this PR fixes**: Fixes #<issue number> **Special notes for your reviewer**: **Checklist** - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](d10549e3ec) --------- Signed-off-by: Edward Welch <edward.welch@grafana.com> Co-authored-by: Paul Rogers <129207811+paul1r@users.noreply.github.com>	2 years ago
Ivana Huckova	eb7dae4583	Loki: Improve error message when step too low (#9641) ... **What this PR does / why we need it**: In https://github.com/grafana/grafana/pull/69648 we are in Grafana introducing a step editor in Loki. Unfortunately, the error message when user sets too low step parameter is hard to understand, so I am proposing following change to make it more understandable and actionable. Let me know what do you think. --------- Co-authored-by: J Stickler <julie.stickler@grafana.com>	2 years ago
Christopher Piggott	415a997fb9	Update CONTRIBUTING.md to point to the new location of the contributi… (#9654) ... **What this PR does / why we need it**: Updating CONTRIBUTING document to point to the location of the LID template, which has moved **Which issue(s) this PR fixes**: none **Special notes for your reviewer**: doc change only **Checklist** - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](d10549e3ec)	2 years ago
Alexandre	806cd73e53	Fix configmap checksum in read sts (#9642) ... **What this PR does / why we need it**: @NilsGriebner : complement to #9579 **Which issue(s) this PR fixes**: Fixes #<issue number> **Special notes for your reviewer**: **Checklist** - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [x] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](d10549e3ec) --------- Co-authored-by: Michel Hollands <42814411+MichelHollands@users.noreply.github.com>	2 years ago