apitech/loki - loki - Gitea: Git with a cup of tea

Commit Graph

Author	SHA1	Message	Date
Susana Ferreira	a894c3ccf0	Bugfix: Fix logic of offset operator in instant query splitting (#9763 ) What this PR does / why we need it: Fix the logic of the `offset` operator for downstream queries on instant query splitting of (range) vector aggregation expressions already containing an offset. Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [x] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`)	3 years ago
Susana Ferreira	b611641fea	Add comments to clarify Grouping struct in Aggregation expressions (#9761 ) What this PR does / why we need it: Add comments to clarify the Grouping struct in Vector Aggregation and Range Vector Aggregations expressions. Related with PR: https://github.com/grafana/loki/pull/9515 Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`)	3 years ago
Ashwanth	bbbbe2440f	changelog: moving unreleased changes to the main section (#9769 ) What this PR does / why we need it: noticed that a couple of unreleased changes are incorrectly showing under 2.8.0 release, moving them to the right place. also adds the missing changelog entry for #7447 Checklist - [X] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) Signed-off-by: Ashwanth Goli <iamashwanth@gmail.com>	3 years ago
Travis Patterson	806674fdaa	Add log-volume feature flag (#9762 ) Adds a feature flag for use with the new log-volume endpoints so associated features can be rolled out incrementally.	3 years ago
Trevor Whitney	dbc3040739	Convert SeriesVolume response to prometheus-style (#9703 ) Changes the response type of the label volume stats endpoint to return volumes as prometheus-style timeseries metrics. It currently only supports instant queries, but is a necessary step to eventually supporting range queries.	3 years ago
Trevor Whitney	d74c67a9d0	Add version to TSDB index builder struct (#9566 ) Add version as a property to the TSDB Index Builder, forcing users of the builder to specify a version for each builder instance. This is the first of many PRs that will add the TSDB index version to the schema version config.	3 years ago
Christian Haudum	ce81895241	Add shuffle sharding to index gateway (#9710 ) This PR introduces shuffle sharding of the index files across index gateways where the shuffle sharding key is the tenant ID. Instead of having a global replication factor on the index gateways which affects all tenants the same, whether they are small or large tenants, the shuffle sharding allows to specify a shard factor per tenant (on top of the replication factor). This a) reduces the amount of indexes that are downloaded on individual index gateways in case of a high replication factor (which is currently the de-factor standard to deal with high load) and b) allows to scale tenants individually. The shuffle sharding algorithm is part of dskit's ring implementation and is not part of this commit. If you have set a high RF for the index gateways, it is required to reduce the RF and instead set the shard factor for the tenants. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	3 years ago
Joao Marcal	3509edd98d	operator: fix replica count for 1x.ExtraSmall (#9751 )	3 years ago
Periklis Tsirakidis	89a094c5fe	operator: Add rules labels filters for openshift application tenant (#9600 ) Co-authored-by: Mohamed-Amine Bouqsimi <bouqsimiamine@gmail.com>	3 years ago
Dylan Guedes	24e9efb907	Loki: Report event for `GetChunkRef` instead of SpanLogging (#9748 ) What this PR does / why we need it: Modify our calls to `index_client.GetChunkRef` to report invocations through a span event instead of through span logging. This way we won't log a new log line for every GetChunkRef call. Which issue(s) this PR fixes: N/A	3 years ago
Tim Collins	4bb622c54d	helm: table_manager configmap volume (#9742 ) What this PR does / why we need it: Which issue(s) this PR fixes: Fixes #9740 Special notes for your reviewer: Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [x] Documentation added - [x] Tests updated - [x] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [x] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [x] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`)	3 years ago
Dylan Guedes	c6e18a2efb	Loki: Humanize downloaded file size (bytes format) (#9746 ) What this PR does / why we need it: Humanize reported size of downloaded file. Which issue(s) this PR fixes: N/A	3 years ago
Marcin Dufrat	ba8388b8fd	Fix config as a secret mount for single binary statefulset (#9733 ) What this PR does / why we need it: With https://github.com/grafana/loki/pull/8526 came support for config stored as a secret, but `single-binary` statefulset was forgotten. Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [x] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [x] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) --------- Co-authored-by: Michel Hollands <42814411+MichelHollands@users.noreply.github.com>	3 years ago
Joao Marcal	636f3843b7	operator: adjusts 1x.extra-small resources according to findings (#9735 )	3 years ago
Danny Kopping	ccf44f8840	Adding 3 existing maintainers, and our newest one: Ashwanth! (#9741 ) Signed-off-by: Danny Kopping <danny.kopping@grafana.com> Co-authored-by: Susana Ferreira <ssncferreira@gmail.com>	3 years ago
Dylan Guedes	c0d9a2b4ba	Log push failures per-entry instead of in batches (#9720 ) What this PR does / why we need it: Modify how we log push errors to do it per-entry instead of accumulating all errors and then logging them all in a single pass. Which issue(s) this PR fixes: N/A	3 years ago
dependabot[bot]	62bb5b27ae	Bump cachix/install-nix-action from 18 to 22 (#9731 ) Bumps [cachix/install-nix-action](https://github.com/cachix/install-nix-action) from 18 to 22. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/cachix/install-nix-action/releases">cachix/install-nix-action's releases</a>.</em></p> <blockquote> <h2>install-nix-action-v22</h2> <ul> <li>Nix 2.16.1</li> <li>Fix issues with System Integrity Protection when using macos-12</li> </ul> <h2>install-nix-action-v21</h2> <ul> <li>pin Nix to 2.15.1 (recent releases broke too many things)</li> <li>fix the action to work on custom containers</li> </ul> <h2>install-nix-action-v20</h2> <p><a href="https://discourse.nixos.org/t/nix-2-14-0-released/25900/4Fix">https://discourse.nixos.org/t/nix-2-14-0-released/25900/4Fix</a> support for Nix 2.14 new profile location: <a href="https://redirect.github.com/NixOS/nix/pull/5226">NixOS/nix#5226</a></p> <h2>install-nix-action-v19</h2> <ul> <li>expose github_access_token option and use the default github token by default to prevent rate limiting</li> <li>support <code>--no-daemon</code> installation</li> </ul> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href="`6ed004b9cc`"><code>6ed004b</code></a> Merge pull request <a href="https://redirect.github.com/cachix/install-nix-action/issues/184">#184</a> from cachix/macos-bump</li> <li><a href="`e27879448e`"><code>e278794</code></a> Nix: 2.15.1 -> 2.16.1</li> <li><a href="`8ab3881720`"><code>8ab3881</code></a> use system certs</li> <li><a href="`16b951426e`"><code>16b9514</code></a> Merge pull request <a href="https://redirect.github.com/cachix/install-nix-action/issues/182">#182</a> from l0b0/feat/configure-editors</li> <li><a href="`2c203fd87b`"><code>2c203fd</code></a> feat: Configure editors</li> <li><a href="`4b933aa7eb`"><code>4b933aa</code></a> Nix: 2.15.1</li> <li><a href="`35806937f1`"><code>3580693</code></a> Merge pull request <a href="https://redirect.github.com/cachix/install-nix-action/issues/179">#179</a> from joergdw/fix-action-path</li> <li><a href="`3eb7a24508`"><code>3eb7a24</code></a> Merge pull request <a href="https://redirect.github.com/cachix/install-nix-action/issues/178">#178</a> from cachix/docs/149</li> <li><a href="`840ed7ce9a`"><code>840ed7c</code></a> Document how to pass env vars to modern nix commands</li> <li><a href="`b2f4229533`"><code>b2f4229</code></a> Fix action to make it work on custom containers;</li> <li>Additional commits viewable in <a href="https://github.com/cachix/install-nix-action/compare/v18...v22">compare view</a></li> </ul> </details> <br /> [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=cachix/install-nix-action&package-manager=github_actions&previous-version=18&new-version=22)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	3 years ago
dependabot[bot]	4c32ff83fc	bump github.com/heroku/x from 0.0.55 to 0.0.59 (#9732 ) [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=github.com/heroku/x&package-manager=go_modules&previous-version=0.0.55&new-version=0.0.59)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	3 years ago
Kaviraj Kanagaraj	74a4dca54e	fix(async_store): Fix the `storageconfig.EnableAsyncStore` config (#9725 ) What this PR does / why we need it: During reverting this [PR](https://github.com/grafana/loki/pull/9715). I missed one important flag switch. Which this PR fixes it. Which issue(s) this PR fixes: Fixes NA Special notes for your reviewer: We found a bug on querier due to async_store and we tracked down to[ this PR](https://github.com/grafana/loki/pull/9476). So I reverted the PR in [this PR ](https://github.com/grafana/loki/pull/9715)(manually. Auto revert was not possible as things changed a lot meanwhile) But during my revert PR I missed switching this async store config. Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com>	3 years ago
Bryan Boreham	695ea32226	stores/series tests: use abstractions over Labels (#9613 ) What this PR does / why we need it: Stop assuming `labels.Labels` is a slice. Special notes for your reviewer: This PR only touches tests. Upstream Prometheus is using a different implementation of `labels.Labels`, so I plan to create a series of PRs which make Loki compatible with that. #9532 was the first. Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - NA Documentation added - [x] Tests updated - NA `CHANGELOG.md` updated - NA Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - NA For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`)	3 years ago
Bryan Boreham	904ea0586a	Remove some unused code (#9611 ) What this PR does / why we need it: Remove some functions I came across while making Loki compatible with different `Labels` structures. Move `MetricToLabels` to the only place it is now used, and make it call abstractions instead of assuming `Labels` is a slice. Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - NA Documentation added - NA Tests updated - NA `CHANGELOG.md` updated - NA Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - NA For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`)	3 years ago
sarah-spang	6157e28998	Loki ECS Client: Update Name in Options config (#9616 ) Configuration options show "Name": "grafana-loki", however, fluentbit specifies this should just be "loki" A few users have reported receiving an error when using the documented "Name" option, but it works as expected when using the Name specified in the fluentbit docs: Reference: https://docs.fluentbit.io/manual/pipeline/outputs/loki#fluent-bit-+-grafana-cloud	3 years ago
zirain	8bf4e12edf	helm: fix resource not respect namespace (#9678 ) What this PR does / why we need it: this PR make all resources have some behavior when running `helm template loki grafana/loki --name istio-system` istio use it here: https://github.com/istio/istio/blob/master/manifests/addons/gen.sh#L89 Which issue(s) this PR fixes: Fixes #<issue number> Special notes for your reviewer: Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) Signed-off-by: hejianpeng <hejianpeng2@huawei.com>	3 years ago
Salva Corts	61d407b4bb	Add keep stage to LogQL (#9693 ) What this PR does / why we need it: This PR adds a new `keep` stage to LogQL. It allows filtering specific labels and helps to avoid hitting the _max streams per query_ limit by keeping only specific labels in the label set. It works the opposite of [drop stage](https://grafana.com/docs/loki/latest/logql/log_queries/#drop-labels-expression). Special notes for your reviewer: We do not drop [special error labels](`edc6b0bff7/pkg/logqlmodel/error.go (L18-L20)`) added by Loki at query time. Users will have to explicitly drop them by using the drop stage. Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [x] Documentation added - [x] Tests updated - [x] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) --------- Co-authored-by: Sandeep Sukhani <sandeep.d.sukhani@gmail.com> Co-authored-by: J Stickler <julie.stickler@grafana.com>	3 years ago
Justin Roberson	262e9d02e1	helm: make ServiceMonitor cluster label respect clusterLabelOverride (#9690 ) What this PR does / why we need it: As of #9218 `clusterLabelOverride` can be used to set the `cluster` label, however the ServiceMonitor still uses `loki.fullname`. This fixes that so it will use `loki.clusterLabel` to be consistent with other components. Which issue(s) this PR fixes: Fixes #<issue number> Special notes for your reviewer: Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`)	3 years ago
Trevor Whitney	7696938928	Update PR template to include add to release notes label (#9691 ) What this PR does / why we need it: Includes a checklist option to the PR template about the `add-to-release-notes` label. Special notes for your reviewer: Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`)	3 years ago
Ed Welch	e4654c9773	Jsonnet: add dns+ to join_members in memberlist config (#9723 ) What this PR does / why we need it: We use a headless service called `gossip-ring` which contains all the memberlist members, previously we passed this headless service into our memberlist code as ``` join_members: - gossip-ring.loki-ops.svc.cluster.local:7946 ``` What this resulted in was the dskit wrapper around the memberlist library thinking there was only 1 join_node, so it passed this directly to the join method in memberlist. The memberlist lib would then do a DNS resolution on this and it would find all the members behind this service. This is generally not noticeable until you get to very large clusters with thousands of memberlist members and it results in new joiners attempting to join to ALL the existing members instead of just a few. This PR prefixes the url with `dns+` which changes the behavior inside dskit in an important way. ``` join_members: - dns+gossip-ring.loki-ops.svc.cluster.local:7946 ``` Now dskit will resolve the address to find all the memberlist members, first, then apply code which selects a subset of all the nodes to be used for first join (based on some logarithmic math) This subset is then passed to the memberlist library which will join just this set instead of all members. Which issue(s) this PR fixes: Fixes #<issue number> Special notes for your reviewer: Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) Signed-off-by: Edward Welch <edward.welch@grafana.com>	3 years ago
J Stickler	32320736db	TOC Sync: Move files under Get Started. (#9717 ) Which issue(s) this PR fixes: Fixes #8711 Special notes for your reviewer: Part of the Database products table of contents alignment project. Moves Getting Started and Fundamentals files under Get Started in TOC. Updates front matter for the topic (headings, page weights, descriptions, aliases to redirect from old URL to new URL) Fixes links broken by the move.	3 years ago
Salva Corts	b7359c5d53	Revert "Add summary stats and metrics for stats cache (#9536 )" (#9721 ) This reverts commit `af287ac3eb`. There is a bug in this PR that inflates the stats returned for the query since we reuse the stats ctx in the query execution engine.	3 years ago
Kaviraj Kanagaraj	4c0e01b93f	revert(9476): Bringback async store like before for the queriers (#9715 ) This seems to cause some issue with `querier`, not getting chunks from ingesters correctly via `async_store`. I narrowed down to this PR. But hard `revert` seems not possible anymore. So created this manually. https://github.com/grafana/loki/pull/9476 Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com>	3 years ago
J Stickler	11e2375e6d	Removing 'please' from the docs. (#9658 ) Which issue(s) this PR fixes: Fixes #8922 Special notes for your reviewer: This is to conform with our Style Guide. https://developers.google.com/style/tone#politeness-and-use-of-please	3 years ago
J Stickler	c2f96221dc	Update that Loki accepts out-of-order rewrites by default. (#9670 ) Which issue(s) this PR fixes: Fixes #7477	3 years ago
Travis Patterson	db97058a84	Series volume endpoint (#9704 ) This changes the `label_volume` endpoint to the `series_volume` endpoint. The new endpoint still returns volumes but now it does it for the requested streams defined by the selector names passed rather than individual labels. All relevant non-requested labels are aggregated into the returned results ex: Assume we have the following streams: ``` {cluster="prod", team="A", component="foo"} {cluster="prod", team="B", component="foo"} {cluster="dev", team="A", component="foo"} {cluster="dev", team="B", component="foo"} ``` - requesting `{cluster="prod"}` returns one result for all streams containing `{cluster="prod"}` - requesting `{cluster=~".+"}` returns two results for the streams containing `{cluster="prod"}` and `{cluster="dev"}` - requesting `{cluster=~".+", team=".+"}` returns four results for the streams containing: ``` {cluster="prod", team="A"} {cluster="prod", team="B"} {cluster="dev", team="A"} {cluster="dev", team="B"} ``` --------- Co-authored-by: Trevor Whitney <trevorjwhitney@gmail.com>	3 years ago
Dan Poltawski	093ec20b43	packaging: fix postinst on debian oses (#9706 ) Previously the systemd version extract pulled in debian string breaking the detection on debian variants What this PR does / why we need it: Fixes postinst error on Debian variants Which issue(s) this PR fixes: Fixes #9701 #9302 Special notes for your reviewer: Tests on various OS variants: Debian/12 ``` $ systemctl --version systemd 252 (252.6-1) +PAM +AUDIT +SELINUX +APPARMOR +IMA +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 -IDN +IPTC +KMOD +LIBCRYPTSETUP +LIBFDISK +PCRE2 -PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD -BPF_FRAMEWORK -XKBCOMMON +UTMP +SYSVINIT default-hierarchy=unified $ systemctl --version \| head -1 \| sed 's/systemd $[0-9]\+$./\1/' 252 ``` CentOS/7 ``` $ systemctl --version systemd 219 +PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 -SECCOMP +BLKID +ELFUTILS +KMOD +IDN $ systemctl --version \| head -1 \| sed 's/systemd $[0-9]\+$./\1/' 219 ``` Debian/11 ``` $ systemctl --version systemd 247 (247.3-7+deb11u2) +PAM +AUDIT +SELINUX +IMA +APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +ZSTD +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=unified $ systemctl --version \| head -1 \| sed 's/systemd $[0-9]\+$./\1/' 247 ``` Ubuntu/20.04 ``` $ systemctl --version systemd 245 (245.4-4ubuntu3.20) +PAM +AUDIT +SELINUX +IMA +APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=hybrid $ systemctl --version \| head -1 \| sed 's/systemd $[0-9]\+$./\1/' 245 ``` Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) Co-authored-by: Michel Hollands <42814411+MichelHollands@users.noreply.github.com>	3 years ago
Michel Hollands	79485cd30d	Add handling of timezones in unixToTime tests (#9707 ) What this PR does / why we need it: The unit tests for unixToTime do not take the current timezone into account. Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [x] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) --------- Signed-off-by: Michel Hollands <michel.hollands@grafana.com>	3 years ago
Jan Jansen	8abe080fd6	Promtail: improve behavior of partial lines (#9508 ) What this PR does / why we need it: I started to see OOM after upgrading to a promtail version with #8497. So played around to make partial line behavior more stable. You can now truncate partial lines and configure max partial lines buffer. Which issue(s) this PR fixes: Fixes #<issue number> Special notes for your reviewer: Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [x] Documentation added - [x] Tests updated - [x] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` Signed-off-by: Jan Jansen <jan.jansen@gdata.de> Co-authored-by: Michel Hollands <42814411+MichelHollands@users.noreply.github.com>	3 years ago
Jan-Otto Kröpke	14f96c0c7b	Add loki.memberlistConfig to fully control the memberlist configuration (#9646 ) What this PR does / why we need it: Add loki.memberlistConfig to fully control the memberlist configuration On singleBinary with replica 1, memberlist isn't need and its counter-productive since it delays the start of the loki binary. At the moment, it isn't possible to remove the memberlist defaults. Which issue(s) this PR fixes: Fixes #<issue number> Special notes for your reviewer: Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [x] Documentation added - [ ] Tests updated - [x] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [x] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`)	3 years ago
Jack Baldry	8391e557b1	Update doc-validator to v3.0.0 (#9666 ) ## 3.0.0 ### Added - Structured output for use with [`reviewdog`](https://github.com/reviewdog/reviewdog). You can achieve the original error output by piping the output to the following `jq` expression: `jq -r '"ERROR: \(.location.path):\(.location.range.start.line):\(.location.range.start.column): \(.message)"'`. - Suggestions for simple link fixes. In GitHub, [`reviewdog`](https://github.com/reviewdog/reviewdog) comments these suggestions for convenient replacement. - Support for anchors referring to repeated headings. In the case that a page has multiple headings that share the same text, the renderer appends a zero indexed, numbered suffix to the identifier. For example, when there are two headings that are both "Heading text", the first anchor identifier is `heading-text` and the second is `heading-text-1`. - Error when running `doc-validator` on no files. Notes for reviewer: This also updates the `make-docs` procedure. Please ensure you can run `make docs` from the `docs/` directory before approving and merging this PR. --------- Signed-off-by: Jack Baldry <jack.baldry@grafana.com>	3 years ago
Terje Sannum	986d4ca756	Helm: add support for additional labels on loki-canary pods (#9673 ) Add support for setting additional labels on the loki-canary pods --------- Signed-off-by: Terje Sannum <terje.sannum@nav.no>	3 years ago
Deepansh Mathur	50126731f8	Update README.md (#9698 ) Backticks were missing for a code block What this PR does / why we need it: Which issue(s) this PR fixes: Fixes #<issue number> Special notes for your reviewer: Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`)	3 years ago
Mohamed-Amine Bouqsimi	7069b897e5	Fix Ruler remote write panic (#9662 ) What this PR does / why we need it: This PR fixes the Ruler remote write panic, caused by the no-op scrape manager not being correctly configured.	3 years ago
Robert Jacob	74b16fb71d	operator: Fix availability of demo LokiStack size (#9689 )	3 years ago
Trevor Whitney	4a56445686	Upgrade `golangci-lint` and fix linting errors (#9601 ) What this PR does / why we need it: Upgrade `golangci-lint` and fixes all the errors. The upgrade includes some stricter linting.	3 years ago
Travis Patterson	065bee7e72	Label Volume Endpoint (#9588 ) For a given set of matchers, returns the top N associated label/value pairs by volume. A query for `{cluster=prod}` will return ``` cluster=prod: size (total logs matching this matcher) . . . nth-label=nth-value ``` This is to service use cases where users want to understand where their log volume has come from by label without making multiple requests to the stats endpoint. Note: This PR is a monster but it's mostly plumbing. I've pointed out the most interesting bits that actually get the volumes from ingesters/indexs	3 years ago
Piotr	4d997a5a10	Fix promtail cluster template not finding all clusters. (#9684 ) What this PR does / why we need it: In promtail-mixin, the dropdown template for clusters would only include clusters that run loki. So if a cluster only run promtail and not loki, it doesn't appear.	3 years ago
Kaviraj Kanagaraj	57f9452231	Revert 9217 chaudum/tsdb chunkrefs pool (#9685 ) Revert #9217 (potential bug in query result) --------- Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com>	3 years ago
Salva Corts	73ac208981	Improve docs for empty value in cache compression config (#9649 ) What this PR does / why we need it: Follow up PR for https://github.com/grafana/loki/pull/9535#discussion_r1218167670 Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [x] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`)	3 years ago
Christophe Collot	f2394355d8	feat(lambda-promtail): add cloudfront log file ingestion support (#9573 ) What this PR does / why we need it: This PR enables ingesting logs from Cloudfront log files stored in s3 (batch). The current setup only supports streaming Cloudfront logs through AWS Kinesis, whereas this PR implements the same flow as for VPC Flow logs, Load Balancer logs, and Cloudtrail logs (s3 --> SQS (optional) --> Lambda Promtail --> Loki) Special notes for your reviewer: + The Cloudfront log file format is different from the already implemented services, meaning we had to build yet another regex. AWS never bothered making all services follow the same log file naming convention but the "good" thing is that it's now very unlikely they will change it in the future. + The Cloudfront file name does not have any mention of the AWS account or the time of log it contains, it means we have to infer the log type from the filename format instead of finding the exact string "cloudfront" in the filename. This is why in `getLabels`, if no `type` parameter is found in the regex, we use the key corresponding to the name of the matching parser. + I introduced a new `parser` struct to group together several parameters specific to a type of log (and avoid relying too much on map key string matching and / or if statements for specific use cases) + I've been successfully running this code in several AWS environments for days. + I corrected a typo from my previous PR #9497 (wrong PR number in Changelog.md) Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [x] Documentation added - [x] Tests updated - [x] `CHANGELOG.md` updated - [x] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [x] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) --------- Co-authored-by: Michel Hollands <42814411+MichelHollands@users.noreply.github.com>	3 years ago
Salva Corts	c6fbff26e1	Add config to avoid caching stats for recent data (#9537 ) What this PR does / why we need it: When we query the stats for recent data, we query both the ingesters and the index gateways for the stats. `ebdb2b1800/pkg/storage/async_store.go (L112-L114)` `ebdb2b1800/pkg/storage/async_store.go (L126-L127)` Then we merge all the responses, which means summing up all the stats `ebdb2b1800/pkg/storage/async_store.go (L157-L158)` `ebdb2b1800/pkg/storage/stores/index/stats/stats.go (L23-L26)` Because we have a replication factor of 3, this means that we will get the stats from the ingesters repeated up to 3 times, hence inflating the stats. In the stats cache, we store the stats for a given matcher set for the whole day, then we extract the stats from the cache by the factor of time from the request that is stored in the cache: `336283acad/pkg/querier/queryrange/index_stats_cache.go (L33)` `336283acad/pkg/querier/queryrange/index_stats_cache.go (L40)` Inflated stats for recent data will be cached, so subsequent stats extracted from the cache will be inflated regardless of the time. This PR adds a new per-tenant limit `max_stats_cache_freshness` to not cache requests with an end time that falls within Now minus this duration. Here's a scenario illustrating this. The graphs below show the bytes stats queried in the sharding middleware. We are running a log filter query that won't match any log, every 5 seconds with a length of 3h. ![image](https://github.com/grafana/loki/assets/8354290/45c2e6e9-185c-4a18-b290-47da27fc3e39) As can be seen, after enabling the stats cache and configuring`do_not_cache_request_within` to not cache stats for requests within 30m, the bytes stats used in the sharding middleware stopped increasing. In both cases the stats cache hit ration was 100%. ![image](https://github.com/grafana/loki/assets/8354290/cd35bcb8-0c77-4693-a06b-502741fd6e23) Special notes for your reviewer: - Blocked by https://github.com/grafana/loki/pull/9535 - Note that this PR doesn't fix the root issue of inflated stats form the ingesters, but rather buys us some time to work on that. Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [x] Documentation added - [x] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`)	3 years ago
Michel Hollands	22779e13c5	Fix date template function with epoch times (#8886 ) What this PR does / why we need it: Adds new toUnixEpoch... functions to convert from a string with a Unix/Epoch time to an integer that can be used in the existing `toDate` function. Note that these are the opposites of some of the functions introduced in https://github.com/grafana/loki/pull/8774. Which issue(s) this PR fixes: Fixes https://github.com/grafana/loki/issues/8624. Special notes for your reviewer: Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [X] Documentation added - [X] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` --------- Signed-off-by: Michel Hollands <michel.hollands@grafana.com>	3 years ago

1 2 3 4 5 ...

5215 Commits (a894c3ccf06526b82779668abeaca03691c56879) All Branches Search

5215 Commits (a894c3ccf06526b82779668abeaca03691c56879)

All Branches