121 Commits (89985399817dde28fe5b94de990133d904bc3e5b)

Author	SHA1	Message	Date
Eng Zer Jun	e455a11b49	Ruler: compare header with `strings.EqualFold` (#10103)	3 years ago
Danny Kopping	8fa4134ef9	Update dskit (#10091)	3 years ago
Mohamed-Amine Bouqsimi	5a7d6789b2	Fix missing alibabacloud ruler storage type (#10026) ... **What this PR does / why we need it**: This PR fixes the missing `alibabacloud` storage type for the ruler. **Which issue(s) this PR fixes**: Fixes #9892 **Special notes for your reviewer**: @chaudum **Checklist** - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [ ] Documentation added - [ ] Tests updated - [x] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](d10549e3ec) --------- Co-authored-by: Salva Corts <salva.corts@grafana.com>	3 years ago
Francois Gouteroux	2880b5055a	fix: parse expression error message based on record or alert (#10065)	3 years ago
Ed Welch	6d2b3e8b76	Loki: Fix a merge error when getting rules from sharded rulers using the `/prometheus/api/v1/rules` endpoint (#9973) ... **What this PR does / why we need it**: The `/prometheus/api/v1/rules` api endpoint will return all the rules currently loaded and being evaluated by the ruler. Previously when we merged the rule groups from all rulers in a sharded ruler setup, we only used the rule group name as the key in the map used for merging the results. This introduced a bug when multiple rule groups have the same name but in different namespaces, leading to some rules being reported in the incorrect namespace. This is just a visualization bug, the rules are in the correct space and operating as defined, however Grafana uses this API in conjunction with `/loki/api/v1/rules` to track the state of a rule being added or removed. The `/loki/api/v1/rules` endpoint returns rules as they exist in storage. When someone has multiple rule groups with the same name in different namespaces this can lead to the Grafana UI showing rules in a perpetual state of creating and deleting as it's possible for rules to be returned from `/prometheus/api/v1/rules` in the wrong group. Adding the namespace to the key used in the map for merging rule results prevents this issue. **Which issue(s) this PR fixes**: Fixes #<issue number> **Special notes for your reviewer**: **Checklist** - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](d10549e3ec) Signed-off-by: Edward Welch <edward.welch@grafana.com>	3 years ago
Vladyslav Diachenko	be1a9c4ec5	Revert "updated to the latest version of dskit" (#9953) ... Reverts grafana/loki#9920	3 years ago
Vladyslav Diachenko	3243676f4c	updated to the latest version of dskit (#9920) ... **What this PR does / why we need it**: Updated Loki with the latest version of dskit **Checklist** - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](d10549e3ec) --------- Signed-off-by: Vladyslav Diachenko <vlad.diachenko@grafana.com>	3 years ago
Bryan Boreham	904ea0586a	Remove some unused code (#9611) ... **What this PR does / why we need it**: Remove some functions I came across while making Loki compatible with different `Labels` structures. Move `MetricToLabels` to the only place it is now used, and make it call abstractions instead of assuming `Labels` is a slice. **Checklist** - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - NA Documentation added - NA Tests updated - NA `CHANGELOG.md` updated - NA Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - NA For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](d10549e3ec)	3 years ago
Mohamed-Amine Bouqsimi	7069b897e5	Fix Ruler remote write panic (#9662) ... **What this PR does / why we need it**: This PR fixes the Ruler remote write panic, caused by the no-op scrape manager not being correctly configured.	3 years ago
Trevor Whitney	4a56445686	Upgrade `golangci-lint` and fix linting errors (#9601) ... **What this PR does / why we need it**: Upgrade `golangci-lint` and fixes all the errors. The upgrade includes some stricter linting.	3 years ago
Danny Kopping	4cd0aedc76	Fix output of limited response body when remote rule evaluation fails (#9253)	3 years ago
Periklis Tsirakidis	3abd260975	Bump dskit from 528d8a7 to 1725bcc (#9184)	3 years ago
Mohamed-Amine Bouqsimi	62572b4f83	Change GeneratorURL to use configurable Grafana explore URL (#8500) ... **What this PR does / why we need it**: This PR changes the `GeneratorURL` associated with alerts generated by Loki. The new `GeneratorURL` uses a Grafana URL path.	3 years ago
Danny Kopping	af896eb5f6	Ruler: log rule evaluation detail (#9206) ... **What this PR does / why we need it**: This logs some extra detail around rule evaluation to enable more effective troubleshooting when a rule is not behaving as expected.	3 years ago
Peter Štibraný	90a1d4593e	Update Prometheus dependency (#9205)	3 years ago
Athira Varghese	ffb961c439	feat(storage): add support for IBM cloud object storage as storage client (#8826) ... **What this PR does / why we need it**: Add support for IBM cloud object storage as storage client **Which issue(s) this PR fixes**: Fixes NA **Checklist** - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [x] Documentation added - [x] Tests updated - [ ] `CHANGELOG.md` updated --------- Signed-off-by: Shahul <shahulsonhal@gmail.com> Co-authored-by: Aman Kumar Singh <amankrsingh2110@gmail.com> Co-authored-by: Suruthi-G-K <shruthi.suruthi@gmail.com> Co-authored-by: tareqmamari <tariq.mamari@de.ibm.com> Co-authored-by: shahulsonhal <shahulsonhal@gmail.com> Co-authored-by: Aditya C S <aditya.gnu@gmail.com> Co-authored-by: Tareq Al-Maamari <tariq.mamari@gmail.com>	3 years ago
Danny Kopping	c9d5a91206	Ruler: Implement consistent rule evaluation jitter (#8896) ... **What this PR does / why we need it**: This PR replaces the previous random jitter with a consistent jitter. While both are random, having the random jitter be applied _consistently_ is essential for evaluating rules on a predictable cadence. If a rule is supposed to evaluate every minute, whether it evaluates at (e.g.) `01:00` or `01:03.234` is irrelevant because the evaluation _instant_ is not adjusted, so it will produce the same result whether run at `01:00` or `01:03.234`. However, if 1000 rules are set to evaluate at `01:00`, this will create a resource contention issue.	3 years ago
Danny Kopping	528ed22dc2	Ruler: remote rule evaluation hardening (#8785) ... **What this PR does / why we need it**: This PR is part 2 of 2 implementing remote rule evaluation. See [part 1](https://github.com/grafana/loki/pull/8744) for more context.	3 years ago
Danny Kopping	5c3d204ebf	Ruler: rule evaluation jitter (#8848) ... **What this PR does / why we need it**: This PR introduces a configurable random sleep before each rule evaluation to prevent contention on the query engine should multiple runs execute concurrently. It's quite likely that rules will be evaluated ~concurrently if they have the same interval.	3 years ago
Trevor Whitney	b46e392adb	gofmt is needed for the build (#8678) ... **What this PR does / why we need it**: `gofmt` is needed for the build, `make format` fails without it.	3 years ago
Danny Kopping	33e44ed39d	Ruler: remote rule evaluation (#8744) ... **What this PR does / why we need it**: Adds the ability to evaluate recording & alerting rules against a given `query-frontend`, allowing these queries to be executed with all the parallelisation & optimisation that regular adhoc queries have. This is important because with `local` evaluation all queries are single-threaded, and rules that evaluate a large range/volume of data may timeout or OOM the `ruler` itself, leading to missed metrics or alerts. When `remote` evaluation mode is enabled, the `ruler` effectively just becomes a gRPC client for the `query-frontend`, which will dramatically improve the reliability of the `ruler` and also drastically reduce its resource requirements. **Which issue(s) this PR fixes**: This PR implements the feature discussed in https://github.com/grafana/loki/pull/8129 (**LID 0002: Remote Rule Evaluation**).	3 years ago
Danny Kopping	e2ac2d50e2	Changes from linter (#8769) ... **What this PR does / why we need it**: We had a few outstanding linter issues, and running `make lint` fixes them all automatically. Signed-off-by: Danny Kopping <danny.kopping@grafana.com>	3 years ago
Danny Kopping	6bf782b0a7	Ruler: do not keep appending logger keys (#8650) ... **What this PR does / why we need it**: Corrects a logger which was accidentally appending keys in every loop iteration, causing massive lines to be ingested	3 years ago
Bryan Boreham	6fd4b5e89b	Update prometheus/prometheus from 2.41 to 2.42 (#8571) ... **What this PR does / why we need it**: Brings in the latest updates from upstream. These open up some opportunities for optimisations in TSDB indexing. Dependencies updated: * github.com/Azure/go-autorest/autorest/adal v0.9.21 -> v0.9.22 (comment-only change) * github.com/docker/docker v20.10.21 -> v20.10.23 (fixing filters bug) * golang.org/x/exp fae10dda9338 -> d38c7dcee874 (optimisations in `BinarySearch` function) Indirect dependencies also updated: * github.com/digitalocean/godo v1.91.1 -> v1.95.0 (nothing alarming in [release notes](https://github.com/digitalocean/godo/releases)) * github.com/google/pprof aee1124e3a93 -> 76d1ae5aea2b (no changes pulled into vendor) * golang.org/x/tools v0.4.0 -> v0.5.0 (relating to Go compiler utilities) * google.golang.org/genproto 76db0878b65f -> 31e0e69b6fc2 * k8s.io/api v0.26.0 -> v0.26.1 (comment-only change) * k8s.io/apimachinery v0.26.0 -> v0.26.1 (no changes pulled into vendor) * k8s.io/client-go v0.26.0 -> v0.26.1 (small fixes) **Special notes for your reviewer**: A couple of interfaces changed; these have required matching changes in Loki code. Those changes are split into separate commits. I also note that most calls to `relabel` ignore when the rule says "drop". Maybe this is wrong?	3 years ago
Travis Patterson	eb1b72ab80	Fix ruler deadlock (#8486) ... A circular dependency between the Prometheus rules.Manager and the MemStore causes occasional deadlock in the rulers. This change makes both the manager and the memstore rely on the `GroupLoader` interface to break the circular dependency. The manger holding the `GroupLoader` only ever adds rules to it so we add another layer to cleanup any leftover state after an update happens. This supersedes https://github.com/grafana/loki/pull/8453 --------- Co-authored-by: Michel Hollands <42814411+MichelHollands@users.noreply.github.com>	3 years ago
李国忠	1fc00fa02d	[new] chunk backend: Integrate Alibaba Cloud oss (#8410)	3 years ago
Danny Kopping	ebfeba5de3	Ruler: implement rule-based and shuffle sharding (#8092)	3 years ago
Danny Kopping	fd3d362e0c	Removing duplicated limits file and refactored references (#8210)	3 years ago
Susana Ferreira	e07fe7c338	Deprecate CLI flag -ruler.wal-cleaer.period in favor of CLI flag -ruler.wal-cleaner.period (#7937) ... **What this PR does / why we need it**: Deprecate CLI flag `-ruler.wal-cleaer.period` in favor of CLI flag `-ruler.wal-cleaner.period`. **Which issue(s) this PR fixes**: Fixes https://github.com/grafana/loki/pull/7916#issuecomment-1348276269 and https://github.com/grafana/loki/pull/7916#discussion_r1047043534 **NOTE:** correctly registring the `ruler.wal-cleaner` configuration flags is not a breaking change and doesn't require an upgrade guide, because the ruler WAL cleaner initialization code already guarantees that the default values are used: c71620ae94/pkg/ruler/storage/cleaner/cleaner.go (L76-L101) **Special notes for your reviewer**: **Checklist** - [x] Reviewed the `CONTRIBUTING.md` guide - [x] Documentation added - [ ] Tests updated - [x] `CHANGELOG.md` updated - [x] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	3 years ago
Susana Ferreira	68a4d3271a	Fix documentation typos from #7916 (#7934) ... **What this PR does / why we need it**: Fix documentation typos from https://github.com/grafana/loki/pull/7916#pullrequestreview-1217192817 **Which issue(s) this PR fixes**: Fixes https://github.com/grafana/loki/pull/7916#pullrequestreview-1217192817 **Special notes for your reviewer**: **Checklist** - [ ] Reviewed the `CONTRIBUTING.md` guide - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	3 years ago
Susana Ferreira	f93b91bfb5	Add configuration documentation generation tool (#7916) ... **What this PR does / why we need it**: Add a tool to generate configuration flags documentation based on the flags properties defined on registration on the code. This tool is based on the [Mimir doc generation tool](https://github.com/grafana/mimir/tree/main/tools/doc-generator) and adapted according to Loki configuration specifications. Prior to this PR, the configuration flags documentation was dispersed across two sources: * [_index.md](5550cd65ec/docs/sources/configuration/_index.md) * configuration flags registration in the code This meant that there was no single source of truth. In this PR, the previous `_index.md` file is replaced with the new file generated by the tool. The next step includes adding a CI step that validates if the _index.md file was generated according to the flags settings. This will be done in a follow-up PR. **NOTE:** this is not a documentation update PR. Apart from some minor typo fixes, the documentation changes on the code, were copied from the `_index.md` file. **Which issue(s) this PR fixes**: Fixes https://github.com/grafana/loki-private/issues/83 **Special notes for your reviewer**: Files: * [docs/sources/configuration/index.template](5550cd65ec/docs/sources/configuration/index.template): template used to generate the final configuration file * [/docs/sources/configuration/_index.md](c32e5d0acb/docs/sources/configuration/_index.md): file generated by tool * `loki/pkg` directory files updated with up-to-date documentation from `_index.md` file * [tools/doc-generator](5550cd65ec/tools/doc-generator) directory with documentation generation tool. **Checklist** - [ ] Reviewed the `CONTRIBUTING.md` guide - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	3 years ago
Mohamed-Amine Bouqsimi	03a34db8eb	Fix alertmanager per tenant override config behavior (#7805) ... **Context:** #7359 implement multi tenancy for the alertmanager config. However, the behavior of the per-tenant alertmanager config is erroneous, as it tries to merge the global alertmanager config with the per-tenant config. This does not work because there is no way of knowing if certain values were set by the user or if they were undefined and have the default Golang zero value (typically booleans). This PR changes the behavior of the alertmanager config to override all the configuration instead of trying to merge the global config with the per-tenant config. It also applies default values when appropriate.	3 years ago
Robert Fratto	85392a9728	Update Prometheus dependency to latest release (v2.40.4) (#7826) ... Closes #7811, which is needed for Grafana Agent to update to v2.40 and add support for native histograms. I did not add support for native histograms to Loki, sorry :)	3 years ago
Karsten Jeschkies	7ec51e8b9d	Log rule changes through API. (#7790) ... **What this PR does / why we need it**: This change should make it easier to detect changes to rules via the API. **Which issue(s) this PR fixes**: Fixes #7789 **Special notes for your reviewer**: **Checklist** - [ ] Reviewed the `CONTRIBUTING.md` guide - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` Co-authored-by: Christian Haudum <christian.haudum@gmail.com>	3 years ago
Mohamed-Amine Bouqsimi	7debe7c235	Ruler: enable Loki to configure Alertmanagers per tenant (#7359)	3 years ago
Robert Fratto	2208ea7147	Update Prometheus dependency to latest release (v2.39.1) (#7425)	3 years ago
Owen Diehl	b59277b472	Revert "Revert "Update Thanos dependency to latest version"" (#7372) ... Reverts grafana/loki#7371 PR re re-enable Thanos dependency bump.	3 years ago
李国忠	8dcc4b3ec8	Do not log 'unable to read rules directory' at startup if directory hasn't been created yet (#7434)	3 years ago
Owen Diehl	8331785a23	Revert "Update Thanos dependency to latest version" (#7371) ... Reverts grafana/loki#7348 I expect we'll merge this again in the future, but we'll need some more time for downstream projects to be ready for this change.	3 years ago
Robert Fratto	c657857e95	Update Thanos dependency to latest version (#7348)	3 years ago
Danny Kopping	c3e92b876c	Restoring lint checks that were previously disabled (#7340) ... I removed these checks in https://github.com/grafana/loki/pull/7337 to not make https://github.com/grafana/loki/pull/7243 overly complex to review. Refactored code to fix valid lint issues, ignored the irrelevant/dangerous ones. I also removed `unused` and `unparam` as these produce low-quality results that are largely useless.	3 years ago
Florian Lehner	4861914c81	pkg/ruler/rulestore/configdb: Preallocate memory (#7326) ... Preallocate memory instead of enforcing an incremental growth. This will result in less work for the garbage collector.	3 years ago
Eng Zer Jun	e2842c69c5	Replace deprecated `io/ioutil` with `io` and `os` packages (#7212) ... The `io/ioutil` package has been deprecated in Go 1.16 (See https://pkg.go.dev/io/ioutil). This PR replaces the existing `io/ioutil` functions with their new definitions in `io` and `os` packages. Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>	3 years ago
Kaviraj Kanagaraj	6f0690e7bf	fix(ruler): Initialize multi remote clients config (#7151) ... <!-- Thanks for sending a pull request! Before submitting: 1. Read our CONTRIBUTING.md guide 2. Name your PR as `<Feature Area>: Describe your change`. a. Do not end the title with punctuation. It will be added in the changelog. b. Start with an imperative verb. Example: Fix the latency between System A and System B. c. Use sentence case, not title case. d. Use a complete phrase or sentence. The PR title will appear in a changelog, so help other people understand what your change will be. 3. Rebase your PR if it gets out of sync with main --> **What this PR does / why we need it**: To be backward compatible with single client config. ``` ts=2022-09-14T08:24:04.549961234Z caller=memberlist_logger.go:74 level=debug msg="Initiating push/pull sync with: 10.132.18.167:7946" │ │ panic: assignment to entry in nil map │ │ goroutine 1 [running]: │ │ github.com/grafana/loki/pkg/ruler.NewRuler({{{0xc00002a120}, {0x0, 0x0, 0x0}, {0x6400000, 0x6400000, {0x0, 0x0}, 0x0, 0x0, ...}, ...}, ...}, ...) │ │ /src/enterprise-logs/vendor/github.com/grafana/loki/pkg/ruler/ruler.go:21 +0x1b7 ``` **Which issue(s) this PR fixes**: Fixes #<issue number> **Special notes for your reviewer**: <!-- Note about CHANGELOG entries, if a change adds: * an important feature * fixes an issue present in a previous release, * causes a change in operation that would be useful for an operator of Loki to know then please add a CHANGELOG entry. For documentation changes, build changes, simple fixes etc please skip this step. We are attempting to curate a changelog of the most relevant and important changes to be easier to ingest by end users of Loki. Note about the upgrade guide, if this changes: * default configuration values * metric names or label names * changes existing log lines such as the metrics.go query output line * configuration parameters * anything to do with any API * any other change that would require special attention or extra steps to upgrade Please document clearly what changed AND what needs to be done in the upgrade guide. --> **Checklist** - [ ] Documentation added - [ ] Tests updated - [ ] Is this an important fix or new feature? Add an entry in the `CHANGELOG.md`. - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	3 years ago
Mohamed-Amine Bouqsimi	3d4788fb51	Add ability for Ruler to remote write to multiple clients (#6823)	3 years ago
Danny Kopping	87e20d9399	Setting sane defaults for ruler in SSD "read" mode (#6551) ... Also removing support for configdb, since this is not supported but was the default storage type Signed-off-by: Danny Kopping <danny.kopping@grafana.com> Signed-off-by: Danny Kopping <danny.kopping@grafana.com>	3 years ago
Jesus Vazquez	db9dbe8e1d	Bump prometheus dependency (#6403) ... * Bump prometheus dependency We had to add a replace of prometheus in go mod because thanos is importing an older version and then replacing it. * Undo textparser.New variable mismatch * modtimevfs path update * Fix labels withlabels and withoutlabels usages WithoutLabels and WithLabels methods have dissappeared. Now labels.NewBuilder has to be used. * Fix RuleManager interface for Update() Update now accepts a new param called ruleGroupPostProcessFunc which is an optional function to be executed after rule group processing. In this case I've set it to nil in the usages i've found. * NewQueryRange now accepts QueryOpts The NewQueryRange function now accepts QueryOptions so I had to update all the calls. I've set the opts to nil since we're not using them. * gofmt	4 years ago
Trevor Wood	aed11c25e6	fix(loki): allow global and per tenant sigv4 config (#6358) ... Signed-off-by: Trevor Wood <Trevor.G.Wood@gmail.com>	4 years ago
Aditya C S	410c1a0d38	feat(ruler): support alert relabeling (#6220)	4 years ago
Periklis Tsirakidis	6e9517e4b5	ruler: Add support for alertmanager header authorization (#6136)	4 years ago