apitech/loki - loki - Gitea: Git with a cup of tea

Commit Graph

Author	SHA1	Message	Date
Shantanu Alshi	145319e5f5	Remove experimental enable ruler api toggle (#10380 ) Remove `-experimental.ruler.enable-api` in favour of `-ruler.enable-api`. The former was already deprecated for some time.	2 years ago
Danny Kopping	33e44ed39d	Ruler: remote rule evaluation (#8744 ) What this PR does / why we need it: Adds the ability to evaluate recording & alerting rules against a given `query-frontend`, allowing these queries to be executed with all the parallelisation & optimisation that regular adhoc queries have. This is important because with `local` evaluation all queries are single-threaded, and rules that evaluate a large range/volume of data may timeout or OOM the `ruler` itself, leading to missed metrics or alerts. When `remote` evaluation mode is enabled, the `ruler` effectively just becomes a gRPC client for the `query-frontend`, which will dramatically improve the reliability of the `ruler` and also drastically reduce its resource requirements. Which issue(s) this PR fixes: This PR implements the feature discussed in https://github.com/grafana/loki/pull/8129 (LID 0002: Remote Rule Evaluation).	2 years ago
Susana Ferreira	e07fe7c338	Deprecate CLI flag -ruler.wal-cleaer.period in favor of CLI flag -ruler.wal-cleaner.period (#7937 ) What this PR does / why we need it: Deprecate CLI flag `-ruler.wal-cleaer.period` in favor of CLI flag `-ruler.wal-cleaner.period`. Which issue(s) this PR fixes: Fixes https://github.com/grafana/loki/pull/7916#issuecomment-1348276269 and https://github.com/grafana/loki/pull/7916#discussion_r1047043534 NOTE: correctly registring the `ruler.wal-cleaner` configuration flags is not a breaking change and doesn't require an upgrade guide, because the ruler WAL cleaner initialization code already guarantees that the default values are used: `c71620ae94/pkg/ruler/storage/cleaner/cleaner.go (L76-L101)` Special notes for your reviewer: Checklist - [x] Reviewed the `CONTRIBUTING.md` guide - [x] Documentation added - [ ] Tests updated - [x] `CHANGELOG.md` updated - [x] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	2 years ago
Susana Ferreira	68a4d3271a	Fix documentation typos from #7916 (#7934 ) What this PR does / why we need it: Fix documentation typos from https://github.com/grafana/loki/pull/7916#pullrequestreview-1217192817 Which issue(s) this PR fixes: Fixes https://github.com/grafana/loki/pull/7916#pullrequestreview-1217192817 Special notes for your reviewer: Checklist - [ ] Reviewed the `CONTRIBUTING.md` guide - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	2 years ago
Susana Ferreira	f93b91bfb5	Add configuration documentation generation tool (#7916 ) What this PR does / why we need it: Add a tool to generate configuration flags documentation based on the flags properties defined on registration on the code. This tool is based on the [Mimir doc generation tool](https://github.com/grafana/mimir/tree/main/tools/doc-generator) and adapted according to Loki configuration specifications. Prior to this PR, the configuration flags documentation was dispersed across two sources: * [_index.md](`5550cd65ec/docs/sources/configuration/_index.md`) * configuration flags registration in the code This meant that there was no single source of truth. In this PR, the previous `_index.md` file is replaced with the new file generated by the tool. The next step includes adding a CI step that validates if the _index.md file was generated according to the flags settings. This will be done in a follow-up PR. NOTE: this is not a documentation update PR. Apart from some minor typo fixes, the documentation changes on the code, were copied from the `_index.md` file. Which issue(s) this PR fixes: Fixes https://github.com/grafana/loki-private/issues/83 Special notes for your reviewer: Files: * [docs/sources/configuration/index.template](`5550cd65ec/docs/sources/configuration/index.template`): template used to generate the final configuration file * [/docs/sources/configuration/_index.md](`c32e5d0acb/docs/sources/configuration/_index.md`): file generated by tool * `loki/pkg` directory files updated with up-to-date documentation from `_index.md` file * [tools/doc-generator](`5550cd65ec/tools/doc-generator`) directory with documentation generation tool. Checklist - [ ] Reviewed the `CONTRIBUTING.md` guide - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	2 years ago
Kaviraj Kanagaraj	6f0690e7bf	fix(ruler): Initialize multi remote clients config (#7151 ) <!-- Thanks for sending a pull request! Before submitting: 1. Read our CONTRIBUTING.md guide 2. Name your PR as `<Feature Area>: Describe your change`. a. Do not end the title with punctuation. It will be added in the changelog. b. Start with an imperative verb. Example: Fix the latency between System A and System B. c. Use sentence case, not title case. d. Use a complete phrase or sentence. The PR title will appear in a changelog, so help other people understand what your change will be. 3. Rebase your PR if it gets out of sync with main --> What this PR does / why we need it: To be backward compatible with single client config. ``` ts=2022-09-14T08:24:04.549961234Z caller=memberlist_logger.go:74 level=debug msg="Initiating push/pull sync with: 10.132.18.167:7946" │ │ panic: assignment to entry in nil map │ │ goroutine 1 [running]: │ │ github.com/grafana/loki/pkg/ruler.NewRuler({{{0xc00002a120}, {0x0, 0x0, 0x0}, {0x6400000, 0x6400000, {0x0, 0x0}, 0x0, 0x0, ...}, ...}, ...}, ...) │ │ /src/enterprise-logs/vendor/github.com/grafana/loki/pkg/ruler/ruler.go:21 +0x1b7 ``` Which issue(s) this PR fixes: Fixes #<issue number> Special notes for your reviewer: <!-- Note about CHANGELOG entries, if a change adds: * an important feature * fixes an issue present in a previous release, * causes a change in operation that would be useful for an operator of Loki to know then please add a CHANGELOG entry. For documentation changes, build changes, simple fixes etc please skip this step. We are attempting to curate a changelog of the most relevant and important changes to be easier to ingest by end users of Loki. Note about the upgrade guide, if this changes: * default configuration values * metric names or label names * changes existing log lines such as the metrics.go query output line * configuration parameters * anything to do with any API * any other change that would require special attention or extra steps to upgrade Please document clearly what changed AND what needs to be done in the upgrade guide. --> Checklist - [ ] Documentation added - [ ] Tests updated - [ ] Is this an important fix or new feature? Add an entry in the `CHANGELOG.md`. - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	3 years ago
Mohamed-Amine Bouqsimi	3d4788fb51	Add ability for Ruler to remote write to multiple clients (#6823 )	3 years ago
Kaviraj	dddace862d	Add nil check for Ruler BasicAuth config. (#5168 ) Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com>	3 years ago
Danny Kopping	0368a09d4c	Ruler: Fix remote write basic auth credentials (#5144 ) * Retrieving original password to avoid marshaling issues Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Fixing test Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Adding CHANGELOG entry Signed-off-by: Danny Kopping <danny.kopping@grafana.com>	3 years ago
Dylan Guedes	3d7e9cee36	Copy `cortex/pkg/ruler` package dependency into Loki (#5089 ) * Fork cortex `ruler/rulestore/local`. * Fork cortex ruler into Loki as package `ruler/base`. * Use forked cortex ruler instead of vendored. * Update `vendor/modules.txt` and Makefile instruction. * Merge `pkg/ruler/base/rulestore` with `pkg/ruler/rulestore`. * Move `pkg/ruler/base/rulespb` to `pkg/ruler/rulespb`. * Rename `cortex_ruler` usage to `base_ruler`. * Fix lint issues. * Add forked ruler tests. * Use own `pkg/storage/bucket` instead of Cortex one. * Swap to `pkg/storage/chunk/{aws,gcp,etc}`. * Update with correct prometheus import. * Change error comparison. * Code style changes	3 years ago
Danny Kopping	5b30589acf	Documenting recording rules per-tenant WAL (#4566 ) * WIP Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Document ruler config options Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Self-review updates Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Clarifying WAL benefit in response to review Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Adding dashboard link Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Update docs/sources/configuration/_index.md Co-authored-by: Owen Diehl <ow.diehl@gmail.com> * Ensuring ruler and ingester default WAL paths do not conflict Signed-off-by: Danny Kopping <danny.kopping@grafana.com> Co-authored-by: Owen Diehl <ow.diehl@gmail.com>	4 years ago
Trevor Whitney	ed025e0cd6	Change a few default config values and improve application of common storage config (#4543 ) * further config simplifications and better defaults * default ruler api to on * register defaults for storage configs in the common config (ie. signature version for S3) * enable ingester wal by default * disable chunk retries by default * allow common path prefix to include or omit trailing slash * allow partial storage config to be defined for overriding things like bucket names per component * fix tests * add changelog and upgrade docs * explain impact of wal change in upgrading.md Co-authored-by: Owen Diehl <ow.diehl@gmail.com> * rename add WithPrefix to RegisterFlags * fix (frontend OR worker) and scheduler boot order when both are running (#4545) Co-authored-by: Owen Diehl <ow.diehl@gmail.com>	4 years ago
Danny Kopping	8a546b31a9	Ruler: per-tenant WAL (#4344 ) Signed-off-by: Danny Kopping <danny.kopping@grafana.com>	4 years ago
Owen Diehl	f17b189a3f	Promote/ruler api (#4132 ) * unmark the ruler api as experimental * deprecation notice for experimental prefix * removes vendor changes, adds -ruler.enable-api (without experimental prefix) on our ruler.Config	4 years ago
Danny Kopping	babed450c7	Ruler: Recording Rules (#3766 ) * WIP: hack to get recording rules working and pushing to Cortex/Prometheus Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Refactoring Adding remote_write config for ruler Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Minor refactorings Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Moving manager subpackage into ruler package to avoid dependency cycles This also mirrors Cortex's package structure Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Minor refactorings Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Skipping commit if remote-write client is not defined Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Updating use of cortex client Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Memoizing appenders, using queue for samples & labels Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Adding buffer size configurability Refactoring Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Adding metric to show current buffer size Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Refactoring for better responsibility separation & testability Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Adding per-tenant overrides of remote-write queue capacity Renaming "buffer size" to "queue capacity" to be more accurate Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Adding tests for evicting queue Minor refactoring Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Adding more tests and refactoring Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Adding queue benchmark Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Reducing redundancy in metric names Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Testing that only metric queries can be run Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Minor fixes pre-review Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Appeasing the linter Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Guarding against unprotected nil pointer dereference in Prometheus remote.Client Adding remote-write client validation Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Appeasing the linter Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Setting tenant ID header on remote-write client Adding User-Agent string Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Updating benchmark to use complex struct rather than int to be more reflective of usage Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Registering flags Removing extraneous checks Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Adding metric to track remote-write commit errors Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Refactoring based on review Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Performance improvements based on review Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Return error on invalid queue capacity Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Removing global queue capacity config - using limits Minor refactoring Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Reusing memory in request preparation Refactoring for testability Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Moving remote-write metrics into struct Refactoring Unexporting and refactoring memstore metrics to match Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Applying review suggestions Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Allowing for runtime changing of per-tenant remote-write queue capacity Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Appeasing the linter Signed-off-by: Danny Kopping <danny.kopping@grafana.com>	4 years ago

15 Commits (46813867758729df3e18763ad7366f141291f3c9)