174 Commits (bfa2a1df65e233ce99fccd01df2130f440ebd166)

Author	SHA1	Message	Date
Christian Haudum	32745ba006	chore(deps): Update grafana/dskit to latest commit (#17235) ... Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	9 months ago
Karsten Jeschkies	75593e0ed7	refactor(stringlabels): Support stringlabels in loghttp, pattern and ruler tests. (#17102)	9 months ago
Christian Haudum	1705f98a9f	chore(engine): Wire up new execution engine (#17032) ... This PR connects the already existing components of the new query engine and wires them up in the querier API. The feature flag `-querier.engine.enable-v2-engine` controls whether supported queries should be executed using the new engine. Note that this implementation currently does not execute any queries. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	9 months ago
Trevor Whitney	eda3ba865b	feat(ruler): add per-tenant configuration to disable WAL replay (#16717)	10 months ago
Paul Rogers	24c260e3e4	chore(deps): Update dskit (#16782)	10 months ago
Cyril Tovena	8976709f24	fix(ruler): ensure test service shutdown is properly awaited (#16468)	11 months ago
renovate[bot]	b37eefe24f	fix(deps): update github.com/prometheus/prometheus (main) (#15950) ... Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> Co-authored-by: Paul Rogers <paul.rogers@grafana.com>	11 months ago
Ashwanth	1cd95f5212	docs(thanos): unhide configuration docs for thanos storage clients (#16051)	11 months ago
Avanish Vaghela	a96a695882	docs: Add clarifications for ruler remote write config options (#15167) ... The update provides better understanding of the ruler remote write config behaviour. If a user specifies an 'X-Scope-OrgID' header under the 'headers' section of RemoteWriteConfig, the config parser silently drops this. This is not currently clear to the user and can lead to confusion. This update aims to clarify this. It also adds clarification about what tenant ID value will be added to the 'X-Scope-OrgID' header in the remote write request if 'add_org_id_header' config parameter is set to 'true'.	12 months ago
Aaron	b272573efc	feat: Add Rule Name and Type to Query Tags (#15055) ... Signed-off-by: Callum Styan <callumstyan@gmail.com> Co-authored-by: Callum Styan <callumstyan@gmail.com>	1 year ago
matthewhudsonedb	e347eb7c80	fix: Fix loki ruler generator url left parameter url encoding (#15601) ... The generate URL `left` parameter is not URL encoded. The double quotes in str variable around "queries" break hyperlinking.	1 year ago
Paul Rogers	f2c2a22bdc	chore: Preparation for incoming static code analysis CI check (#15164) ... Co-authored-by: Danny Cooper <danny.cooper@grafana.com>	1 year ago
renovate[bot]	e0ac0d0efd	fix(deps): update module github.com/grpc-ecosystem/go-grpc-middleware to v2 (#15045) ... Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> Co-authored-by: Paul Rogers <paul.rogers@grafana.com>	1 year ago
renovate[bot]	be24862f53	fix(deps): update module github.com/imdario/mergo to v1 (#15035) ... Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> Co-authored-by: Paul Rogers <paul.rogers@grafana.com>	1 year ago
Joao Marcal	cfc38194b4	feat(thanos): make use of the new function IterWithAttributes (#14793)	1 year ago
Nikolay Andreev	b7ac5b4dcd	docs: add info about remotewrite client config in ruler (#14401) ... Co-authored-by: J Stickler <julie.stickler@grafana.com>	1 year ago
Ashwanth	8bca2e7608	feat(ruler): enables ruler store that uses clients from thanos-io/objstore pkg (#11713)	1 year ago
Joao Marcal	c059ace53e	feat(storage): GCS backend using thanos.io/objstore (#11132) ... Signed-off-by: Joao Marcal <jmarcal@redhat.com> Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> Signed-off-by: Kaviraj Kanagaraj <kavirajkanagaraj@gmail.com> Signed-off-by: Ashwanth Goli <iamashwanth@gmail.com> Co-authored-by: Kaviraj Kanagaraj <kavirajkanagaraj@gmail.com> Co-authored-by: Ashwanth <iamashwanth@gmail.com>	1 year ago
Trevor Whitney	b5462b6639	chore: remove copied loop vars (#14383)	1 year ago
Christian Haudum	3beb8ff9cf	fix(build): Use Debian Bullseye base image for build image (#14368) ... This change is necessary to allow the binaries built using the Loki build image to run on operating systems with older libc version. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	1 year ago
Paul Rogers	18cef217fc	chore: Linting updates in prep for Go 1.23 (#14071)	1 year ago
Paul Rogers	8fb7b488bd	chore(deps): update dskit 20240819 (#13924)	1 year ago
Marco Pracucci	b88583da7d	feat: upgrade prometheus (#13671) ... Signed-off-by: Marco Pracucci <marco@pracucci.com> Signed-off-by: Vladyslav Diachenko <vlad.diachenko@grafana.com> Co-authored-by: Vladyslav Diachenko <vlad.diachenko@grafana.com>	1 year ago
Joel Verezhak	e9a9c60c22	fix: protect ruler remote-write overrides map with a mutex when creating new appenders (#13676)	1 year ago
Christian Haudum	d8cc1cea83	chore: Make dedicated iterator package (#13273) ... Iterators of various types are widely used throughout the Loki code base. With the recent code additions of the bloom filters, a new set of utility functions for iterators emerged in `github.com/grafana/loki/v3/pkg/storage/bloom/v1`. The package defines interfaces for various common types, but also provides implementations to create and compose new iterators that implement these interfaces. This new package uses Go Generics. - However, at the current state, there are multiple iterator interfaces and implementations, which adds cognitive overhead in understanding how they work and which ones should be used. The idea is to unify them into a single iterator lib for Loki. - As a first step towards a single iterator library for the Loki code base, this PR moves the utilities from the `pkg/storage/bloom/v1` package to the `pkg/iter/v2` package. - Second, it changes the existing `EntryIterator` and `SampleIterator` iterators to "inherit" `v2.CloseIterator` in order to expose the same function names. - And lastly, naming conventions of iterator interfaces and structs are unified. The basic iterator interface (defined in `pkg/iter/v2/interface.go`) looks like so: ```go // Usage: // // for it.Next() { // curr := it.At() // // do something // } // if it.Err() != nil { // // do something // } type Iterator[T any] interface { Next() bool Err() error At() T } ``` --- Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
George Robinson	384f451193	chore: Change default version to Alertmanager APIv2 (#13263)	2 years ago
jackyin	a46d14fb05	fix: Optimize regular initialization (#12926)	2 years ago
MarkDaveny	3a46d3717d	style: fix function names (#12817) ... Signed-off-by: MarkDaveny <peicuiping@aliyun.com>	2 years ago
Owen Diehl	bc78d13d9b	feat(blooms): compute chunks once (#12664) ... Signed-off-by: Owen Diehl <ow.diehl@gmail.com>	2 years ago
Trevor Whitney	af09f53a33	ci: release workflows get build image from makefile (#12778)	2 years ago
Ed Welch	4c88be0ef2	chore: update loki modules for 3.0 release (#12433) ... Signed-off-by: Edward Welch <edward.welch@grafana.com>	2 years ago
Ashwanth	774b01dac0	fix(ruler): pass noop analyseRules to rules manager (#12353)	2 years ago
Paulin Todev	d54e0871d4	chore(deps): Update Prometheus in Loki and Promtail (#12245) ... This PR updates the Prometheus dependency in Loki and Promtail. The PR is required so that we can also update Grafana Agent to the latest Prometheus. Unfortunately, Promtail and Loki share the same go.mod file.	2 years ago
Travis Patterson	fcd3e3342a	fix: pass correctly formatted query tags from the ruler to the querier (#12265)	2 years ago
Justin Burnham	1b4d23f9b7	Ruler: Disable x-scope-orgid header append in remote write (#11819) ... Co-authored-by: Michel Hollands <42814411+MichelHollands@users.noreply.github.com> Co-authored-by: Danny Kopping <dannykopping@gmail.com>	2 years ago
Ashwanth	69919a303b	ruler: add filtering support for prometheus rules api (#11817) ... **What this PR does / why we need it**: Adds `rule_name`, `rule_group`, `file` and `type` query parameters for filtering the response of `/prometheus/api/v1/rules` endpoint. Replicates mimir's functionality: https://github.com/grafana/mimir/pull/5291 - all of them are optional. - `type` paremeter accepts either `alert` or `record` - `rule_name`, `rule_group`, `file` can accept multiple values and they filter the response accordingly. There is a minor change in behavior: `/prometheus/api/v1/rules` endpoint will no longer return empty rule groups which is inline with both [prometheus](https://github.com/prometheus/prometheus/pull/12270/files#diff-315f251cdd7e93fcec1e7e9505744da1d1828f30d2b61d1f4ce963fa26bf1909R1403) and [mimir](https://github.com/grafana/mimir/pull/5291/files#diff-e5424c21c0e827bd1c9d3f669ed605897696bdc27993bc8bfd7113eba787b49dR1120). This is not a breaking change since rule groups with [no rules fail validation](27fbd62505/pkg/ruler/base/manager.go (L295)) and cannot be created. **Which issue(s) this PR fixes**: Fixes https://github.com/grafana/loki/issues/9295 **Special notes for your reviewer**: **Checklist** - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [x] Documentation added - [x] Tests updated - [x] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](d10549e3ec) - [ ] If the change is deprecating or removing a configuration option, update the `deprecated-config.yaml` and `deleted-config.yaml` files respectively in the `tools/deprecated-config-checker` directory. [Example PR](0d4416a4b0)	2 years ago
Danny Kopping	61a4205313	Ruler: catering for another data race in tenant config (#11612) ... Expands on #11601 **What this PR does / why we need it**: Turns out the previous tests didn't expose all possible causes for data races (another one occurs at 5a55158cc7/pkg/ruler/registry.go (L204)). Moving the mutex to the calling function adds more safety. **Which issue(s) this PR fixes**: Fixes #11569 Signed-off-by: Danny Kopping <danny.kopping@grafana.com>	2 years ago
Danny Kopping	cd3cf6291f	Ruler: protect overrides map with mutex when accessing tenant configs (#11601) ... **What this PR does / why we need it**: A ruler handling many hundreds of rules can provoke a situation where the WAL appender reads & modifies tenant configs concurrently in an unsafe way; this PR protects that with a mutex. **Which issue(s) this PR fixes**: Fixes #11569	2 years ago
Karsten Jeschkies	5b97fcfd93	Send query plan to querier. (#11246) ... **What this PR does / why we need it**: Following https://github.com/grafana/loki/pull/11123 and in order to enable https://github.com/grafana/loki/pull/10417 the query frontend should send the serialized LogQL AST instead of the query string to the queriers. This enables the frontend to change the AST and inject expressions that are not expressible in LogQL. **Checklist** - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [ ] Documentation added - [x] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](d10549e3ec) - [ ] If the change is deprecating or removing a configuration option, update the `deprecated-config.yaml` and `deleted-config.yaml` files respectively in the `tools/deprecated-config-checker` directory. [Example PR](0d4416a4b0) --------- Signed-off-by: Callum Styan <callumstyan@gmail.com> Co-authored-by: Callum Styan <callumstyan@gmail.com>	2 years ago
Kaviraj Kanagaraj	30d0030bf9	inflight-logging: Add extra metadata to inflight requests logging (#11243) ... **What this PR does / why we need it**: logging: Add extra metadata to inflight requests This adds extra metadata (similar to what we have in `metrics.go`) but for queries in in-flight (both started and retrying) Changes: Adds following data 1. Query Hash 2. Start and End time 3. Start and End delta 4. Length of the query 5. Moved the helper util to `queryutil` package because of cyclic dependencies with `logql` package. **Which issue(s) this PR fixes**: Fixes #<issue number> **Special notes for your reviewer**: Find the screenshots of log entries looks like (both in `retry.go` and `roundtrip.go`)   **Checklist** - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [ ] Documentation added - [x] Tests updated - [x] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](d10549e3ec) - [ ] If the change is deprecating or removing a configuration option, update the `deprecated-config.yaml` and `deleted-config.yaml` files respectively in the `tools/deprecated-config-checker` directory. [Example PR](0d4416a4b0) --------- Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com>	2 years ago
Ashwanth	32e9ee8209	chore: follow-up to #11151 and #11025 (#11225) ... **What this PR does / why we need it**: - remove usage of `enforce_metric_name` in ksonnet - https://github.com/grafana/loki/pull/11151 - make `cortex_ruler_client_request_duration_seconds` prefix configurable - updates upgrade guide to remove reference to metrics that loki does not export **Which issue(s) this PR fixes**: Fixes #<issue number> **Special notes for your reviewer**: **Checklist** - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](d10549e3ec) - [ ] If the change is deprecating or removing a configuration option, update the `deprecated-config.yaml` and `deleted-config.yaml` files respectively in the `tools/deprecated-config-checker` directory. [Example PR](0d4416a4b0)	2 years ago
Ashwanth	766f27645d	config: remove flags & YAML settings deprecated in 2.9.x (#11151) ... **What this PR does / why we need it**: - remove `-ruler.evaluation-delay-duration` - remove `boltdb.shipper.compactor.deletion-mode` - remove compactor flags with prefix`-boltdb.shipper.compactor.*` - remove `validation.enforce-metric-name` - remove `log.use-buffered`, `log.use-sync` **Special notes for your reviewer**: `log.use-buffered`, `log.use-sync` are not mentioned in the upgrade guide since they were hidden fields **Checklist** - [X] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [ ] Documentation added - [X] Tests updated - [x] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [X] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](d10549e3ec) - [X] If the change is deprecating or removing a configuration option, update the `deprecated-config.yaml` and `deleted-config.yaml` files respectively in the `tools/deprecated-config-checker` directory. [Example PR](0d4416a4b0) --------- Co-authored-by: J Stickler <julie.stickler@grafana.com>	2 years ago
Periklis Tsirakidis	6385b19573	fix: All lifecycler cfgs ref a valid IPv6 addr and port combination (#11121) ... **What this PR does / why we need it**: Ensure that all constructors of LifeCyclerConfig are using `net.JoinHostPort` for instance addr and port combinations. This enables IPv6 usage compatible across the code based. Currently only distributors provide IPv6 compatibility because they use `Lifecycler` (that pulls the addr/port in the `NewLifeCycler`) while everything else uses `BasicLifeCycler` delegate this to the user defining an addr/port combination in `LifecyclerConfig`.	2 years ago
Michel Hollands	e93f5bfd93	Add dashboard changes for migrating from cortex_ metrics to loki_ metrics (#11082) ... **What this PR does / why we need it**: - add missing namespace param for ruler manager metrics - update the loki mixin to use the renamed metrics so both are supported - created the compiled mixin - make the prefix of the cortex_distributor_ingester_clients metrics configurable **Checklist** - [X] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](d10549e3ec) - [ ] If the change is deprecating or removing a configuration option, update the `deprecated-config.yaml` and `deleted-config.yaml` files respectively in the `tools/deprecated-config-checker` directory. [Example PR](0d4416a4b0) --------- Signed-off-by: Michel Hollands <michel.hollands@gmail.com>	2 years ago
Michel Hollands	8628b15624	Use metrics namespace for more metrics (#11025) ... **What this PR does / why we need it**: Use the metrics namespace setting instead of hardcoding to `cortex`. This is a follow up to (and based on) https://github.com/grafana/loki/pull/11014. **Checklist** - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [ ] Documentation added - [X] Tests updated - [x] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](d10549e3ec) - [ ] If the change is deprecating or removing a configuration option, update the `deprecated-config.yaml` and `deleted-config.yaml` files respectively in the `tools/deprecated-config-checker` directory. <!-- TODO(salvacorts): Add example PR --> --------- Signed-off-by: Michel Hollands <michel.hollands@gmail.com> Co-authored-by: Ashwanth <iamashwanth@gmail.com>	2 years ago
Kaviraj Kanagaraj	52a7c0fd36	ruler: Support `writeNotify` on Loki ruler (#10906) ... **What this PR does / why we need it**: prometheus ruler added a feature of notifying the reader when a sample is appended, instead of waiting in a loop burning the CPU cycles. https://github.com/prometheus/prometheus/pull/11949 This changes a default behaviour a bit. Now if `notify` is not enabled, next read is done only when next readTicker is triggered. **Which issue(s) this PR fixes**: Also should fix https://github.com/grafana/loki/issues/10859 **Special notes for your reviewer**: Adding few more details for the sake of completeness. We found this via more frequent failures of rule-evaluation integration tests linked on the issue above. After some investigation, we tracked down to prometheus changes. Prometheus introduced new type `wlog.WriteNotified` interface with `Notify()` method with a goal to notify any waiting readers, that some write is done. Two types implements this type `wlog.Watcher` and `remote.Storage`. `remote.Storage` implements `Notify()` by just calling it's queues `wlog.Watcher`'s `Notify()` under the hood. How are these types impacts Loki ruler? Loki ruler also uses `remote.Storage`. So when any samples got committed via `appender`, we have to notify the remote storage. **Checklist** - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](d10549e3ec) --------- Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com>	2 years ago
Shantanu Alshi	183d6f2c39	Upgrade prometheus to 0.47.1 and dskit (#10814)	2 years ago
Paschalis Tsilias	07cbef9226	Update Prometheus dependency to v2.46.0 (#10777) ... This PR updates the Prometheus dependency to v2.46.0. We need this to update the Agent to use Prometheus' Native Histograms after some bug fixes in the latest version. I bumped the version in go.mod, ran `go mod tidy && go mod vendor` and fixed compilation issues. **Special notes for your reviewer**: The only changes needed to have `make all` successfully compile was to use the new `wlog.NewSize` to use wlog.CompressionNone (instead of false), and wlog.CompressionSnappy (the default, instead of true), as well as add a new argument to the textparse.New constructor to work as it did without scraping native histograms via protobuf. I don't think there are any other user-facing changes or changes related to Loki. --------- Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com>	2 years ago
Shantanu Alshi	145319e5f5	Remove experimental enable ruler api toggle (#10380) ... Remove `-experimental.ruler.enable-api` in favour of `-ruler.enable-api`. The former was already deprecated for some time.	2 years ago
Shantanu Alshi	92ba055ae5	Remove ruler.wal-cleaer.period (#10378) ... The config with typo is now replaced by `-ruler.wal-cleaner.period`	2 years ago