1088 Commits (416e01ca8e8f6e606135ea7b18f9db3393345ede)

Author	SHA1	Message	Date
MOZGIII	08e741b7c6	helm: Allow altering the canary label name (#9945) ... **What this PR does / why we need it**: Allows changing the label name at the canary in the helm chart. **Which issue(s) this PR fixes**: Idk, none? **Special notes for your reviewer**: This is a very trivial change. **Checklist** - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [x] Documentation added - [x] Tests updated - [x] `CHANGELOG.md` updated - [x] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [x] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [x] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](d10549e3ec) --------- Co-authored-by: Michel Hollands <42814411+MichelHollands@users.noreply.github.com> Co-authored-by: Travis Patterson <travis.patterson@grafana.com>	3 years ago
J Stickler	cc2aeac860	Break up and move Best Practices topic. (#9912) ... I realized that I should have moved the Best Practices topic as part of #8711, since most of the content is best practices for using labels. This PR: * Splits the file into two parts (Best practices for labels and best practices for configuration) * Moves the two files to Get Started and Configure, respectively. * Updates page weights for both sections as needed.	3 years ago
Vladyslav Diachenko	28e9e15639	[jsonnet] removed deprecated policy.v1beta1 (#9978) ... **What this PR does / why we need it**: in k8s version 1.25 some k8s APIs are deprecated, such as policy.v1beta1 which is moved to policy.v1 **Checklist** - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [ ] Documentation added - [ ] Tests updated - [x] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](d10549e3ec) --------- Signed-off-by: Vladyslav Diachenko <vlad.diachenko@grafana.com>	3 years ago
Adrien	c3da1ca04b	Docs : fix promtal typo (#9951) ... **What this PR does / why we need it**: Fix a typo in installation documentation, Promtal instead of Promtail **Which issue(s) this PR fixes**: Fixes #<issue number> **Special notes for your reviewer**: **Checklist** - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [x] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](d10549e3ec) Co-authored-by: J Stickler <julie.stickler@grafana.com>	3 years ago
Vladyslav Diachenko	be1a9c4ec5	Revert "updated to the latest version of dskit" (#9953) ... Reverts grafana/loki#9920	3 years ago
J Stickler	8e59fcac91	Update redirects to use new alias format. (part 2 of 2) (#9890) ... Breaking the alias updates into two PRs to make for smaller reviews. **Special notes for your reviewer**: @jdbaldry commented on #9717 that the alias format had changed, but we [agreed](https://github.com/grafana/loki/pull/9717#issuecomment-1594960563) that I could update it in a follow up PR. This is that follow up. Also updating previously merged file moves that used the old alias format. <strike>I could not get relative redirects to work for the LIDS or Maintaining files, so had to use the older format, which did work.</strike> Updated weights for some pages. Updated a couple of `menuTitles` that I missed updating on previous PRs. --------- Co-authored-by: Jack Baldry <jack.baldry@grafana.com>	3 years ago
Karsten Jeschkies	2dfac72835	Clarify that label filter expressions are not supported in promtail. (#9894) ... **What this PR does / why we need it**: Some users of Promtail seem to be confused that certain types of filter expressions are not supported in Promtail's `match` stage. This change clarifies what kind of filter expressions are supported. **Which issue(s) this PR fixes**: Fixes #5209 **Special notes for your reviewer**: **Checklist** - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [x] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](d10549e3ec) --------- Co-authored-by: J Stickler <julie.stickler@grafana.com>	3 years ago
J Stickler	b3b4317861	Fix image links in moved files. (#9910) ... Noticed the other day that images in files that have been moved as part of #8710 were not displaying in `/next/`. This PR updates the image references.	3 years ago
J Stickler	1c9160bc4b	Remove aliases to current location. (#9911) ... Re: [this comment ](https://github.com/grafana/loki/pull/9890#pullrequestreview-1521562982)from @jdbaldry on #9890 that "Relative aliases that refer to the current page location won't be effective and should be removed because they can cause confusion and might result in mistaken redirects in the future if pages are moved again." Removing aliases that refer to the current location of the pages.	3 years ago
Vladyslav Diachenko	3243676f4c	updated to the latest version of dskit (#9920) ... **What this PR does / why we need it**: Updated Loki with the latest version of dskit **Checklist** - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](d10549e3ec) --------- Signed-off-by: Vladyslav Diachenko <vlad.diachenko@grafana.com>	3 years ago
Travis Patterson	cb2b3ca9ea	Add limit to the series volume endpoint (#9833) ... To protect index gateways, this PR adds a per-tenant limit to the `series_volume` endpoint so the accumulation of series can't grow without bound.	3 years ago
SeamusGrafana	72ea9d778c	[Docs] - Add example of Conditional OR to IP Filter (#9848) ... **What this PR does / why we need it**: This updates the Docs with an example of how to use an OR conditional filter when filtering by IP addresses. Example of this working on [Grafana Play can be found here](https://play.grafana.org/explore?orgId=1&left=%7B%22datasource%22:%22grafanacloud-logs%22,%22queries%22:%5B%7B%22refId%22:%22A%22,%22expr%22:%22%7Bcontainer%3D%5C%22grafana%5C%22%7D%20%7C%20logfmt%20%7C%20remote_addr%3Dip%28%6010.10.0.0%2F24%60%29%20or%20remote_addr%3Dip%28%6010.4.1.0%2F24%60%29%22,%22queryType%22:%22range%22,%22datasource%22:%7B%22type%22:%22loki%22,%22uid%22:%22grafanacloud-logs%22%7D,%22editorMode%22:%22code%22%7D%5D,%22range%22:%7B%22from%22:%221688342400000%22,%22to%22:%221688428799000%22%7D%7D) **Which issue(s) this PR fixes**: Fixes #<issue number> **Special notes for your reviewer**: **Checklist** - [X] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [X] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](d10549e3ec)	3 years ago
Karsten Jeschkies	02027e442c	Document deadlocked Docker daemon workaround. (#9896) ... **What this PR does / why we need it**: This pulls @Pandry's [workaround](https://github.com/grafana/loki/issues/2361#issuecomment-718024318) for the seemingly deadlocked Docker daemon into the documentation. **Special notes for your reviewer**: **Checklist** - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [x] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](d10549e3ec)	3 years ago
Jara Suárez de Puga García	10e38b8b74	Request to change the loki-overview-2 image (#9359) ... **What this PR does / why we need it**: We in support are doing some review of public documents in general, the Overview is okay but the image seemed quite outdated, I was wondering if it would be nice to have it updated to this or something else ( not sure if it is the correct format). **Checklist** - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [x] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	3 years ago
J Stickler	0d79b219b2	Update redirects to use new alias format. (part 1 of 2) (#9881) ... **Special notes for your reviewer**: @jdbaldry commented on #9717 that the alias format had changed, but we [agreed](https://github.com/grafana/loki/pull/9717#issuecomment-1594960563) that I could update it in a follow up PR. This is that follow up. Also updating previously merged file moves that used the old alias format. Breaking the alias updates into two PRs to make for smaller reviews.	3 years ago
J Stickler	f70f9110aa	Update version in docker driver command. (#9870) ... **Which issue(s) this PR fixes**: Fixes #9802 There is an 2.5 year old image in Docker Hub with the tag `latest`, so updating the documentation to not reference this tag, as it pulls the incorrect image.	3 years ago
J Stickler	7797a98074	TOC Sync: Move configuration files under Configure (#9879) ... **Which issue(s) this PR fixes**: Fixes #8713 **Special notes for your reviewer**: Replaces PR [#8817](https://github.com/grafana/loki/pull/8817), which was stale and badly out of date. Part of the Database products table of contents alignment project. Moves configuration files from Configuration to new Configure heading in TOC. Updates front matter for the topic. Fixes links broken by the move. Updates the MakeFile to point to the new Configure DOCS_SOURCES_PATH.	3 years ago
David Peleg	c129cfe93c	Docs: add reference to flog CLI args (#9638) ... **What this PR does / why we need it**: **Which issue(s) this PR fixes**: Fixes #<issue number> **Special notes for your reviewer**: **Checklist** - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [x] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](d10549e3ec)	3 years ago
Sandeep Sukhani	e089b4b60d	address feedback on pr 9798 (#9842) ... **What this PR does / why we need it**: PR #9798 got merged by mistake without me addressing the feedback. There is some critical feedback which needs to be addressed, particularly https://github.com/grafana/loki/pull/9798#discussion_r1243526743.	3 years ago
Jonathan Duncan	1d3f0b1001	Documentation: limits_config.enforce_metric_name removed (#9393) ... **What this PR does / why we need it**: ["The config still exists though, it's just not used anywhere."](https://github.com/grafana/loki/issues/2096) **Checklist** - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	3 years ago
aryalireza	dc4277fd86	Update tanka.md (#9820) ... Add delete loki's environment and resources from k8s cluster. **What this PR does / why we need it**: **Which issue(s) this PR fixes**: Fixes #<issue number> **Special notes for your reviewer**: **Checklist** - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](d10549e3ec) --------- Co-authored-by: Travis Patterson <masslessparticle@gmail.com> Co-authored-by: J Stickler <julie.stickler@grafana.com> Co-authored-by: Travis Patterson <travis.patterson@grafana.com>	3 years ago
Karsten Jeschkies	b35bbd80d6	Support content negotiation between query frontend and querier. (#9813) ... **What this PR does / why we need it**: Currently, the querier sends results to the query frontend in JSON which is then decoded to Protobuf. It is more efficient to send the results as Protobuf. This will also allow to extend the results with custom data structures. The change is backwards compatible through content negotiation. **Special notes for your reviewer**: **Checklist** - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [x] Documentation added - [x] Tests updated - [x] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](d10549e3ec)	3 years ago
ZhenYi Liang	0711580154	helm: support endpoint_suffix for azure storage (#9791) ... **What this PR does / why we need it**: According the [official document](https://grafana.com/docs/loki/latest/storage/#using-account-name-and-key), support storage_config.azure.endpoint_suffix, example blob.core.chinacloudapi.cn **Which issue(s) this PR fixes**: Fixes #<issue number> **Special notes for your reviewer**: **Checklist** - [X] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [x] Documentation added - [ ] Tests updated - [x] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [x] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](d10549e3ec) --------- Co-authored-by: Michel Hollands <42814411+MichelHollands@users.noreply.github.com> Co-authored-by: J Stickler <julie.stickler@grafana.com>	3 years ago
Tiago Posse	a24ef7b206	Helm: support metricRelabelings for loki serviceMonitors (#9784) ... **What this PR does / why we need it**: Add support for `metricRelabelings` in the `ServiceMonitor` resources when monitoring Loki. **Checklist** - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [x] Documentation added - [ ] Tests updated - [x] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [x] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](d10549e3ec) --------- Signed-off-by: Tiago Posse <tiagoposse@gmail.com> Co-authored-by: Tiago Posse <tiago.posse@hi.health>	3 years ago
manohar-koukuntla	999259a4d3	jsonnet: Add TSDB shipper configuration flag (#9790) ... This commit adds the TSDB equivalent of `use_boltdb_shipper` flag to be able to configure `tsdb_shipper` section. Fixes #9789	3 years ago
Ashwanth	f9fc1df4c0	logfmt: add --strict flag to enable strict parsing, perform nostrict parsing by default (#9626) ... **What this PR does / why we need it**: Introduces `--strict` flag for logfmt parser to enable strict parsing. With strict parsing enabled, logfmt parser stops scanning the log line and returns early with an error if it encounters any poorly formatted key=value pair. This pr updates the parser to perform non-strict parsing by default where it skips over the invalid tokens. This offers the flexibility to parse semi-structured log lines. Also adds `--keep-empty` flag to retain standalone keys(with empty value) as labels with value set to empty string **Special notes for your reviewer**: **Checklist** - [X] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [X] Documentation added - [X] Tests updated - [x] `CHANGELOG.md` updated - [x] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` --------- Co-authored-by: Michel Hollands <42814411+MichelHollands@users.noreply.github.com>	3 years ago
Owen Diehl	e2a63e582c	adds tsdb-max-bytes-per-shard limit (#9811) ... Adds the per tenant limit `tsdb-max-bytes-per-shard` which is used in configuring the shard size for tsdb subqueries. This effectively gives control over how big subqueries should be (assuming they're shardable). The default is no different (`600MB`) than the previously hardcoded initial value. This should help us iterate to find optimal shard sizes to improve operations in the long term.	3 years ago
Victor Login	30fff3cd65	upgrade version nginxinc/nginx-unprivileged to 1.23 (#8984) ... **What this PR does / why we need it**: This PR upgrade nginxinc/nginx-unprivileged to 1.23 **Which issue(s) this PR fixes**: Fixes #<issue number> **Special notes for your reviewer**: **Checklist** - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` Co-authored-by: Travis Patterson <travis.patterson@grafana.com>	3 years ago
Alexandru Gologan	f785faa820	Helm: add additionalRuleLabels to PrometheusRule alerts (#9020) ... **What this PR does / why we need it**: Adds the ability to specify other labels on the Prometheus alerts. Specifying: ```yaml monitoring: rules: additionalRuleLabels: custom_label: custom_value ``` will add it to all alerts: ```yaml - alert: LokiRequestErrors annotations: ... expr: ... for: 15m labels: severity: critical custom_label: custom_value ``` This approach is inspired by: [kube-prometheus-stack/rules](https://github.com/prometheus-community/helm-charts/blob/main/charts/kube-prometheus-stack/templates/prometheus/rules-1.14/general.rules.yaml) Though in their case they run [sync_prometheus_rules.py](https://github.com/prometheus-community/helm-charts/blob/main/charts/kube-prometheus-stack/hack/sync_prometheus_rules.py) to create the `tpl`. I can understand if this is not the approach (manual) you want to take as editing the tpl adding the labels everywhere and escaping go tpl `{{ }}` is quite painful. **Which issue(s) this PR fixes**: N/A **Special notes for your reviewer**: **Checklist** - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [x] Documentation added - [x] Tests updated - [ ] `CHANGELOG.md` updated - [x] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	3 years ago
Nyaundi Brian	5add792487	Update _index.md (#9028) ... **What this PR does / why we need it**: Typo fix **Which issue(s) this PR fixes**: None	3 years ago
Hans Kristian Flaatten	86d943d8f4	helm: add updateStategy for canary pods (#9760) ... **What this PR does / why we need it**: Allows configuring Canary DaemonSet rollout. **Which issue(s) this PR fixes**: N/A **Special notes for your reviewer**: **Checklist** - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [x] Documentation added - [x] Tests updated - [x] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [x] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [x] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](d10549e3ec)	3 years ago
Steven Miller	e1002624bd	Helm: Add priorityClassName for Grafana Agent and Loki Canary (#8506) ... **What this PR does / why we need it**: - Allow configuring PriorityClassName for the GrafanaAgent daemonset - Allow configuring PriorityClassName for the loki canary daemonset without also configuring for read deployment - Helpful when an administrator wants to prioritize logging on their nodes **Special notes for your reviewer**: - Used _coalese_ style like it was already in the case of loki canary - For loki canary, specifying on the canary takes higher precedence than on the reader, but lower than global - For grafana agent, global takes precedence over grafana agent to match the existing style, but maintainers might consider flipping that around so that most-specific is also most-prioritized **Checklist** - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide --------- Co-authored-by: Michel Hollands <42814411+MichelHollands@users.noreply.github.com>	3 years ago
Travis Patterson	806674fdaa	Add log-volume feature flag (#9762) ... Adds a feature flag for use with the new log-volume endpoints so associated features can be rolled out incrementally.	3 years ago
Christian Haudum	ce81895241	Add shuffle sharding to index gateway (#9710) ... This PR introduces shuffle sharding of the index files across index gateways where the shuffle sharding key is the tenant ID. Instead of having a global replication factor on the index gateways which affects all tenants the same, whether they are small or large tenants, the shuffle sharding allows to specify a shard factor per tenant (on top of the replication factor). This a) reduces the amount of indexes that are downloaded on individual index gateways in case of a high replication factor (which is currently the de-factor standard to deal with high load) and b) allows to scale tenants individually. The shuffle sharding algorithm is part of dskit's ring implementation and is not part of this commit. If you have set a high RF for the index gateways, it is required to reduce the RF and instead set the shard factor for the tenants. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	3 years ago
Danny Kopping	ccf44f8840	Adding 3 existing maintainers, and our newest one: Ashwanth! (#9741) ... Signed-off-by: Danny Kopping <danny.kopping@grafana.com> Co-authored-by: Susana Ferreira <ssncferreira@gmail.com>	3 years ago
sarah-spang	6157e28998	Loki ECS Client: Update Name in Options config (#9616) ... Configuration options show "Name": "grafana-loki", however, fluentbit specifies this should just be "loki" A few users have reported receiving an error when using the documented "Name" option, but it works as expected when using the Name specified in the fluentbit docs: Reference: https://docs.fluentbit.io/manual/pipeline/outputs/loki#fluent-bit-+-grafana-cloud	3 years ago
Salva Corts	61d407b4bb	Add keep stage to LogQL (#9693) ... **What this PR does / why we need it**: This PR adds a new `keep` stage to LogQL. It allows filtering specific labels and helps to avoid hitting the _max streams per query_ limit by keeping only specific labels in the label set. It works the opposite of [drop stage](https://grafana.com/docs/loki/latest/logql/log_queries/#drop-labels-expression). **Special notes for your reviewer**: We do not drop [special error labels](edc6b0bff7/pkg/logqlmodel/error.go (L18-L20)) added by Loki at query time. Users will have to explicitly drop them by using the drop stage. **Checklist** - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [x] Documentation added - [x] Tests updated - [x] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](d10549e3ec) --------- Co-authored-by: Sandeep Sukhani <sandeep.d.sukhani@gmail.com> Co-authored-by: J Stickler <julie.stickler@grafana.com>	3 years ago
J Stickler	32320736db	TOC Sync: Move files under Get Started. (#9717) ... **Which issue(s) this PR fixes**: Fixes #8711 **Special notes for your reviewer**: Part of the Database products table of contents alignment project. Moves Getting Started and Fundamentals files under Get Started in TOC. Updates front matter for the topic (headings, page weights, descriptions, aliases to redirect from old URL to new URL) Fixes links broken by the move.	3 years ago
J Stickler	11e2375e6d	Removing 'please' from the docs. (#9658) ... **Which issue(s) this PR fixes**: Fixes #8922 **Special notes for your reviewer**: This is to conform with our Style Guide. https://developers.google.com/style/tone#politeness-and-use-of-please	3 years ago
J Stickler	c2f96221dc	Update that Loki accepts out-of-order rewrites by default. (#9670) ... **Which issue(s) this PR fixes**: Fixes #7477	3 years ago
Jan Jansen	8abe080fd6	Promtail: improve behavior of partial lines (#9508) ... **What this PR does / why we need it**: I started to see OOM after upgrading to a promtail version with #8497. So played around to make partial line behavior more stable. You can now truncate partial lines and configure max partial lines buffer. **Which issue(s) this PR fixes**: Fixes #<issue number> **Special notes for your reviewer**: **Checklist** - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [x] Documentation added - [x] Tests updated - [x] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` Signed-off-by: Jan Jansen <jan.jansen@gdata.de> Co-authored-by: Michel Hollands <42814411+MichelHollands@users.noreply.github.com>	3 years ago
Jan-Otto Kröpke	14f96c0c7b	Add loki.memberlistConfig to fully control the memberlist configuration (#9646) ... **What this PR does / why we need it**: Add loki.memberlistConfig to fully control the memberlist configuration On singleBinary with replica 1, memberlist isn't need and its counter-productive since it delays the start of the loki binary. At the moment, it isn't possible to remove the memberlist defaults. **Which issue(s) this PR fixes**: Fixes #<issue number> **Special notes for your reviewer**: **Checklist** - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [x] Documentation added - [ ] Tests updated - [x] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [x] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](d10549e3ec)	3 years ago
Jack Baldry	8391e557b1	Update doc-validator to v3.0.0 (#9666) ... ## 3.0.0 ### Added - Structured output for use with [`reviewdog`](https://github.com/reviewdog/reviewdog). You can achieve the original error output by piping the output to the following `jq` expression: `jq -r '"ERROR: \(.location.path):\(.location.range.start.line):\(.location.range.start.column): \(.message)"'`. - Suggestions for simple link fixes. In GitHub, [`reviewdog`](https://github.com/reviewdog/reviewdog) comments these suggestions for convenient replacement. - Support for anchors referring to repeated headings. In the case that a page has multiple headings that share the same text, the renderer appends a zero indexed, numbered suffix to the identifier. For example, when there are two headings that are both "Heading text", the first anchor identifier is `heading-text` and the second is `heading-text-1`. - Error when running `doc-validator` on no files. **Notes for reviewer:** This also updates the `make-docs` procedure. Please ensure you can run `make docs` from the `docs/` directory before approving and merging this PR. --------- Signed-off-by: Jack Baldry <jack.baldry@grafana.com>	3 years ago
Terje Sannum	986d4ca756	Helm: add support for additional labels on loki-canary pods (#9673) ... Add support for setting additional labels on the loki-canary pods --------- Signed-off-by: Terje Sannum <terje.sannum@nav.no>	3 years ago
Salva Corts	73ac208981	Improve docs for empty value in cache compression config (#9649) ... **What this PR does / why we need it**: Follow up PR for https://github.com/grafana/loki/pull/9535#discussion_r1218167670 **Checklist** - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [x] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](d10549e3ec)	3 years ago
Christophe Collot	f2394355d8	feat(lambda-promtail): add cloudfront log file ingestion support (#9573) ... **What this PR does / why we need it**: This PR enables ingesting logs from Cloudfront log files stored in s3 (batch). The current setup only supports streaming Cloudfront logs through AWS Kinesis, whereas this PR implements the same flow as for VPC Flow logs, Load Balancer logs, and Cloudtrail logs (s3 --> SQS (optional) --> Lambda Promtail --> Loki) **Special notes for your reviewer**: + The Cloudfront log file format is different from the already implemented services, meaning we had to build yet another regex. AWS never bothered making all services follow the same log file naming convention but the "good" thing is that it's now very unlikely they will change it in the future. + The Cloudfront file name does not have any mention of the AWS account or the time of log it contains, it means we have to infer the log type from the filename format instead of finding the exact string "cloudfront" in the filename. This is why in `getLabels`, if no `type` parameter is found in the regex, we use the key corresponding to the name of the matching parser. + I introduced a new `parser` struct to group together several parameters specific to a type of log (and avoid relying too much on map key string matching and / or if statements for specific use cases) + I've been successfully running this code in several AWS environments for days. + I corrected a typo from my previous PR #9497 (wrong PR number in Changelog.md) **Checklist** - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [x] Documentation added - [x] Tests updated - [x] `CHANGELOG.md` updated - [x] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [x] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](d10549e3ec) --------- Co-authored-by: Michel Hollands <42814411+MichelHollands@users.noreply.github.com>	3 years ago
Salva Corts	c6fbff26e1	Add config to avoid caching stats for recent data (#9537) ... **What this PR does / why we need it**: When we query the stats for recent data, we query both the ingesters and the index gateways for the stats. ebdb2b1800/pkg/storage/async_store.go (L112-L114) ebdb2b1800/pkg/storage/async_store.go (L126-L127) Then we merge all the responses, which means summing up all the stats ebdb2b1800/pkg/storage/async_store.go (L157-L158) ebdb2b1800/pkg/storage/stores/index/stats/stats.go (L23-L26) Because we have a replication factor of 3, this means that we will get the stats from the ingesters repeated up to 3 times, hence inflating the stats. In the stats cache, we store the stats for a given matcher set for the whole day, then we extract the stats from the cache by the factor of time from the request that is stored in the cache: 336283acad/pkg/querier/queryrange/index_stats_cache.go (L33) 336283acad/pkg/querier/queryrange/index_stats_cache.go (L40) Inflated stats for recent data will be cached, so subsequent stats extracted from the cache will be inflated regardless of the time. This PR adds a new per-tenant limit `max_stats_cache_freshness` to not cache requests with an end time that falls within Now minus this duration. Here's a scenario illustrating this. The graphs below show the bytes stats queried in the sharding middleware. We are running a log filter query that won't match any log, every 5 seconds with a length of 3h.  As can be seen, after enabling the stats cache and configuring`do_not_cache_request_within` to not cache stats for requests within 30m, the bytes stats used in the sharding middleware stopped increasing. In both cases the stats cache hit ration was 100%.  **Special notes for your reviewer**: - Blocked by https://github.com/grafana/loki/pull/9535 - Note that this PR doesn't fix the root issue of inflated stats form the ingesters, but rather buys us some time to work on that. **Checklist** - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [x] Documentation added - [x] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](d10549e3ec)	3 years ago
Michel Hollands	22779e13c5	Fix date template function with epoch times (#8886) ... **What this PR does / why we need it**: Adds new toUnixEpoch... functions to convert from a string with a Unix/Epoch time to an integer that can be used in the existing `toDate` function. Note that these are the opposites of some of the functions introduced in https://github.com/grafana/loki/pull/8774. **Which issue(s) this PR fixes**: Fixes https://github.com/grafana/loki/issues/8624. **Special notes for your reviewer**: **Checklist** - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [X] Documentation added - [X] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` --------- Signed-off-by: Michel Hollands <michel.hollands@grafana.com>	3 years ago
Bruno FERNANDO	1b410dbc01	feat(promtail): add CF ClientRequestSource field (#9669) ... **What this PR does / why we need it**: Hey folks 👋 Little contribution here to add a useful log field for cloudflare users. Indeed I add the [ClientRequestSource field](https://developers.cloudflare.com/logs/reference/clientrequestsource/ ) which is pretty useful when debugging some specific traffic handled by cloudflare Extra: Since I was on the documentation I fixed an indentation issue that I spotted Don't hesitate to reach me if you have any questions Cheers 😉 **Which issue(s) this PR fixes**: Fixes #<issue number> **Special notes for your reviewer**: Loki rocks 🚀 **Checklist** - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [x] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](d10549e3ec)	3 years ago
Gregor Zeitlinger	b1917a6178	add "alignLeft" and "alignRight" functions (#9672) ... Fixes https://github.com/grafana/loki/issues/9667	3 years ago