1144 Commits (0f139e2cb35daa9e56f8e0bbdaca472dbf802201)

Author	SHA1	Message	Date
Susana Ferreira	0f139e2cb3	Update log queries regex documentation (#8002) ... **What this PR does / why we need it**: Update log queries regex documentation with a clarification on the difference between the newline character in: * label matching operator: `.*` does not match newlines * line filter expressions: `.*` matches newlines **Checklist** - [x] Documentation updated	3 years ago
Michel Hollands	59eff5ca58	Add count to template functions (#7951) ... **What this PR does / why we need it**: Add a `count` template function to count occurrences of a substring. For example if `label="abc abd abc"` ``` {{ .label | count "abc" }} ``` would return 2. Signed-off-by: Michel Hollands <michel.hollands@grafana.com> Co-authored-by: Danny Kopping <dannykopping@gmail.com>	3 years ago
Susana Ferreira	d086c237df	Add missing period_config root block to doc generator (#7981) ... **What this PR does / why we need it**: Update doc generator tool to validate if slice element is a root block. Add missing `period_config` root block to doc generator tool. **Special notes for your reviewer**: **Checklist** - [ ] Documentation updated	4 years ago
Danny Kopping	a4f306399a	Add store & cache download statistics (#7982)	4 years ago
Javier Vela	fe69443a05	Fix links to configuration page (#7164) ... **What this PR does / why we need it**: Fix links to configuration page: - kafka - Cloudflare - Kubernetes Discovery - Windows Event Log - Heroku Drain - HTTP client options Signed-off-by: Javier Vela <fjvela@gmail.com>	4 years ago
Sandeep Sukhani	9e7ac3d853	fix query-frontend request load balancing when using k8s service (#7966) ... **What this PR does / why we need it**: We noticed an imbalance in the requests sent to the Query Frontend pods using `query-frontend` k8s service. This seems to be caused by `query-frontend` being a headless service that resolves to each QF pod IP and leaves it up to the client to load balance the requests as mentioned [here](https://kubernetes.io/docs/concepts/services-networking/service/#headless-services). This PR fixes the issue by creating two separate services for pod IP discovery and load balancing of queries: * `query-frontend` to be used for load balancing incoming Loki queries. * `query-frontend-headless` to be used for discovering QF pod IPs from queriers to connect as workers. **Checklist** - [x] `CHANGELOG.md` updated - [x] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	4 years ago
Michel Hollands	cc4ab7487a	Fix typo (#7956) ... **What this PR does / why we need it**: Fix docs as reported by user **Which issue(s) this PR fixes**: Fixes https://github.com/grafana/loki/issues/7893 Signed-off-by: Michel Hollands <michel.hollands@grafana.com>	4 years ago
mfordjody	b6ae4b4b73	fix error helm install monolithic path (#7948)	4 years ago
Karsten Jeschkies	0f3eecd209	Show help tooltip on hover in sizing tool. (#7919)	4 years ago
Susana Ferreira	86d33a4e4a	Add CI step to check generated documentation to check pipeline (#7938) ... **What this PR does / why we need it**: Follow up from PR: https://github.com/grafana/loki/pull/7916 to add a CI step to check if the configuration flag documentation file `doc/sources/configuration/_index.md` was generated successfully. Requirements: * CI `check` pipeline should fail if the documentation file is updated manually * CI `check` pipeline should fail if registration flags are changed in the code, but the documentation file is not generated. **Which issue(s) this PR fixes**: Fixes https://github.com/grafana/loki-private/issues/83 **Special notes for your reviewer**: **Checklist** - [ ] Reviewed the `CONTRIBUTING.md` guide - [ ] Documentation added - [ ] Tests updated - [x] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	4 years ago
Susana Ferreira	e07fe7c338	Deprecate CLI flag -ruler.wal-cleaer.period in favor of CLI flag -ruler.wal-cleaner.period (#7937) ... **What this PR does / why we need it**: Deprecate CLI flag `-ruler.wal-cleaer.period` in favor of CLI flag `-ruler.wal-cleaner.period`. **Which issue(s) this PR fixes**: Fixes https://github.com/grafana/loki/pull/7916#issuecomment-1348276269 and https://github.com/grafana/loki/pull/7916#discussion_r1047043534 **NOTE:** correctly registring the `ruler.wal-cleaner` configuration flags is not a breaking change and doesn't require an upgrade guide, because the ruler WAL cleaner initialization code already guarantees that the default values are used: c71620ae94/pkg/ruler/storage/cleaner/cleaner.go (L76-L101) **Special notes for your reviewer**: **Checklist** - [x] Reviewed the `CONTRIBUTING.md` guide - [x] Documentation added - [ ] Tests updated - [x] `CHANGELOG.md` updated - [x] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	4 years ago
Kaviraj Kanagaraj	cbd6ec15ce	feat(logql): Supporting prettifying LogQL expressions (#7906) ... **What this PR does / why we need it**: Changes 1. Added `Pretty()` method to `Expr` interface. So that every expression would know how to render themselfs as pretty :) 2. Implemented `Pretty()` for every single LogQL expressions 3. Exposed `/api/v1/format_query` endpoint that takes `query` argument and returns formatted version. 4. Integerated into `logcli`. echo `'<query>' | logcli fmt` would format the query. Why? 1. Readability, helpful in debugging and **more importantly, to understand the execution flow** example **before** ``` quantile_over_time(0.99,{container="ingress-nginx",service="hosted-grafana"}| json| unwrap response_latency_seconds| __error__=""[1m]) by (cluster) ``` ``` sum(rate({job="loki", namespace="loki", cluster="loki-dev-us"} |= "err" [5m])) + sum(rate({job="loki-dev", namespace="loki", cluster="loki-dev-eu"}|logfmt | level != "info" [5m])) / sum(rate({job="loki-prod", namespace="loki", cluster="loki-prod-us"} |logfmt | level="error"[5m])) ``` ``` label_replace(rate({job="api-server",service="a:c"}|= "err" [5m]), "foo", "$1", "service", "(.*):.*") ``` **after** ``` quantile_over_time( 0.99, {container="ingress-nginx", service="hosted-grafana"} | json | unwrap response_latency_seconds | __error__="" [1m] ) by (cluster) ``` ``` sum( rate( {job="loki", namespace="loki", cluster="loki-dev-us"} |= "err" [5m] ) ) + sum( rate( {job="loki-dev", namespace="loki", cluster="loki-dev-eu"} | logfmt | level!="info" [5m] ) ) / sum( rate( {job="loki-prod", namespace="loki", cluster="loki-prod-us"} | logfmt | level="error" [5m] ) ) ``` ``` label_replace( rate({job="api-server", service="a:c"} |= "err"[5m]), "foo", "$1", "service", "(.*):.*" ) ``` You can find more examples in the `prettier_test.go` Future plans * Integerate into LogQL analyzer * Integrate into Grafana UI. **Which issue(s) this PR fixes**: Fixes # NA **Special notes for your reviewer**: This whole idea was inspired from last [PromCon lighting talk](https://youtu.be/pjkWzDVxWk4?t=24469) **Checklist** - [x] Reviewed the `CONTRIBUTING.md` guide - [x] Documentation added - [x] Tests updated - [x] `CHANGELOG.md` updated Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> Co-authored-by: Christian Haudum <christian.haudum@gmail.com> Co-authored-by: Christian Simon <simon@swine.de>	4 years ago
Susana Ferreira	68a4d3271a	Fix documentation typos from #7916 (#7934) ... **What this PR does / why we need it**: Fix documentation typos from https://github.com/grafana/loki/pull/7916#pullrequestreview-1217192817 **Which issue(s) this PR fixes**: Fixes https://github.com/grafana/loki/pull/7916#pullrequestreview-1217192817 **Special notes for your reviewer**: **Checklist** - [ ] Reviewed the `CONTRIBUTING.md` guide - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	4 years ago
Susana Ferreira	f93b91bfb5	Add configuration documentation generation tool (#7916) ... **What this PR does / why we need it**: Add a tool to generate configuration flags documentation based on the flags properties defined on registration on the code. This tool is based on the [Mimir doc generation tool](https://github.com/grafana/mimir/tree/main/tools/doc-generator) and adapted according to Loki configuration specifications. Prior to this PR, the configuration flags documentation was dispersed across two sources: * [_index.md](5550cd65ec/docs/sources/configuration/_index.md) * configuration flags registration in the code This meant that there was no single source of truth. In this PR, the previous `_index.md` file is replaced with the new file generated by the tool. The next step includes adding a CI step that validates if the _index.md file was generated according to the flags settings. This will be done in a follow-up PR. **NOTE:** this is not a documentation update PR. Apart from some minor typo fixes, the documentation changes on the code, were copied from the `_index.md` file. **Which issue(s) this PR fixes**: Fixes https://github.com/grafana/loki-private/issues/83 **Special notes for your reviewer**: Files: * [docs/sources/configuration/index.template](5550cd65ec/docs/sources/configuration/index.template): template used to generate the final configuration file * [/docs/sources/configuration/_index.md](c32e5d0acb/docs/sources/configuration/_index.md): file generated by tool * `loki/pkg` directory files updated with up-to-date documentation from `_index.md` file * [tools/doc-generator](5550cd65ec/tools/doc-generator) directory with documentation generation tool. **Checklist** - [ ] Reviewed the `CONTRIBUTING.md` guide - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	4 years ago
Kaviraj Kanagaraj	4768b6d997	doc(api): Default value for `delete_ring_tokens` on `/ingester/shutdown` endpoint (#7921) ... Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> **What this PR does / why we need it**: PR documents the default value for `delete_ring_tokens` params on `/ingester/shutdown` endpoint. **Which issue(s) this PR fixes**: Fixes #NA **Special notes for your reviewer**: **Checklist** - [x] Documentation added Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com>	4 years ago
Tobias Wolf	79e0d96978	Helm: Decouple the Canary from self-monitoring (#7757) ... **What this PR does / why we need it**: The Canary has no dependency on the Grafana Agent-based self-monitoring. This PR decouples that dependency, allowing the canary to be deployed with the other self-monitoring features. **Which issue(s) this PR fixes**: Fixes #7364	4 years ago
Karsten Jeschkies	c4eb8c87ed	Remove duplicate sizing tool (#7917) ... **What this PR does / why we need it**: The sizing tool was checked in twice. Also, there was a function error in fetching the nodes. **Checklist** - [ ] Reviewed the `CONTRIBUTING.md` guide - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	4 years ago
Karsten Jeschkies	9b90aad7ce	Fix sizing tool. (#7899) ... **What this PR does / why we need it** The current sizing tool is broken because it would not allow cross origin requests. See LogQL Analyzer (https://github.com/grafana/loki/blob/main/docs/sources/logql/analyzer/script.js#L115) for the proper way to call its API.	4 years ago
Justin Avery Rexroad	7546c4041e	Fix typo in helm install monolithic docs (#7909) ... **What this PR does / why we need it**: Add missing letter to end of word. **Special notes for your reviewer**: **Checklist** - [ ] Reviewed the `CONTRIBUTING.md` guide - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	4 years ago
Maxim Ivanov	b773a7a3cf	promtail: ratelimiting by label (#7597) ... Limit stage can now takes optional `by_label_name` param, which tracks distinct values of that label and ratelimit entries independently. Log lines without expected label are not considered for ratelimiting and passed to the next pipeline stage.	4 years ago
Trevor Whitney	99b7c14e09	Prepare changelog and release notes for 2.7.1 (#7895) ... **What this PR does / why we need it**: Prepares changelog and release notes for 2.7.1 release. Co-authored-by: Dylan Guedes <djmgguedes@gmail.com>	4 years ago
Karsten Jeschkies	95ec710168	Introduce sizing tool to documentation page. (#7830) ... **What this PR does / why we need it**: The sizing tool should help to find a good cluster sizing when starting out with Loki. It's a dynamic form withing the documentation.  This pull request will be followed by help on hover. **Special notes for your reviewer**: **Checklist** - [ ] Reviewed the `CONTRIBUTING.md` guide - [x] Documentation added - [x] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` Co-authored-by: Karsten Jeschkies <karste.jeschkies@grafana.com>	4 years ago
Felix Ehrenpfort	8423f56537	fix small typo (#7874) ... This change fixes a small typo in case of a missing space.	4 years ago
Chris James	527ee12220	Fixing a typo in configuration.md (#7878) ... **What this PR does / why we need it**: Fixes a typo in configuration.md, improves reader clarify and confidence **Which issue(s) this PR fixes**: Fixes #<issue number> **Special notes for your reviewer**: **Checklist** - [x] Reviewed the `CONTRIBUTING.md` guide - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	4 years ago
Steven Dungan	fee2374532	Update v2-7.md (#7857)	4 years ago
Robert Fratto	679c9e122f	rerun make drone so the newest build image is used (#7849) ... Follow up on #7848, where the Drone config needed to be updated as well	4 years ago
Trevor Whitney	652e216a85	[helm] Bump helm Loki and GEL version (#7760) ... **What this PR does / why we need it**: Bumps Loki to 2.7.0 and GEL to 1.6.0 in the helm chart.	4 years ago
cadrake	ea9ad336dc	Loki: Modified heroku drain target to make any url query parameters available as labels (#7619) ... **What this PR does / why we need it**: This PR reads the url query parameters included in Heroku drain POST request and creates `__param_<name>` labels similar to how the parameters work with prometheus metrics and metric endpoints as a way to allow setting the true application name when ingesting logs from a Heroku drain. **Which issue(s) this PR fixes**: Fixes #<issue number> **Special notes for your reviewer**: The existing `app` field coming from Heroku only specifies application source (either the application itself or Heroku) and thus is insufficient for uniquely identifying a log lines source. **Checklist** - [x] Reviewed the `CONTRIBUTING.md` guide - [x] Documentation added - [x] Tests updated - [x] `CHANGELOG.md` updated - [x] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	4 years ago
Anders Bennedsgaard	b5563cee08	[Helm] Remove unused value and set default values explicitly (#7576) ... **What this PR does / why we need it**: `monitoring.alerts` are not used in the Loki Helm chart. This removes it. Furthermore, some values are referenced in the templates, but not used in the values file. This adds the default values. **Which issue(s) this PR fixes**: Fixes #7575 **Special notes for your reviewer**: https://github.com/AndersBennedsgaard/loki/blob/main/production/helm/loki/templates/single-binary/pdb.yaml#L15 should probably also be updated since `podDisruptionBudget` is not referenced in the values file. However, I am unsure what default values should be used. https://github.com/AndersBennedsgaard/loki/blob/main/production/helm/loki/templates/networkpolicy.yaml#L124 should probably also be updated since the `loki.rulerSelectorLabels` is not a valid helper function. However, I am unsure which selector labels to use. Especially since it seems like `ruler` is not used. Perhaps any reference to `ruler` should be removed? **Checklist** - [x] Reviewed the `CONTRIBUTING.md` guide - [x] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	4 years ago
Jan-Otto Kröpke	5dc60567da	[Helm] Add support for azure blob storage (#7500) ... **What this PR does / why we need it**: Support Azure Blob Storage in helm chart. See https://grafana.com/docs/loki/latest/storage/#azure-storage-account-1 **Which issue(s) this PR fixes**: **Special notes for your reviewer**: **Checklist** - [ ] Reviewed the `CONTRIBUTING.md` guide - [x] Documentation added - [ ] Tests updated - [x] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` Signed-off-by: Jan-Otto Kröpke <jok@cloudeteer.de>	4 years ago
Sandeep Sukhani	1410808ee9	use grpc for communicating with compactor for query time filtering of data requested for deletion (#7804) ... **What this PR does / why we need it**: Add grpc support to compactor for getting delete requests and gen number for query time filtering. Since these requests are internal to Loki, it would be good to use grpc instead of HTTP same as all the internal requests we do in Loki. I have added a new config for accepting the grpc address of the compactor. I tried having just the existing config and detecting if it is a grpc server, but it was hard to do it reliably, considering the different deployment modes we support. I think it is safe to keep it the same and eventually deprecate the existing config. **Checklist** - [x] Documentation added - [x] Tests updated - [x] `CHANGELOG.md` updated	4 years ago
Karsten Jeschkies	691b8be6c7	Provide Terraform script for setting up S3 and document role based access. (#7656) ... **What this PR does / why we need it**: Role based access to S3 on an EKS cluster via service account annotation was not documented well and let to some confusion. This change documents how a role based accessis achieved with providing an AWS secret access key and access key id. Thus there is no change of accidentally leaking credentials. A Terraform module is provided under `production/terraform/modules/s3` that provisions and S3 bucket and IAM role and policy. **Which issue(s) this PR fixes**: Fixes #7279 **Special notes for your reviewer**: **Checklist** - [ ] Reviewed the `CONTRIBUTING.md` guide - [x] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` Co-authored-by: Pablo Angulo <pablo.angulo@grafana.com>	4 years ago
Teddy Andrieux	217a914f98	helm/loki: Allow to override default loki `target` (#7791) ... **What this PR does / why we need it**: Add the ability to change the Loki `target` argument. E.g. one might want to use the single-binary which has target `all` but also the `table-manager` which is not part of it. So one can override the target with `all,table-manager`. **Which issue(s) this PR fixes**: Fixes #7788 **Special notes for your reviewer**: **Checklist** - [x] Reviewed the `CONTRIBUTING.md` guide - [x] `CHANGELOG.md` updated	4 years ago
Marc Sensenich	409c13131c	docs(embedded-cache): config uses embedded_cache (#7797)	4 years ago
Danny Kopping	6c8d04fe78	Completing step 2 of updating loki-build-image (#7796)	4 years ago
Danny Kopping	bd110d2b92	Improvements to loki-build-image build & release process (#7795)	4 years ago
Duc Tran	71b21c924e	[Docs] Typo: Fix build path of promtail binary (#7776)	4 years ago
Danny Kopping	a63ad06509	Querier/Ruler: query blocker (#7785) ... Block malicious or expensive queries using a per-tenant runtime configuration.	4 years ago
Vladyslav Diachenko	c6b55e7512	[docs] removed static URL to LogQL Analyzer API (#7773) ... added logic to determine logql-analyzer version dynamically based on the value selected in docs version dropdown. Adding this logic to the docs, allows us to skip the step on release when we manually update the link to the new version of LogQL Analyzer	4 years ago
Wade Fitzpatrick	8a5c8f0a3d	Add closing code block backticks (#7772)	4 years ago
Trevor Whitney	ea4a764143	[helm] Fix incorrect ingress paths when using single binary deployment (#7530) ... **What this PR does / why we need it**: The helm ingress template was copied over from the simple scalable chart and does not work when deploying the chart in single binary configuration. This fixes that by adding a set of single binary routes, and conditionally configuring the ingress resource based on deployment type. **Which issue(s) this PR fixes**: Fixes #7080 **Special notes for your reviewer**: **Checklist** - [ ] Reviewed the `CONTRIBUTING.md` guide - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	4 years ago
Trevor Whitney	1aa7991828	Update release instructions (#7663) ... We usually tag releases from the HEAD of the release branch	4 years ago
Teddy Andrieux	220770c693	helm/loki: Allow to provide selector for Loki persistence volume (#7743)	4 years ago
Karsten Jeschkies	54320e827c	Correct Helm documentation aliases. (#7537) ... **What this PR does / why we need it**: #7034 introduced a regression in the documentation. This changes corrects the aliases of the Helm documentation pages. **Checklist** - [ ] Reviewed the `CONTRIBUTING.md` guide - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	4 years ago
Kaviraj Kanagaraj	3e90d5f972	docs(v2.7.0): Minor tweaks on 2.7 release notes (#7672) ... Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> **What this PR does / why we need it**: 1. Tweak entry about `fifocache` 2. Also fix upgrade guide ref on release notes. `#260 -> #270` **Which issue(s) this PR fixes**: Fixes #<issue number> **Special notes for your reviewer**: **Checklist** - [x] Documentation added - [x] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com>	4 years ago
Kaviraj Kanagaraj	7073ea8702	docs(embedded-cache): Add missing docs on config page. (#7684) ... **What this PR does / why we need it**: Adds `embedded-cache` config under `cache_config` section. Also adds message about deprecation of `fifocache` **Which issue(s) this PR fixes**: Fixes #7677 **Special notes for your reviewer**: **Checklist** - [x] Documentation added - [x] `CHANGELOG.md` updated	4 years ago
Dominik Philp	3039ca06f4	Docs: Fix typo in helm scalable installation documentation, object storage section (#7676) ... **What this PR does / why we need it**: fixes a small typo in the helm install documentation regarding object storage. In a reference to more documentation it says 'otehr' instead of 'other' **Which issue(s) this PR fixes**: n/a **Special notes for your reviewer**: n/a **Checklist** - [x] Reviewed the `CONTRIBUTING.md` guide - [x] Documentation added - [x] Tests updated - [x] `CHANGELOG.md` updated - [x] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	4 years ago
Dan Poltawski	705acd3462	docs/upgrading: Add a note about windows event log fix (#7668) ... **What this PR does / why we need it**: Add note to upgrade guide for a bug fix with log query impact **Which issue(s) this PR fixes**: Fixes #7667	4 years ago
Trevor Whitney	ed5ecd2bdf	Add 2.7 release notes (#7661) ... **What this PR does / why we need it**: Adds 2.7 release notes Co-authored-by: JordanRushing <rushing.jordan@gmail.com>	4 years ago
Trevor Whitney	04c7fbb531	Add config changes to upgrade guide (#7662) ... **What this PR does / why we need it**: This add config differences I noticed when running the config diff tool between 2.6 and 2.7 Co-authored-by: JordanRushing <rushing.jordan@gmail.com>	4 years ago