apitech/loki - loki - Gitea: Git with a cup of tea

Commit Graph

Author	SHA1	Message	Date
Raul Gonzales	7d5475541c	promtail: enable structured_metadata convert labels (#10752 ) What this PR does / why we need it: There are several labels that can be extracted by the supports service discovery providers, however the structured metadata wouldn't allow to parse labels to it. This PR allows the users to update the configuration to read from labels discovered, down below are some of the metadata from a pod, that could be of interested of anyone to have that pushed to Loki without creating several streams due to either high-carnality or proliferation of idle chunks: ``` __address__="10.0.165.166:3100" __meta_kubernetes_namespace="loki" __meta_kubernetes_pod_annotation_checksum_config="f008ef39c4f3230c5e153c6e9fde51a678e6f0b1087b3be7d91b8142a03dbe7c" __meta_kubernetes_pod_annotation_kubectl_kubernetes_io_default_container="loki" __meta_kubernetes_pod_annotationpresent_checksum_config="true" __meta_kubernetes_pod_annotationpresent_kubectl_kubernetes_io_default_container="true" __meta_kubernetes_pod_container_id="containerd://e6aef5c39c3fe806d39ea536b198c8c39cdee01d23141fed91e8fe1573cc5e08" __meta_kubernetes_pod_container_image="docker.io/grafana/loki:2.9.1" __meta_kubernetes_pod_container_init="false" __meta_kubernetes_pod_container_name="loki" __meta_kubernetes_pod_container_port_name="http-metrics" __meta_kubernetes_pod_container_port_number="3100" __meta_kubernetes_pod_container_port_protocol="TCP" __meta_kubernetes_pod_controller_kind="StatefulSet" __meta_kubernetes_pod_controller_name="loki-write" __meta_kubernetes_pod_host_ip="10.249.129.191" __meta_kubernetes_pod_ip="10.0.165.166" __meta_kubernetes_pod_label_app_kubernetes_io_component="write" __meta_kubernetes_pod_label_app_kubernetes_io_instance="loki" __meta_kubernetes_pod_label_app_kubernetes_io_name="loki" __meta_kubernetes_pod_label_app_kubernetes_io_part_of="memberlist" __meta_kubernetes_pod_label_controller_revision_hash="loki-write-75847dc945" __meta_kubernetes_pod_label_statefulset_kubernetes_io_pod_name="loki-write-1" __meta_kubernetes_pod_labelpresent_app_kubernetes_io_component="true" __meta_kubernetes_pod_labelpresent_app_kubernetes_io_instance="true" __meta_kubernetes_pod_labelpresent_app_kubernetes_io_name="true" __meta_kubernetes_pod_labelpresent_app_kubernetes_io_part_of="true" __meta_kubernetes_pod_labelpresent_controller_revision_hash="true" __meta_kubernetes_pod_labelpresent_statefulset_kubernetes_io_pod_name="true" __meta_kubernetes_pod_name="loki-write-1" __meta_kubernetes_pod_node_name="ip-10-249-129-191.ec2.internal" __meta_kubernetes_pod_phase="Running" __meta_kubernetes_pod_ready="false" __meta_kubernetes_pod_uid="b3937321-fe90-4e15-ac94-495c8fdb9202" ``` Which issue(s) this PR fixes: https://github.com/grafana/loki/issues/10766 Special notes for your reviewer: Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [x] Documentation added - [x] Tests updated - [x] `CHANGELOG.md` updated - [x] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) --------- Co-authored-by: J Stickler <julie.stickler@grafana.com>	2 years ago
Gergő Rubint	93cd29f261	docs: fix promtail configuration sample for compressed files (#10784 ) What this PR does / why we need it: Fixes a typo which references an unknown field in [scrapeconfig.DecompressionConfig](https://github.com/grafana/loki/blob/main/clients/pkg/promtail/scrapeconfig/scrapeconfig.go#L56) [Search](https://github.com/search?q=repo%3Agrafana%2Floki+initial_sleep&type=code ) for `initial_sleep` yields only this line in the docs, while [search](https://github.com/search?q=repo%3Agrafana%2Floki+initial_delay&type=code) for `initial_deplay` yields code matches. Co-authored-by: J Stickler <julie.stickler@grafana.com>	2 years ago
Christian Haudum	a3b9534f37	Restructure API reference documentation (#10654 ) * Removed unsupported APIs * Change order so it matches the list on top * Write better headings * Make cURL examples consistent with command-response-blocks Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
Christian Haudum	d3d7ab7d82	Promtail: Remove deprecated stream_lag_labels setting (#10677 ) The setting was deprecated and already caused an error on startup when used. This behaviour was noticed when looking at the reference configuration docs of Promtail. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
Christian Haudum	760ad8691a	Docs: Remove experimental flag from GELF support in Promtail (#10684 ) GELF support in Promtail is not experimental (any more). Special notes for your reviewer: Nothing really worth mentioning in the changelog or release notes. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
Kaviraj Kanagaraj	1b24d2af9a	chore: Missed version updates. (#10676 ) What this PR does / why we need it: ``` LOKI_NEW_VERSION=2.9.1 ./tools/release_update_tags.sh ``` This script was introduced as the part of release docs PR. https://github.com/grafana/loki/pull/10646 Which issue(s) this PR fixes: Fixes #<issue number> Special notes for your reviewer: Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [x] Documentation added --------- Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> Co-authored-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
J Stickler	aed39c89ee	[docs] Replace ampersand with the word "and" (#10637 ) Replacing the ampersand character (&) with the word `and` to conform to the Style Guide https://grafana.com/docs/writers-toolkit/write/style-guide/capitalization-punctuation/#punctuation-guidelines Do not abbreviate and with an ampersand (&).	2 years ago
Šimon Lukašík	f184c5db51	docs: master tag hasn't been pushed for almost 2 years (#10633 )	2 years ago
Pablo	222ffc52d3	Add CloudFormation template for lambda + EventBridge setup (#10540 ) What this PR does / why we need it: https://github.com/grafana/loki/pull/10449 added support so that lambda-promtail can handle s3 events through AWS EventBridge. This PR is a follow up adding a CloudFormation template to deploy lambda-promtail, with the corresponding EventBridge rule to deliver s3 events to the lambda. Which issue(s) this PR fixes: Rel https://github.com/grafana/loki/issues/10209 --------- Co-authored-by: J Stickler <julie.stickler@grafana.com> Co-authored-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
Christian Haudum	f17e55030a	Reflect release 2.9.1 on main branch (#10597 ) What this PR does / why we need it: Reflect Loki 2.9.1 release on main branch to be up-to-date with latest released version. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
Raul Gonzales	a7b83bf9bb	chore(docs): Fix structured_metadata usage (#10526 ) What this PR does / why we need it: According to the code the pipeline name uses `_` not `-` for the name https://github.com/grafana/loki/blob/main/clients/pkg/logentry/stages/stage.go#L46C38-L46C38 Co-authored-by: J Stickler <julie.stickler@grafana.com>	2 years ago
J Stickler	247c312dd8	[Docs] Separate command and result (#10463 ) Now that we've implemented a copy-paste feature for code blocks on the docs website, put commands and the results in separate code blocks in the docs for ease of use. --------- Co-authored-by: Christian Haudum <christian.haudum@gmail.com> Co-authored-by: Ashwanth <iamashwanth@gmail.com>	2 years ago
J Stickler	71d526a23a	[Style updates] Capitalize Loki when referring to project. (#10462 ) What this PR does / why we need it: Went through and looked for places where Loki should be capitalized because we are talking about the product or project (vs. in configuration examples, etc. where lower case is acceptable or required).	2 years ago
Owen Diehl	2cc80c59c4	per tenant gate for structured metadata feature (#10402 ) Adds a per tenant configuration to block access to the upcoming structured-metadata push API extension. --------- Co-authored-by: Salva Corts <salva.corts@grafana.com>	2 years ago
Vladyslav Diachenko	00a3c5b026	renamed `non indexed labels` feature to `structured metadata` (#10432 ) Loki team has made a decision to rename the feature because the new name is more precise. --------- Signed-off-by: Vladyslav Diachenko <vlad.diachenko@grafana.com> Co-authored-by: Salva Corts <salva.corts@grafana.com>	2 years ago
Guillaume Grossetie	e5c8aed774	Fix typo "most be" -> "must be" (#10427 )	2 years ago
Salva Corts	603eb637f6	Add docs for non-indexed labels (#10073 ) What this PR does / why we need it: Adds documentation for the new experimental non-indexed labels feature. --------- Co-authored-by: J Stickler <julie.stickler@grafana.com> Co-authored-by: Sandeep Sukhani <sandeep.d.sukhani@gmail.com> Co-authored-by: Vladyslav Diachenko <82767850+vlad-diachenko@users.noreply.github.com>	2 years ago
Anthony Goddard	509896137d	[docs] replace use of "examplifies" (#10296 ) minor spelling correction What this PR does / why we need it: Trivial patch for grammar, updated sentence to match sentence structure used elsewhere in the docs Checklist - [ x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required)	2 years ago
William Dumont	a1c1152b79	Promtail: Add additional fields in cloudflare config (#10301 ) What this PR does / why we need it: In this PR the cloudflare config is enhanced by adding a custom fields type and by having a new property "additional_types" to give the customer the ability to specify the fields that he wants. This functionality has been added in the grafana agent and therefore should also be added in promtail. Which issue(s) this PR fixes: Fixes https://github.com/grafana/agent/issues/4860 Special notes for your reviewer: Checklist - [X] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [X] Documentation added - [X] Tests updated - [x] `CHANGELOG.md` updated - [x] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [x] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md`	2 years ago
J Stickler	d0545bff2d	[docs] Create top level Send data section, part 2 (#10247 ) Part of the database information architecture Epic # 8710 Which issue(s) this PR fixes: Second half of work for issue # 8741 that was started in PR #10192 When closed, fixes #8741 Move the following files under Send data: ./sources/clients/promtail/_index.md ./sources/clients/promtail/configuration.md ./sources/clients/promtail/gcplog-cloud.md ./sources/clients/promtail/installation.md ./sources/clients/promtail/logrotation/_index.md ./sources/clients/promtail/pipelines.md ./sources/clients/promtail/scraping.md ./sources/clients/promtail/stages/_index.md ./sources/clients/promtail/stages/cri.md ./sources/clients/promtail/stages/decolorize.md ./sources/clients/promtail/stages/docker.md ./sources/clients/promtail/stages/drop.md ./sources/clients/promtail/stages/json.md ./sources/clients/promtail/stages/labelallow.md ./sources/clients/promtail/stages/labeldrop.md ./sources/clients/promtail/stages/labels.md ./sources/clients/promtail/stages/limit.md ./sources/clients/promtail/stages/logfmt.md ./sources/clients/promtail/stages/match.md ./sources/clients/promtail/stages/metrics.md ./sources/clients/promtail/stages/multiline.md ./sources/clients/promtail/stages/output.md ./sources/clients/promtail/stages/pack.md ./sources/clients/promtail/stages/regex.md ./sources/clients/promtail/stages/replace.md ./sources/clients/promtail/stages/static_labels.md ./sources/clients/promtail/stages/template.md ./sources/clients/promtail/stages/tenant.md ./sources/clients/promtail/stages/timestamp.md ./sources/clients/promtail/troubleshooting/_index.md This PR also - Revises the Clients landing page to clarify supported clients. - Updates the metadata (descriptions, weights) - Adds aliases to redirect from the old URLs. - Updates cross-references broken by the move/renaming. - A few other small fixes (headings, typos, etc.) Special notes for your reviewer: Please review the updates to the Clients landing page (now called Send Data) as I've made some extensive edits to try to clarify recommended/supported clients. The file is docs/sources/send-data/_index.md --------- Co-authored-by: Michel Hollands <42814411+MichelHollands@users.noreply.github.com>	2 years ago
Elliot Dobson	6a8225d533	lambda-promtail: s3: update log parsing to support NLBs (#7194 ) <!-- Thanks for sending a pull request! Before submitting: 1. Read our CONTRIBUTING.md guide 2. Name your PR as `<Feature Area>: Describe your change`. a. Do not end the title with punctuation. It will be added in the changelog. b. Start with an imperative verb. Example: Fix the latency between System A and System B. c. Use sentence case, not title case. d. Use a complete phrase or sentence. The PR title will appear in a changelog, so help other people understand what your change will be. 3. Rebase your PR if it gets out of sync with main --> What this PR does / why we need it: Fixes the filename and timestamp regex for AWS Network Load Balancer (NLB) logs and adds logic to make NLB log timestamps RFC3339 compatible. Also adds a test to check the parsed timestamp is as expected. Which issue(s) this PR fixes: Fixes #6455 Special notes for your reviewer: <!-- Note about CHANGELOG entries, if a change adds: * an important feature * fixes an issue present in a previous release, * causes a change in operation that would be useful for an operator of Loki to know then please add a CHANGELOG entry. For documentation changes, build changes, simple fixes etc please skip this step. We are attempting to curate a changelog of the most relevant and important changes to be easier to ingest by end users of Loki. Note about the upgrade guide, if this changes: * default configuration values * metric names or label names * changes existing log lines such as the metrics.go query output line * configuration parameters * anything to do with any API * any other change that would require special attention or extra steps to upgrade Please document clearly what changed AND what needs to be done in the upgrade guide. --> Checklist - [x] Documentation added - [x] Tests updated - [ ] Is this an important fix or new feature? Add an entry in the `CHANGELOG.md`. - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` --------- Signed-off-by: Callum Styan <callumstyan@gmail.com> Co-authored-by: Callum Styan <callumstyan@gmail.com>	2 years ago
J Stickler	1951eef914	[Docs] Create top level Send data section, part 1 (#10192 ) Part of the database information architecture Epic # 8710 Which issue(s) this PR fixes: First half of work for issue # 8741 Move the following files under Send data: /sources/clients/_index.md ./sources/clients/aws/_index.md ./sources/clients/aws/ec2/_index.md ./sources/clients/aws/ecs/_index.md ./sources/clients/aws/eks/_index.md ./sources/clients/docker-driver/_index.md ./sources/clients/docker-driver/configuration.md ./sources/clients/fluentbit/_index.md ./sources/clients/fluentd/_index.md ./sources/clients/k6/_index.md ./sources/clients/k6/log-generation.md ./sources/clients/k6/query-scenario.md ./sources/clients/k6/write-scenario.md ./sources/clients/lambda-promtail/_index.md ./sources/clients/logstash/_index.md Updates the metadata (descriptions, weights) Adds aliases to redirect from the old URLs. Updates cross-references broken by the move/renaming. A few other small fixes (headings, typos, etc.) NOTE:The promtail files will move in a second PR.	2 years ago

22 Commits (a1387c78c71498e9017ce508e47816d007497475)