apitech/loki - loki - Gitea: Git with a cup of tea

Commit Graph

Author	SHA1	Message	Date
Sandeep Sukhani	6069df8f7d	ingestion: native otlp ingestion support (#10727 ) What this PR does / why we need it: Add support for natively supporting logs ingestion in OTLP format. `/otlp/v1/logs` is the new endpoint where users can push logs in OTLP format. It accepts logs serialized in JSON or proto format. Since OTEL format is very different than what Loki storage model, here is how data in OTEL format will be mapped to Loki data model: * Index labels: The Resource Attributes map quite well to Index labels in Loki since both usually identify the source of the logs. The problem however is that Resource attributes in OTLP can have an unbounded number of values while Loki has a default limit of having up to 30 labels. Since Index labels in Loki can largely drive the kind of querying experience the users are going to have, we have chosen select attributes which would be picked as Index Labels. The ones that are not picked up as Index labels would be stored as Structured Metadata with each log entry. * Timestamp: LogRecord.TimeUnixNano * LogLine: LogRecord.Body holds the body of the log. However, since Loki only supports Log body in string format, we will stringify non-string values using [AsString method from OTEL collector lib](`ab3d6c5b64/pdata/pcommon/value.go (L353)`). * Structured Metadata: Anything which can’t be stored in Index labels and LogLine. Here is a non-exhaustive list of what will be stored in Structured Metadata to give a sense of what it will hold: * Resource Attributes not stored as Index labels is replicated and stored with each log entry. * Everything under InstrumentationScope is replicated and stored with each log entry. * Everything under LogRecord except LogRecord.Body, LogRecord.TimeUnixNano and sometimes LogRecord.ObservedTimestamp. NOTES: * Since Loki does not support `.` or any other special characters other than `_` in label names, we replace all non-supported characters with `_`. * Since Loki only supports string in values of Index Labels and Structured Metadata, all the complex types are converted as follows: * Map would be flattened into label keys using `_` as separator, same as how we do it in [json parser in LogQL](https://grafana.com/docs/loki/latest/query/log_queries/#json). * Everything else is stringified using [AsString method from OTEL collector lib](`ab3d6c5b64/pdata/pcommon/value.go (L353)`) Special notes for your reviewer: I will open follow-up PRs for: * Documentation * Make blessed attributes list configurable per tenant. Checklist - [x] Tests updated - [x] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label	2 years ago
Kevin Burke	32f0dbc2a5	promtail/client: note client config is not deprecated (#10705 ) What this PR does / why we need it: Because the block of code immediately below `Config` indicates that all arguments are deprecated, I incorrectly assumed that the `Config` itself was deprecated and went hunting for a replacement. But the `Config` struct is still how a client is expected to be configured. Which issue(s) this PR fixes: Updates #10702. Checklist* - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [x] Documentation added - [x] Tests updated - [x] `CHANGELOG.md` updated - [x] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [x] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [x] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`)	2 years ago
Šimon Lukašík	fed9f14970	clients: update fluentd base container (#10966 ) 1.14.0 is over 2 years old and the latest 1.16 continues to work just fine with loki Co-authored-by: Vladyslav Diachenko <82767850+vlad-diachenko@users.noreply.github.com>	2 years ago
Raul Gonzales	7d5475541c	promtail: enable structured_metadata convert labels (#10752 ) What this PR does / why we need it: There are several labels that can be extracted by the supports service discovery providers, however the structured metadata wouldn't allow to parse labels to it. This PR allows the users to update the configuration to read from labels discovered, down below are some of the metadata from a pod, that could be of interested of anyone to have that pushed to Loki without creating several streams due to either high-carnality or proliferation of idle chunks: ``` __address__="10.0.165.166:3100" __meta_kubernetes_namespace="loki" __meta_kubernetes_pod_annotation_checksum_config="f008ef39c4f3230c5e153c6e9fde51a678e6f0b1087b3be7d91b8142a03dbe7c" __meta_kubernetes_pod_annotation_kubectl_kubernetes_io_default_container="loki" __meta_kubernetes_pod_annotationpresent_checksum_config="true" __meta_kubernetes_pod_annotationpresent_kubectl_kubernetes_io_default_container="true" __meta_kubernetes_pod_container_id="containerd://e6aef5c39c3fe806d39ea536b198c8c39cdee01d23141fed91e8fe1573cc5e08" __meta_kubernetes_pod_container_image="docker.io/grafana/loki:2.9.1" __meta_kubernetes_pod_container_init="false" __meta_kubernetes_pod_container_name="loki" __meta_kubernetes_pod_container_port_name="http-metrics" __meta_kubernetes_pod_container_port_number="3100" __meta_kubernetes_pod_container_port_protocol="TCP" __meta_kubernetes_pod_controller_kind="StatefulSet" __meta_kubernetes_pod_controller_name="loki-write" __meta_kubernetes_pod_host_ip="10.249.129.191" __meta_kubernetes_pod_ip="10.0.165.166" __meta_kubernetes_pod_label_app_kubernetes_io_component="write" __meta_kubernetes_pod_label_app_kubernetes_io_instance="loki" __meta_kubernetes_pod_label_app_kubernetes_io_name="loki" __meta_kubernetes_pod_label_app_kubernetes_io_part_of="memberlist" __meta_kubernetes_pod_label_controller_revision_hash="loki-write-75847dc945" __meta_kubernetes_pod_label_statefulset_kubernetes_io_pod_name="loki-write-1" __meta_kubernetes_pod_labelpresent_app_kubernetes_io_component="true" __meta_kubernetes_pod_labelpresent_app_kubernetes_io_instance="true" __meta_kubernetes_pod_labelpresent_app_kubernetes_io_name="true" __meta_kubernetes_pod_labelpresent_app_kubernetes_io_part_of="true" __meta_kubernetes_pod_labelpresent_controller_revision_hash="true" __meta_kubernetes_pod_labelpresent_statefulset_kubernetes_io_pod_name="true" __meta_kubernetes_pod_name="loki-write-1" __meta_kubernetes_pod_node_name="ip-10-249-129-191.ec2.internal" __meta_kubernetes_pod_phase="Running" __meta_kubernetes_pod_ready="false" __meta_kubernetes_pod_uid="b3937321-fe90-4e15-ac94-495c8fdb9202" ``` Which issue(s) this PR fixes: https://github.com/grafana/loki/issues/10766 Special notes for your reviewer: Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [x] Documentation added - [x] Tests updated - [x] `CHANGELOG.md` updated - [x] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) --------- Co-authored-by: J Stickler <julie.stickler@grafana.com>	2 years ago
Rens Groothuijsen	f51ee849b0	fix(promtail): correctly parse list of drop stage sources from YAML (#10848 ) What this PR does / why we need it: This PR fixes YAML parsing of the `source` field in the drop stage when said field is a list of strings. Because this list is not being recognized in the code as a list of strings, but rather as a list of generic `interface{}` elements, it fails a type check and throws an error. In order to fix this, an `interface{}` list is manually converted to a `string` list. The problem was already reproducible with existing tests, but the test YAML mistakenly referred to the field as `sources` rather than `source`. Which issue(s) this PR fixes: Fixes #10095 Special notes for your reviewer: Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [x] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) --------- Co-authored-by: Michel Hollands <42814411+MichelHollands@users.noreply.github.com>	2 years ago
Đỗ Trọng Hải	47e2c5884f	fix(promtail): prevent panic due to duplicate metric registration after reloaded (#10798 ) What this PR does / why we need it: Prevent Promtail panicking after getting reloaded. Which issue(s) this PR fixes: Fixes #10796 Special notes for your reviewer: Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [x] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) --------- Co-authored-by: Michel Hollands <42814411+MichelHollands@users.noreply.github.com>	2 years ago
Ashwanth	98551ceb9a	use go 1.21.3 and go-grpc v1.56.3 (#10869 ) What this PR does / why we need it: Fixes CVE-2023-39325 / CVE-2023-44487 https://github.com/grpc/grpc-go/releases/tag/v1.56.3 Which issue(s) this PR fixes: Fixes #<issue number> Special notes for your reviewer: Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`)	2 years ago
Kevin Burke	c071447900	clients/pkg/promtail/targets/gcplog: avoid listening on 0.0.0.0 (#10411 ) What this PR does / why we need it: Previously on Mac, running these tests triggers a dialog box asking me to accept all incoming network connections. Edit the test to listen on localhost only. Which issue(s) this PR fixes: Updates #10407 Special notes for your reviewer: Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [x] Documentation added (Just test changes, no documentation changes) - [X] Tests updated - [x] `CHANGELOG.md` updated (Not worth adding to CHANGELOG) - [x] If the change is worth mentioning in the release notes, add `add-to-release-notes` label (Not worth adding to release notes) - [x] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` (No user attention required) - [x] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) (No need to do this)	2 years ago
renovate[bot]	b2c4511a44	chore(deps): update dependency fluentd to v1.15.3 [security] (main) (#10839 ) [![Mend Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com) This PR contains the following updates: \| Package \| Change \| Age \| Adoption \| Passing \| Confidence \| \|---\|---\|---\|---\|---\|---\| \| [fluentd](https://www.fluentd.org/) ([source](https://togithub.com/fluent/fluentd)) \| `'1.14.2'` -> `'1.15.3'` \| [![age](https://developer.mend.io/api/mc/badges/age/rubygems/fluentd/1.15.3?slim=true)](https://docs.renovatebot.com/merge-confidence/) \| [![adoption](https://developer.mend.io/api/mc/badges/adoption/rubygems/fluentd/1.15.3?slim=true)](https://docs.renovatebot.com/merge-confidence/) \| [![passing](https://developer.mend.io/api/mc/badges/compatibility/rubygems/fluentd/'1.14.2'/1.15.3?slim=true)](https://docs.renovatebot.com/merge-confidence/) \| [![confidence](https://developer.mend.io/api/mc/badges/confidence/rubygems/fluentd/'1.14.2'/1.15.3?slim=true)](https://docs.renovatebot.com/merge-confidence/) \| ### GitHub Vulnerability Alerts #### [CVE-2022-39379](https://togithub.com/fluent/fluentd/security/advisories/GHSA-fppq-mj76-fpj2) ### Impact A remote code execution (RCE) vulnerability in non-default configurations of Fluentd allows unauthenticated attackers to execute arbitrary code via specially crafted JSON payloads. Fluentd setups are only affected if the environment variable `FLUENT_OJ_OPTION_MODE` is explicitly set to `object`. Please note: The option FLUENT_OJ_OPTION_MODE was introduced in Fluentd version 1.13.2. Earlier versions of Fluentd are not affected by this vulnerability. ### Patches v1.15.3 ### Workarounds Do not use `FLUENT_OJ_OPTION_MODE=object`. ### References * GHSL-2022-067 --- ### Release Notes <details> <summary>fluent/fluentd (fluentd)</summary> ### [`v1.15.3`](https://togithub.com/fluent/fluentd/blob/HEAD/CHANGELOG.md#Release-v1153---20221102) [Compare Source](https://togithub.com/fluent/fluentd/compare/v1.15.2...v1.15.3) ##### Bug Fix - Support glob for `!include` directive in YAML config format [https://github.com/fluent/fluentd/pull/3917](https://togithub.com/fluent/fluentd/pull/3917)3917 - Remove meaningless oj options [https://github.com/fluent/fluentd/pull/3929](https://togithub.com/fluent/fluentd/pull/3929)3929 - Fix log initializer to correctly create per-process files on Windows [https://github.com/fluent/fluentd/pull/3939](https://togithub.com/fluent/fluentd/pull/3939)3939 - out_file: Fix the multi-worker check with `<worker 0-N>` directive [https://github.com/fluent/fluentd/pull/3942](https://togithub.com/fluent/fluentd/pull/3942)3942 ##### Misc - Fix broken tests on Ruby 3.2 [https://github.com/fluent/fluentd/pull/3883](https://togithub.com/fluent/fluentd/pull/3883)3[https://github.com/fluent/fluentd/pull/3922](https://togithub.com/fluent/fluentd/pull/3922)ull/3922 ### [`v1.15.2`](https://togithub.com/fluent/fluentd/blob/HEAD/CHANGELOG.md#Release-v1152---20220822) [Compare Source](https://togithub.com/fluent/fluentd/compare/v1.15.1...v1.15.2) ##### Enhancement - Add a new system configuration `enable_jit` [https://github.com/fluent/fluentd/pull/3857](https://togithub.com/fluent/fluentd/pull/3857)3857 ##### Bug Fix - out_file: Fix append mode with `--daemon` flag [https://github.com/fluent/fluentd/pull/3864](https://togithub.com/fluent/fluentd/pull/3864)3864 - child_process: Plug file descriptor leak [https://github.com/fluent/fluentd/pull/3844](https://togithub.com/fluent/fluentd/pull/3844)3844 ##### Misc - Drop win32-api gem to support Ruby 3.2 [https://github.com/fluent/fluentd/pull/3849](https://togithub.com/fluent/fluentd/pull/3849)3[https://github.com/fluent/fluentd/pull/3866](https://togithub.com/fluent/fluentd/pull/3866)ull/3866 ### [`v1.15.1`](https://togithub.com/fluent/fluentd/blob/HEAD/CHANGELOG.md#Release-v1151---20220727) [Compare Source](https://togithub.com/fluent/fluentd/compare/v1.15.0...v1.15.1) ##### Bug Fix - Add support for concurrent append in out_file [https://github.com/fluent/fluentd/pull/3808](https://togithub.com/fluent/fluentd/pull/3808)3808 ##### Misc - in_tail: Show more information on skipping update_watcher [https://github.com/fluent/fluentd/pull/3829](https://togithub.com/fluent/fluentd/pull/3829)3829 ### [`v1.15.0`](https://togithub.com/fluent/fluentd/blob/HEAD/CHANGELOG.md#Release-v1150---20220629) [Compare Source](https://togithub.com/fluent/fluentd/compare/v1.14.6...v1.15.0) ##### Enhancement - in_tail: Add log throttling in files based on group rules [https://github.com/fluent/fluentd/pull/3535](https://togithub.com/fluent/fluentd/pull/3535)3[https://github.com/fluent/fluentd/pull/3771](https://togithub.com/fluent/fluentd/pull/3771)ull/3771 - Add `dump` command to fluent-ctl [https://github.com/fluent/fluentd/pull/3680](https://togithub.com/fluent/fluentd/pull/3680)3680 - Handle YAML configuration format on configuration file [https://github.com/fluent/fluentd/pull/3712](https://togithub.com/fluent/fluentd/pull/3712)3712 - Add `restart_worker_interval` parameter in `<system>` directive to set interval to restart workers that has stopped for some reas[https://github.com/fluent/fluentd/pull/3768](https://togithub.com/fluent/fluentd/pull/3768)ull/3768 ##### Bug fixes - out_forward: Fix to update timeout of cached sockets [https://github.com/fluent/fluentd/pull/3711](https://togithub.com/fluent/fluentd/pull/3711)3711 - in_tail: Fix a possible crash on file rotation when `follow_inodes true` [https://github.com/fluent/fluentd/pull/3754](https://togithub.com/fluent/fluentd/pull/3754)3754 - output: Fix a possible crash of flush thread [https://github.com/fluent/fluentd/pull/3755](https://togithub.com/fluent/fluentd/pull/3755)3755 - in_tail: Fix crash bugs on Ruby 3.1 on Windows [https://github.com/fluent/fluentd/pull/3766](https://togithub.com/fluent/fluentd/pull/3766)3766 - in_tail: Fix a bug that in_tail cannot open non-ascii path on Windows [https://github.com/fluent/fluentd/pull/3774](https://togithub.com/fluent/fluentd/pull/3774)3774 - Fix a bug that fluentd doesn't release its own log file even after rotated by external to[https://github.com/fluent/fluentd/pull/3782](https://togithub.com/fluent/fluentd/pull/3782)ull/3782 ##### Misc - in_tail: Simplify TargetInfo related code [https://github.com/fluent/fluentd/pull/3489](https://togithub.com/fluent/fluentd/pull/3489)3489 - Fix a wrong issue number in CHANGELOG [https://github.com/fluent/fluentd/pull/3700](https://togithub.com/fluent/fluentd/pull/3700)3700 - server helper: Add comments to linger_timeout behavior about Windows [https://github.com/fluent/fluentd/pull/3701](https://togithub.com/fluent/fluentd/pull/3701)3701 - service_discovery: Fix typo [https://github.com/fluent/fluentd/pull/3724](https://togithub.com/fluent/fluentd/pull/3724)3724 - test: Fix unstable tests and warnings [https://github.com/fluent/fluentd/pull/3745](https://togithub.com/fluent/fluentd/pull/3745)3[https://github.com/fluent/fluentd/pull/3753](https://togithub.com/fluent/fluentd/pull/3753)u[https://github.com/fluent/fluentd/pull/3767](https://togithub.com/fluent/fluentd/pull/3767)t[https://github.com/fluent/fluentd/pull/3783](https://togithub.com/fluent/fluentd/pull/3783)l[https://github.com/fluent/fluentd/pull/3784](https://togithub.com/fluent/fluentd/pull/3784)n[https://github.com/fluent/fluentd/pull/3785](https://togithub.com/fluent/fluentd/pull/3785)f[https://github.com/fluent/fluentd/pull/3787](https://togithub.com/fluent/fluentd/pull/3787)com/fluent/fluentd/pull/3787 ### [`v1.14.6`](https://togithub.com/fluent/fluentd/blob/HEAD/CHANGELOG.md#Release-v1146---20220331) [Compare Source](https://togithub.com/fluent/fluentd/compare/v1.14.5...v1.14.6) ##### Enhancement - Enable server plugins to specify socket-option `SO_LINGER` [https://github.com/fluent/fluentd/pull/3644](https://togithub.com/fluent/fluentd/pull/3644)3644 - Add `--umask` command line parameter [https://github.com/fluent/fluentd/pull/3671](https://togithub.com/fluent/fluentd/pull/3671)3[https://github.com/fluent/fluentd/pull/3679](https://togithub.com/fluent/fluentd/pull/3679)ull/3679 ##### Bug fixes - Fix metric name typo [https://github.com/fluent/fluentd/pull/3630](https://togithub.com/fluent/fluentd/pull/3630)3[https://github.com/fluent/fluentd/pull/3673](https://togithub.com/fluent/fluentd/pull/3673)ull/3673 - Apply modifications in pipeline to the records being passed to `@ERROR` label [https://github.com/fluent/fluentd/pull/3631](https://togithub.com/fluent/fluentd/pull/3631)3631 - Fix wrong calculation of retry interval [https://github.com/fluent/fluentd/pull/3640](https://togithub.com/fluent/fluentd/pull/3640)3[https://github.com/fluent/fluentd/pull/3649](https://togithub.com/fluent/fluentd/pull/3649)u[https://github.com/fluent/fluentd/pull/3685](https://togithub.com/fluent/fluentd/pull/3685)t[https://github.com/fluent/fluentd/pull/3686](https://togithub.com/fluent/fluentd/pull/3686)luentd/pull/3686 - Support IPv6 address for `rpc_endpoint` in `system` config [https://github.com/fluent/fluentd/pull/3641](https://togithub.com/fluent/fluentd/pull/3641)3641 ##### Misc - CI: Support Ruby 3.1 except Windows [https://github.com/fluent/fluentd/pull/3619](https://togithub.com/fluent/fluentd/pull/3619)3619 - Switch to GitHub Discussions [https://github.com/fluent/fluentd/pull/3654](https://togithub.com/fluent/fluentd/pull/3654)3654 - Fix CHANGELOG.md heading styles [https://github.com/fluent/fluentd/pull/3648](https://togithub.com/fluent/fluentd/pull/3648)3648 - Declare `null_value_pattern` as `regexp` [https://github.com/fluent/fluentd/pull/3650](https://togithub.com/fluent/fluentd/pull/3650)3650 ### [`v1.14.5`](https://togithub.com/fluent/fluentd/blob/HEAD/CHANGELOG.md#Release-v1145---20220209) [Compare Source](https://togithub.com/fluent/fluentd/compare/v1.14.4...v1.14.5) ##### Enhancement - Add support for "application/x-ndjson" to `in_http` [https://github.com/fluent/fluentd/pull/3616](https://togithub.com/fluent/fluentd/pull/3616)3616 - Add support for ucrt binary for Windows [https://github.com/fluent/fluentd/pull/3613](https://togithub.com/fluent/fluentd/pull/3613)3613 ##### Bug fixes - Don't retry when `retry_max_times == 0` [https://github.com/fluent/fluentd/pull/3608](https://togithub.com/fluent/fluentd/pull/3608)3608 - Fix hang-up issue during TLS handshake in `out_forward` [https://github.com/fluent/fluentd/pull/3601](https://togithub.com/fluent/fluentd/pull/3601)3601 - Bump up required ServerEngine to v2.2.5 [https://github.com/fluent/fluentd/pull/3599](https://togithub.com/fluent/fluentd/pull/3599)3599 - Fix "invalid byte sequence is replaced" warning on Kubernetes [https://github.com/fluent/fluentd/pull/3596](https://togithub.com/fluent/fluentd/pull/3596)3596 - Fix "ArgumentError: unknown keyword: :logger" on Windows with Ruby 3.1 [https://github.com/fluent/fluentd/pull/3592](https://togithub.com/fluent/fluentd/pull/3592)3592 ### [`v1.14.4`](https://togithub.com/fluent/fluentd/blob/HEAD/CHANGELOG.md#Release-v1144---20220106) [Compare Source](https://togithub.com/fluent/fluentd/compare/v1.14.3...v1.14.4) ##### Enhancement - `in_tail`: Add option to skip long lines (`max_line_size`) [https://github.com/fluent/fluentd/pull/3565](https://togithub.com/fluent/fluentd/pull/3565)3565 ##### Bug fix - Incorrect BufferChunkOverflowError when each event size is < `chunk_limit_size` [https://github.com/fluent/fluentd/pull/3560](https://togithub.com/fluent/fluentd/pull/3560)3560 - On macOS with Ruby 2.7/3.0, `out_file` fails to write events if `append` is true. [https://github.com/fluent/fluentd/pull/3579](https://togithub.com/fluent/fluentd/pull/3579)3579 - test: Fix unstable test cases [https://github.com/fluent/fluentd/pull/3574](https://togithub.com/fluent/fluentd/pull/3574)3[https://github.com/fluent/fluentd/pull/3577](https://togithub.com/fluent/fluentd/pull/3577)ull/3577 ### [`v1.14.3`](https://togithub.com/fluent/fluentd/blob/HEAD/CHANGELOG.md#Release-v1143---20211126) [Compare Source](https://togithub.com/fluent/fluentd/compare/v1.14.2...v1.14.3) ##### Enhancement - Changed to accept `http_parser.rb` 0.8.0. `http_parser.rb` 0.8.0 is ready for Ractor. [https://github.com/fluent/fluentd/pull/3544](https://togithub.com/fluent/fluentd/pull/3544)3544 ##### Bug fix - in_tail: Fixed a bug that no new logs are read when `enable_stat_watcher true` and `enable_watch_timer false` is set. [https://github.com/fluent/fluentd/pull/3541](https://togithub.com/fluent/fluentd/pull/3541)3541 - in_tail: Fixed a bug that the beginning and initial lines are lost after startup when `read_from_head false` and path includes wildcard '\'.[https://github.com/fluent/fluentd/pull/3542](https://togithub.com/fluent/fluentd/pull/3542)/3542 - Fixed a bug that processing messages were lost when BufferChunkOverflowError was thrown even though only a specific message size exceeds chunk_limi[https://github.com/fluent/fluentd/pull/3553](https://togithub.com/fluent/fluentd/pull/3553)t[https://github.com/fluent/fluentd/pull/3562](https://togithub.com/fluent/fluentd/pull/3562)luentd/pull/3562 ##### Misc - Bump up required version of `win32-service` gem. newer version is required to implement additional `fluent-ctl` commands. [https://github.com/fluent/fluentd/pull/3556](https://togithub.com/fluent/fluentd/pull/3556)3556 </details> --- ### Configuration 📅 Schedule: Branch creation - "" (UTC), Automerge - At any time (no schedule defined). 🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied. ♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 Ignore*: Close this PR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR has been generated by [Mend Renovate](https://www.mend.io/free-developer-tools/renovate/). View repository job log [here](https://developer.mend.io/github/grafana/loki). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy44LjEiLCJ1cGRhdGVkSW5WZXIiOiIzNy44LjEiLCJ0YXJnZXRCcmFuY2giOiJtYWluIn0=--> Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>	2 years ago
renovate[bot]	1715e645fe	chore(deps): update dependency fluentd to v1.14.2 [security] (main) (#10818 ) [![Mend Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com) This PR contains the following updates: \| Package \| Change \| Age \| Adoption \| Passing \| Confidence \| \|---\|---\|---\|---\|---\|---\| \| [fluentd](https://www.fluentd.org/) ([source](https://togithub.com/fluent/fluentd)) \| `'1.9.0'` -> `'1.14.2'` \| [![age](https://developer.mend.io/api/mc/badges/age/rubygems/fluentd/1.14.2?slim=true)](https://docs.renovatebot.com/merge-confidence/) \| [![adoption](https://developer.mend.io/api/mc/badges/adoption/rubygems/fluentd/1.14.2?slim=true)](https://docs.renovatebot.com/merge-confidence/) \| [![passing](https://developer.mend.io/api/mc/badges/compatibility/rubygems/fluentd/'1.9.0'/1.14.2?slim=true)](https://docs.renovatebot.com/merge-confidence/) \| [![confidence](https://developer.mend.io/api/mc/badges/confidence/rubygems/fluentd/'1.9.0'/1.14.2?slim=true)](https://docs.renovatebot.com/merge-confidence/) \| --- ### ⚠ Dependency Lookup Warnings ⚠ Warnings were logged while processing this repo. Please check the Dependency Dashboard for more information. ### GitHub Vulnerability Alerts #### [CVE-2021-41186](https://togithub.com/fluent/fluentd/security/advisories/GHSA-hwhf-64mh-r662) ### Impact parser_apache2 plugin in Fluentd v0.14.14 to v1.14.1 suffers from a regular expression denial of service (ReDoS) vulnerability. A broken apache log with a certain pattern of string can spend too much time in a regular expression, resulting in the potential for a DoS attack. ### Patches v1.14.2 ### Workarounds Either of the following: * Don't use parser_apache2 for parsing logs which cannot guarantee generated by Apache. * Put patched version of parser_apache2.rb into /etc/fluent/plugin directory (or any other directories specified by the environment variable `FLUENT_PLUGIN` or `--plugin` option of fluentd). ### References * [CVE-2021-41186](http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-41186) * [GHSA-hwhf-64mh-r662](https://togithub.com/fluent/fluentd/security/advisories/GHSA-hwhf-64mh-r662) * [GHSL-2021-102](https://securitylab.github.com/advisories/GHSL-2021-102-fluent-fluentd/) * https://github.com/fluent/fluentd/blob/master/CHANGELOG.md#v1142 --- ### Release Notes <details> <summary>fluent/fluentd (fluentd)</summary> ### [`v1.14.2`](https://togithub.com/fluent/fluentd/blob/HEAD/CHANGELOG.md#Release-v1142---20211029) [Compare Source](https://togithub.com/fluent/fluentd/compare/v1.14.1...v1.14.2) IMPORTANT: This release contain the fix for CVE-2021-41186 - ReDoS vulnerability in `parser_apache2`. This vulnerability is affected from Fluentd v0.14.14 to v1.14.1. We recommend to upgrade Fluentd to v1.14.2 or use patched version of `parser_apache2` plugin. ##### Enhancement - fluent-cat: Add `--event-time` option to send specified event time for testing. [https://github.com/fluent/fluentd/pull/3528](https://togithub.com/fluent/fluentd/pull/3528)3528 ##### Bug fix - Fixed to generate correct epoch timestamp even after switching Daylight Saving Time [https://github.com/fluent/fluentd/pull/3524](https://togithub.com/fluent/fluentd/pull/3524)3524 - Fixed ReDoS vulnerability in parser_apache2. This vulnerability is caused by a certain pattern of a broken apache log. ### [`v1.14.1`](https://togithub.com/fluent/fluentd/blob/HEAD/CHANGELOG.md#Release-v1141---20210929) [Compare Source](https://togithub.com/fluent/fluentd/compare/v1.14.0...v1.14.1) ##### Enhancement - in_tail: Added file related metrics. These metrics should be collected same as fluent-bit's in_ta[https://github.com/fluent/fluentd/pull/3504](https://togithub.com/fluent/fluentd/pull/3504)ull/3504 - out_forward: Changed to use metrics mechanism for node statistics [https://github.com/fluent/fluentd/pull/3506](https://togithub.com/fluent/fluentd/pull/3506)3506 ##### Bug fix - in_tail: Fixed a crash bug that it raise undefined method of eof? error. This error may happen only when `read_bytes_limit_per_second` was specified. [https://github.com/fluent/fluentd/pull/3500](https://togithub.com/fluent/fluentd/pull/3500)3500 - out_forward: Fixed a bug that node statistics information is not included correctly. [https://github.com/fluent/fluentd/pull/3503](https://togithub.com/fluent/fluentd/pull/3503)3[https://github.com/fluent/fluentd/pull/3507](https://togithub.com/fluent/fluentd/pull/3507)ull/3507 - Fixed a error when using `@include` directive It was occurred when http/https scheme URI is used in `@include` directive with Ruby 3. [https://github.com/fluent/fluentd/pull/3517](https://togithub.com/fluent/fluentd/pull/3517)3517 - out_copy: Fixed to suppress a wrong warning for `ignore_if_prev_success` It didn't work even if a user set [https://github.com/fluent/fluentd/pull/3515](https://togithub.com/fluent/fluentd/pull/3515)ull/3515 - Fixed not to output nanoseconds field of next retry time in warning log Then, inappropriate labels in log are also fixed. (retry_time -> retry_times, next_retry_seconds -> next_retr[https://github.com/fluent/fluentd/pull/3518](https://togithub.com/fluent/fluentd/pull/3518)td/pull/3518 ### [`v1.14.0`](https://togithub.com/fluent/fluentd/blob/HEAD/CHANGELOG.md#Release-v1140---20210830) [Compare Source](https://togithub.com/fluent/fluentd/compare/v1.13.3...v1.14.0) ##### Enhancement - Added `enable_input_metrics`, `enable_size_metrics` system configuration parameter This feature might need to pay higher CPU cost, so input event metrics features are disabled by default. These features are also enabled by `--enable-input-metrics`,`--enable-size-metrics` command line opti[https://github.com/fluent/fluentd/pull/3440](https://togithub.com/fluent/fluentd/pull/3440)ull/3440 - Added reserved word `@ROOT` for getting root router. This is incompatible change. Do not use `@ROOT` for label name. [https://github.com/fluent/fluentd/pull/3358](https://togithub.com/fluent/fluentd/pull/3358)3358 - in_syslog: Added `send_keepalive_packet` option [https://github.com/fluent/fluentd/pull/3474](https://togithub.com/fluent/fluentd/pull/3474)3474 - in_http: Added `cors_allow_credentials` option. This option tells browsers whether to expose the response to frontend when the credentials mode is "in[https://github.com/fluent/fluentd/pull/3481](https://togithub.com/fluent/fluentd/pull/3481)t[https://github.com/fluent/fluentd/pull/3491](https://togithub.com/fluent/fluentd/pull/3491)luentd/pull/3491 ##### Bug fix - in_tail: Fixed a bug that deleted paths are not removed from pos file by file compaction at start[https://github.com/fluent/fluentd/pull/3467](https://togithub.com/fluent/fluentd/pull/3467)ull/3467 - in_tail: Revived a warning message of retrying unaccessible file [https://github.com/fluent/fluentd/pull/3478](https://togithub.com/fluent/fluentd/pull/3478)3478 - TLSServer: Fixed a crash bug on logging peer host name errors [https://github.com/fluent/fluentd/pull/3483](https://togithub.com/fluent/fluentd/pull/3483)3483 ##### Misc - Added metrics plugin mechanism The implementations is changed to use metrics plugin. In the future, 3rd party plugin will be able to handle these m[https://github.com/fluent/fluentd/pull/3471](https://togithub.com/fluent/fluentd/pull/3471)t[https://github.com/fluent/fluentd/pull/3473](https://togithub.com/fluent/fluentd/pull/3473)l[https://github.com/fluent/fluentd/pull/3479](https://togithub.com/fluent/fluentd/pull/3479)n[https://github.com/fluent/fluentd/pull/3484](https://togithub.com/fluent/fluentd/pull/3484)fluent/fluentd/pull/3484 ### [`v1.13.3`](https://togithub.com/fluent/fluentd/blob/HEAD/CHANGELOG.md#Release-v1133---20210727) [Compare Source](https://togithub.com/fluent/fluentd/compare/v1.13.2...v1.13.3) ##### Bug fix - in_tail: Care DeletePending state on Windows [https://github.com/fluent/fluentd/pull/3457](https://togithub.com/fluent/fluentd/pull/3457)3[https://github.com/fluent/fluentd/pull/3460](https://togithub.com/fluent/fluentd/pull/3460)ull/3460 - in_tail: Fix some pos_file bugs. Avoid deleting pos_file entries unexpectedly when both `pos_file_compaction_interval` and `follow_inode` are enabled. Use `bytesize` instead of `size` for path length. [https://github.com/fluent/fluentd/pull/3459](https://togithub.com/fluent/fluentd/pull/3459)3459 - in_tail: Fix detecting rotation twice on `follow_inode`. [https://github.com/fluent/fluentd/pull/3466](https://togithub.com/fluent/fluentd/pull/3466)3466 ##### Misc - Remove needless spaces in a sample config file [https://github.com/fluent/fluentd/pull/3456](https://togithub.com/fluent/fluentd/pull/3456)3456 ### [`v1.13.2`](https://togithub.com/fluent/fluentd/blob/HEAD/CHANGELOG.md#Release-v1132---20210712) [Compare Source](https://togithub.com/fluent/fluentd/compare/v1.13.1...v1.13.2) ##### Enhancement - fluent-plugin-generate: Storage plugin was supported. [https://github.com/fluent/fluentd/pull/3426](https://togithub.com/fluent/fluentd/pull/3426)3426 - parser_json: Added support to customize configuration of oj options. Use `FLUENT_OJ_OPTION_BIGDECIMAL_LOAD`, `FLUENT_OJ_OPTION_MAX_NESTING`, `FLUENT_OJ_OPTION_MODE`, and `FLUENT_OJ_OPTION_USE_TO_JSON` environment variable to configure [https://github.com/fluent/fluentd/pull/3315](https://togithub.com/fluent/fluentd/pull/3315)ull/3315 ##### Bug fix - binlog_reader: Fixed a crash bug by missing "fluent/env" dependency. [https://github.com/fluent/fluentd/pull/3443](https://togithub.com/fluent/fluentd/pull/3443)3443 - Fixed a crash bug on outputting log at the early stage when parsing config file. This is a regression since v1.13.0. If you use invalid '@' prefix parameter, remove it as [https://github.com/fluent/fluentd/pull/3451](https://togithub.com/fluent/fluentd/pull/3451)/fluentd/pull/3451 - in_tail: Fixed a bug that when rotation is occurred, remaining lines will be discarded if the throttling feature is enabl[https://github.com/fluent/fluentd/pull/3390](https://togithub.com/fluent/fluentd/pull/3390)ull/3390 - fluent-plugin-generate: Fixed a crash bug during gemspec generation. It was unexpectedly introduced by [#3305](https://togithub.com/fluent/fluentd/issues/3305), thus this bug was a regression since 1.12[https://github.com/fluent/fluentd/pull/3444](https://togithub.com/fluent/fluentd/pull/3444)ull/3444 ##### Misc - Fixed the runtime dependency version of http_parse.rb to 0.7.0. It was fixed because false positive detection is occurred frequently by security scanning[https://github.com/fluent/fluentd/pull/3450](https://togithub.com/fluent/fluentd/pull/3450)td/pull/3450 ### [`v1.13.1`](https://togithub.com/fluent/fluentd/blob/HEAD/CHANGELOG.md#Release-v1131---20210625) [Compare Source](https://togithub.com/fluent/fluentd/compare/v1.13.0...v1.13.1) ##### Bug fix - out_forward: Fixed a race condition on handshake It's caused by using a same unpacker from multiple threa[https://github.com/fluent/fluentd/pull/3405](https://togithub.com/fluent/fluentd/pull/3405)u[https://github.com/fluent/fluentd/pull/3406](https://togithub.com/fluent/fluentd/pull/3406)td/pull/3406 - in_tail: Fixed to remove too much verbose debugging logs It was unexpectedly introduced by [#3185](https://togithub.com/fluent/fluentd/issues/3185) log throttling feature. [https://github.com/fluent/fluentd/pull/3418](https://togithub.com/fluent/fluentd/pull/3418)3418 - Fixed not to echo back the provides path as is on a 404 error There was a potential cross-site scripting vector even though it is quite difficult to e[https://github.com/fluent/fluentd/pull/3427](https://togithub.com/fluent/fluentd/pull/3427)td/pull/3427 ##### Misc - Pretty print for Fluent::Config::Section has been supported for debugg[https://github.com/fluent/fluentd/pull/3398](https://togithub.com/fluent/fluentd/pull/3398)ull/3398 - CI: Dropped to run CI for Ruby 2.5 [https://github.com/fluent/fluentd/pull/3412](https://togithub.com/fluent/fluentd/pull/3412)3412 ### [`v1.13.0`](https://togithub.com/fluent/fluentd/blob/HEAD/CHANGELOG.md#Release-v1130---20210529) [Compare Source](https://togithub.com/fluent/fluentd/compare/v1.12.4...v1.13.0) ##### Enhancement - in_tail: Handle log throttling per file feature [https://github.com/fluent/fluentd/pull/3185](https://togithub.com/fluent/fluentd/pull/3185)3[https://github.com/fluent/fluentd/pull/3364](https://togithub.com/fluent/fluentd/pull/3364)u[https://github.com/fluent/fluentd/pull/3379](https://togithub.com/fluent/fluentd/pull/3379)td/pull/3379 - Extend to support service discovery manager in simpler way [https://github.com/fluent/fluentd/pull/3299](https://togithub.com/fluent/fluentd/pull/3299)3[https://github.com/fluent/fluentd/pull/3362](https://togithub.com/fluent/fluentd/pull/3362)ull/3362 - in_http: HTTP GET requests has been supported [https://github.com/fluent/fluentd/pull/3373](https://togithub.com/fluent/fluentd/pull/3373)3373 - The log rotate settings in system configuration has been supported [https://github.com/fluent/fluentd/pull/3352](https://togithub.com/fluent/fluentd/pull/3352)3352 ##### Bug fix - Fix to disable `trace_instruction` when `RubyVM::InstructionSequence` is available. It improves compatibility with `truffleruby` some extent. [https://github.com/fluent/fluentd/pull/3376](https://togithub.com/fluent/fluentd/pull/3376)3376 - in_tail: Safely skip files which are used by another process on Windows. It improves exception handling about `ERROR_SHARING_VIOLATION` on Windows. [https://github.com/fluent/fluentd/pull/3378](https://togithub.com/fluent/fluentd/pull/3378)3378 - fluent-cat: the issue resending secondary file in specific format has been fi[https://github.com/fluent/fluentd/pull/3368](https://togithub.com/fluent/fluentd/pull/3368)ull/3368 - in_tail: Shutdown immediately & safely even if reading huge files Note that `skip_refresh_on_startup` must be enabled. [https://github.com/fluent/fluentd/pull/3380](https://togithub.com/fluent/fluentd/pull/3380)3380 ##### Misc - example: Change a path to backup_path in counter_server correctly [https://github.com/fluent/fluentd/pull/3359](https://togithub.com/fluent/fluentd/pull/3359)3359 - README: Update link to community forum to discuss.fluentd.org [https://github.com/fluent/fluentd/pull/3360](https://togithub.com/fluent/fluentd/pull/3360)3360 ### [`v1.12.4`](https://togithub.com/fluent/fluentd/blob/HEAD/CHANGELOG.md#Release-v1124---20210526) [Compare Source](https://togithub.com/fluent/fluentd/compare/v1.12.3...v1.12.4) ##### Bug fix - in_tail: Fix a bug that refresh_watcher fails to handle file rotations [https://github.com/fluent/fluentd/pull/3393](https://togithub.com/fluent/fluentd/pull/3393)3393 ### [`v1.12.3`](https://togithub.com/fluent/fluentd/blob/HEAD/CHANGELOG.md#Release-v1123---20210423) [Compare Source](https://togithub.com/fluent/fluentd/compare/v1.12.2...v1.12.3) ##### Enhancement - plugin_helper: Allow TLS to use keep-alive socket option [https://github.com/fluent/fluentd/pull/3308](https://togithub.com/fluent/fluentd/pull/3308)3308 ##### Bug fix - parser_csv, parser_syslog: Fix a naming conflict on parser_type [https://github.com/fluent/fluentd/pull/3302](https://togithub.com/fluent/fluentd/pull/3302)3302 - in_tail: Fix incorrect error code & message on Windows [https://github.com/fluent/fluentd/pull/3325](https://togithub.com/fluent/fluentd/pull/3325)3[https://github.com/fluent/fluentd/pull/3329](https://togithub.com/fluent/fluentd/pull/3329)u[https://github.com/fluent/fluentd/pull/3331](https://togithub.com/fluent/fluentd/pull/3331)t[https://github.com/fluent/fluentd/pull/3337](https://togithub.com/fluent/fluentd/pull/3337)luentd/pull/3337 - in_tail: Fix a crash bug on catching a short-lived log [https://github.com/fluent/fluentd/pull/3328](https://togithub.com/fluent/fluentd/pull/3328)3328 - storage_local: Fix position file corruption issue on concurrent gracefulReloads [https://github.com/fluent/fluentd/pull/3335](https://togithub.com/fluent/fluentd/pull/3335)3335 - Fix incorrect warnings about ${chunk_id} with out_s3 [https://github.com/fluent/fluentd/pull/3339](https://togithub.com/fluent/fluentd/pull/3339)3339 - TLS Server: Add peer information to error log message [https://github.com/fluent/fluentd/pull/3330](https://togithub.com/fluent/fluentd/pull/3330)3330 ##### Misc - fluent-plugin-generate: add note about plugin name [https://github.com/fluent/fluentd/pull/3303](https://togithub.com/fluent/fluentd/pull/3303)3303 - fluent-plugin-generate: Use same depended gem version with fluentd [https://github.com/fluent/fluentd/pull/3305](https://togithub.com/fluent/fluentd/pull/3305)3305 - Fix some broken unit tests and improve CI's stability [https://github.com/fluent/fluentd/pull/3304](https://togithub.com/fluent/fluentd/pull/3304)3[https://github.com/fluent/fluentd/pull/3307](https://togithub.com/fluent/fluentd/pull/3307)u[https://github.com/fluent/fluentd/pull/3312](https://togithub.com/fluent/fluentd/pull/3312)t[https://github.com/fluent/fluentd/pull/3313](https://togithub.com/fluent/fluentd/pull/3313)l[https://github.com/fluent/fluentd/pull/3314](https://togithub.com/fluent/fluentd/pull/3314)n[https://github.com/fluent/fluentd/pull/3316](https://togithub.com/fluent/fluentd/pull/3316)f[https://github.com/fluent/fluentd/pull/3336](https://togithub.com/fluent/fluentd/pull/3336)com/fluent/fluentd/pull/3336 - Permit to install with win32-service 2.2.0 on Windows [https://github.com/fluent/fluentd/pull/3343](https://togithub.com/fluent/fluentd/pull/3343)3343 ### [`v1.12.2`](https://togithub.com/fluent/fluentd/blob/HEAD/CHANGELOG.md#Release-v1122---20210329) [Compare Source](https://togithub.com/fluent/fluentd/compare/v1.12.1...v1.12.2) ##### Enhancement - out_copy: Add ignore_if_prev_successes [https://github.com/fluent/fluentd/pull/3190](https://togithub.com/fluent/fluentd/pull/3190)3[https://github.com/fluent/fluentd/pull/3287](https://togithub.com/fluent/fluentd/pull/3287)ull/3287 - Support multiple kind of timestamp format [https://github.com/fluent/fluentd/pull/3252](https://togithub.com/fluent/fluentd/pull/3252)3252 - formatter_ltsv: suppress delimiters in output [https://github.com/fluent/fluentd/pull/1666](https://togithub.com/fluent/fluentd/pull/1666)1[https://github.com/fluent/fluentd/pull/3288](https://togithub.com/fluent/fluentd/pull/3288)u[https://github.com/fluent/fluentd/pull/3289](https://togithub.com/fluent/fluentd/pull/3289)td/pull/3289 ##### Bug fix - in_tail: Expect ENOENT during stat [https://github.com/fluent/fluentd/pull/3275](https://togithub.com/fluent/fluentd/pull/3275)3275 - out_forward: Prevent transferring duplicate logs on restart [https://github.com/fluent/fluentd/pull/3267](https://togithub.com/fluent/fluentd/pull/3267)3[https://github.com/fluent/fluentd/pull/3285](https://togithub.com/fluent/fluentd/pull/3285)ull/3285 - in_tail: Handle to send rotated logs when mv is used for rotating [https://github.com/fluent/fluentd/pull/3294](https://togithub.com/fluent/fluentd/pull/3294)3294 - fluent-plugin-config-format: Fill an uninitialized instance variable [https://github.com/fluent/fluentd/pull/3297](https://togithub.com/fluent/fluentd/pull/3297)3297 - Fix MessagePackEventStream issue with Enumerable methods [https://github.com/fluent/fluentd/pull/2116](https://togithub.com/fluent/fluentd/pull/2116)2116 ##### Misc - Add webrick to support Ruby 3.0 [https://github.com/fluent/fluentd/pull/3257](https://togithub.com/fluent/fluentd/pull/3257)3257 - Suggest Discource instead of Google Groups [https://github.com/fluent/fluentd/pull/3261](https://togithub.com/fluent/fluentd/pull/3261)3261 - Update MAINTAINERS.md [https://github.com/fluent/fluentd/pull/3282](https://togithub.com/fluent/fluentd/pull/3282)3282 - Introduce DeepSource to check code quality [https://github.com/fluent/fluentd/pull/3286](https://togithub.com/fluent/fluentd/pull/3286)3[https://github.com/fluent/fluentd/pull/3259](https://togithub.com/fluent/fluentd/pull/3259)u[https://github.com/fluent/fluentd/pull/3291](https://togithub.com/fluent/fluentd/pull/3291)td/pull/3291 - Migrate to GitHub Actions and stabilize tests [https://github.com/fluent/fluentd/pull/3266](https://togithub.com/fluent/fluentd/pull/3266)3[https://github.com/fluent/fluentd/pull/3268](https://togithub.com/fluent/fluentd/pull/3268)u[https://github.com/fluent/fluentd/pull/3281](https://togithub.com/fluent/fluentd/pull/3281)t[https://github.com/fluent/fluentd/pull/3283](https://togithub.com/fluent/fluentd/pull/3283)l[https://github.com/fluent/fluentd/pull/3290](https://togithub.com/fluent/fluentd/pull/3290)nt/fluentd/pull/3290 ### [`v1.12.1`](https://togithub.com/fluent/fluentd/blob/HEAD/CHANGELOG.md#Release-v1121---20210218) [Compare Source](https://togithub.com/fluent/fluentd/compare/v1.12.0...v1.12.1) ##### Enhancement - out_http: Add `headers_from_placeholders` parameter [https://github.com/fluent/fluentd/pull/3241](https://togithub.com/fluent/fluentd/pull/3241)3241 - fluent-plugin-config-format: Add `--table` option to use markdown table [https://github.com/fluent/fluentd/pull/3240](https://togithub.com/fluent/fluentd/pull/3240)3240 - Add `--disable-shared-socket`/`disable_shared_socket` to disable ServerEngine's shared socket setup [https://github.com/fluent/fluentd/pull/3250](https://togithub.com/fluent/fluentd/pull/3250)3250 ##### Bug fix - ca_generate: Fix creating TLS certification files which include broken extensions [https://github.com/fluent/fluentd/pull/3246](https://togithub.com/fluent/fluentd/pull/3246)3246 - test: Drop TLS 1.1 tests [https://github.com/fluent/fluentd/pull/3256](https://togithub.com/fluent/fluentd/pull/3256)3256 - Remove old gem constraints to support Ruby 3 ##### Misc - Use GitHub Actions [https://github.com/fluent/fluentd/pull/3233](https://togithub.com/fluent/fluentd/pull/3233)3[https://github.com/fluent/fluentd/pull/3255](https://togithub.com/fluent/fluentd/pull/3255)ull/3255 ### [`v1.12.0`](https://togithub.com/fluent/fluentd/blob/HEAD/CHANGELOG.md#Release-v1120---20210105) [Compare Source](https://togithub.com/fluent/fluentd/compare/v1.11.5...v1.12.0) ##### New feature - in_tail: Add `follow_inode` to support log rotation with wild card [https://github.com/fluent/fluentd/pull/3182](https://togithub.com/fluent/fluentd/pull/3182)3182 - in_tail: Handle linux capability [https://github.com/fluent/fluentd/pull/3155](https://togithub.com/fluent/fluentd/pull/3155)3[https://github.com/fluent/fluentd/pull/3162](https://togithub.com/fluent/fluentd/pull/3162)ull/3162 - windows: Add win32 events alternative to unix signals [https://github.com/fluent/fluentd/pull/3131](https://togithub.com/fluent/fluentd/pull/3131)3131 ##### Enhancement - buffer: Enable metadata comparison optimization on all platforms [https://github.com/fluent/fluentd/pull/3095](https://togithub.com/fluent/fluentd/pull/3095)3095 - fluent-plugin-config-formatter: Handle `service_discovery` type [https://github.com/fluent/fluentd/pull/3178](https://togithub.com/fluent/fluentd/pull/3178)3178 - in_http: Add `add_query_params` parameter to add query params to event record [https://github.com/fluent/fluentd/pull/3197](https://togithub.com/fluent/fluentd/pull/3197)3197 - inject: Support `unixtime_micros` and `unixtime_nanos` in `time_type` [https://github.com/fluent/fluentd/pull/3220](https://togithub.com/fluent/fluentd/pull/3220)3220 - Refactoring code [https://github.com/fluent/fluentd/pull/3167](https://togithub.com/fluent/fluentd/pull/3167)3[https://github.com/fluent/fluentd/pull/3170](https://togithub.com/fluent/fluentd/pull/3170)u[https://github.com/fluent/fluentd/pull/3180](https://togithub.com/fluent/fluentd/pull/3180)t[https://github.com/fluent/fluentd/pull/3196](https://togithub.com/fluent/fluentd/pull/3196)l[https://github.com/fluent/fluentd/pull/3213](https://togithub.com/fluent/fluentd/pull/3213)n[https://github.com/fluent/fluentd/pull/3222](https://togithub.com/fluent/fluentd/pull/3222)fluent/fluentd/pull/3222 ##### Bug fix - output: Prevent retry.step from being called too many times in a short time [https://github.com/fluent/fluentd/pull/3203](https://togithub.com/fluent/fluentd/pull/3203)3203 ### [`v1.11.5`](https://togithub.com/fluent/fluentd/blob/HEAD/CHANGELOG.md#Release-v1115---20201106) [Compare Source](https://togithub.com/fluent/fluentd/compare/v1.11.4...v1.11.5) ##### Enhancement - formatter: Provide `newline` parameter to support `CRLF` [https://github.com/fluent/fluentd/pull/3152](https://togithub.com/fluent/fluentd/pull/3152)3152 - out_http: adding support for intermediate certificates [https://github.com/fluent/fluentd/pull/3146](https://togithub.com/fluent/fluentd/pull/3146)3146 - Update serverengine dependency to 2.2.2 or later ##### Bug fix - Fix a bug that windows service isn't stopped gracefuly [https://github.com/fluent/fluentd/pull/3156](https://togithub.com/fluent/fluentd/pull/3156)3156 ### [`v1.11.4`](https://togithub.com/fluent/fluentd/blob/HEAD/CHANGELOG.md#Release-v1114---20201013) [Compare Source](https://togithub.com/fluent/fluentd/compare/v1.11.3...v1.11.4) ##### Enhancement - inject: Support `unixtime_millis` in `time_type` parameter [https://github.com/fluent/fluentd/pull/3145](https://togithub.com/fluent/fluentd/pull/3145)3145 ##### Bug fix - out_http: Fix broken data with `json_array true` [https://github.com/fluent/fluentd/pull/3144](https://togithub.com/fluent/fluentd/pull/3144)3144 - output: Fix wrong logging issue for `${chunk_id}` [https://github.com/fluent/fluentd/pull/3134](https://togithub.com/fluent/fluentd/pull/3134)3134 ### [`v1.11.3`](https://togithub.com/fluent/fluentd/blob/HEAD/CHANGELOG.md#Release-v1113---20200930) [Compare Source](https://togithub.com/fluent/fluentd/compare/v1.11.2...v1.11.3) ##### Enhancement - in_exec: Add `connect_mode` parameter to read stderr [https://github.com/fluent/fluentd/pull/3108](https://togithub.com/fluent/fluentd/pull/3108)3108 - parser_json: Improve the performance [https://github.com/fluent/fluentd/pull/3109](https://togithub.com/fluent/fluentd/pull/3109)3109 - log: Add `ignore_same_log_interval` parameter [https://github.com/fluent/fluentd/pull/3119](https://togithub.com/fluent/fluentd/pull/3119)3119 - Upgrade win32 gems [https://github.com/fluent/fluentd/pull/3100](https://togithub.com/fluent/fluentd/pull/3100)3100 - Refactoring code [https://github.com/fluent/fluentd/pull/3094](https://togithub.com/fluent/fluentd/pull/3094)3[https://github.com/fluent/fluentd/pull/3118](https://togithub.com/fluent/fluentd/pull/3118)ull/3118 ##### Bug fix - buffer: Fix calculation of timekey stats [https://github.com/fluent/fluentd/pull/3018](https://togithub.com/fluent/fluentd/pull/3018)3018 - buffer: fix binmode usage for prevent gc [https://github.com/fluent/fluentd/pull/3138](https://togithub.com/fluent/fluentd/pull/3138)3138 ### [`v1.11.2`](https://togithub.com/fluent/fluentd/blob/HEAD/CHANGELOG.md#Release-v1112---20200804) [Compare Source](https://togithub.com/fluent/fluentd/compare/v1.11.1...v1.11.2) ##### Enhancement - `in_dummy` renamed to `in_sample` [https://github.com/fluent/fluentd/pull/3065](https://togithub.com/fluent/fluentd/pull/3065)3065 - Allow regular expression in filter/match directive [https://github.com/fluent/fluentd/pull/3071](https://togithub.com/fluent/fluentd/pull/3071)3071 - Refactoring code [https://github.com/fluent/fluentd/pull/3051](https://togithub.com/fluent/fluentd/pull/3051)3051 ##### Bug fix - buffer: Fix log message for `chunk_limit_records` case [https://github.com/fluent/fluentd/pull/3079](https://togithub.com/fluent/fluentd/pull/3079)3079 - buffer: Fix timekey optimization for non-windows platform [https://github.com/fluent/fluentd/pull/3092](https://togithub.com/fluent/fluentd/pull/3092)3092 - cert: Raise an error for broken certificate file [https://github.com/fluent/fluentd/pull/3086](https://togithub.com/fluent/fluentd/pull/3086)3086 - cert: Set TLS ciphers list correcty on older OpenSSL [https://github.com/fluent/fluentd/pull/3093](https://togithub.com/fluent/fluentd/pull/3093)3093 ### [`v1.11.1`](https://togithub.com/fluent/fluentd/blob/HEAD/CHANGELOG.md#Release-v1111---20200622) [Compare Source](https://togithub.com/fluent/fluentd/compare/v1.11.0...v1.11.1) ##### Enhancement - in_http: Add `dump_error_log` parameter [https://github.com/fluent/fluentd/pull/3035](https://togithub.com/fluent/fluentd/pull/3035)3035 - in_http: Improve time field handling [https://github.com/fluent/fluentd/pull/3046](https://togithub.com/fluent/fluentd/pull/3046)3046 - Refactoring code [https://github.com/fluent/fluentd/pull/3047](https://togithub.com/fluent/fluentd/pull/3047)3047 ##### Bug fix - in_tail: Use actual path instead of based pattern for ignore list [https://github.com/fluent/fluentd/pull/3042](https://togithub.com/fluent/fluentd/pull/3042)3042 - child_process helper: Fix child process failure due to SIGPIPE if the command uses stdout [https://github.com/fluent/fluentd/pull/3044](https://togithub.com/fluent/fluentd/pull/3044)3044 ### [`v1.11.0`](https://togithub.com/fluent/fluentd/blob/HEAD/CHANGELOG.md#Release-v1110---20200604) [Compare Source](https://togithub.com/fluent/fluentd/compare/v1.10.4...v1.11.0) ##### New feature - in_unix: Use v1 API [https://github.com/fluent/fluentd/pull/2992](https://togithub.com/fluent/fluentd/pull/2992)2992 ##### Enhancement - parser_syslog: Support any `time_format` for RFC3164 string parser [https://github.com/fluent/fluentd/pull/3014](https://togithub.com/fluent/fluentd/pull/3014)3014 - parser_syslog: Add new parser for RFC5424 [https://github.com/fluent/fluentd/pull/3015](https://togithub.com/fluent/fluentd/pull/3015)3015 - Refactoring code [https://github.com/fluent/fluentd/pull/3019](https://togithub.com/fluent/fluentd/pull/3019)3019 ##### Bug fix - in_gc_stat: Add `use_symbol_keys` parameter to emit string key record [https://github.com/fluent/fluentd/pull/3008](https://togithub.com/fluent/fluentd/pull/3008)3008 ### [`v1.10.4`](https://togithub.com/fluent/fluentd/blob/HEAD/CHANGELOG.md#Release-v1104---20200512) [Compare Source](https://togithub.com/fluent/fluentd/compare/v1.10.3...v1.10.4) ##### Enhancement - out_http: Support single json array payload [https://github.com/fluent/fluentd/pull/2973](https://togithub.com/fluent/fluentd/pull/2973)2973 - Refactoring [https://github.com/fluent/fluentd/pull/2988](https://togithub.com/fluent/fluentd/pull/2988)2988 ##### Bug fix - supervisor: Call `File.umask(0)` for standalone worker [https://github.com/fluent/fluentd/pull/2987](https://togithub.com/fluent/fluentd/pull/2987)2987 - out_forward: Fix ZeroDivisionError issue with `weight 0` [https://github.com/fluent/fluentd/pull/2989](https://togithub.com/fluent/fluentd/pull/2989)2989 ### [`v1.10.3`](https://togithub.com/fluent/fluentd/blob/HEAD/CHANGELOG.md#Release-v1103---20200501) [Compare Source](https://togithub.com/fluent/fluentd/compare/v1.10.2...v1.10.3) ##### Enhancement - record_accessor: Add `set` method [https://github.com/fluent/fluentd/pull/2977](https://togithub.com/fluent/fluentd/pull/2977)2977 - config: Ruby DSL format is deprecated [https://github.com/fluent/fluentd/pull/2958](https://togithub.com/fluent/fluentd/pull/2958)2958 - Refactor code [https://github.com/fluent/fluentd/pull/2961](https://togithub.com/fluent/fluentd/pull/2961)2[https://github.com/fluent/fluentd/pull/2962](https://togithub.com/fluent/fluentd/pull/2962)u[https://github.com/fluent/fluentd/pull/2965](https://togithub.com/fluent/fluentd/pull/2965)t[https://github.com/fluent/fluentd/pull/2966](https://togithub.com/fluent/fluentd/pull/2966)l[https://github.com/fluent/fluentd/pull/2978](https://togithub.com/fluent/fluentd/pull/2978)nt/fluentd/pull/2978 ##### Bug fix - out_forward: Disable `linger_timeout` setting on Windows [https://github.com/fluent/fluentd/pull/2959](https://togithub.com/fluent/fluentd/pull/2959)2959 - out_forward: Fix warning of service discovery manager when fluentd stops [https://github.com/fluent/fluentd/pull/2974](https://togithub.com/fluent/fluentd/pull/2974)2974 ### [`v1.10.2`](https://togithub.com/fluent/fluentd/blob/HEAD/CHANGELOG.md#Release-v1102---20200415) [Compare Source](https://togithub.com/fluent/fluentd/compare/v1.10.1...v1.10.2) ##### Enhancement - out_copy: Add plugin_id to log message [https://github.com/fluent/fluentd/pull/2934](https://togithub.com/fluent/fluentd/pull/2934)2934 - socket: Allow cert chains in mutual auth [https://github.com/fluent/fluentd/pull/2930](https://togithub.com/fluent/fluentd/pull/2930)2930 - system: Add ignore_repeated_log_interval parameter [https://github.com/fluent/fluentd/pull/2937](https://togithub.com/fluent/fluentd/pull/2937)2937 - windows: Allow to launch fluentd from whitespace included path [https://github.com/fluent/fluentd/pull/2920](https://togithub.com/fluent/fluentd/pull/2920)2920 - Refactor code [https://github.com/fluent/fluentd/pull/2935](https://togithub.com/fluent/fluentd/pull/2935)2[https://github.com/fluent/fluentd/pull/2936](https://togithub.com/fluent/fluentd/pull/2936)u[https://github.com/fluent/fluentd/pull/2938](https://togithub.com/fluent/fluentd/pull/2938)t[https://github.com/fluent/fluentd/pull/2939](https://togithub.com/fluent/fluentd/pull/2939)l[https://github.com/fluent/fluentd/pull/2946](https://togithub.com/fluent/fluentd/pull/2946)nt/fluentd/pull/2946 ##### Bug fix - in_syslog: Fix octet-counting mode bug [https://github.com/fluent/fluentd/pull/2942](https://togithub.com/fluent/fluentd/pull/2942)2942 - out_forward: Create timer for purging obsolete sockets when keepalive_timeout is not set [https://github.com/fluent/fluentd/pull/2943](https://togithub.com/fluent/fluentd/pull/2943)2943 - out_forward: Need authentication when sending tcp heartbeat with keepalive [https://github.com/fluent/fluentd/pull/2945](https://togithub.com/fluent/fluentd/pull/2945)2945 - command: Fix fluent-debug start failure [https://github.com/fluent/fluentd/pull/2948](https://togithub.com/fluent/fluentd/pull/2948)2948 - command: Fix regression of supervisor's worker and `--daemon` combo [https://github.com/fluent/fluentd/pull/2950](https://togithub.com/fluent/fluentd/pull/2950)2950 ### [`v1.10.1`](https://togithub.com/fluent/fluentd/blob/HEAD/CHANGELOG.md#Release-v1101---20200402) [Compare Source](https://togithub.com/fluent/fluentd/compare/v1.10.0...v1.10.1) ##### Enhancement - command: `--daemon` and `--no-supervisor` now work together [https://github.com/fluent/fluentd/pull/2912](https://togithub.com/fluent/fluentd/pull/2912)2912 - Refactor code [https://github.com/fluent/fluentd/pull/2913](https://togithub.com/fluent/fluentd/pull/2913)2913 ##### Bug fix - in_tail: `Fix pos_file_compaction_interval` parameter type [https://github.com/fluent/fluentd/pull/2921](https://togithub.com/fluent/fluentd/pull/2921)2921 - in_tail: Fix seek position update after compaction [https://github.com/fluent/fluentd/pull/2922](https://togithub.com/fluent/fluentd/pull/2922)2922 - parser_syslog: Fix regression in the `with_priority` and RFC5424 case [https://github.com/fluent/fluentd/pull/2923](https://togithub.com/fluent/fluentd/pull/2923)2923 ##### Misc - Add document for security audit [https://github.com/fluent/fluentd/pull/2911](https://togithub.com/fluent/fluentd/pull/2911)2911 ### [`v1.10.0`](https://togithub.com/fluent/fluentd/blob/HEAD/CHANGELOG.md#Release-v1100---20200324) [Compare Source](https://togithub.com/fluent/fluentd/compare/v1.9.3...v1.10.0) ##### New feature - sd plugin: Add SRV record plugin [https://github.com/fluent/fluentd/pull/2876](https://togithub.com/fluent/fluentd/pull/2876)2876 ##### Enhancement - server: Add `cert_verifier` parameter for TLS transport [https://github.com/fluent/fluentd/pull/2888](https://togithub.com/fluent/fluentd/pull/2888)2888 - parser_syslog: Support customized time format [https://github.com/fluent/fluentd/pull/2886](https://togithub.com/fluent/fluentd/pull/2886)2886 - in_dummy: Delete `suspend` parameter [https://github.com/fluent/fluentd/pull/2897](https://togithub.com/fluent/fluentd/pull/2897)2897 - Refactor code [https://github.com/fluent/fluentd/pull/2858](https://togithub.com/fluent/fluentd/pull/2858)2[https://github.com/fluent/fluentd/pull/2862](https://togithub.com/fluent/fluentd/pull/2862)u[https://github.com/fluent/fluentd/pull/2864](https://togithub.com/fluent/fluentd/pull/2864)t[https://github.com/fluent/fluentd/pull/2869](https://togithub.com/fluent/fluentd/pull/2869)l[https://github.com/fluent/fluentd/pull/2870](https://togithub.com/fluent/fluentd/pull/2870)n[https://github.com/fluent/fluentd/pull/2874](https://togithub.com/fluent/fluentd/pull/2874)f[https://github.com/fluent/fluentd/pull/2881](https://togithub.com/fluent/fluentd/pull/2881)c[https://github.com/fluent/fluentd/pull/2885](https://togithub.com/fluent/fluentd/pull/2885)h[https://github.com/fluent/fluentd/pull/2894](https://togithub.com/fluent/fluentd/pull/2894)/[https://github.com/fluent/fluentd/pull/2896](https://togithub.com/fluent/fluentd/pull/2896)p[https://github.com/fluent/fluentd/pull/2898](https://togithub.com/fluent/fluentd/pull/2898) [https://github.com/fluent/fluentd/pull/2899](https://togithub.com/fluent/fluentd/pull/2899) [https://github.com/fluent/fluentd/pull/2900](https://togithub.com/fluent/fluentd/pull/2900)2[https://github.com/fluent/fluentd/pull/2901](https://togithub.com/fluent/fluentd/pull/2901)u[https://github.com/fluent/fluentd/pull/2906](https://togithub.com/fluent/fluentd/pull/2906)td/pull/2901 https://github.com/fluent/fluentd/pull/2906 ##### Bug fix - out_forward: windows: Permit to specify `linger_timeout` [https://github.com/fluent/fluentd/pull/2868](https://togithub.com/fluent/fluentd/pull/2868)2868 - parser_syslog: Fix syslog format detection [https://github.com/fluent/fluentd/pull/2879](https://togithub.com/fluent/fluentd/pull/2879)2879 - buffer: Fix `available_buffer_space_ratio` calculation [https://github.com/fluent/fluentd/pull/2882](https://togithub.com/fluent/fluentd/pull/2882)2882 - tls: Support CRLF based X.509 certificates [https://github.com/fluent/fluentd/pull/2890](https://togithub.com/fluent/fluentd/pull/2890)2890 - msgpack_factory mixin: Fix performance penalty for deprecation log [https://github.com/fluent/fluentd/pull/2903](https://togithub.com/fluent/fluentd/pull/2903)2903 ### [`v1.9.3`](https://togithub.com/fluent/fluentd/blob/HEAD/CHANGELOG.md#Release-v193---20200305) [Compare Source](https://togithub.com/fluent/fluentd/compare/v1.9.2...v1.9.3) ##### Enhancement - in_tail: Emit buffered lines as `unmatched_line` at shutdown phase when `emit_unmatched_lines true` [https://github.com/fluent/fluentd/pull/2837](https://togithub.com/fluent/fluentd/pull/2837)2837 - Specify directory mode explicitly [https://github.com/fluent/fluentd/pull/2827](https://togithub.com/fluent/fluentd/pull/2827)2827 - server helper: Change SSLError log level to warn in accept [https://github.com/fluent/fluentd/pull/2861](https://togithub.com/fluent/fluentd/pull/2861)2861 - Refactor code [https://github.com/fluent/fluentd/pull/2829](https://togithub.com/fluent/fluentd/pull/2829)2[https://github.com/fluent/fluentd/pull/2830](https://togithub.com/fluent/fluentd/pull/2830)u[https://github.com/fluent/fluentd/pull/2832](https://togithub.com/fluent/fluentd/pull/2832)t[https://github.com/fluent/fluentd/pull/2836](https://togithub.com/fluent/fluentd/pull/2836)l[https://github.com/fluent/fluentd/pull/2838](https://togithub.com/fluent/fluentd/pull/2838)n[https://github.com/fluent/fluentd/pull/2842](https://togithub.com/fluent/fluentd/pull/2842)f[https://github.com/fluent/fluentd/pull/2843](https://togithub.com/fluent/fluentd/pull/2843)com/fluent/fluentd/pull/2843 ##### Bug fix - buffer: Add seq to metadata that it can be unique [https://github.com/fluent/fluentd/pull/2824](https://togithub.com/fluent/fluentd/pull/2824)2[https://github.com/fluent/fluentd/pull/2853](https://togithub.com/fluent/fluentd/pull/2853)ull/2853 - buffer: Use `Tempfile` as binmode for decompression [https://github.com/fluent/fluentd/pull/2847](https://togithub.com/fluent/fluentd/pull/2847)2847 ##### Misc - Add `.idea` to git ignore file [https://github.com/fluent/fluentd/pull/2834](https://togithub.com/fluent/fluentd/pull/2834)2834 - appveyor: Fix tests [https://github.com/fluent/fluentd/pull/2853](https://togithub.com/fluent/fluentd/pull/2853)2[https://github.com/fluent/fluentd/pull/2855](https://togithub.com/fluent/fluentd/pull/2855)ull/2855 - Update pem for test [https://github.com/fluent/fluentd/pull/2839](https://togithub.com/fluent/fluentd/pull/2839)2839 ### [`v1.9.2`](https://togithub.com/fluent/fluentd/blob/HEAD/CHANGELOG.md#Release-v192---20200213) [Compare Source](https://togithub.com/fluent/fluentd/compare/v1.9.1...v1.9.2) ##### Enhancement - in_tail: Add `pos_file_compaction_interval` parameter for auto compaction [https://github.com/fluent/fluentd/pull/2805](https://togithub.com/fluent/fluentd/pull/2805)2805 - command: Use given encoding when RUBYOPT has `-E` [https://github.com/fluent/fluentd/pull/2814](https://togithub.com/fluent/fluentd/pull/2814)2814 ##### Bug fix - command: Accept RUBYOPT with two or more options [https://github.com/fluent/fluentd/pull/2807](https://togithub.com/fluent/fluentd/pull/2807)2807 - command: Fix infinite loop bug when RUBYOPT is invalid [https://github.com/fluent/fluentd/pull/2813](https://togithub.com/fluent/fluentd/pull/2813)2813 - log: serverengine's log should be formatted with the same format of fluentd [https://github.com/fluent/fluentd/pull/2812](https://togithub.com/fluent/fluentd/pull/2812)2812 - in_http: Fix `NoMethodError` when `OPTIONS` request doesn't have 'Origin' header [https://github.com/fluent/fluentd/pull/2823](https://togithub.com/fluent/fluentd/pull/2823)2823 - parser_syslog: Improved for parsing RFC5424 structured data in `parser_syslog` [https://github.com/fluent/fluentd/pull/2816](https://togithub.com/fluent/fluentd/pull/2816)2816 ### [`v1.9.1`](https://togithub.com/fluent/fluentd/blob/HEAD/CHANGELOG.md#Release-v191---20200131) [Compare Source](https://togithub.com/fluent/fluentd/compare/v1.9.0...v1.9.1) ##### Enhancement - http_server helper: Support HTTPS [https://github.com/fluent/fluentd/pull/2787](https://togithub.com/fluent/fluentd/pull/2787)2787 - in_tail: Add `path_delimiter` to split with any char [https://github.com/fluent/fluentd/pull/2796](https://togithub.com/fluent/fluentd/pull/2796)2796 - in_tail: Remove an entry from PositionaFile when it is unwatched [https://github.com/fluent/fluentd/pull/2803](https://togithub.com/fluent/fluentd/pull/2803)2803 - out_http: Add warning for `retryable_response_code` [https://github.com/fluent/fluentd/pull/2809](https://togithub.com/fluent/fluentd/pull/2809)2809 - parser_syslog: Add multiline RFC5424 support [https://github.com/fluent/fluentd/pull/2767](https://togithub.com/fluent/fluentd/pull/2767)2767 - Add TLS module to unify TLS related code [https://github.com/fluent/fluentd/pull/2802](https://togithub.com/fluent/fluentd/pull/2802)2802 ##### Bug fix - output: Add `EncodingError` to unrecoverable errors [https://github.com/fluent/fluentd/pull/2808](https://togithub.com/fluent/fluentd/pull/2808)2808 - tls: Fix TLS version handling in secure mode [https://github.com/fluent/fluentd/pull/2802](https://togithub.com/fluent/fluentd/pull/2802)2802 </details> --- ### Configuration 📅 Schedule: Branch creation - "" (UTC), Automerge - At any time (no schedule defined). 🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied. ♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 Ignore: Close this PR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR has been generated by [Mend Renovate](https://www.mend.io/free-developer-tools/renovate/). View repository job log [here](https://developer.mend.io/github/grafana/loki). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy44LjEiLCJ1cGRhdGVkSW5WZXIiOiIzNy44LjEiLCJ0YXJnZXRCcmFuY2giOiJtYWluIn0=--> Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>	2 years ago
Michel Hollands	eb3cff4625	Update Go version to 1.21.2 in Dockerfiles (#10805 ) What this PR does / why we need it: Update the go version in the Dockerfiles to 1.21.2 Special notes for your reviewer: Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) --------- Signed-off-by: Michel Hollands <michel.hollands@gmail.com>	2 years ago
Kevin Burke	da175771c0	clients/pkg/promtail: tests no longer listen on 0.0.0.0 (#10599 ) As a result, when you run the tests on a Mac, they no longer prompt the user to ask if they are OK with the tests listening on an external address. This makes the tests substantially easier to run on a Mac, in addition to improving this library's security posture. Updates #10407. Checklist - [X] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [x] Documentation added - [x] Tests updated These are not required - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) Co-authored-by: Michel Hollands <42814411+MichelHollands@users.noreply.github.com>	2 years ago
Paschalis Tsilias	07cbef9226	Update Prometheus dependency to v2.46.0 (#10777 ) This PR updates the Prometheus dependency to v2.46.0. We need this to update the Agent to use Prometheus' Native Histograms after some bug fixes in the latest version. I bumped the version in go.mod, ran `go mod tidy && go mod vendor` and fixed compilation issues. Special notes for your reviewer: The only changes needed to have `make all` successfully compile was to use the new `wlog.NewSize` to use wlog.CompressionNone (instead of false), and wlog.CompressionSnappy (the default, instead of true), as well as add a new argument to the textparse.New constructor to work as it did without scraping native histograms via protobuf. I don't think there are any other user-facing changes or changes related to Loki. --------- Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com>	2 years ago
Christian Haudum	d3d7ab7d82	Promtail: Remove deprecated stream_lag_labels setting (#10677 ) The setting was deprecated and already caused an error on startup when used. This behaviour was noticed when looking at the reference configuration docs of Promtail. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
Šimon Lukašík	209b4d83af	fluentd docs: fix broken link (#10640 ) Co-authored-by: J Stickler <julie.stickler@grafana.com>	2 years ago
Piotr	a8d5815510	Fix race condition when stopping to tail files (#10631 ) What this PR does / why we need it: Users are getting `PromtailFileMissing` alerts, which are defined as: ``` promtail_file_bytes_total{namespace!=""} unless promtail_read_bytes_total{namespace!=""} ``` This appears to be due to a race condition where a file can be deleted while the file target is being stopped.	2 years ago
Kevin Burke	9dad6a1350	clients,pkg: use http.NewRequestWithContext (#10616 ) What this PR does / why we need it: It saves one line, and the new function (vs. http.NewRequest) performs additional validation of the request. Which issue(s) this PR fixes: None Special notes for your reviewer: None Checklist - [X] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) Not necessary: - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) Co-authored-by: Michel Hollands <42814411+MichelHollands@users.noreply.github.com>	2 years ago
Kevin Burke	23d8c7c63c	clients/pkg/promtail/client: better error on nil metrics (#10611 ) Fixes #10610. Checklist - [X] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) Not necessary: - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`)	2 years ago
Kevin Burke	1256692f95	clients/pkg/promtail/targets/gcplog: fix spelling error (#10615 ) What this PR does / why we need it: Fix spelling error Which issue(s) this PR fixes: none Checklist - [X] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) Not necessary: - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`)	2 years ago
Dylan Guedes	4681386775	Update alpine versions (#10573 ) What this PR does / why we need it: Update alpine versions used by our images to `3.18.3` Which issue(s) this PR fixes: N/A Special notes for your reviewer: Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`)	2 years ago
Christian Haudum	2b6bdc2a29	chore(storage): Simplify store interfaces and abstractions (pt 2) (#10454 ) What this PR does / why we need it: * Remove preemptive interface from indexReaderWriter and make it an exported struct * Compose IngesterStore and QuerierStore The ingester and querier do not require a store that implements the full interface, but can accept stores that are a subset of the interfaces. Therefore we can make specific interfaces that are accepted by the ingester and querier constructors. * Split up ChunkFetcher interface This commit splits the ChunkFetcher interface into a ChunkFetcher and a ChunkFetcherProvider interface. This has the advantage that methods can allow accepting smaller interfaces. This commit also renames the `GetChunkRefs()` method of the ChunkFetcher interface into `GetChunks()`, because it returns a `[][]chunk.Chunk`, not a `[]chunk.ChunkRef` type. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
Vladyslav Diachenko	00a3c5b026	renamed `non indexed labels` feature to `structured metadata` (#10432 ) Loki team has made a decision to rename the feature because the new name is more precise. --------- Signed-off-by: Vladyslav Diachenko <vlad.diachenko@grafana.com> Co-authored-by: Salva Corts <salva.corts@grafana.com>	2 years ago
Đurica Yuri Nikolić	37ec68eebb	Remove unnecessary code after dskit log.Interface has been dereleased (#10434 ) This PR removes unnecessary code used to convert the dereleased `dslog.Interface` (see https://github.com/grafana/dskit/pull/359) into `go-kit.Logger`. Additionally, it modifies the signature and implementation of `pkg/util/log/log.go`'s `InitLogger` as follows: - previous signature ``` InitLogger(server.Config, prometheus.Registerer, bool, bool) ``` has been replaced with ``` InitLogger(server.Config, prometheus.Registerer, bool, bool) log.Logger ``` Namely, the new `InitLogger` initialises the global logger and returns it. - the new implementation initializes the global logger, but it does not override the default logger for the server. This is now done by the callers of `InitLogger`. Previously, it was done inside `InitLogger` for convenience, since the types of the 2 loggers were not the same. Signed-off-by: Yuri Nikolic <durica.nikolic@grafana.com>	2 years ago
Vladyslav Diachenko	6387bf765c	[promtail] include the size of non-indexed labels into batch size (#10412 ) What this PR does / why we need it: When we added non-indexed-labels support to the promtail, we forgot to adjust the batch size calculation which leads to the case when promtail thinks that batch size is `X` but when it sends it to Loki, Loki rejects it because the actual size of the payload is `X`+`N`(size of non-indexed labels). Special notes for your reviewer: This feature is not released yet, so, changelog update is not needed. Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [x] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) Signed-off-by: Vladyslav Diachenko <vlad.diachenko@grafana.com>	2 years ago
Shantanu Alsi	586b9042c6	Upgrade Thanos objstore, dskit and other modules (#10366 ) What this PR does / why we need it: Upgrades various common modules including dskit, thanos objstore, prometheus, etc. Fixes errors/ changes resulting due to module upgrades. This is required in order to eventually upgrade BEL dependency in GEL to point to master. Which issue(s) this PR fixes: None Special notes for your reviewer: Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [x] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`)	2 years ago
William Dumont	a1c1152b79	Promtail: Add additional fields in cloudflare config (#10301 ) What this PR does / why we need it: In this PR the cloudflare config is enhanced by adding a custom fields type and by having a new property "additional_types" to give the customer the ability to specify the fields that he wants. This functionality has been added in the grafana agent and therefore should also be added in promtail. Which issue(s) this PR fixes: Fixes https://github.com/grafana/agent/issues/4860 Special notes for your reviewer: Checklist - [X] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [X] Documentation added - [X] Tests updated - [x] `CHANGELOG.md` updated - [x] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [x] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md`	2 years ago
Arve Knudsen	a60e0167a8	Upgrade to latest dskit main (#10300 ) What this PR does / why we need it: Upgrade to latest dskit main branch, which mainly has changes to its `log` package, which requires corresponding fixing up in Loki.	2 years ago
Shantanu Alsi	b57d260ddc	Upgrade alpine 3.16.5 -> 3.16.7 (#10188 ) What this PR does / why we need it: Which issue(s) this PR fixes: Fixes #10181 Fixes Open SSL vulnerabilities [CVE-2023-1255](https://security.alpinelinux.org/vuln/CVE-2023-1255), [CVE-2023-2650](https://security.alpinelinux.org/vuln/CVE-2023-2650), [CVE-2023-2975](https://security.alpinelinux.org/vuln/CVE-2023-2975), [CVE-2023-3446](https://security.alpinelinux.org/vuln/CVE-2023-3446), [CVE-2023-3817](https://security.alpinelinux.org/vuln/CVE-2023-3817) Alpine Release Notes: [3.16.7](https://www.alpinelinux.org/posts/Alpine-3.15.10-3.16.7-3.17.5-3.18.3-released.html) [3.16.6](https://www.alpinelinux.org/posts/Alpine-3.15.9-3.16.6-3.17.4-3.18.2-released.html) Special notes for your reviewer: Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [x] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`)	2 years ago
Charles Korn	bde65667f7	Upgrade dskit to use packages migrated from weaveworks/common (#10164 ) What this PR does / why we need it: This PR upgrades dskit and replaces use of packages from weaveworks/common with their migrated equivalents in dskit. See https://github.com/grafana/dskit/pull/342 for more details. Note that Loki still uses some packages from weaveworks/common that I haven't migrated (`aws` and `test`) - I'll migrate these separately. If this PR needs to be rebuilt, I used `rewrite.sh` ([source](https://gist.github.com/charleskorn/48efe62a09d6d70f3de30327003df5c5#file-rewrite-sh)) to generate most of these changes. Which issue(s) this PR fixes: (none) Special notes for your reviewer: (none) Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [n/a] Documentation added - [n/a] Tests updated - [n/a] `CHANGELOG.md` updated - [n/a] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [n/a] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [n/a] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) Co-authored-by: Kaviraj Kanagaraj <kavirajkanagaraj@gmail.com>	2 years ago
Vladyslav Diachenko	0fc934e4ea	non_indexed_labels stage: additional test-case (#10170 ) added additional test-case to cover match and non_indexed_labels stages together. Signed-off-by: Vladyslav Diachenko <vlad.diachenko@grafana.com>	2 years ago
Michel Hollands	d484d31856	Add support for cross arch for fluent-bit-plugin-loki image (#10081 ) Copied from https://github.com/grafana/loki/pull/9991 which has problems running CI. What this PR does / why we need it: Currently the `fluent-bit-plugin-loki` image is only build and published for amd64. With more use of arm-based processors, it would be good to add support for those as well. I've build the image using `docker buildx` on my M1 Mac and pushed it to Dockerhub, should you wish to try it first: https://hub.docker.com/r/ebosveld/fluent-bit-plugin-loki/ On AWS ECS Graviton based images it works well with the above mentioned image. I've also fixed #9932 as it needed the full bullseye support. Which issue(s) this PR fixes: Fixes #6738 Fixes #9932 Special notes for your reviewer: Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) --------- Signed-off-by: Michel Hollands <michel.hollands@gmail.com> Co-authored-by: Edwin Bosveld <edwin.bosveld@persgroep.net> Co-authored-by: Travis Patterson <travis.patterson@grafana.com> Co-authored-by: Edwin Bosveld <ebosveld@users.noreply.github.com>	2 years ago
Bruno FERNANDO	a8a349690b	feat(promtail): retrieve BotTags field from cloudflare (#10010 ) What this PR does / why we need it: Hey folks 👋 Little contribution here to add a useful log field for cloudflare users. Indeed I add the [BotTags field](https://developers.cloudflare.com/bots/concepts/cloudflare-bot-tags/) which is pretty useful when debugging some specific traffic handled by cloudflare Don't hesitate to reach me if you have any questions Cheers 😉 Which issue(s) this PR fixes: Fixes #<issue number> Special notes for your reviewer: Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [x] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) --------- Co-authored-by: Michel Hollands <42814411+MichelHollands@users.noreply.github.com>	2 years ago
J Stickler	7b671348a9	Delete incomplete comment. (#10046 ) Which issue(s) this PR fixes: Fixes #9960 Deletes incomplete comment in code that was introduced from Prometheus as part of `fb26baa5b1`	2 years ago
Pablo	5f04eedc03	Refactor Promtail's `client.Manager` to do multi client instantiation (#9908 ) What this PR does / why we need it: This PR refactors Promtails `client.Manager` to include both WAL-enabled client, and WAL disabled. This will make the transition easier towards the agent codebase, only needing to instantiate one primitive to support both WAL enabled/disabled clients under the `loki.write` component. Also, re-uses a bunch of logic shared by both. Which issue(s) this PR fixes: Part of https://github.com/grafana/loki/issues/8197 Special notes for your reviewer: Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [x] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`)	2 years ago
Vladyslav Diachenko	416e01ca8e	[promtail] non_indexed_labels stage (#9986 ) What this PR does / why we need it: added non_indexed_labels stage to copy values from extracted data tonon-indexed-labels field of the Entry. Special notes for your reviewer: Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) --------- Signed-off-by: Vladyslav Diachenko <vlad.diachenko@grafana.com>	2 years ago
Karsten Jeschkies	2dfac72835	Clarify that label filter expressions are not supported in promtail. (#9894 ) What this PR does / why we need it: Some users of Promtail seem to be confused that certain types of filter expressions are not supported in Promtail's `match` stage. This change clarifies what kind of filter expressions are supported. Which issue(s) this PR fixes: Fixes #5209 Special notes for your reviewer: Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [x] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) --------- Co-authored-by: J Stickler <julie.stickler@grafana.com>	2 years ago
Michel Hollands	bad691b509	Update go build image (#9914 ) Update the go build image --------- Signed-off-by: Michel Hollands <michel.hollands@grafana.com>	2 years ago
Michel Hollands	7be9deacf3	Update go version to 1.20.6 (#9913 ) Update the go version. Another PR will follow that uses the new build image --------- Signed-off-by: Michel Hollands <michel.hollands@grafana.com>	2 years ago
Ashwanth	de937e72cd	promtail: apply defaults to HTTP client config (#9863 ) What this PR does / why we need it: Apply defaults to HTTPClientConfig and validate it. similar to how it's done [here](`e9644d8186/vendor/github.com/prometheus/prometheus/config/config.go (L524)`) in prometheus. promtail [configuration page](https://grafana.com/docs/loki/latest/clients/promtail/configuration/) incorrectly claims that `follow_redirects` is true, but it is being set to false given that the defaults are not being applied currently Which issue(s) this PR fixes: Fixes #<issue number> Special notes for your reviewer: Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [x] Tests updated - [x] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) --------- Signed-off-by: Ashwanth Goli <iamashwanth@gmail.com> Co-authored-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
René Scheibe	954df433e9	Promtail: Fix file descriptor leak (#8987 ) This reduces the amount of open file descriptors per followed journal file from 3 to 2. It also releases file descriptors for removed files (journal vacuuming). The `defaultJournalEntryFunc` is only called once upon startup. It creates a `Journal` instance to only read a single entry to determine the age of the entry referenced in the positions file. Right after, this instance is not required anymore. The actually used `Journal` instance used to follow the journal is created inside `journalTargetWithReader`.	2 years ago
Jan Jansen	8abe080fd6	Promtail: improve behavior of partial lines (#9508 ) What this PR does / why we need it: I started to see OOM after upgrading to a promtail version with #8497. So played around to make partial line behavior more stable. You can now truncate partial lines and configure max partial lines buffer. Which issue(s) this PR fixes: Fixes #<issue number> Special notes for your reviewer: Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [x] Documentation added - [x] Tests updated - [x] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` Signed-off-by: Jan Jansen <jan.jansen@gdata.de> Co-authored-by: Michel Hollands <42814411+MichelHollands@users.noreply.github.com>	2 years ago
Trevor Whitney	4a56445686	Upgrade `golangci-lint` and fix linting errors (#9601 ) What this PR does / why we need it: Upgrade `golangci-lint` and fixes all the errors. The upgrade includes some stricter linting.	2 years ago
Travis Patterson	065bee7e72	Label Volume Endpoint (#9588 ) For a given set of matchers, returns the top N associated label/value pairs by volume. A query for `{cluster=prod}` will return ``` cluster=prod: size (total logs matching this matcher) . . . nth-label=nth-value ``` This is to service use cases where users want to understand where their log volume has come from by label without making multiple requests to the stats endpoint. Note: This PR is a monster but it's mostly plumbing. I've pointed out the most interesting bits that actually get the volumes from ingesters/indexs	2 years ago
Bruno FERNANDO	1b410dbc01	feat(promtail): add CF ClientRequestSource field (#9669 ) What this PR does / why we need it: Hey folks 👋 Little contribution here to add a useful log field for cloudflare users. Indeed I add the [ClientRequestSource field](https://developers.cloudflare.com/logs/reference/clientrequestsource/ ) which is pretty useful when debugging some specific traffic handled by cloudflare Extra: Since I was on the documentation I fixed an indentation issue that I spotted Don't hesitate to reach me if you have any questions Cheers 😉 Which issue(s) this PR fixes: Fixes #<issue number> Special notes for your reviewer: Loki rocks 🚀 Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [x] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`)	2 years ago
Piotr	ac4a442291	Promtail: Fix goroutine leak in file tailer (#9495 ) What this PR does / why we need it: There's a goroutine leak in promtail file tailer, which has been encountered in https://github.com/grafana/agent/issues/3885. The issue is the following: - There are two goroutines that tailer manages: one for reading lines, the other for updating positions: https://github.com/grafana/loki/blob/main/clients/pkg/promtail/targets/file/tailer.go#L103 - The tailer can be cleanly stopped & drained via `.Stop()` https://github.com/grafana/loki/blob/main/clients/pkg/promtail/targets/file/tailer.go#L221 - But if the goroutine reading lines encounters an error from the underlying `tail`, the tailer is stopped without draining using the defer statement here: https://github.com/grafana/loki/blob/main/clients/pkg/promtail/targets/file/tailer.go#L151 - The above defer statement does not shut down the position updating goroutine and the `.Stop()` won't be called, so we end up with a leaked goroutine. Which issue(s) this PR fixes: Fixes https://github.com/grafana/agent/issues/3885 As can be seen in the [goroutine dump attached to the issue](https://gist.github.com/amseager/e37f31984facd9cbc51ce0f2153d6864), there were 933 update position goroutines, while we only had 21 files being tailed. Special notes for your reviewer: Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [x] Tests updated - [x] `CHANGELOG.md` updated	2 years ago
Pablo	306c96ce8a	Notification based Promtail WAL Watcher polling approach (#9135 ) What this PR does / why we need it: This PR implements a new mechanism for the wal Watcher in Promtail, to know there are new records to be read. It uses a combination of: - https://github.com/prometheus/prometheus/pull/11950 - https://github.com/prometheus/prometheus/pull/11949 The main idea is that the primary mechanism is a notification channel between the `wal.Writer` and `wal.Watcher`. The Watcher subscribes to write events the writer publishes, getting notified if the wal has been written. The same subscriptions design is used for cleanup events. As a backup, the watcher has a timer that implements an exponential backoff strategy, which is constrained by a minimum and maximum that the user can configure. Below the cpu difference is shown of running both main and this branch against the same scrape target. <img width="2496" alt="image" src="https://user-images.githubusercontent.com/2617411/232099483-7e5c36fa-9360-4eb9-8240-687adf46e330.png"> The yellow line is the latest main build from where this branch started, and the green line is this branch. Both promtails tailing docker logs, and using the following metrics to get cpu usage from cadvisor: ``` avg by (name) (rate(container_cpu_usage_seconds_total{job=~".+", instance=~".+", name=~"promtail-wal-test_promtail.+"}[$__rate_interval])) ``` Which issue(s) this PR fixes: Part of https://github.com/grafana/loki/issues/8197 Special notes for your reviewer: Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	2 years ago
cui fliter	520e434446	fix function names in comments (#9415 )	2 years ago
Jan Jansen	c4048a3adf	Promtail: Break on iterate journal failure (#9155 )	2 years ago
Danny Kopping	9495b6cc56	Update loki-build-image with go1.20.4 (#9372 ) Continuation of #9370 Signed-off-by: Danny Kopping <danny.kopping@grafana.com>	2 years ago
Danny Kopping	f5c0fdf028	Upgrade to latest security release: go1.20.4 (#9370 ) What this PR does / why we need it: https://groups.google.com/g/golang-announce/c/MEb0UyuSMsU Also updated the build image. --------- Signed-off-by: Danny Kopping <danny.kopping@grafana.com>	2 years ago

1 2 3 4 5 ...

296 Commits (8628b15624e0c1f5cd730cc9ee585416926c09c3)