272 Commits (8abe080fd6d04dc5efb074ee0764a9bfd39c82d2)

Author	SHA1	Message	Date
Jan Jansen	8abe080fd6	Promtail: improve behavior of partial lines (#9508) ... **What this PR does / why we need it**: I started to see OOM after upgrading to a promtail version with #8497. So played around to make partial line behavior more stable. You can now truncate partial lines and configure max partial lines buffer. **Which issue(s) this PR fixes**: Fixes #<issue number> **Special notes for your reviewer**: **Checklist** - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [x] Documentation added - [x] Tests updated - [x] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` Signed-off-by: Jan Jansen <jan.jansen@gdata.de> Co-authored-by: Michel Hollands <42814411+MichelHollands@users.noreply.github.com>	3 years ago
Jack Baldry	8391e557b1	Update doc-validator to v3.0.0 (#9666) ... ## 3.0.0 ### Added - Structured output for use with [`reviewdog`](https://github.com/reviewdog/reviewdog). You can achieve the original error output by piping the output to the following `jq` expression: `jq -r '"ERROR: \(.location.path):\(.location.range.start.line):\(.location.range.start.column): \(.message)"'`. - Suggestions for simple link fixes. In GitHub, [`reviewdog`](https://github.com/reviewdog/reviewdog) comments these suggestions for convenient replacement. - Support for anchors referring to repeated headings. In the case that a page has multiple headings that share the same text, the renderer appends a zero indexed, numbered suffix to the identifier. For example, when there are two headings that are both "Heading text", the first anchor identifier is `heading-text` and the second is `heading-text-1`. - Error when running `doc-validator` on no files. **Notes for reviewer:** This also updates the `make-docs` procedure. Please ensure you can run `make docs` from the `docs/` directory before approving and merging this PR. --------- Signed-off-by: Jack Baldry <jack.baldry@grafana.com>	3 years ago
Christophe Collot	f2394355d8	feat(lambda-promtail): add cloudfront log file ingestion support (#9573) ... **What this PR does / why we need it**: This PR enables ingesting logs from Cloudfront log files stored in s3 (batch). The current setup only supports streaming Cloudfront logs through AWS Kinesis, whereas this PR implements the same flow as for VPC Flow logs, Load Balancer logs, and Cloudtrail logs (s3 --> SQS (optional) --> Lambda Promtail --> Loki) **Special notes for your reviewer**: + The Cloudfront log file format is different from the already implemented services, meaning we had to build yet another regex. AWS never bothered making all services follow the same log file naming convention but the "good" thing is that it's now very unlikely they will change it in the future. + The Cloudfront file name does not have any mention of the AWS account or the time of log it contains, it means we have to infer the log type from the filename format instead of finding the exact string "cloudfront" in the filename. This is why in `getLabels`, if no `type` parameter is found in the regex, we use the key corresponding to the name of the matching parser. + I introduced a new `parser` struct to group together several parameters specific to a type of log (and avoid relying too much on map key string matching and / or if statements for specific use cases) + I've been successfully running this code in several AWS environments for days. + I corrected a typo from my previous PR #9497 (wrong PR number in Changelog.md) **Checklist** - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [x] Documentation added - [x] Tests updated - [x] `CHANGELOG.md` updated - [x] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [x] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](d10549e3ec) --------- Co-authored-by: Michel Hollands <42814411+MichelHollands@users.noreply.github.com>	3 years ago
Bruno FERNANDO	1b410dbc01	feat(promtail): add CF ClientRequestSource field (#9669) ... **What this PR does / why we need it**: Hey folks 👋 Little contribution here to add a useful log field for cloudflare users. Indeed I add the [ClientRequestSource field](https://developers.cloudflare.com/logs/reference/clientrequestsource/ ) which is pretty useful when debugging some specific traffic handled by cloudflare Extra: Since I was on the documentation I fixed an indentation issue that I spotted Don't hesitate to reach me if you have any questions Cheers 😉 **Which issue(s) this PR fixes**: Fixes #<issue number> **Special notes for your reviewer**: Loki rocks 🚀 **Checklist** - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [x] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](d10549e3ec)	3 years ago
Pepe Cano	4cebc2d85c	Docs: replace `k6 Cloud` mention (#9599) ... k6 is now available as a managed service on Grafana Cloud. This is a small doc changes to remove the mention of `k6 Cloud`. --------- Co-authored-by: J Stickler <julie.stickler@grafana.com>	3 years ago
Eric	c6f809a738	Fix missing ~ in the regexp match for namespace (#9618) ... **What this PR does / why we need it**: Fix documentation which has a missing `~` for the regexp match. I was following the documentation and found it odd that this query didn't follow [LogQL](https://grafana.com/docs/loki/latest/logql/log_queries/#log-queries), but it actually does and was just a typo in the docs. I've tested this with my own Loki 2.8 release and with the `~` this works great! I also added a similar description block to the above example explaining the pipeline, but if this is not desired, I'm happy to remove it. **Which issue(s) this PR fixes**: NA, trivial issue, just a small fix in the docs. **Special notes for your reviewer**: **Checklist** - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [x] Documentation added - [x] Tests updated - [x] `CHANGELOG.md` updated - [x] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [x] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](d10549e3ec)	3 years ago
Christophe Collot	b709b32d6a	feat(lambda-promtail): add cloudtrail log ingestion support (#9497) ... **What this PR does / why we need it**: ### Add support for AWS CloudTrail audit logs ingestion using lambda-promtail Calls to AWS APIs are logged in AWS Cloudtrail and are helpful for security and debugging purposes. However, I've experienced difficulties with it: + The AWS CloudTrail service is not well integrated with Prometheus (no metrics, no alerts) and I don't want to manage alerts in CloudWatch Alerts + The search experience is painful with CloudTrail via the AWS Console (I will not elaborate 😅). This PR allows ingesting CloudTrail audit logs sent to an S3 bucket using the same approach as VPC flow logs or Load Balancer logs. **Special notes for your reviewer**: + Because the Cloudtrail file format is not text but json, we stream the json CloudTrail records instead of using the already existing scanner. + Because the Cloudtrail filename format is not the same as for the Flow log or the Load balancer log files, we need to split the regexes by service (although many AWS services seem to share the same `defaultFilenameRegex`). + In the `getLabels` function, we expect the `type` parameter to be found in the filename using the Regex. For some log files (ex: Cloudfront log files). The file name has no reference to the service name. This is why, as a default, when no type is found in the name of the file, I set it to use the key of the matching Regex expression. **Checklist** - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [x] Documentation added - [x] Tests updated - [x] `CHANGELOG.md` updated - [x] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	3 years ago
BlakeFernandez	905d14bee7	Add Prometheus Link for scrape_configs (#9270) ... **What this PR does / why we need it**: Adds some context to the scrape configs section in Promtail Config Docs to ensure clarity around the connection to Prometheus Here is a link to a slack thread where the customer was confused on this: https://raintank-corp.slack.com/archives/C04KY6NJSCE/p1681761335754199 --------- Co-authored-by: Michel Hollands <42814411+MichelHollands@users.noreply.github.com> Co-authored-by: J Stickler <julie.stickler@grafana.com>	3 years ago
Alexandre de Verteuil	e38ccb4418	Uniformize "Available Labels" headers in Promtail config reference doc (#9436) ... **What this PR does / why we need it**: There were various formats for the "Available Labels" headings. This PR makes them all the same heading level and spelling.	3 years ago
bluesky6529	a9cde84c95	Fix typo on lambda-promtail docs (#9472) ... **What this PR does / why we need it**: fix for doc typo **Which issue(s) this PR fixes**:N/A **Special notes for your reviewer**: N/A	3 years ago
MattiasSegerdahl	56593efa56	Update geoip.md (#9334) ... - Corrected spelling mistake, databse - Added link to Maxmind's website - Added link to Grafana blog post **What this PR does / why we need it**: **Which issue(s) this PR fixes**: Fixes #<issue number> **Special notes for your reviewer**: **Checklist** - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` --------- Co-authored-by: Travis Patterson <travis.patterson@grafana.com>	3 years ago
J Stickler	1bb489c438	TOC Sync: Move LogQL and LogCLI files under Query. (#9200)	3 years ago
J Stickler	8ee9a2c3c7	TOC Sync: Move API docs under Reference. (#9203) ... **Which issue(s) this PR fixes**: Fixes #8719 **Special notes for your reviewer**: Part of the Database products table of contents alignment project. Moves API doc under Reference heading in TOC. Updates front matter for the topic. Fixes links broken by the move. Adds aliases for redirects.	3 years ago
Ed Welch	25c4b4ce35	Update docs to make file rotation support clearer (#9391)	3 years ago
MattiasSegerdahl	233e52546e	Update _index.md (#9331) ... **What this PR does / why we need it**: **Which issue(s) this PR fixes**: Fixes #<issue number> **Special notes for your reviewer**: **Checklist** - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	3 years ago
MattiasSegerdahl	c4e5d0d322	Update write-scenario.md (#9312) ... **What this PR does / why we need it**: **Which issue(s) this PR fixes**: Fixes #<issue number> **Special notes for your reviewer**: **Checklist** - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	3 years ago
MattiasSegerdahl	24cce4f799	Update gcplog-cloud.md (#9304) ... **What this PR does / why we need it**: **Which issue(s) this PR fixes**: Fixes #<issue number> **Special notes for your reviewer**: **Checklist** - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	3 years ago
MattiasSegerdahl	329c0d0d33	Update _index.md (#9305) ... **What this PR does / why we need it**: **Which issue(s) this PR fixes**: Fixes #<issue number> **Special notes for your reviewer**: **Checklist** - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	3 years ago
MattiasSegerdahl	c5bf444d7b	Update geoip.md (#9311) ... **What this PR does / why we need it**: **Which issue(s) this PR fixes**: Fixes #<issue number> **Special notes for your reviewer**: **Checklist** - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	3 years ago
MattiasSegerdahl	d14669d20b	Update cri.md (#9310) ... **What this PR does / why we need it**: **Which issue(s) this PR fixes**: Fixes #<issue number> **Special notes for your reviewer**: **Checklist** - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	3 years ago
steve-caron-grafana	86d628e827	Update _index.md (#9347) ... **What this PR does / why we need it**: Doc update: Adding Cribl and Vector plugins as unofficial clients - this is something we are seeing more and more in the field, customer using either Vector and Cribl as observability data pipelines to decide which log data will be sent to which log backend (including Loki). **Which issue(s) this PR fixes**: **Special notes for your reviewer**: **Checklist** - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	3 years ago
Labesse Kévin	0215a17148	gcplog: improve formatter (#9044) ... **What this PR does / why we need it**: Add two improvement to the target `gcplog` * capability to ignore the `textPayload` from the log line -> don't trunk important metadata * add new source labels (severity and root labels of log entry) -> more relabel_config possibilities **Special notes for your reviewer**: I'm not sure about the naming of the option `useFullLine`, feel free to suggest any name, but I guess this feature must be disable by default for compatibility? **Checklist** - [X] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [X] Documentation added - [X] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` --------- Signed-off-by: Kevin Labesse <kevin@labesse.me> Co-authored-by: J Stickler <julie.stickler@grafana.com>	3 years ago
logan	556af7609a	Fixing spelling errors and incorrect units (#9082) ... **What this PR does / why we need it**: mild grammar changes **Which issue(s) this PR fixes**: None **Special notes for your reviewer**: **Checklist** - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [x] Documentation added - [ ] ~~Tests updated~~ - [ ] ~~`CHANGELOG.md` updated~~ - [ ] ~~Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`~~	3 years ago
nikuljain	b5946df30e	Update configuration.md (#9085) ... **What this PR does / why we need it**: **Which issue(s) this PR fixes**: Fixes #<issue number> **Special notes for your reviewer**: **Checklist** - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	3 years ago
Mitsuru Kariya	926a4e148e	Docs: add missing configuration option (#9092) ... **What this PR does / why we need it**: Promtail can set the tenant id from a label with PR 6290 (commit a1e0298a5). However, while docs/sources/clients/promtail/stages/tenant.md was appropriately updated, docs/sources/clients/promtail/configuration.md was left untouched. **Which issue(s) this PR fixes**: **Special notes for your reviewer**: **Checklist** - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	3 years ago
René Scheibe	e050d5c494	Docs: Update Loki GitHub URLs (#9080) ... * use `main` instead of `master` branch * fix outdated URLs	3 years ago
Robert Fratto	7bec727c6d	promtail: support configurable polling interval (#8874) ... Currently, Promtail always polls files every 250ms. This PR allows the polling period to be customized, starting at a minimum polling period and exponentially increasing the polling frequency to a bounded maximum if the file has no changes. When file changes are detected, the polling frequency resets to the minimum. The default behavior is the existing behavior; both the minimum and maximum polling intervals are set to 250ms. I'm opening this in draft to get early feedback. There's a few open questions: * Where can I add validations to ensure that the min/max polling intervals are configured properly? * Is this the right approach for implementation? Do I need to make any changes? * Does this need tests? I'll update the CHANGELOG and Documentation after we're agreed on an approach.	3 years ago
Dylan Guedes	6dfcd59c1b	Promtail: Add `decompression` configuration (#8994) ... **What this PR does / why we need it**: Adds a new `compression` configuration to Promtail to customize decompression behavior. As of now, two options are supported: `enabled`: false by default but if truthy, Promtail will infer a compression algorithm to use based on the file extension `initialDelay`: 0 by default but if higher, Decompressor will sleep for the given duration before trying to decompress. This is to avoid scenarios where the Decompressor starts to work on a file that is still being compressed/generated. **Which issue(s) this PR fixes**: Fixes https://github.com/grafana/loki/issues/8784	3 years ago
kernc	e773c004f5	Docs: Mention python-logging-loki as an unofficial client (#8983) ... **What this PR does / why we need it**: Mention python-logging-loki as an unofficial client, since it looks a reasonably well done client. This PR is a continuation from https://github.com/grafana/loki/pull/8636. 😅 CLA has been signed!	3 years ago
Andrii Kushch	9f41182da2	Fix the link in the Azure Event Hubs doc (#8904) ... **What this PR does / why we need it**: This PR fixes the link to the configuration in the Azure Event Hubs scraping doc. Link to the discussion https://github.com/grafana/loki/pull/8787#discussion_r1145298591 **Which issue(s) this PR fixes**: Not available **Checklist** - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [x] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	3 years ago
Erik Sjölund	2cccc1bc68	Docs: Fix typos and grammar (#8966) ... **What this PR does / why we need it**: Fixes typos and grammar. Most of them were occurrences of `it's` that should be `its`. **Which issue(s) this PR fixes**: None **Special notes for your reviewer**: **Checklist** - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` Signed-off-by: Erik Sjölund <erik.sjolund@gmail.com>	3 years ago
chrisharbro	9efbd5ca45	typo and phrasing (#8942) ... The first sentence doesn't sound natural, tried to massage it to have a more natural sound. Fixed a minor typo.	3 years ago
李国忠	73f9ece867	[doc] client : ilogtail loki integration (#8970) ... **What this PR does / why we need it**: https://github.com/alibaba/ilogtail/pull/685 ilogtail is a very popular log agent in China, with tens of millions of installations. The open source community of ilogtail has completed the integration of loki. This PR is completed as a supplement to loki doc demo config，Type: flusher_loki ```yaml enable: true inputs: - Type: file_log LogPath: . FilePattern: simple.log flushers: - Type: flusher_loki URL: http://localhost:3100/loki/api/v1/push ExternalLabels: source: ilogtail ``` **Which issue(s) this PR fixes**: Fixes #<issue number> **Special notes for your reviewer**: flusher_loki PR snapshot  **Checklist** - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [x] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	3 years ago
Andrii Kushch	17e05d28e4	Promtail: Add a new target for the Azure Event Hub (#8787) ... **What this PR does / why we need it**: We want to allow receiving logs from the Azure Cloud. To solve this problem, we need to be able to configure Promtail to consume logs from the Azure Events Hub. To achieve this goal, we need to add a new configuration option to the scrape configuration in Promtail and implement the component that, given this configuration, can connect to the Azure Event Hub, parse received data, and prepare it to be stored in Loki. **Which issue(s) this PR fixes**: Fix [#8788](https://github.com/grafana/loki/issues/8788) # **Special notes for your reviewer**: With the current implementation, I would like to leverage Azure Events Hub's compatibility with a Kafka protocol and reuse code for a Kafka target. ### Why I decided to create a new target instead of just reusing the Kafka target: 1. Configuration for Kafka in this scenario requires fewer parameters. For example, version and auth should have some specific values. 2. Message handling is different for Azure Logs. In addition to what we are doing for the Kafka target, we want to fix JSON and split incoming messages into multiple log lines before propagating it further. ### From the implementation perspective, there are two goals: - Extract interfaces that are not Kafka configuration specific from the Kafka target module. Splitting Kafka config parsing and target creation. - To make the message handler/parser injectable or configurable: we would like to fix JSON and split a message into multiple messages. **Checklist** - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [x] Documentation added - [x] Tests updated - [x] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` --------- Co-authored-by: Pablo <2617411+thepalbi@users.noreply.github.com> Co-authored-by: Paschalis Tsilias <tpaschalis@users.noreply.github.com> Co-authored-by: J Stickler <julie.stickler@grafana.com>	3 years ago
J Stickler	b8221dc68f	Correcting typos merged in #8870. (#8873) ... **Which issue(s) this PR fixes**: Fixes typos introduced in #8870	3 years ago
annelaurefroment	01936330f4	update on Running Promtail on AWS EC2 tutorial (#8870) ... Adding more information on what to do if `__meta_ec2_private_dns_name` doesn't match the hostname.	3 years ago
李国忠	0295fd4698	[new feature] promtail: Add a Promtail stage for probabilistic sampling (#7127) ... <!-- Thanks for sending a pull request! Before submitting: 1. Read our CONTRIBUTING.md guide 2. Name your PR as `<Feature Area>: Describe your change`. a. Do not end the title with punctuation. It will be added in the changelog. b. Start with an imperative verb. Example: Fix the latency between System A and System B. c. Use sentence case, not title case. d. Use a complete phrase or sentence. The PR title will appear in a changelog, so help other people understand what your change will be. 3. Rebase your PR if it gets out of sync with main --> **What this PR does / why we need it**: The sampling stage can be directly sampled. The implementation of sampling is to use the algorithm in jaeger go client ``` pipeline_stages: - sampling: rate: 0.1 ``` or it can be used with match for precise sampling. ``` pipeline_stages: - json: expressions: app: - match: pipeline_name: "app2" selector: "{app=\"poki\"}" stages: - sampling: rate: 0.1 ``` **Which issue(s) this PR fixes**: Fixes #6654 **Special notes for your reviewer**: The promtail 'rate' stage is also used with the 'match' stage for log filtering.This design makes the code very clean. Other log agents vector have log filtering built into the sampling operator, which I think is too complicated https://vector.dev/docs/reference/configuration/transforms/sample/ ``` transforms: my_transform_id: type: sample inputs: - my-source-or-transform-id exclude: null rate: 10 ``` 'rate' stage review suggestions . https://github.com/grafana/loki/pull/5051  <!-- Note about CHANGELOG entries, if a change adds: * an important feature * fixes an issue present in a previous release, * causes a change in operation that would be useful for an operator of Loki to know then please add a CHANGELOG entry. For documentation changes, build changes, simple fixes etc please skip this step. We are attempting to curate a changelog of the most relevant and important changes to be easier to ingest by end users of Loki. Note about the upgrade guide, if this changes: * default configuration values * metric names or label names * changes existing log lines such as the metrics.go query output line * configuration parameters * anything to do with any API * any other change that would require special attention or extra steps to upgrade Please document clearly what changed AND what needs to be done in the upgrade guide. --> **Checklist** - [x] Documentation added - [x] Tests updated - [ ] Is this an important fix or new feature? Add an entry in the `CHANGELOG.md`. - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` --------- Co-authored-by: J Stickler <julie.stickler@grafana.com>	3 years ago
Langdi	9b5bbf3fb0	Fix word confusions (#8778) ... Fixes two word confusion in the docs: * in multiple occasions `\` is referred to as slash instead of backslash * "most be convertible [...]" probably should be "must be convertible [...]"	3 years ago
Paul Tirk	540380fa94	Promtail: fix GeoIP typo (#8783) ... **What this PR does / why we need it**: Fixes a typo for GeoIP stage. From `geoip_continet_name` to `geoip_continent_name`.	3 years ago
J Stickler	673bfd603b	Fixing broken link (404) (#8688) ... This was reported to the docs team via Slack. Fixing broken link from Promtail Scraping file to Promtail Configuration file.	3 years ago
Aditya C S	b94c8aaf16	feature: geoip stage in promtail (#3493) ... Co-authored-by: Ed Welch <edward.welch@grafana.com>	3 years ago
Christophe Collot	a013e9f342	Lambda-promtail: Add support for processing SQS messages, add promtailClient Type, add logger, upgrade dependencies and fix unexpected flushing behaviors (#8231)	3 years ago
Yasmine El wahbi	c6542e61fa	Promtail: Add ready endoint to loki-push-api server (#8552) ... When trying to expose loki_push_api service (in our case using GKE ingress), we need a health check target endpoint that would return a 200 response code. This PR contains a simple addition for a HTTP handler that handles the /ready path, along with the necessary tests. Fixes #8551 Co-authored-by: Christian Haudum <christian.haudum@gmail.com>	3 years ago
Nick Moore	85b3e5d4ab	Promtail: Add event log message stage (#8382) ... The Windows Event Log includes a field named Message that include a large amount of additional data, stored in a simple key-value format: key-value pairs are separated with Windows-style \r\n newlines, the key and value are separated with : (or just : if the key has no associated value). The key-value data in the Message field is commonly utilised when investigating Windows events, e.g., in security use-cases, but the existing windows_events pipeline stage keeps the Message data as a string, leading to requiring significant effort at the query time to extract that data to be used in field filters. This PR request introduces a new promtail stage that parses a specified Source (defaulting to message used by the windows_events) and extracts all the key-value pairs into the extracted fields, which can then be used in latter stages in the pipeline.	3 years ago
Jack King	d434e80877	Promtail: Add custom headers on push requests (#7470) (#7472)	3 years ago
Witold Duranek	ee1aee0af3	feat(promtail): add more complex options of dropping logs at the drop stage (#6371)	3 years ago
nicoche	80ea621ff3	promtail: Add `max-line-size-truncate` (#8233)	3 years ago
Danny Kopping	d0fd4e8ad1	Do not exit when using `stream_lag_labels` setting (#8243)	3 years ago
Travis Patterson	1c954166b8	Fix documentation linter errors (#8229)	3 years ago
Kaviraj Kanagaraj	86461b3faa	promtail: Add doc for `max-line-size` flag. (#8202) ... Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> **What this PR does / why we need it**: Follow up from #8153 Changes: * Change type of the flag from `int` to `flagext.ByteSize` (to be consistent with similar flag in `distributor`) * Add doc to promtail config page * Fix changelog entries **Which issue(s) this PR fixes**: Fixes #NA **Special notes for your reviewer**: **Checklist** - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [x] Documentation added - [x] `CHANGELOG.md` updated Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com>	3 years ago