152 Commits (8154f3f2b4e7f40b81e4a8374ca31f4b51a92cbc)

Author	SHA1	Message	Date
Callum Styan	01d63c8ae8	[promtail] Refactor promtail client metrics so that we can't have duplicate metrics collected for the lag metric. (#5521) ... * Refactor promtail client metrics so that we can't have duplicate metrics collected for the lag metric. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Set up new metrics passing for other client types. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Fix lint issues. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Update changelog and docs, also rename nested client configs struct to configs. Signed-off-by: Callum Styan <callumstyan@gmail.com> Co-authored-by: Owen Diehl <ow.diehl@gmail.com>	4 years ago
Lukas Monkevicius	3dc1319a36	Update Pub/Sub docs section (#5205) ... * Update Pub/Sub section * Update docs/sources/clients/promtail/gcplog-cloud.md Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> * apply corrections * Update docs/sources/clients/promtail/gcplog-cloud.md Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> * Update docs/sources/clients/promtail/gcplog-cloud.md Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com>	4 years ago
Josh Risley	39e9c14611	Use output plugin grafana-loki (#5386) ... Reflects name change in v2.1.0. - If an ECS user configures `"Name": "loki"` then the native output plugin will be used and the task configuration crashes in ECS.	4 years ago
Jonathan Bédard Schami	776d5c41c7	Lambda-promtail: Enhance lambda-promtail to support adding extra labels from an environment variable value (#5359) ... * updated with ability to add extra tags * updated CHANGELOG.md * simplified extra label env variable to a simple comma seperated list in the form of "name1,value1,name2,value2,..." * refactored parsing of extra labels into its own method. Added unit tests. * removed invalid test, exit early on parseExtraLabels, refactored apply extra labels code to utility method, replaced hardcoded string with const global var in module. * updated docs, terraform template and cloud formation template * updated fmt.Println to fmt.Printf with line termination * updated to validate extra labels and added unit test for extra label validation behaviour * removed unused ExtraLabel struct * updated to use the new applyExtraLabels * pull latest * updated lambda promtail docs with instructions and example on how to use extra labels.	4 years ago
Vladyslav Diachenko	0e518967a7	updated documentation for xk6-loki extension (#5321) ... * documented custom k6 metrics that xk6-loki extension reports for instant and range queries * Update docs/sources/clients/k6/query-scenario.md Co-authored-by: Christian Haudum <christian.haudum@gmail.com> * changed description * documented new metrics Co-authored-by: Christian Haudum <christian.haudum@gmail.com>	4 years ago
Andre Ziviani	699fffe9e5	lambda-promtail: Add ability to ingest logs from S3 (#5065) ... * Add ability to ingest logs from S3 on lambda-promtail * fix ci * fix typo * bump golang and alpine version * update changelog * add s3 permissions on terraform * use for_each instead of count * fix typo * improve function naming * add documentation and an example of a s3 file path * refact logic to identify event type * add missing iam permission to allow lambda to run inside a vpc * fix typo * allow lambda to access only specified s3 buckets * configure a default log retention policy on log group * add missing depends_on to make sure iam role is created before lambda function * update docs * fix label naming convention * fix merge conflict * use new backoff lib and update dependencies * add option to limit batch size * cache s3 client * update docs and terraform * address some feedback on PR * fix typo	4 years ago
Karen Miller	d787a0fe28	Docs: remove "regular" from string definition (#5309)	4 years ago
Vladyslav Diachenko	f0ef4662fe	Documentation for load testing Loki with xk6-loki (#5182) ... * Add initial page for xk6-loki docs Signed-off-by: Christian Haudum <christian.haudum@gmail.com> * added documentation for xk6-loki query scenario * Add simple installation and usage instructions Signed-off-by: Christian Haudum <christian.haudum@gmail.com> * Add page about log generation with xk6-loki * Add documentation for k6 write scenario Signed-off-by: Christian Haudum <christian.haudum@gmail.com> * fixup: incorporate feedback from Karen Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> Signed-off-by: Christian Haudum <christian.haudum@gmail.com> * fixup! fixup: incorporate feedback from Karen Signed-off-by: Christian Haudum <christian.haudum@gmail.com> * fixup: @ppcano's suggestions * fixup: @cyriltovena's suggestions * fixup: make read scenario more generic * Apply suggestions from code review Co-authored-by: Pepe Cano <pepe@loadimpact.com> Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> * Apply suggestions from code review Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> * Use correct URL for git clone command Co-authored-by: Kaviraj Kanagaraj <kavirajkanagaraj@gmail.com> Co-authored-by: Christian Haudum <christian.haudum@gmail.com> Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> Co-authored-by: Pepe Cano <pepe@loadimpact.com> Co-authored-by: Kaviraj Kanagaraj <kavirajkanagaraj@gmail.com>	4 years ago
Susana Ferreira	82fac990a1	Update promtail to support duration string formats (#5290) ... * Update promtail to support duration string formats * Address comments	4 years ago
Sashank Agarwal	c88734b72c	Link `replace` pipeline stage to doc (#5200)	4 years ago
Dan Poltawski	379ca6b409	Promtail docs: Fix link to `unpack` LogQL documentation (#5274)	4 years ago
Ed Welch	09a5a2cee3	Logstash: Add config option which allows seting up an allowlist for labels to be mapped to Loki (#5244) ... * add a config called `include_fields` which lets you allowlist a specific set of fields which will be mapped to labels and sent to Loki. Signed-off-by: Edward Welch <edward.welch@grafana.com> * update docs Signed-off-by: Edward Welch <edward.welch@grafana.com> * restore path for Dockerfile so it works in CI Signed-off-by: Edward Welch <edward.welch@grafana.com>	4 years ago
Florian Klink	ca2132d7cb	Update _index.md (#5241) ... We also fixes typo.	4 years ago
Nabeel Saad	d0c6e3d016	Update _index.md (#5159) ... Adding default value for line_format so customers know which is picked up if the option is not provided.	4 years ago
Karsten Jeschkies	4716eea0db	Provide Docker target and discovery in Promtail. (#4911) ... **What this PR does / why we need it**: This patch adds support to fetch Docker container logs through the Docker daemon API. This should be more robust than the Loki Docker driver or scraping the logs files. The new Docker target will also collect meta information of the scraped containers. **Which issue(s) this PR fixes**: Addresses #2361 Closes #4703 **Special notes for your reviewer**: **Checklist** - [x] Documentation added - [x] Tests updated - [x] Add an entry in the `CHANGELOG.md` about the changes.	4 years ago
aengusrooneygrafana	c9d49cea6f	Update to Unofficial clients for JS (#5180) ... Came across these packages for JS with help from Danny - I thought they would be useful to include in our docs for posterity. CC @dannykopping for info!	4 years ago
Kevin	5030a1f463	Promtail: exclude_user_data description fix (#5038) ... Based on https://github.com/grafana/loki/blob/main/clients/pkg/promtail/scrapeconfig/scrapeconfig.go#L224 the option to exclude user data of windows evens is misspelled in the documentation. This puts the correct exclude_user_data option in the documentation.	4 years ago
Karen Miller	94d1550e9c	Docs: improve Promtail installation prose (#5017)	4 years ago
Avtion	1b16bb3d1d	docs: fix yaml grammar mistakes (#5027)	4 years ago
Robby Milo	0b93520994	update promtail daemonset docs (#4946) ... * Update installation.md * Update installation.md	4 years ago
Fabian Eisinger	9a86cc882a	Add default tags for docker-compose (#4916) ... The docker driver parses `compose_project` and `compose_service` when container in deployed with Docker Compose. Relevant code: 24b4c0eed2/clients/cmd/docker-driver/config.go (L260-L268)	4 years ago
nerrehmit	57a85a1a57	add multiline to stages overview (#4851)	4 years ago
Cyril Tovena	7f71c5f051	Promtail pull cloudflare logs (#4813)	4 years ago
Fredrik Ekre	18d8fee439	[doc]: Update URL for LokiLogger.jl (Julia client) (#4784)	4 years ago
Hiroki Sakamoto	d61dd1872a	Expose Kafka message key in labels (#4745) ... * expose kafka message key in labels * add changelog entry * improve performance * goimported * fix label name in test * add empty message key test case * pass actual value in message key	4 years ago
Karen Miller	35ebe967ea	Docs: fix broken links (#4770) ... * Docs: fix broken links * Docs: fix another link name	4 years ago
James Callahan	21556efa87	Promtail: improve GCP labels (#4628) ... * Promtail: GCP resource label prefix is now '__gcp_resource_labels_' * Promtail: turn GCP 'resource_type' into the internal label '__gcp_resource_type' * Promtail: add internal label with logName * Add upgrade notes for gcplog changes	4 years ago
Cyril Tovena	81e67f89b3	Add GELF support for Promtail. (#4744) ... * Add Gelf support for Promtail. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add documentation. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Changelog Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Move change to MAIN Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Update graylog to grafana remote. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add a test and check mod Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * add log messages around startup and shutdown Co-authored-by: Edward Welch <edward.welch@grafana.com>	4 years ago
Jiří Sejkora	925de9e102	Docs: Fix a typo (#4726)	4 years ago
Hiroki Sakamoto	1557dab34f	Add SASL&mTLS authentication support for Kafka in Promtail (#4663) ... * Add xdg-go module * Add mTLS, SASL/PLAIN, SASL/SCRAM authentication support * docs: Add authentication configuration * Add test tools for kafka authentication * Add test stacks for SSL, SASL/PLAIN, SASL/SCRAM, SASL over TLS authentication * Fix shelllint	4 years ago
Ed Welch	3fef08ca1a	2.4 release notes, changelog, upgrade guide (#4658) ... * update changelog, release notes, and upgrade guide for 2.4 * tweak release notes a little * make path relative for kafka config * update upgrade guide * more docs fix the cache size not applied to results cache. * fix prefix name in ring_config	4 years ago
Cyril Tovena	c8f3df33cf	Promtail Kafka target (#4568) ... * Adds a kafka target manager in promtail. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add validations. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Working on tests. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Moar test for the fanout client. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Finishing off tests. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * final adjustement Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Adding topics discovery. Still needs to finish tests. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Finishing off testing it. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Wip Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Revert config changes. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * lint Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add group id as discovered label Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * linter Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add tools for running kafka and testing locally. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * got linted shell Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Update sarama to compile in ARM. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add documentation for kafka target. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Improve code comment. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * add a few s's * Better cancellation support. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * adds `__meta_kafka_` suffix to discovered labels. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> Co-authored-by: Edward Welch <edward.welch@grafana.com>	4 years ago
SurDaft - Jack Stupple	3dd0702b7c	Fix typo in docs (#4612) ... Fix a typo in the documentation	4 years ago
James Callahan	a92541bd84	Promtail: no need for GCP promtail_instance label now that loki supports out-of-order writes (#4556) ... Closes #4474	4 years ago
Karen Miller	96130ae389	Docs: revise incendiary language added in PR 4507 (#4611)	4 years ago
Owen Diehl	6eab7be5b4	a few doc fixes in preparation for 2.4 (#4597)	4 years ago
Callum Styan	e49e318b42	Improve lambda-promtail docs based on Owens review. (#4548) ... * Improve lambda-promtail docs based on Owens review. Signed-off-by: Callum Styan <callumstyan@gmail.com> * One last lambda-promtail docs improvement. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Fix bolding in lambda-promtail docs + include link to drop filter promtail pipeline stage.	4 years ago
Sankalp Rangare	9e2acd656d	add static labels in stages (#4495) ... Signed-off-by: Sankalp Rangare <sankalprangare786@gmail.com>	4 years ago
Callum Styan	aa713f54f4	Modify lambda-promtail docs based on rewrite. (#4539) ... Signed-off-by: Callum Styan <callumstyan@gmail.com>	4 years ago
Danny Kopping	703d68bc76	Promtail: allow for customisable stream lag labels (#4507) ... * Allow for customisable stream lag labels Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Updating test Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Allowing for multiple labels Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Adding field to docs Signed-off-by: Danny Kopping <danny.kopping@grafana.com>	4 years ago
Karen Miller	24accf6a5b	Docs: correctly represent product name (#4416) ... The product name is Grafana Loki. The first use of the product name on each web page should be the full product name. After that, within each page, it is fine to use just Loki. This PR also set up some frontmatter with weights where it was missing on a few files, and correct where "Loki" was not capitalized.	4 years ago
Leo	0a66b897ec	clients/promtail: Add ndjson and plaintext formats to loki_push (#4336) ... * feat(clients/promtail): add ndjson,plaintext formats to loki_push * feat(clients/promtail): add ndjson,plaintext formats to loki_push * feat(clients/promtail): documentation * Address review comments * fmt * fix missing port	4 years ago
Johan Fleury	9b0f61d9d3	promtail: add support for TLS/mTLS in syslog receiver (#3907)	4 years ago
Jason Giedymin	6477970bc0	fix(docs): spelling mistake (#4361) ... fix(docs): spelling mistake in logstash configuration doc	4 years ago
Jordan Rushing	0f0cb9fc99	Add missing `Inc()` to correctly increment the `dropStage.dropCount` metric on valid dropped log line; update related docs (#4246)	4 years ago
Karen Miller	11a0d28b61	Docs: first draft, Loki accepts out-of-order writes (#4237) ... * Docs: first draft, Loki accepts out-of-order writes * Update docs/sources/architecture/_index.md Co-authored-by: Owen Diehl <ow.diehl@gmail.com>	4 years ago
Alexandre Vallarino	b0646e7156	fix: typo on loki-external-labels for labels (#4231)	4 years ago
Karsten Jeschkies	29d2394962	Link Kubernetes service discovery configuration. (#4206) ... * Link Kubernetes service discovery configuration. * Use `configuration`.	4 years ago
Karsten Jeschkies	e25587bfd7	Document known Docker driver issues. (#4190) ... * Document known Docker driver issues. * Update docs/sources/clients/docker-driver/_index.md Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> * Update docs/sources/clients/docker-driver/_index.md Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> * Update docs/sources/clients/docker-driver/_index.md Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com>	4 years ago
Karen Miller	cd79adc8ab	Docs: fluentd client phrasing and formatting (#4147) ... - Adds corrections from closed PR https://github.com/grafana/loki/pull/3880	4 years ago