apitech/loki - loki - Gitea: Git with a cup of tea

Commit Graph

Author	SHA1	Message	Date
Travis Patterson	1c59875420	Update docs/sources/clients/promtail/scraping.md Co-authored-by: J Stickler <julie.stickler@grafana.com>	3 years ago
Travis Patterson	b5cce33957	Update docs/sources/clients/promtail/scraping.md Co-authored-by: J Stickler <julie.stickler@grafana.com>	3 years ago
Travis Patterson	b697309ea9	Update docs/sources/clients/promtail/scraping.md Co-authored-by: J Stickler <julie.stickler@grafana.com>	3 years ago
Alex Close	c473dc847d	Added list of supported scrapes to top of page to save reading the whole thing	3 years ago
Danny Kopping	e001086ac0	Fix CHANGELOG link (#9048 ) Previous was pointing to "main" which will cause confusion Signed-off-by: Danny Kopping <danny.kopping@grafana.com>	3 years ago
Trevor Whitney	fa56f00517	These features did not make the release (#9025 ) What this PR does / why we need it: These features did not actually make it into this release.	3 years ago
Karsten Jeschkies	4fc51ccdd7	Remove unnecessary build image step. (#9019 ) What this PR does / why we need it: > Repeat step 1.3, which will use the new image is the same as calling `make drone`. So it is not required. Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	3 years ago
Kaviraj Kanagaraj	cce3d3bed9	chore(format_query): Change response type to `application/json` (#9016 )	3 years ago
Trevor Whitney	ec4b88955f	Add 2.8 release notes (#9004 ) Adds 2.8 release notes.	3 years ago
Dylan Guedes	6dfcd59c1b	Promtail: Add `decompression` configuration (#8994 ) What this PR does / why we need it: Adds a new `compression` configuration to Promtail to customize decompression behavior. As of now, two options are supported: `enabled`: false by default but if truthy, Promtail will infer a compression algorithm to use based on the file extension `initialDelay`: 0 by default but if higher, Decompressor will sleep for the given duration before trying to decompress. This is to avoid scenarios where the Decompressor starts to work on a file that is still being compressed/generated. Which issue(s) this PR fixes: Fixes https://github.com/grafana/loki/issues/8784	3 years ago
kernc	e773c004f5	Docs: Mention python-logging-loki as an unofficial client (#8983 ) What this PR does / why we need it: Mention python-logging-loki as an unofficial client, since it looks a reasonably well done client. This PR is a continuation from https://github.com/grafana/loki/pull/8636. 😅 CLA has been signed!	3 years ago
Trevor Whitney	bf2e712b32	I do not think we have circleci jobs anymore (#9002 )	3 years ago
Trevor Whitney	5113906c67	Update helm and jsonnet for 2.8 release (#9003 ) Prepare helm and jsonnet for 2.8 release.	3 years ago
Trevor Whitney	e82885da50	Change loki version in docs for 2.8 release (#8985 ) Changes version references in Loki docs for the upcoming release Also adds a script for diffing metrics when doing a release.	3 years ago
Andrii Kushch	9f41182da2	Fix the link in the Azure Event Hubs doc (#8904 ) What this PR does / why we need it: This PR fixes the link to the configuration in the Azure Event Hubs scraping doc. Link to the discussion https://github.com/grafana/loki/pull/8787#discussion_r1145298591 Which issue(s) this PR fixes: Not available Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [x] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	3 years ago
Andrey Em	0f98b45d8d	Loki: Add UsageStatsURL to usagestats Config (#8779 ) What this PR does / why we need it: Reports used to be sent to a fixed URL usageStatsURL. Now this url can be configurable while using the old url as a default. Updated tests to use config for custom URLs instead of changing package variable. Which issue(s) this PR fixes: Fixes #6502 Special notes for your reviewer: Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [x] Documentation added - [x] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	3 years ago
Erik Sjölund	2cccc1bc68	Docs: Fix typos and grammar (#8966 ) What this PR does / why we need it: Fixes typos and grammar. Most of them were occurrences of `it's` that should be `its`. Which issue(s) this PR fixes: None Special notes for your reviewer: Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` Signed-off-by: Erik Sjölund <erik.sjolund@gmail.com>	3 years ago
Gábor Farkas	09f23ecd03	Docs: Improve time range documentation (#8847 ) What this PR does / why we need it: Describes how exactly the `start` and `end` timestamps operate on the time range boundaries. Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [x] Documentation added --------- Co-authored-by: J Stickler <julie.stickler@grafana.com>	3 years ago
Danny Kopping	8233a1560a	Adding release cadence docs (#8893 ) What this PR does / why we need it: Members of our community often ask about our release cadence, so I have documented it for easy future reference. --------- Signed-off-by: Danny Kopping <danny.kopping@grafana.com> Co-authored-by: J Stickler <julie.stickler@grafana.com>	3 years ago
Alfredo	f3a49f4734	Update contains and hasPrefix/hasSuffix with the correct examples (#8929 ) What this PR does / why we need it: WHAT: - Update examples for `contains` and `hasSuffix` - Correct order should be `functionName arg1 arg2 argN`, eg. `contains s src` WHY: - Current example for `contains` and `hasPrefix` / `hasSuffix` has a wrong order that will cause `TemplateFormatErr`. - Examples I'm referring to `{{ if .err hasSuffix "Timeout" }} timeout {{end}}` Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	3 years ago
chrisharbro	9efbd5ca45	typo and phrasing (#8942 ) The first sentence doesn't sound natural, tried to massage it to have a more natural sound. Fixed a minor typo.	3 years ago
李国忠	73f9ece867	[doc] client : ilogtail loki integration (#8970 ) What this PR does / why we need it: https://github.com/alibaba/ilogtail/pull/685 ilogtail is a very popular log agent in China, with tens of millions of installations. The open source community of ilogtail has completed the integration of loki. This PR is completed as a supplement to loki doc demo config，Type: flusher_loki ```yaml enable: true inputs: - Type: file_log LogPath: . FilePattern: simple.log flushers: - Type: flusher_loki URL: http://localhost:3100/loki/api/v1/push ExternalLabels: source: ilogtail ``` Which issue(s) this PR fixes: Fixes #<issue number> Special notes for your reviewer: flusher_loki PR snapshot ![image](https://user-images.githubusercontent.com/9583245/229053024-66fcd286-a2f8-4a32-9ec5-758b63554986.png) Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [x] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	3 years ago
Ed Welch	edc6b0bff7	Loki: Add a limit for the [range] value on range queries (#8343 ) Signed-off-by: Edward Welch <edward.welch@grafana.com> What this PR does / why we need it: Loki does not currently split queries by time to a value smaller than what's in the [range] of a range query. Example ``` sum(rate({job="foo"}[2d])) ``` Imagine now this query being executed over a longer window of a few days with a step of something like 30m. Every step evaluation would query the last [2d] of data. There are use cases where this is desired, specifically if you force the step to match the value in the range, however what is more common is someone accidentally uses `[$__range]` in here instead of `[$__interval]` within Grafana and then sets the query time selector to a large value like 7 days. This PR adds a limit which will fail queries that set the [range] value higher than the configured limit. It's disabled by default. In the future it may be possible for Loki to perform splits within the [range] and remove the need for this limit, but until then this can be an important safeguard in clusters with a lot of data. Which issue(s) this PR fixes: Fixes #8746 Special notes for your reviewer: Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` --------- Signed-off-by: Edward Welch <edward.welch@grafana.com> Co-authored-by: Karsten Jeschkies <karsten.jeschkies@grafana.com> Co-authored-by: Vladyslav Diachenko <82767850+vlad-diachenko@users.noreply.github.com>	3 years ago
Dylan Guedes	9159c1dac3	Loki: Improve spans usage (#8927 ) What this PR does / why we need it: - At different places, inherit the span/spanlogger from the given context instead of instantiating a new one from scratch, which fix spans being orphaned on a read/write operation. - At different places, turn spans into events. Events are lighter than spans and by having fewer spans in the trace, trace visualization will be cleaner without losing any details. - Adds new spans/events to places that might be a bottleneck for our writes/reads.	3 years ago
Suruthi-G-K	e7b0d4d26e	feat(cos): Support authentication with trusted profiles (#8939 ) What this PR does / why we need it: Add trusted profile authentication in COS client Which issue(s) this PR fixes: Fixes NA Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [x] Documentation added - [x] Tests updated - [x] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` --------- Co-authored-by: shahulsonhal <shahulsonhal@gmail.com> Co-authored-by: tareqmamari <tariq.mamari@de.ibm.com>	3 years ago
Danny Kopping	6bb8a071ca	Querier: block query by hash (#8953 ) What this PR does / why we need it: Using the [query blocker](https://grafana.com/docs/loki/next/operations/blocking-queries/) can be unergonomic since queries can be long, require escaping, or hard to copy from logs. This change enables an operator to block queries by their hash.	3 years ago
Karsten Jeschkies	0030cafb16	Remove dedupe cache from operations documentation. (#8957 )	3 years ago
Periklis Tsirakidis	1bcf683513	Expose optional label matcher for label values handler (#8824 )	3 years ago
Grot (@grafanabot)	b97525a448	[CI/CD] Update yaml file `./production/helm/loki/values.yaml` (+1 other) (#8955 ) Here is a summary of the updates contained in this PR: * Update attribute `$.enterprise.version` in yaml file `./production/helm/loki/values.yaml` to the following value: `v1.6.3` * Bump version of Helm Chart Add changelog entry to `./production/helm/loki/CHANGELOG.md` Re-generate docs	3 years ago
Salva Corts	28a7733ede	Rename config for enforcing a minimum number of label matchers (#8940 ) What this PR does / why we need it: Followup PR for https://github.com/grafana/loki/pull/8918 renaming config. See https://github.com/grafana/loki/pull/8918/files#r1151820792. Which issue(s) this PR fixes: Fixes https://github.com/grafana/loki-private/issues/699 Special notes for your reviewer: Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [x] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` --------- Co-authored-by: Dylan Guedes <djmgguedes@gmail.com>	3 years ago
Karsten Jeschkies	43ae1db14c	Give examples for all cache configurations. (#8832 ) What this PR does / why we need it: #8373 documented how to setup Memcached for caching. The documentation is missing some information on a few caches. These are added in this PR. Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [x] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` --------- Co-authored-by: J Stickler <julie.stickler@grafana.com>	3 years ago
Karsten Jeschkies	038a7722d6	Ask users to add node types to the sizing tool. (#8834 ) What this PR does / why we need it: We want to invite users to make contributions to the node types of the sizing tool since we don't have the capacity to do so. Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [x] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` --------- Co-authored-by: J Stickler <julie.stickler@grafana.com>	3 years ago
Salva Corts	45775c82f7	Implement `RequiredNumberLabels` query limit (#8918 ) What this PR does / why we need it: As pointed out in https://github.com/grafana/loki/pull/8851, some queries can impose a great workload on a cluster by selecting too many streams. Similarly to the `RequiredLabels` limit introduced at https://github.com/grafana/loki/pull/8851, here we add a new limit `RequiredNumberLabels` to require queries to specify at least N label. For example, if the limit is set to 2, then the query should contain at least 2 label matchers. This limit can be configured per tenant and at query time. ![image](https://user-images.githubusercontent.com/8354290/228271398-4b9bcc49-f539-4e94-86c1-071e519a30a9.png) Which issue(s) this PR fixes: Fixes https://github.com/grafana/loki-private/issues/699 Special notes for your reviewer: Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [x] Documentation added - [x] Tests updated - [x] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` --------- Co-authored-by: Dylan Guedes <djmgguedes@gmail.com>	3 years ago
Michel Hollands	e7e752378f	Add release notes for v2.7.5 release (#8924 ) What this PR does / why we need it: This adds the release notes for v2.7.5 and v2.7.4. This last release was apparently not added last time. This will be backported to the release branch as well in a separate PR. Checklist - [X] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [X] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` Signed-off-by: Michel Hollands <michel.hollands@grafana.com>	3 years ago
Michel Hollands	035f673e24	Update Loki version to latest (#8926 ) What this PR does / why we need it: Update to v2.7.5 Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [X] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` Signed-off-by: Michel Hollands <michel.hollands@grafana.com>	3 years ago
Athira Varghese	ffb961c439	feat(storage): add support for IBM cloud object storage as storage client (#8826 ) What this PR does / why we need it: Add support for IBM cloud object storage as storage client Which issue(s) this PR fixes: Fixes NA Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [x] Documentation added - [x] Tests updated - [ ] `CHANGELOG.md` updated --------- Signed-off-by: Shahul <shahulsonhal@gmail.com> Co-authored-by: Aman Kumar Singh <amankrsingh2110@gmail.com> Co-authored-by: Suruthi-G-K <shruthi.suruthi@gmail.com> Co-authored-by: tareqmamari <tariq.mamari@de.ibm.com> Co-authored-by: shahulsonhal <shahulsonhal@gmail.com> Co-authored-by: Aditya C S <aditya.gnu@gmail.com> Co-authored-by: Tareq Al-Maamari <tariq.mamari@gmail.com>	3 years ago
Danny Kopping	c9d5a91206	Ruler: Implement consistent rule evaluation jitter (#8896 ) What this PR does / why we need it: This PR replaces the previous random jitter with a consistent jitter. While both are random, having the random jitter be applied _consistently_ is essential for evaluating rules on a predictable cadence. If a rule is supposed to evaluate every minute, whether it evaluates at (e.g.) `01:00` or `01:03.234` is irrelevant because the evaluation _instant_ is not adjusted, so it will produce the same result whether run at `01:00` or `01:03.234`. However, if 1000 rules are set to evaluate at `01:00`, this will create a resource contention issue.	3 years ago
René Scheibe	4c4c7e3010	Promtail: Fix examples how to build it (#8898 ) The build flags have to be provided before the package paths. Otherwise the `build` command fails with this error: > malformed import path "--tags=promtail_journal_enabled": leading dash See `go help build`: > usage: go build [-o output] [build flags] [packages]	3 years ago
Andrii Kushch	17e05d28e4	Promtail: Add a new target for the Azure Event Hub (#8787 ) What this PR does / why we need it: We want to allow receiving logs from the Azure Cloud. To solve this problem, we need to be able to configure Promtail to consume logs from the Azure Events Hub. To achieve this goal, we need to add a new configuration option to the scrape configuration in Promtail and implement the component that, given this configuration, can connect to the Azure Event Hub, parse received data, and prepare it to be stored in Loki. Which issue(s) this PR fixes: Fix [#8788](https://github.com/grafana/loki/issues/8788) # Special notes for your reviewer: With the current implementation, I would like to leverage Azure Events Hub's compatibility with a Kafka protocol and reuse code for a Kafka target. ### Why I decided to create a new target instead of just reusing the Kafka target: 1. Configuration for Kafka in this scenario requires fewer parameters. For example, version and auth should have some specific values. 2. Message handling is different for Azure Logs. In addition to what we are doing for the Kafka target, we want to fix JSON and split incoming messages into multiple log lines before propagating it further. ### From the implementation perspective, there are two goals: - Extract interfaces that are not Kafka configuration specific from the Kafka target module. Splitting Kafka config parsing and target creation. - To make the message handler/parser injectable or configurable: we would like to fix JSON and split a message into multiple messages. Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [x] Documentation added - [x] Tests updated - [x] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` --------- Co-authored-by: Pablo <2617411+thepalbi@users.noreply.github.com> Co-authored-by: Paschalis Tsilias <tpaschalis@users.noreply.github.com> Co-authored-by: J Stickler <julie.stickler@grafana.com>	3 years ago
Salva Corts	d24fe3e68b	Max bytes read limit (#8670 ) What this PR does / why we need it: This PR implements two new per-tenant limits that are enforced on log and metric queries (both range and instant) when TSDB is used: - `max_query_bytes_read`: Refuse queries that would read more than the configured bytes here. Overall limit regardless of splitting/sharding. The goal is to refuse queries that would take too long. The default value of 0 disables this limit. - `max_querier_bytes_read`: Refuse queries in which any of their subqueries after splitting and sharding would read more than the configured bytes here. The goal is to avoid a querier from running a query that would load too much data in memory and can potentially get OOMed. The default value of 0 disables this limit. These new limits can be configured per tenant and per query (see https://github.com/grafana/loki/pull/8727). The bytes a query would read are estimated through TSDB's index stats. Even though they are not exact, they are good enough to have a rough estimation of whether a query is too big to run or not. For more details on this refer to this discussion in the PR: https://github.com/grafana/loki/pull/8670#discussion_r1124858508. Both limits are implemented in the frontend. Even though we considered implementing `max_querier_bytes_read` in the querier, this way, the limits for pre and post splitting/sharding queries are enforced close to each other on the same component. Moreover, this way we can reduce the number of index stats requests issued to the index gateways by reusing the stats gathered while sharding the query. With regard to how index stats requests are issued: - We parallelize index stats requests by splitting them into queries that span up to 24h since our indices are sharded by 24h periods. On top of that, this prevents a single index gateway from processing a single huge request like `{app=~".+"} for 30d`. - If sharding is enabled and the query is shardable, for `max_querier_bytes_read`, we re-use the stats requests issued by the sharding ware. Specifically, we look at the [bytesPerShard][1] to enforce this limit. Note that once we merge this PR and enable these limits, the load of index stats requests will increase substantially and we may discover bottlenecks in our index gateways and TSDB. After speaking with @owen-d, we think it should be fine as, if needed, we can scale up our index gateways and support caching index stats requests. Here's a demo of this working: <img width="1647" alt="image" src="https://user-images.githubusercontent.com/8354290/226918478-d4b6c2fd-de4d-478a-9c8b-e38fe148fa95.png"> <img width="1647" alt="image" src="https://user-images.githubusercontent.com/8354290/226918798-a71b1db8-ea68-4d00-933b-e5eb1524d240.png"> Which issue(s) this PR fixes: This PR addresses https://github.com/grafana/loki-private/issues/674. Special notes for your reviewer: - @jeschkies has reviewed the changes related to query-time limits. - I've done some refactoring in this PR: - Extracted logic to get stats for a set of matches into a new function [getStatsForMatchers][2]. - Extracted the _Handler_ interface implementation for [queryrangebase.roundTripper][3] into a new type [queryrangebase.roundTripperHandler][4]. This is used to create the handler that skips the rest of configured middlewares when sending an index stat quests ([example][5]). Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [x] Documentation added - [x] Tests updated - [x] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` [1]: `ff847305af/pkg/querier/queryrange/shard_resolver.go (L179-L186)` [2]: `ff847305af/pkg/querier/queryrange/shard_resolver.go (L72)` [3]: `3d2fff3a2d/pkg/querier/queryrange/queryrangebase/roundtrip.go (L124)` [4]: `3d2fff3a2d/pkg/querier/queryrange/queryrangebase/roundtrip.go (L163)` [5]: `f422e0a52b/pkg/querier/queryrange/roundtrip.go (L521)`	3 years ago
Karsten Jeschkies	94725e7908	Define `RequiredLabels` query limit. (#8851 ) What this PR does / why we need it: Some end-users can impose great workload on a cluster by selecting too many streams in their queries. We should be able to limit them. Therefore we introduce a new limit `RequiredLabelMatchers` which list label names that must be included in the stream selectors. The implementation follows the same approach as for max query limit. Which issue(s) this PR fixes: Fixes #8745 Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [x] Documentation added - [x] Tests updated - [x] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	3 years ago
Karsten Jeschkies	c6c07a8276	Document how to install Loki and Promtail using APT/DNF. (#8841 ) What this PR does / why we need it: With #6456 we started providing native packages of Loki, Promtail and logcli. They've been added to Grafana's repository in the meantime. In order to spread the knowledge of the packages and the repo it should be documented. Which issue(s) this PR fixes: Fixes #52 Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [x] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` --------- Co-authored-by: René Scheibe <rene.scheibe@gmail.com> Co-authored-by: J Stickler <julie.stickler@grafana.com>	3 years ago
J Stickler	b8221dc68f	Correcting typos merged in #8870 . (#8873 ) Which issue(s) this PR fixes: Fixes typos introduced in #8870	3 years ago
Travis Patterson	9d20bed26a	Short circuit parsers (#8724 ) In many cases we only need a few labels from a line. Because most of our parsers parse lines incrementally, we can stop parsing a line after we have all the labels we want from it. This pr uses `ParserHints` to keep track of the number of extracted labels. It also provides a way for parsers to know when they should stop parsing. Notes: - parsers had inconsistent ordering between the `ShouldExtract` call and adding the `_extracted` label to duplicates. This PR makes appending `_extracted` always happen before `ShouldExtract` to keep counts of what is extracted compared to expected labels consistent. Next Steps: - When the user specifies a query with a grouping containing the `_extracted` label but there is no duplicate between the passed line and labels, short circuiting will not work. I'll address this in a follow up PR. Benchmarks: To try and show a balanced view of what this buys us, this pr picks a label our of the middle of a line. In a best case this might be much better. In the worst case, we have to parse the whole line. ``` benchstat short_circuit_old.txt short_circuit_new.txt name old time/op new time/op delta KeyExtraction/json-8 456ns ± 3% 256ns ± 2% -43.84% (p=0.000 n=9+10) KeyExtraction/logfmt-8 347ns ± 4% 171ns ± 2% -50.86% (p=0.000 n=10+10) KeyExtraction/logfmt-expression-8 552ns ± 2% 368ns ± 2% -33.23% (p=0.000 n=9+10) name old alloc/op new alloc/op delta KeyExtraction/json-8 5.00B ± 0% 5.00B ± 0% ~ (all equal) KeyExtraction/logfmt-8 5.00B ± 0% 5.00B ± 0% ~ (all equal) KeyExtraction/logfmt-expression-8 16.0B ± 0% 16.0B ± 0% ~ (all equal) name old allocs/op new allocs/op delta KeyExtraction/json-8 1.00 ± 0% 1.00 ± 0% ~ (all equal) KeyExtraction/logfmt-8 1.00 ± 0% 1.00 ± 0% ~ (all equal) KeyExtraction/logfmt-expression-8 2.00 ± 0% 2.00 ± 0% ~ (all equal) ``` --------- Co-authored-by: J Stickler <julie.stickler@grafana.com>	3 years ago
annelaurefroment	01936330f4	update on Running Promtail on AWS EC2 tutorial (#8870 ) Adding more information on what to do if `__meta_ec2_private_dns_name` doesn't match the hostname.	3 years ago
W.T. Chang	183fe85f1c	Loki: Add route_randomly to Redis options (#8852 ) What this PR does / why we need it: Current Redis client only reads from master node, which makes master node under heavy pressure. Usually, one or more read replicas are expected on production environments. This PR helps to increase the utilization of read replicas.	3 years ago
Danny Kopping	528ed22dc2	Ruler: remote rule evaluation hardening (#8785 ) What this PR does / why we need it: This PR is part 2 of 2 implementing remote rule evaluation. See [part 1](https://github.com/grafana/loki/pull/8744) for more context.	3 years ago
Danny Kopping	2d93952929	(chore): remove incorrect cache config preample (#8859 ) With the addition of the [config generation tool](https://github.com/grafana/loki/pull/7916), a comment prefix was added for all cache types: `Cache config for index entry writing.` https://grafana.com/docs/loki/next/configuration/#cache_config This appears in the description of each cache config option, and is misleading.	3 years ago
Danny Kopping	5c3d204ebf	Ruler: rule evaluation jitter (#8848 ) What this PR does / why we need it: This PR introduces a configurable random sleep before each rule evaluation to prevent contention on the query engine should multiple runs execute concurrently. It's quite likely that rules will be evaluated ~concurrently if they have the same interval.	3 years ago
Christian Haudum	be8b4eece3	Scheduler: Add query fairness control across multiple actors within a tenant (#8752 ) What this PR does / why we need it: This PR wires up the scheduler with the hierarchical queues. It is the last PR to implement https://github.com/grafana/loki/pull/8585. When these changes are in place, the client performing query requests can control their QoS (query fairness) using the `X-Actor-Path` HTTP header. This header controls in which sub-queue of the tenant's scheduler queue the query request is enqueued. The place within the hierarchy where it is enqueued defines the probability with which the request gets dequeued. A common use-case for this QoS control is giving each Grafana user within a tenant their fair share of query execution time. Any documentation is still missing and will be provided by follow-up PRs. Special notes for your reviewer: ```console $ gotest -count=1 -v ./pkg/scheduler/queue/... -test.run=TestQueryFairness === RUN TestQueryFairness === RUN TestQueryFairness/use_hierarchical_queues_=_false dequeue_qos_test.go:109: duration actor a 2.007765568s dequeue_qos_test.go:109: duration actor b 2.209088331s dequeue_qos_test.go:112: total duration 2.209280772s === RUN TestQueryFairness/use_hierarchical_queues_=_true dequeue_qos_test.go:109: duration actor b 605.283144ms dequeue_qos_test.go:109: duration actor a 2.270931324s dequeue_qos_test.go:112: total duration 2.271108551s --- PASS: TestQueryFairness (4.48s) --- PASS: TestQueryFairness/use_hierarchical_queues_=_false (2.21s) --- PASS: TestQueryFairness/use_hierarchical_queues_=_true (2.27s) PASS ok github.com/grafana/loki/pkg/scheduler/queue 4.491s ``` ```console $ gotest -count=5 -v ./pkg/scheduler/queue/... -bench=Benchmark -test.run=^$ -benchtime=10000x -benchmem goos: linux goarch: amd64 pkg: github.com/grafana/loki/pkg/scheduler/queue cpu: 11th Gen Intel(R) Core(TM) i7-1185G7 @ 3.00GHz BenchmarkGetNextRequest BenchmarkGetNextRequest/without_sub-queues BenchmarkGetNextRequest/without_sub-queues-8 10000 29337 ns/op 1600 B/op 100 allocs/op BenchmarkGetNextRequest/without_sub-queues-8 10000 21348 ns/op 1600 B/op 100 allocs/op BenchmarkGetNextRequest/without_sub-queues-8 10000 21595 ns/op 1600 B/op 100 allocs/op BenchmarkGetNextRequest/without_sub-queues-8 10000 21189 ns/op 1600 B/op 100 allocs/op BenchmarkGetNextRequest/without_sub-queues-8 10000 21602 ns/op 1600 B/op 100 allocs/op BenchmarkGetNextRequest/with_1_level_of_sub-queues BenchmarkGetNextRequest/with_1_level_of_sub-queues-8 10000 33770 ns/op 2400 B/op 200 allocs/op BenchmarkGetNextRequest/with_1_level_of_sub-queues-8 10000 33596 ns/op 2400 B/op 200 allocs/op BenchmarkGetNextRequest/with_1_level_of_sub-queues-8 10000 34432 ns/op 2400 B/op 200 allocs/op BenchmarkGetNextRequest/with_1_level_of_sub-queues-8 10000 33760 ns/op 2400 B/op 200 allocs/op BenchmarkGetNextRequest/with_1_level_of_sub-queues-8 10000 33664 ns/op 2400 B/op 200 allocs/op BenchmarkGetNextRequest/with_2_levels_of_sub-queues BenchmarkGetNextRequest/with_2_levels_of_sub-queues-8 10000 71405 ns/op 3200 B/op 300 allocs/op BenchmarkGetNextRequest/with_2_levels_of_sub-queues-8 10000 59472 ns/op 3200 B/op 300 allocs/op BenchmarkGetNextRequest/with_2_levels_of_sub-queues-8 10000 117163 ns/op 3200 B/op 300 allocs/op BenchmarkGetNextRequest/with_2_levels_of_sub-queues-8 10000 106505 ns/op 3200 B/op 300 allocs/op BenchmarkGetNextRequest/with_2_levels_of_sub-queues-8 10000 64374 ns/op 3200 B/op 300 allocs/op BenchmarkQueueRequest BenchmarkQueueRequest-8 10000 168391 ns/op 320588 B/op 1156 allocs/op BenchmarkQueueRequest-8 10000 166203 ns/op 320587 B/op 1156 allocs/op BenchmarkQueueRequest-8 10000 149518 ns/op 320584 B/op 1156 allocs/op BenchmarkQueueRequest-8 10000 219776 ns/op 320583 B/op 1156 allocs/op BenchmarkQueueRequest-8 10000 185198 ns/op 320597 B/op 1156 allocs/op PASS ok github.com/grafana/loki/pkg/scheduler/queue 64.648s ``` Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	3 years ago

1 2 3 4 5 ...

1328 Commits (Alex3k-patch-6)