310 Commits (80ea621ff3c7be677502d404c1bb89d1435b20ed)

Author	SHA1	Message	Date
nicoche	80ea621ff3	promtail: Add `max-line-size-truncate` (#8233)	2 years ago
Sandeep Sukhani	96a4fc1409	index-store: fix indexing of chunks overlapping multiple schemas (#8251)	2 years ago
Kaviraj Kanagaraj	e18b4fca6f	ingester: Add `ingester_memory_streams_labels_bytes` metric (#8212) ... Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> This is to give visibility on `labels` size of in-memory streams on the ingesters. It caculate the size of labels via `len(labels.String())`. Meaning if you have ```go lbs := labels.Labels{ {Name: "foo", Value: "bar"}, {Name: "instance", Value: "loki"}, } ``` size would be number of bytes in `{foo="bar", instance="loki"}` which is `28`. NOTE: `len(string)` gives [bytes length in Go](https://pkg.go.dev/builtin#len)	2 years ago
Kaviraj Kanagaraj	86461b3faa	promtail: Add doc for `max-line-size` flag. (#8202) ... Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> **What this PR does / why we need it**: Follow up from #8153 Changes: * Change type of the flag from `int` to `flagext.ByteSize` (to be consistent with similar flag in `distributor`) * Add doc to promtail config page * Fix changelog entries **Which issue(s) this PR fixes**: Fixes #NA **Special notes for your reviewer**: **Checklist** - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [x] Documentation added - [x] `CHANGELOG.md` updated Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com>	2 years ago
Kaviraj Kanagaraj	5031ae1b4e	doc(promtail): Doc about how log rotate works with promtail. (#8096) ... Changes 1. Why log rotation matters? 2. Different components involving in reading, writing and rotating logs 3. Different types of log rotation. 4. How to configure log rotation effectively to work with promtail Fixes # Should resolve all confusions around log rotation errors with promtail Related: #7967 Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com>	2 years ago
Kaviraj Kanagaraj	2597429c89	promtail: Add `max-line-size` limit to drop on client side (#8153)	2 years ago
Karsten Jeschkies	4504ed2701	Cross build Promtail for ARM with journald support. (#8131) ... This is the follow up to #8111 and adds journald support for Promatil ARM and ARM64 builds. I've tested it locally with my Raspberry Pi and it works. Fixes #1459	2 years ago
Jake Herbst	02e0b3ae89	Fix logentry drop to more intuitively match numeric fields (fixes #7011) (#7292) ... **What this PR does / why we need it**: 1. Makes the `drop` stage matching for numeric fields more intuitive. 2. Adds tests scenarios to confirm fix **Which issue(s) this PR fixes**: Fixes #7011	2 years ago
Aditya C S	8df5803d9a	feat(logql): Support drop labels in logql pipeline (#7975) ... This PR introduces `drop` stage in logql pipeline. Fixes #7870, Fixes #7368	2 years ago
Ashwanth	361e7f7f8a	scheduler: fix panic on hitting /scheduler/ring when ring is disabled (#8120) ... Fixes nil pointer dereference that occurs on calling `/scheduler/ring` endpoint when `use_scheduler_ring` is set to false. ``` /usr/local/go/src/runtime/panic.go:884 +0x212 github.com/grafana/dskit/ring.(*Ring).ServeHTTP(...) /src/enterprise-logs/vendor/github.com/grafana/dskit/ring/ring.go:858 github.com/grafana/loki/pkg/scheduler.(*Scheduler).ServeHTTP(0xc00056ba00?, {0x2ca0068?, 0xc001d95140?}, 0xc0005752a8?) /src/enterprise-logs/vendor/github.com/grafana/loki/pkg/scheduler/scheduler.go:765 +0x25 ``` Signed-off-by: Ashwanth Goli <iamashwanth@gmail.com>	2 years ago
Chris Hodges	eb6ba615f4	Promtail: Add configuration to drop batches when rate limited by Loki (#7973) ... This change will allow for more deterministic behavior by multitenant Promtail instances where one or more tenants are being heavily throttled due to limits exceeded within Loki. Specifically, it will reduce the performance impact of having exponential backoff cause HOL blocking for rate limit response codes while other tenants are ready to send batches. Co-authored-by: chodges15 <chris.hodges@gmail.org>	2 years ago
Ashwanth	b3e038c7a8	config_wrapper: Add support for named stores (#7946) ... Signed-off-by: Ashwanth Goli <iamashwanth@gmail.com> **What this PR does / why we need it**: This PR adds support for named stores which allows users to define multiple object store configurations for a storage type. Named stores can be referred to from other sections of the config similar to how provider type is being used `i.e aws, gcs...`. Named store reference should be of the format: `<store_type>.<key_name>` Below is an example configuration with two named stores defined for aws `store-1, store-2`. Schema config is referring to one of them as `aws.store-1` ``` storage_config: aws: endpoint: s3://common-bucket region: us-east1 access_key_id: abc123 secret_access_key: def789 named_stores: aws: store-1: endpoint: s3://foo-bucket region: us-west1 access_key_id: 123abc secret_access_key: 789def store-2: endpoint: s3://bar-bucket region: us-west2 access_key_id: 456def secret_access_key: 789abc schema_config: configs: - from: "2020-07-31" index: period: 24h prefix: loki_prod_index_ object_store: aws.store-1 schema: v11 store: boltdb-shipper ``` **Which issue(s) this PR fixes**: Fixes #7276 **Special notes for your reviewer**: Configuration defined in common storage config has no effect on the named stores. Common storage config only overrides storageConfig if it's not explicitly set - same behavior as before. This should be a non-intrusive change. Signed-off-by: Ashwanth Goli <iamashwanth@gmail.com>	2 years ago
Bryan Boreham	2bad98dc8e	dashboards: add k8s resource requests to CPU and memory panels (#8047) ... **What this PR does / why we need it**: This lets administrators see at a glance if requests are too low, which risks overloading nodes, or too high which wastes provisioning. Example: <img width="356" alt="image" src="https://user-images.githubusercontent.com/8125524/210851875-f9ccbb60-1a3c-4276-b65f-645aec0268be.png"> **Checklist** - [x] Reviewed the `CONTRIBUTING.md` guide - NA Documentation added - NA Tests updated - [x] `CHANGELOG.md` updated - NA Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` Co-authored-by: Owen Diehl <ow.diehl@gmail.com>	2 years ago
Kaviraj Kanagaraj	c31c85bdcd	chore(ci): Remove CircleCI (#8061)	2 years ago
Jijo Thomas John	b022f5eee3	[loki-canary] Fix changelog (#8052) ... **What this PR does / why we need it**: #8024 added an enhancement to loki-canary to pickup config from env variables. However the change was wrongly added under loki instead of loki-canary in the changelog. This PR fixes the changelog. **Which issue(s) this PR fixes**: NA **Special notes for your reviewer**: **Checklist** - [x] Reviewed the `CONTRIBUTING.md` guide - [ ] Documentation added - [ ] Tests updated - [x] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	2 years ago
Ed Welch	2b66c6b126	Loki: add a 'since' query parameter for easier querying using relative time (#7964) ... Signed-off-by: Edward Welch <edward.welch@grafana.com> **What this PR does / why we need it**: logcli has a command line parameter called `since` which makes it easy to query logs relative to the current time, however this is some syntactic sugar used in logcli to calculate `start` and `end` query parameters for the query sent to Loki. This PR adds native support for a `since` query parameter to make it a little easier to consume the Loki query API without having to calculate precise start/end times with every request. This is useful if you are polling the Loki API from a third party app which doesn't let you easily build dynamic URL's and all you want to do is poll for say the last 3 hours of logs. **Which issue(s) this PR fixes**: Fixes #<issue number> **Special notes for your reviewer**: **Checklist** - [x] Reviewed the `CONTRIBUTING.md` guide - [x] Documentation added - [x] Tests updated - [x] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` Signed-off-by: Edward Welch <edward.welch@grafana.com> Co-authored-by: Owen Diehl <ow.diehl@gmail.com>	2 years ago
Jijo Thomas John	f9ed9cb7d9	[loki-canary] Support passing loki address as environment variable (#8024) ... **What this PR does / why we need it**: - Replace flag with kingpin which has more capabilities to override defaults - Override default for `addr`, `user` and `pass` from environment the variables `LOKI_ADDRESS`, `LOKI_USERNAME` and `LOKI_PASSWORD` respectively. **Which issue(s) this PR fixes**: Fixes #8021 **Special notes for your reviewer**: **Checklist** - [x] Reviewed the `CONTRIBUTING.md` guide - [x] Documentation added - [ ] Tests updated - [x] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	2 years ago
Maciej Strzelecki	b8d3256fd9	Helm: Use stable autoscaling/v2 if possible (#7416) ... **What this PR does / why we need it**: It checks if the Kubernetes cluster, on which Loki is being installed using Helm, supports the stable version of the autoscaling API group. [`HorizontalPodAutoscaler` v2 graduated to GA in Kubernetes 1.23](https://kubernetes.io/blog/2021/12/07/kubernetes-1-23-release-announcement/#horizontalpodautoscaler-v2-graduates-to-ga). The v2beta1 and v2beta2 became deprecated, and they will be removed in Kubernetes 1.26 which is going to be released in December 2022.	2 years ago
李国忠	b4440911e0	[new feature] logql:support sort and sort_desc (#7989) ... **What this PR does / why we need it**: sort() sort(v instant-vector) returns vector elements sorted by their sample values, in ascending order. sort_desc() Same as sort, but sorts in descending order. ex: `sort(rate(({app=~"foo|bar"} |~".+bar")[1m])) without (app) ` prometheud doc: https://prometheus.io/docs/prometheus/latest/querying/functions/#sort **Which issue(s) this PR fixes**: Fixes #7933 **Special notes for your reviewer**: **Checklist** - [x] Reviewed the `CONTRIBUTING.md` guide - [x] Documentation added - [x] Tests updated - [x] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` Co-authored-by: Owen Diehl <ow.diehl@gmail.com>	2 years ago
Sandeep Sukhani	24deb6ed3b	fix bugs in logs results caching and its tests (#7925) ... **What this PR does / why we need it**: When a logs query results in an empty response, we cache it to avoid doing that query again and respond straight away with an empty response. However, we cache a single entry per time split interval with the query itself to keep things simple. For example, if the time split config for tenant `A` is `30m`, then queries for intervals `10m`-`20m` and `21m`-`25m` would have the same cache key. Here is roughly how cache hit is handled: * If the new query is within the cached query bounds, return empty results * If the start of new query is before the start time of the cached query, do a query from `newQuery.Start` to `cachedQuery.Start` * If the response of last query is also empty, set `cachedQuery.Start` = `newQuery.Start` * If the end of new query is after the end time of the cached query, do a query from `cachedQuery.End` to `newQuery.Start` * If the response of last query is also empty, set `cachedQuery.End` = `newQuery.End` * If we have changes in `cachedQuery.Start/End`, update it in the cache. The problem here is when we do queries to fill the gap, we sometimes do queries for the range outside of what the user requested and respond back without reducing the response to what the user requested. For example, if the cached query is from `21m`-`25m` and the user query is from `10m`-`15m`, we will query for the whole gap i.e `10m`-`21m`. If there are logs from `15m`-`21m` in the response, we will unexpectedly send it back to the user. This PR takes care of this issue by extracting the data and sending back only the user's requested logs. I have also found the tests for logs results cache were incorrect. They heavily use [mergeLokiResponse](e2842c69c5/pkg/querier/queryrange/codec.go (L1014)) for building the test data, but they were not setting the `limit` in test queries which was causing `mergeLokiResponse` to send back empty results. This means were were always comparing two empty results, which would always be the same. This PR also takes care of fixing it, and relevant changes to correct the test. **Checklist** - [x] Tests updated - [x] `CHANGELOG.md` updated	2 years ago
Christian Haudum	17577b4d47	Shut down query frontend gracefully (#7978) ... **What this PR does / why we need it**: This PR fixes a bug in the shutdown process of the query frontend. In order to shut down the frontend gracefully without cancelling any in-flight operations/sub-requests, the frontend service has to wait until all in-flight operations are finished. Only then its sub-services may be stopped. In order to control the deterministic order of the shutdown of the service and its child services, child services need to be started with their own context (not with their parent context). Otherwise, if the context of the frontend is cancelled, also the contexts of the child services are cancelled, leading to a parallel stopping of the parent and its child services. During the time of the shutdown, the `RoundTripGRCP` function still needs to accept any round-tripping requests. This is, because the frontend round tripper is wrapped into a query range tripper ware, which splits incoming (HTTP) query requests into multiple (gRPC) query requests if needed. **Which issue(s) this PR fixes**: This fixes a race condition where queries fail, because the query frontend shut down before the queriers that processed in-flight request could send the query response to the frontend. This manifested in an increased error rate of queries during restarts (rollouts) of the query frontend. **Special notes for your reviewer**: **Checklist** - [x] Reviewed the `CONTRIBUTING.md` guide - [x] Documentation added - [x] Tests updated - [x] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` Signed-off-by: Danny Kopping <danny.kopping@grafana.com> Signed-off-by: Christian Haudum <christian.haudum@gmail.com> Co-authored-by: Danny Kopping <danny.kopping@grafana.com>	2 years ago
Kaviraj Kanagaraj	1375fa0654	chore(promtail): Make `batchwait` and `batchsize` config explicit with yaml tags. (#8027) ... Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> **What this PR does / why we need it**: These configs can be used via `batchwait` and `batchsize` keys in yaml. This PR just makes it explicit via yaml tags. **Which issue(s) this PR fixes**: Fixes #NA **Special notes for your reviewer**: We already have test to cover those keys. https://github.com/grafana/loki/blob/kavirajk%2Fbatch-configs-make-yaml-explicit/clients/pkg/promtail/client/config_test.go#L28-L29 Also we have config pages for those keys. https://grafana.com/docs/loki/latest/clients/promtail/configuration/#clients **Checklist** - [x] Reviewed the `CONTRIBUTING.md` guide - [x] `CHANGELOG.md` updated Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com>	2 years ago
Michel Hollands	59eff5ca58	Add count to template functions (#7951) ... **What this PR does / why we need it**: Add a `count` template function to count occurrences of a substring. For example if `label="abc abd abc"` ``` {{ .label | count "abc" }} ``` would return 2. Signed-off-by: Michel Hollands <michel.hollands@grafana.com> Co-authored-by: Danny Kopping <dannykopping@gmail.com>	2 years ago
Ashwanth	dc5cc4c494	store: write overlapping chunks to multiple stores (#7988) ... Signed-off-by: Ashwanth Goli <iamashwanth@gmail.com> **What this PR does / why we need it**: When a chunk falls on the period boundary it should be written to both the stores as it gets referred to by the indexes on both sides. But we currently skip the writes for subsequent periods if the chunk is found in the cache. This PR ensures that overlapping chunks get written to both the stores **Which issue(s) this PR fixes**: Fixes #7847 **Special notes for your reviewer**: Downside of this approach is that we end up writing the overlapping chunk multiple times, once from each instance in the replica set. But these duplicate writes would be infrequent given this occurs only on schema changes **Checklist** - [X] Reviewed the `CONTRIBUTING.md` guide - [X] Tests updated - [x] `CHANGELOG.md` updated Signed-off-by: Ashwanth Goli <iamashwanth@gmail.com>	2 years ago
Kaviraj Kanagaraj	d5b68c0257	fix(promtail): Fix cri tags extra new lines. (#7997)	2 years ago
Sandeep Sukhani	9e7ac3d853	fix query-frontend request load balancing when using k8s service (#7966) ... **What this PR does / why we need it**: We noticed an imbalance in the requests sent to the Query Frontend pods using `query-frontend` k8s service. This seems to be caused by `query-frontend` being a headless service that resolves to each QF pod IP and leaves it up to the client to load balance the requests as mentioned [here](https://kubernetes.io/docs/concepts/services-networking/service/#headless-services). This PR fixes the issue by creating two separate services for pod IP discovery and load balancing of queries: * `query-frontend` to be used for load balancing incoming Loki queries. * `query-frontend-headless` to be used for discovering QF pod IPs from queriers to connect as workers. **Checklist** - [x] `CHANGELOG.md` updated - [x] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	2 years ago
Michel Hollands	5fe9c964db	Fix validation for pattern and rexexp parsers to they don't panic in query-frontend (#7926) ... Running the query `topk(10,sum by(namespace)(count_over_time({application="nginx", site!="eu-west-1-dev"} |= "/artifactory/" != "api" != "binarystore" | regexp `` [1d])))` causes a panic. This PR adds validation for regexp and pattern parsers at parse time. If they don't have a (valid) parameter an error will be returned. Signed-off-by: Michel Hollands <michel.hollands@grafana.com> Co-authored-by: Susana Ferreira <ssncferreira@gmail.com>	2 years ago
Susana Ferreira	86d33a4e4a	Add CI step to check generated documentation to check pipeline (#7938) ... **What this PR does / why we need it**: Follow up from PR: https://github.com/grafana/loki/pull/7916 to add a CI step to check if the configuration flag documentation file `doc/sources/configuration/_index.md` was generated successfully. Requirements: * CI `check` pipeline should fail if the documentation file is updated manually * CI `check` pipeline should fail if registration flags are changed in the code, but the documentation file is not generated. **Which issue(s) this PR fixes**: Fixes https://github.com/grafana/loki-private/issues/83 **Special notes for your reviewer**: **Checklist** - [ ] Reviewed the `CONTRIBUTING.md` guide - [ ] Documentation added - [ ] Tests updated - [x] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	2 years ago
Susana Ferreira	e07fe7c338	Deprecate CLI flag -ruler.wal-cleaer.period in favor of CLI flag -ruler.wal-cleaner.period (#7937) ... **What this PR does / why we need it**: Deprecate CLI flag `-ruler.wal-cleaer.period` in favor of CLI flag `-ruler.wal-cleaner.period`. **Which issue(s) this PR fixes**: Fixes https://github.com/grafana/loki/pull/7916#issuecomment-1348276269 and https://github.com/grafana/loki/pull/7916#discussion_r1047043534 **NOTE:** correctly registring the `ruler.wal-cleaner` configuration flags is not a breaking change and doesn't require an upgrade guide, because the ruler WAL cleaner initialization code already guarantees that the default values are used: c71620ae94/pkg/ruler/storage/cleaner/cleaner.go (L76-L101) **Special notes for your reviewer**: **Checklist** - [x] Reviewed the `CONTRIBUTING.md` guide - [x] Documentation added - [ ] Tests updated - [x] `CHANGELOG.md` updated - [x] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	2 years ago
Kaviraj Kanagaraj	cbd6ec15ce	feat(logql): Supporting prettifying LogQL expressions (#7906) ... **What this PR does / why we need it**: Changes 1. Added `Pretty()` method to `Expr` interface. So that every expression would know how to render themselfs as pretty :) 2. Implemented `Pretty()` for every single LogQL expressions 3. Exposed `/api/v1/format_query` endpoint that takes `query` argument and returns formatted version. 4. Integerated into `logcli`. echo `'<query>' | logcli fmt` would format the query. Why? 1. Readability, helpful in debugging and **more importantly, to understand the execution flow** example **before** ``` quantile_over_time(0.99,{container="ingress-nginx",service="hosted-grafana"}| json| unwrap response_latency_seconds| __error__=""[1m]) by (cluster) ``` ``` sum(rate({job="loki", namespace="loki", cluster="loki-dev-us"} |= "err" [5m])) + sum(rate({job="loki-dev", namespace="loki", cluster="loki-dev-eu"}|logfmt | level != "info" [5m])) / sum(rate({job="loki-prod", namespace="loki", cluster="loki-prod-us"} |logfmt | level="error"[5m])) ``` ``` label_replace(rate({job="api-server",service="a:c"}|= "err" [5m]), "foo", "$1", "service", "(.*):.*") ``` **after** ``` quantile_over_time( 0.99, {container="ingress-nginx", service="hosted-grafana"} | json | unwrap response_latency_seconds | __error__="" [1m] ) by (cluster) ``` ``` sum( rate( {job="loki", namespace="loki", cluster="loki-dev-us"} |= "err" [5m] ) ) + sum( rate( {job="loki-dev", namespace="loki", cluster="loki-dev-eu"} | logfmt | level!="info" [5m] ) ) / sum( rate( {job="loki-prod", namespace="loki", cluster="loki-prod-us"} | logfmt | level="error" [5m] ) ) ``` ``` label_replace( rate({job="api-server", service="a:c"} |= "err"[5m]), "foo", "$1", "service", "(.*):.*" ) ``` You can find more examples in the `prettier_test.go` Future plans * Integerate into LogQL analyzer * Integrate into Grafana UI. **Which issue(s) this PR fixes**: Fixes # NA **Special notes for your reviewer**: This whole idea was inspired from last [PromCon lighting talk](https://youtu.be/pjkWzDVxWk4?t=24469) **Checklist** - [x] Reviewed the `CONTRIBUTING.md` guide - [x] Documentation added - [x] Tests updated - [x] `CHANGELOG.md` updated Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> Co-authored-by: Christian Haudum <christian.haudum@gmail.com> Co-authored-by: Christian Simon <simon@swine.de>	2 years ago
Charlie N	9d5665e34a	[Jsonnet] Fix memberlist when using a stateful ruler (#6662) ... Signed-off-by: Whyeasy <Whyeasy@users.noreply.github.com> <!-- Thanks for sending a pull request! Before submitting: 1. Read our CONTRIBUTING.md guide 2. Name your PR as `<Feature Area>: Describe your change`. a. Do not end the title with punctuation. It will be added in the changelog. b. Start with an imperative verb. Example: Fix the latency between System A and System B. c. Use sentence case, not title case. d. Use a complete phrase or sentence. The PR title will appear in a changelog, so help other people understand what your change will be. 3. Rebase your PR if it gets out of sync with main --> **What this PR does / why we need it**: When using stateful rulers and the memberlist as its ring, generating the resources with jsonnet throws in an error regarding the `ruler_deployment`. The `ruler_statefulset` didn't receive the meberlist labels either. <!-- Note about CHANGELOG entries, if a change adds: * an important feature * fixes an issue present in a previous release, * causes a change in operation that would be useful for an operator of Loki to know then please add a CHANGELOG entry. For documentation changes, build changes, simple fixes etc please skip this step. We are attempting to curate a changelog of the most relevant and important changes to be easier to ingest by end users of Loki. Note about the upgrade guide, if this changes: * default configuration values * metric names or label names * changes existing log lines such as the metrics.go query output line * configuration parameters * anything to do with any API * any other change that would require special attention or extra steps to upgrade Please document clearly what changed AND what needs to be done in the upgrade guide. --> **Checklist** - [ ] Documentation added - [ ] Tests updated - [x] Is this an important fix or new feature? Add an entry in the `CHANGELOG.md`. - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` Signed-off-by: Whyeasy <Whyeasy@users.noreply.github.com>	2 years ago
Maxim Ivanov	b773a7a3cf	promtail: ratelimiting by label (#7597) ... Limit stage can now takes optional `by_label_name` param, which tracks distinct values of that label and ratelimit entries independently. Log lines without expected label are not considered for ratelimiting and passed to the next pipeline stage.	2 years ago
Trevor Whitney	99b7c14e09	Prepare changelog and release notes for 2.7.1 (#7895) ... **What this PR does / why we need it**: Prepares changelog and release notes for 2.7.1 release. Co-authored-by: Dylan Guedes <djmgguedes@gmail.com>	2 years ago
Sandeep Sukhani	e9c93cd0f5	consider range and offset in queries while looking for schema config for query sharding (#7880) ... **What this PR does / why we need it**: We disable query sharding when a query touches multiple schema configs since they might have different sharding factors and might handle sharding differently. This check was done purely on the start and end time of the query without considering `range` and `offset`, which would change the length of the actual data being queried. Besides being incorrect, it also causes us to fail queries when migrating between tsdb and non-tsdb stores since both handle query sharding differently. For e.g., if the prev schema is `boltdb-shipper` and starting today, `tsdb` index is being used, so if we do a query `sum(rate({foo="bar"}[24h])` even with start and end within `tsdb` range, we will fail the query with error `incompatible index shard` if dynamic sharding goes with shard factor other than `32`(default shard factor in ingester for inverted index). The reason being the `range` here is `24h` which causes us to process data from previous schema as well. This PR takes care of the issue by factoring in `range` and `offset` in the queries while looking for schema config for query sharding. **Checklist** - [x] `CHANGELOG.md` updated	2 years ago
cadrake	ea9ad336dc	Loki: Modified heroku drain target to make any url query parameters available as labels (#7619) ... **What this PR does / why we need it**: This PR reads the url query parameters included in Heroku drain POST request and creates `__param_<name>` labels similar to how the parameters work with prometheus metrics and metric endpoints as a way to allow setting the true application name when ingesting logs from a Heroku drain. **Which issue(s) this PR fixes**: Fixes #<issue number> **Special notes for your reviewer**: The existing `app` field coming from Heroku only specifies application source (either the application itself or Heroku) and thus is insufficient for uniquely identifying a log lines source. **Checklist** - [x] Reviewed the `CONTRIBUTING.md` guide - [x] Documentation added - [x] Tests updated - [x] `CHANGELOG.md` updated - [x] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	3 years ago
Kaviraj Kanagaraj	4efdc131be	fix(memcached): panic on `send on closed channel`. (#7817)	3 years ago
Sandeep Sukhani	1410808ee9	use grpc for communicating with compactor for query time filtering of data requested for deletion (#7804) ... **What this PR does / why we need it**: Add grpc support to compactor for getting delete requests and gen number for query time filtering. Since these requests are internal to Loki, it would be good to use grpc instead of HTTP same as all the internal requests we do in Loki. I have added a new config for accepting the grpc address of the compactor. I tried having just the existing config and detecting if it is a grpc server, but it was hard to do it reliably, considering the different deployment modes we support. I think it is safe to keep it the same and eventually deprecate the existing config. **Checklist** - [x] Documentation added - [x] Tests updated - [x] `CHANGELOG.md` updated	3 years ago
George Tsilias	d3615f7e9e	promtail: Handle nil error on target Details() call (#7771) ... **What this PR does / why we need it**: This PR addresses issue [#6930](https://github.com/grafana/loki/issues/6930). Promtail should not panic when the Details() method is called for a target that has no errors. **Which issue(s) this PR fixes**: Fixes #6930	3 years ago
Josef Johansson	b05f4fced3	Remove hardwired '127.0.0.1' values in default addresses (#7784) ... While running all-in-one Loki instance, it's not possible to listen on anything other than IPv4 since compactor_address is then set to http://127.0.0.1. This enables listening on only IPv6. Also if frontend_address, scheduler_ring and scheduler_address is not set, the default value is to connect to '127.0.0.1:<grpc_listen_port>'. Which means that grpc_listen_address is ignored. **What this PR does / why we need it**: It's not possible to use compactor while using an all-in-one instance and listening to IPv6. Let's not ignore grpc_listen_address to not confuse users. **Special notes for your reviewer**: As noted in the first commit, the default behavior is changed from defaulting to `127.0.0.1` to `localhost`. Not sure how major that change is though. My /etc/hosts says ``` $ grep localhost /etc/hosts 127.0.0.1 localhost ``` Signed-off-by: Josef Johansson <josef@oderland.se>	3 years ago
李国忠	6d05ade6a8	[performance] metrics query: range vector support streaming agg when no overlap (#7380) ... metrics query: range vector support streaming agg when no overlap	3 years ago
MarNicGit	59c1abe259	Docs - Move PR's in CHANGELOG.md that weren't actually implemented in 2.7.x (#7690) ... Fixes the changelog. - PR #7462 and #7461 were **not** included in the 2.7.x release (they were submitted right after the cutoff and did not get backported) and accidentally ended up in the 2.7.x changelog. Moved them up to Main/Unreleased. - Moved #7602 to enhancements section of Promtail	3 years ago
Chris Hodges	e80b4deb43	Promtail: Add tenant label to client drop metrics and logs (#7593) ... **What this PR does / why we need it**: Provides better visibility for per-tenant drops/retries in Promtail **Which issue(s) this PR fixes**: Fixes #7570 **Special notes for your reviewer**: Does anything depend on these metrics only having 1 label that this change would break? I decided against a reason label due to cardinality concerns given how widely promtail is often deployed. **Checklist** - [x] Reviewed the `CONTRIBUTING.md` guide - [x] Documentation added - [x] Tests updated - [x] `CHANGELOG.md` updated - [x] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	3 years ago
Danny Kopping	a63ad06509	Querier/Ruler: query blocker (#7785) ... Block malicious or expensive queries using a per-tenant runtime configuration.	3 years ago
Dylan Guedes	ad2260aec2	Loki: Fix multitenant querying (#7708) ... **What this PR does / why we need it**: We recently broke multitenant querying because of recent changes to how timeouts working across Loki. This PR fixes this by: - Adapt the timeout wrapper to work with multitenant queries. It will take the shortest timeout across all given tenants - Adapt the query engine timeout assigning to work with multitenant queries. It will take the shortest timeout between all the tenants - Adapt query sharding to use smallest max query parallelism across given tenants - Add a functional test to ensure multitenant is behaving as expected Signed-off-by: DylanGuedes <djmgguedes@gmail.com> Signed-off-by: Mehmet Burak Devecí <mhmtbrkdvc@gmail.com> **Which issue(s) this PR fixes**: https://github.com/grafana/loki/issues/7696 **Special notes for your reviewer**: The regression was probably introduced by https://github.com/grafana/loki/pull/7555	3 years ago
Kaviraj Kanagaraj	3a0be48975	config: Improve error message loading config with ENV variables. (#7759) ... **What this PR does / why we need it**: Before: ``` Unable to parse config: ./clients/cmd/promtail/promtail-local-config.yaml: parse "http://${LOKI_URL}localhost:3100/loki/api/v1/push": invalid character "{" in host name ``` After: ``` Unable to parse config: ./clients/cmd/promtail/promtail-local-config.yaml: parse "http://${LOKI_URL}localhost:3100/loki/api/v1/push": invalid character "{" in host name. Use `-config.expand-env=true` flag if you want to expand environment variables in your config file ``` **Which issue(s) this PR fixes**: Fixes #NA **Special notes for your reviewer**: **Checklist** - [x] Reviewed the `CONTRIBUTING.md` guide - [x] `CHANGELOG.md` updated	3 years ago
Rui Fernandes	1ae1318fce	Examples: Add healthckecks in docker example (#7731) ... **What this PR does / why we need it**: Adds healthchecks for the loki `docker compose` example, this makes the getting started example more complete, and can be now used with the `--wait` command. **Special notes for your reviewer**: Use the following commands to test: ```sh cd examples/getting-started docker compose up -d --wait docker compose ps ``` **Checklist** - [x] Reviewed the `CONTRIBUTING.md` guide - [x] `CHANGELOG.md` updated	3 years ago
Sandeep Sukhani	d3e42959a5	fix bugs in processing delete requests with line filters and add proper tests (#7720) ... **What this PR does / why we need it**: It fixes the following issues: * Whole chunks being deleted with a line filter were skipped. * Chunks partially covered by delete requests with a line filter were not properly processed. The part that **was not covered** by the delete request was rewritten with a line filter, while the part that **was covered** by the delete request was completely deleted. The issue went unnoticed because it seems we continue to do query time filtering of data even after the delete requests are processed. **Which issue(s) this PR fixes**: Fixes #7590 **Special notes for your reviewer**: If multiple delete requests with a line filter touch the same chunk, then I am considering just the first request that marks it for deletion to keep this PR simple. I have left a to-do to see if we can make it work efficiently. **Checklist** - [x] Tests updated - [x] `CHANGELOG.md` updated	3 years ago
Kaviraj Kanagaraj	7073ea8702	docs(embedded-cache): Add missing docs on config page. (#7684) ... **What this PR does / why we need it**: Adds `embedded-cache` config under `cache_config` section. Also adds message about deprecation of `fifocache` **Which issue(s) this PR fixes**: Fixes #7677 **Special notes for your reviewer**: **Checklist** - [x] Documentation added - [x] `CHANGELOG.md` updated	3 years ago
Trevor Whitney	04c7fbb531	Add config changes to upgrade guide (#7662) ... **What this PR does / why we need it**: This add config differences I noticed when running the config diff tool between 2.6 and 2.7 Co-authored-by: JordanRushing <rushing.jordan@gmail.com>	3 years ago
MarNicGit	1f1dd81555	Promtail: Exclude event message (#7462) ... **What this PR does / why we need it**: Windows Event Logs have an Event Message field that is intended for human eyes, and often contains data that already present in the event data XML. Omitting this field the same way we can already omit `user_data` and `event_data` can easily save a lot of bytes of data per event - Event ID 4264 alone has ~2KB of just text that is already present in `event_data`. **Which issue(s) this PR fixes**: Fixes #7395 **Special notes for your reviewer**: I also took the liberty to improve upon the existing test '`Test_renderEntries` by using unique values for each field rather than 10's everywhere. I expect this to conflict with my other PR, #7461.	3 years ago