4127 Commits (b523ff336e91e1e8a607073551b8a3525b6ba917)

 

All Branches   

Search

Author	SHA1	Message	Date
Trevor Whitney	b523ff336e	[Helm] Fix invalid ruler config when filesystem storage is used (#7528) ... **What this PR does / why we need it**: When using filesystem storage, there's a bug in the configmap helm template that sets `ruler.storage` to `null`. This fixes that bug. **Which issue(s) this PR fixes**: Fixes #7267	3 years ago
Callum Styan	46b40a5582	Hardcode ring.WriteNoExtend for distributor push operations (#7517) ... With zone awareness enabled we allow restarting multiple ingesters within a single zone at the same time, this means that for many more write requests there's potentially only 2 of the 3 replicas for a stream up at the time the write request is received. This lead to a large increase in 500s during rollouts. My suspicion is that the 500s on rollouts was always related to the following; the replica set ring code has something referred to as "extending the replica set". This essentially is the adding of ingesters in the ring that are not in an ACTIVE state (for example, are in LEAVING) to the set of replicas considered valid for an operation. Adding of the instance in a state other than ACTIVE can be controlled by using a different operation type, which Mimir was using but we were not. Here are the relevant functions, [ring.Get](8d6d914ef6/ring/ring.go (L335-L403)) (particularly these lines [1](8d6d914ef6/ring/ring.go (L381-L389)) and [2](8d6d914ef6/ring/ring.go (L394-L397))) and [replicationStrategy.Filter](789ec0ca4a/ring/replication_strategy.go (L22-L71)) (particularly these lines [1](789ec0ca4a/ring/replication_strategy.go (L32-L36)) and [2](789ec0ca4a/ring/replication_strategy.go (L54-L67))) The tl; dr of the above code snippets is that we seem to always have required a minSuccess from replicationFactor/2 + 1 except we overwrite replicationFactor to be the # of instances which is likely always going to be more than just 3 ingester replicas because of the replica set extension. At least for zone aware rollouts this seems to be the case. This is because many ingesters will be in the LEAVING state. We can avoid this by just using `WriteNoExtend` operations instead of `Write` operations ([here](789ec0ca4a/ring/ring.go (L83-L93))) In this PR I've hardcoded our call to the ring in `distributor.Push` to always use `ring.WriteNoExtend`. IMO this is the better option to allowing `ring.WriteNoExtend` to be used optionally and defaulting to `ring.Write`. Extending the replication set on writes feels like a footgun for write outages and there's even some feelings internally that allowing that extension is a bug. For more background information see: https://github.com/grafana/mimir/issues/1854 Signed-off-by: Callum Styan <callumstyan@gmail.com> Signed-off-by: Callum Styan <callumstyan@gmail.com>	3 years ago
Karsten Jeschkies	b16ad8cb38	Document new Helm Charts (#7034) ... **What this PR does / why we need it**: This change describes various Helm related tasks such as installing Loki and configuring meta monitoring. It also explains different concepts related to the new Helm chart and provides a reference of the new Helm values file. **Which issue(s) this PR fixes**: Fixes #7035	3 years ago
Cyril Tovena	7b01864a91	Improves performance of distributor validations (#7534) ... Most of the time we don't need to create those strings which create an alloc per entry.	3 years ago
Pablo	1df72ee209	Add GCP Logs push target Promtail docs (#7511) ... **What this PR does / why we need it**: Some months ago, a new Promtail target was added to support receiving GCP Logs in a push manner. This PR updates the setting up GCP guide, for adding the new required push subscription. Also, adds some notes on details of the setup procedure. This PR will have a follow up expanding on the terraform code needed to setup the GCP resources in a IaaC way. **Which issue(s) this PR fixes**: Fixes #<issue number> **Special notes for your reviewer**: **Checklist** - [ ] Reviewed the `CONTRIBUTING.md` guide - [x] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	3 years ago
Trevor Whitney	705e16fbea	[helm] Fix included rules (#7430) ... **What this PR does / why we need it**: Adds the correct cluster label to our recording rules so they work correctly in the included dashboards. Co-authored-by: Karsten Jeschkies <k@jeschkies.xyz>	3 years ago
Ed Welch	3ecfe2478c	Fix drone signature (#7512) ... Signed-off-by: Edward Welch <edward.welch@grafana.com> **What this PR does / why we need it**: **Which issue(s) this PR fixes**: Fixes #<issue number> **Special notes for your reviewer**: **Checklist** - [ ] Reviewed the `CONTRIBUTING.md` guide - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` Signed-off-by: Edward Welch <edward.welch@grafana.com>	3 years ago
Ed Welch	45caba4459	Loki: Remove the bypass for "limited" queries (#7510) ... **What this PR does / why we need it**: Limited queries are queries which don't have a filter expression. Now all log queries will be handled by the LogFilterTripperware which will result in them being split by time (they previously were not) Signed-off-by: Edward Welch <edward.welch@grafana.com> **Which issue(s) this PR fixes**: We've found that very large timeframe `limited` queries will be sent to queriers and then ingesters and can stall out the read path because of the large time ranges. Splitting these by time avoids this problem by keeping any subquery limited in length to the `split_by_interval`. It's important to note that this will likely increase some extra work done by limited queries as more work for these will be parallelized and can result in extra data being processed. This will be the same as how filter queries are handled now, so it will be no worse than that. In fact this was an optimization based on the premise that it's advantageous not to split/shard limited queries but we are seeing this not be the case when limited queries are run over very large time ranges matching streams with huge volumes and combined with parsers like `| json` **Special notes for your reviewer**: **Checklist** - [ ] Reviewed the `CONTRIBUTING.md` guide - [ ] Documentation added - [x] Tests updated - [x] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` Signed-off-by: Edward Welch <edward.welch@grafana.com>	3 years ago
Trevor Whitney	f83988a061	Add 2.7.0 section to upgrading doc (#7457) ... Add 2.7.0 section to Upgrading doc. Co-authored-by: Ursula Kallio <ursula.kallio@grafana.com>	3 years ago
Trevor Whitney	57c3a704b8	Few small changelog changes (#7460) ... A few small fixes to changelog. Removed one entry that was just about updating docs, and reorganized a few others to the correct section.	3 years ago
Trevor Whitney	6681923a64	Make 2.7.0 section (#7456) ... Update the 2.7 changelog to move unreleased section into 2.7.	3 years ago
Jan-Otto Kröpke	77225e8a7c	Remove prometheus CRDs (#7499) ... **What this PR does / why we need it**: Remove Prometheus CRDs from loki chart. Signed-off-by: Jan-Otto Kröpke <jok@cloudeteer.de> Signed-off-by: Jan-Otto Kröpke <mail@jkroepke.de>	3 years ago
Trevor Whitney	6c53113be1	[helm] Add a loki canary test to the helm chart (#7229) ... **What this PR does / why we need it**: Add a helm chart test that utilizes the Loki canary to confirm the Loki cluster is functioning correctly. **Which issue(s) this PR fixes**: Fixes #7228 Co-authored-by: Karsten Jeschkies <k@jeschkies.xyz>	3 years ago
Karsten Jeschkies	9bc6e85604	Helm: Allow users to define remote write for metrics. (#7399)	3 years ago
Goutham Veeramachaneni	1260adc756	[mixin] Add cortex-gw-internal to dashboards (#7481)	3 years ago
Sandeep Sukhani	bb2927172a	tsdb post compaction cleanup (#7502) ... **What this PR does / why we need it**: Previously when we were reading the whole tsdb index in memory, there was not much cleanup required after we were done with the index. Now that we are mmaping the tsdb index, we need to close the mmap. I had missed calling `index.Close()` on the compacted-away multi-tenant index. This causes the compactor to retain the files on disk, leading to the compactor running away from disk space. This PR takes care of it. While doing the changes, I noticed that some of the tests failed due to `LabelValues` tsdb implementation returning strings valid only until the underlying byte slice stays valid. Since the underlying byte slice could become invalid when we close the tsdb, we should return a copy instead of yolostring. This can also happen on the query path when a sync closes the index before we get to return the response. This PR also fixes that.	3 years ago
Ed Welch	23cc448881	Loki: log when a query starts in the logql engine (#7469) ... Signed-off-by: Edward Welch <edward.welch@grafana.com> **What this PR does / why we need it**: There are cases where a query cannot finish (OOM of a querier/ruler) and most of our logs are emitted upon completion of a query. This can help us find queries which are OOM crashing a component. Note: It only covers queries executed by the engine, so only instant and range queries. Another PR will be necessary to cover labels/series endpoints. However this approach seemed the best to be able to get this information from ruler executed queries. **Which issue(s) this PR fixes**: Fixes #<issue number> **Special notes for your reviewer**: **Checklist** - [ ] Reviewed the `CONTRIBUTING.md` guide - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` Signed-off-by: Edward Welch <edward.welch@grafana.com>	3 years ago
Owen Diehl	70f6bf2e45	Index gw lock bugfix (#7497) ... While I'm still in favor of a longer refactor to address the complexity described in https://github.com/grafana/loki/pull/7490, I'm opening this PR as a shorter, more immediate alternative that should fix the deadlocks. This also reduces the number of total lock calls in many cases as we lock to act over entire groups rather than individual items.	3 years ago
Travis Patterson	45d59ba571	Improve rate store refresh latency (#7492) ... Rather than iterating streams and getting rates per stream, keep all necessary information in the rate calculator and just hand back the current rates when asked. This change took p99 latencies from ~1.8s to ~400ms and p50 latences from ~1s to 6ms.	3 years ago
Michel Hollands	fd93a6c999	Add how to connect to Azure using Service Principal (#7485) ... Add documentation for https://github.com/grafana/loki/pull/7179 which added authentication using a Service Principal for Azure.	3 years ago
Travis Patterson	b4ed595461	Remove groupcache (#7126) ... As discussed during the caching retro, this PR removes the experimental distributed cache based on Mailgun's fork of groupcache. Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> Co-authored-by: Kaviraj <kavirajkanagaraj@gmail.com>	3 years ago
Trevor Whitney	ae2a6f31ae	[Helm] Add flags needed for migration from distributed (#7331) ... **What this PR does / why we need it**: Adds flags to the helm charts that are needed when doing a zero-downtime migration from a `loki-distributed` helm chart deployment. The flags allow for an additional memberlist join member to be specified. This allows the new `grafana/loki` cluster to join the old `grafana/loki-distributed` cluster, forming one big cluster. The ingesters in the `grafana/loki-distributed` cluster can then be taken down 1-by-1, allowing the whole `grafana/loki-distributed` cluster to be deleted once all the ingesters have scaled down. **Which issue(s) this PR fixes**: Fixes #7330 Co-authored-by: Karsten Jeschkies <k@jeschkies.xyz>	3 years ago
Dylan Guedes	da5378c927	Loki: Remove RatelimitedStream proto (#7235) ... **What this PR does / why we need it**: Remove an unused proto (RateLimitedStream). It was only used for an older approach to deal with rate-limited streams but since we changed approaches this isn't necessary anymore.	3 years ago
Dylan Guedes	67de0334e9	Docs: Add max_streams to Promtail configuration doc (#7489) ... **What this PR does / why we need it**: Document `max_streams` configuration, introduced in 2.7. Co-authored-by: Ursula Kallio <ursula.kallio@grafana.com>	3 years ago
Trevor Whitney	16862b1edc	[helm] Mimic loki service account with canary account (#7236) ... **What this PR does / why we need it**: Bring the Loki Canary service account in line with the Loki service account, removing the post install hook, and sharing the top level properties for `automountServiceAccountToken` and `imagePullSecrets`. I'm open to creating canary specific values for these setting if needed, but want to start by sharing the top level values to reduce config if possible. Also default canary to Appversion to be in sync with default Loki version **Which issue(s) this PR fixes**: Fixes two issues raised in comments from @towolf (thanks!) on this [PR](https://github.com/grafana/loki/pull/7173#issuecomment-1256410181). Fixes #7237	3 years ago
Tim Schwenke	ff406bd4a9	Helm: Replace incorrect key loki.authEnabled in example override values (#7432) ... **What this PR does**: Replace `loki.authEnabled` in one of the example override YAMLs with `loki.auth_enabled` to match the actual key name. Also adds missing newline at end of file.	3 years ago
Mohamed-Amine Bouqsimi	7debe7c235	Ruler: enable Loki to configure Alertmanagers per tenant (#7359)	3 years ago
Christian Haudum	0f5dd2bceb	Revert "fluentd: Add un-escaping of control characters in JSON" (#7468) ... Reverts grafana/loki#7396 according to discussing on PR	3 years ago
Michel Hollands	488e64e3ff	Update helm chart version (#7484) ... Update the Loki helm chart version after the changes made in https://github.com/grafana/loki/pull/7479 Signed-off-by: Michel Hollands <michel.hollands@grafana.com>	3 years ago
Michel Hollands	ec541d70e9	Add envFrom to the tokengen job (#7479) ... Add an envFrom section to the tokengen job. Fixes https://github.com/grafana/loki/issues/7411 Signed-off-by: Michel Hollands <michel.hollands@grafana.com>	3 years ago
Mohamed-Amine Bouqsimi	7e7436d482	operator: Expose and migrate query_timeout in limits config (#7471)	3 years ago
Dylan Guedes	28e192f22e	Loki: Implement random entries assignment when sharding (#7427) ... **What this PR does / why we need it**: Modify sharding to assign random shards when the number of shards is bigger than the number of entries. This helps us to avoid hot-spotting the first shards. By randomizing the entries assignment we help spread the load between all shards.	3 years ago
Ursula Kallio	709c8508da	Update MAINTAINERS.md (#7467) ... This change removes removes Karen Miller, and includes at least one alternative, interim technical writer. Co-authored-by: Christian Haudum <christian.haudum@gmail.com>	3 years ago
Ursula Kallio	7fa3e500c3	Change `master` to `main` and include `.md` ext.	3 years ago
BenjaminVouillaumeJT	2d649a7fdd	Promtail: Add additional fields for cloudflare (#7463)	3 years ago
Periklis Tsirakidis	f8604ba0a7	operator: Upgrade operator-sdk to v1.24.1 (#7449)	3 years ago
Pablo	5f665651d5	Promtail: Add basic tracing support (#7414) ... We've been using Promtail's HTTP/gRPC server-based targets with real workloads for some weeks now. The server that these spin out is based on [weaveworks commons](https://github.com/weaveworks/common) HTTP/gRPC servers. These include by default some middlewares that if enabled, inject tracing support. We found this useful when tracking down request handling time, or even more, if the caller also support Jaeger tracing support, cross-correlated requests between the upstream server and Promtail's. This PR adds support for enabling tracing the same way Loki does. At the moment, this will just generate traces for the before-mentioned targets. In the future, the quality and completeness of tracing support can grow organically to enrich spans.	3 years ago
Robert Fratto	2208ea7147	Update Prometheus dependency to latest release (v2.39.1) (#7425)	3 years ago
Periklis Tsirakidis	4176de17d1	Add compactor delete client tls client config (#7426)	3 years ago
Michel Hollands	109b9dce9a	Add remote schema option (#7442) ... Signed-off-by: Michel Hollands <michel.hollands@grafana.com>	3 years ago
Mohamed-Amine Bouqsimi	fa305a4a7a	operator: Fix Custom TLS profile setting for LokiStack on OpenShift (#7437)	3 years ago
Periklis Tsirakidis	26d2989086	Expose ring and memberlist handlers through internal server listener (#7436)	3 years ago
Travis Patterson	c1bccac141	Results cache fix improvements (#7444) ... Move middleware to someplace more sensible and incorporate feedback missed in the review for this.	3 years ago
Owen Diehl	b59277b472	Revert "Revert "Update Thanos dependency to latest version"" (#7372) ... Reverts grafana/loki#7371 PR re re-enable Thanos dependency bump.	3 years ago
Travis Patterson	f2297d3d2a	Fix result cache misses on sharded queries (#7429) ... Headers weren't being propagated from query responses on sharded queries causing result cache misses. This PR: - Adds headers to `logproto.Result` so we can save response headers as they come back from queries - Metric-query results are turned into numbers well away from where responses are sent to the frontend: save all headers in the context so they can be pulled out when the final result is calculated - Propagate headers in the final responses	3 years ago
Danny Kopping	c6b2294bbf	Add Li Guozhong as a maintainer (#7440) ... A formal vote has taken place and `liguozhong` has been accepted! 🎉	3 years ago
Richard Treu	5fe714e466	fluentd: Add un-escaping of control characters in JSON (#7396)	3 years ago
Robert Jacob	895e95c4fc	operator: Regenerate manifests (#7438)	3 years ago
李国忠	8dcc4b3ec8	Do not log 'unable to read rules directory' at startup if directory hasn't been created yet (#7434)	3 years ago
Periklis Tsirakidis	9c8a827576	Fix enable of internal server for modules (#7423) ... **What this PR does / why we need it**: Makes adding the internal server modules to the module map more robust. The previous implementation accidentally dropped other modules. **Which issue(s) this PR fixes**: Fixes #<issue number> **Special notes for your reviewer**: **Checklist** - [x] Reviewed the `CONTRIBUTING.md` guide - [ ] Documentation added - [x] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	3 years ago