3830 Commits (15f8f42295c48968fbb5209b46360b79b4108020)

 

All Branches   

Search

Author	SHA1	Message	Date
Dylan Guedes	15f8f42295	Adapt queryTimeout to be a per-tenant configuration (#6835) ... What this PR does / why we need it: * Add a new per-tenant query timeout * Adds new middleware to query calls. This new middleware will timeout requests based on the new per-tenant query timeout * Add span logs to query calls missing it * Deprecate `engine.timeout` configuration The motivation to change this configuration to be per-tenant instead of global is that it makes sense to alleviate the timeout for particular tenants or make it more strict for others. Especially useful for scenarios where the Loki client doesn't handle context cancellations correctly; in such scenarios, since Loki would still process expensive queries that were canceled, this allows one to have small timeouts for most tenants, which will help mitigate unnecessary work without having a timeout that is too short for important tenants.	3 years ago
Travis Patterson	e15da437f0	Shard deletes requests based on time ranges (#6903) ... * Refactor the store API to be in terms of DeleteRequest * backfill remove tests * backfill GetAll tests * Make the delete request store deal in groups of requests instead of single ones * Make cancelation aware of request groups * remove cancelation deadline * shard deletes based on a query parameter * merge request group on gets * global configuration for max query range * sort delete requests by start time so batches are grouped to similar chunks * Fix status when some deletes haven't happened yet * fix tests * Review feedback * review feedback * make max_interval > deletion interval a validation error * list partially completed deletes * list partially completed deletes * fix flaky test * clarify naming * disallow cancelation of partial deletes unless the 'force' flag is passed * more comment specificity for the delete request hash	3 years ago
Salva Corts	7961ef6a4e	Evenly spread distributors across available nodes (#6964) ... * Evenly spread distributors across available nodes * Add note about reliability implications	3 years ago
liukch	59be5e036c	Fix Loki distributor in production Nomad job and docs. (#6963)	3 years ago
Trevor Lauder	e125252ee7	lambda-promtail: allow numbers on LB name regex (#6961)	3 years ago
Danny Kopping	449dd81981	Restoring metrics removed by #6957, simplifying implementation (#6962) ... Adding all available metrics from Go Signed-off-by: Danny Kopping <danny.kopping@grafana.com> Signed-off-by: Danny Kopping <danny.kopping@grafana.com>	3 years ago
Danny Kopping	2321da49b5	Upgrade prometheus/client_golang and reconfigure to restore `go_sched.*` metrics (#6957) ... * Upgrading prometheus/client_golang to restore go_sched.* metrics These metrics were removed unintentionally in #6403 Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Fixing test to match interface correctly Signed-off-by: Danny Kopping <danny.kopping@grafana.com> Signed-off-by: Danny Kopping <danny.kopping@grafana.com>	3 years ago
Alex Close	288e644894	docs: Fixed examples within Loki and Promtail docs (#6591) ... * Updated the promtail install example to both fix the Daemonset yaml file and provide a more concrete example for deploying to kubernetes * Removed the sidecar documentation for promtail install as it was both broken and didn't do what it said it did * Updated the simple scalable chart docs to include an example of using a overrides values file and to point to a working example of deploying promtail in a k8s cluster * Updated promtail and loki SSD install docs based on PR feedback * Update docs/sources/clients/promtail/installation.md Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> * Update docs/sources/installation/simple-scalable-helm.md Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> * Update docs/sources/installation/simple-scalable-helm.md Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> * Update docs/sources/installation/simple-scalable-helm.md Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> * Update docs/sources/installation/simple-scalable-helm.md Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com>	3 years ago
Trevor Whitney	c6f7a41225	Bring helm chart back into Loki repo (#6829) ... **What this PR does / why we need it**: This is the first step of the process to bring the helm chart back into the Loki repo. We currently have ~7 public helm charts. This creates a lot of confusion for the community on which ones to start with, and is also a lot of additional surface area that is hard for Loki contributors to keep track of. By bringing the helm chart that we use ourselves back into the Loki repo, it will be a more clear starting point for helm users. It will also increase visibility for Loki contributors into helm PRs and issues. This will _not_ affect where the helm chart is published. It will still be published to the `grafana/helm-charts` following the same pattern as the Mimir helm chart. The helm chart that we are currently using and most actively supporting is the `loki-simple-scalable` chart. Part of this effort is to rename that chart to just `loki`, which means it needs to be expanded to support the single binary deployment topology currently deployed by the existing `loki` helm chart. This PR is the first attempt to merge those two charts. This chart currently works for a basic install of each deployment topology. There is more work to be done before we publish it. This work will include: * Making sure there is a migration path from the existing `loki` chart to this new one * Making sure there is a migration path from the existing `loki-simple-scalable` chart to this new one * Making sure the single binary deployment works in both `filesystem` and minio-backed multi-replica mode * Migrating CI from `grafana/helm-charts` for testing helm chart changes * Integrating with the github action in the `grafana/helm-charts` repo (this will publish it, so we will tackle this last) A question I have for reviewers is what we should name each deployment mode (single binary vs. simple scalable)? We have received feedback from the community that some did not think "simple scalable" was suitable for production loads since it had "simple" in the name. This might be an opportunity to adjust that perception. Some options I've thought of include: * `single-binary` and `simple-scalable` (currently how it's coded) * `basic` and `scalable` (leaving open the option for `advanced` if we ever choose to support micro-services topology in this chart as well) * `ha`, `redundant` or similar vs. not Fixes #6873	3 years ago
Jonathan	1b6d0bf3a0	feat: add kms ecryption (#6926)	3 years ago
Danny Kopping	87e20d9399	Setting sane defaults for ruler in SSD "read" mode (#6551) ... Also removing support for configdb, since this is not supported but was the default storage type Signed-off-by: Danny Kopping <danny.kopping@grafana.com> Signed-off-by: Danny Kopping <danny.kopping@grafana.com>	3 years ago
Owen Diehl	c340f3e202	[TSDB] disables bloom filters for tsdb index on ingesters (#6939) ... * disables bloom filters for tsdb index on ingesters Signed-off-by: Owen Diehl <ow.diehl@gmail.com> * gofmt -s Signed-off-by: Owen Diehl <ow.diehl@gmail.com> * faillint exception for proto sync/atomic Signed-off-by: Owen Diehl <ow.diehl@gmail.com>	3 years ago
Ed Welch	ded2614823	Remove comparisons page, it's not very acurate and the information is outdated. (#6938) ... Signed-off-by: Edward Welch <edward.welch@grafana.com> Signed-off-by: Edward Welch <edward.welch@grafana.com>	3 years ago
Salva Corts	83081653d1	Add operational guide for autoscaling (#6801) ... * Add operational guide for autoscaling * Add promql tags to code snippets * Apply feedback from Karen * Update docs/sources/operations/autoscaling_queriers.md Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> * Update docs/sources/operations/autoscaling_queriers.md Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> * Update docs/sources/operations/autoscaling_queriers.md Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> * Update docs/sources/operations/autoscaling_queriers.md Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> * Update docs/sources/operations/autoscaling_queriers.md Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> * Update docs/sources/operations/autoscaling_queriers.md Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> * Update docs/sources/operations/autoscaling_queriers.md Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> * CR feedback * Add note about needing to use the query-scheduler. * Update docs/sources/operations/autoscaling_queriers.md Co-authored-by: Christopher Moyer <35463610+chri2547@users.noreply.github.com> * Update docs/sources/operations/autoscaling_queriers.md Co-authored-by: Christopher Moyer <35463610+chri2547@users.noreply.github.com> * Update docs/sources/operations/autoscaling_queriers.md Co-authored-by: Christopher Moyer <35463610+chri2547@users.noreply.github.com> * Update docs/sources/operations/autoscaling_queriers.md Co-authored-by: Christopher Moyer <35463610+chri2547@users.noreply.github.com> * Update docs/sources/operations/autoscaling_queriers.md Co-authored-by: Christopher Moyer <35463610+chri2547@users.noreply.github.com> * Update docs/sources/operations/autoscaling_queriers.md Co-authored-by: Christopher Moyer <35463610+chri2547@users.noreply.github.com> * Update docs/sources/operations/autoscaling_queriers.md Co-authored-by: Christopher Moyer <35463610+chri2547@users.noreply.github.com> * Update docs/sources/operations/autoscaling_queriers.md Co-authored-by: Christopher Moyer <35463610+chri2547@users.noreply.github.com> * Update docs/sources/operations/autoscaling_queriers.md Co-authored-by: Christopher Moyer <35463610+chri2547@users.noreply.github.com> * Update docs/sources/operations/autoscaling_queriers.md Co-authored-by: Christopher Moyer <35463610+chri2547@users.noreply.github.com> * Update docs/sources/operations/autoscaling_queriers.md Co-authored-by: Christopher Moyer <35463610+chri2547@users.noreply.github.com> * Update docs/sources/operations/autoscaling_queriers.md Co-authored-by: Christopher Moyer <35463610+chri2547@users.noreply.github.com> * Update docs/sources/operations/autoscaling_queriers.md Co-authored-by: Christopher Moyer <35463610+chri2547@users.noreply.github.com> * Update docs/sources/operations/autoscaling_queriers.md Co-authored-by: Christopher Moyer <35463610+chri2547@users.noreply.github.com> * CR changes * Apply suggestions from code review Co-authored-by: Christopher Moyer <35463610+chri2547@users.noreply.github.com> * Update docs/sources/operations/autoscaling_queriers.md Co-authored-by: Christopher Moyer <35463610+chri2547@users.noreply.github.com> * CR feedback Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> Co-authored-by: Christopher Moyer <35463610+chri2547@users.noreply.github.com>	3 years ago
André Tadeu de Carvalho	9694776537	docs:Add documentation for TLS configuration parameters for HTTP and gRPC (#6933) ... * Add documentation for TLS configuration parameters for HTTP and gRPC. * Removed possible values from client-auth parameters.	3 years ago
Susana Ferreira	3f3f4eda1d	Fix topk and bottomk operations with int <= 0 (#6937) ... * Fix topk and bottomk operations with int <= 0 * Fix VectorAggregationExpr String method * Update rangemapper_test expression * Add tests * Update CHANGELOG.md	3 years ago
Bayan Taani	2e21896aa5	Change Loki Operator namespace to openshift-logging in the CSV (#6935)	3 years ago
Dylan Guedes	2d7ef07494	Fix `increment_duplicate_timestamps` for multiple dupped timestamps (#6924)	3 years ago
Owen Diehl	72d8627a55	log query which causes panic in logql.clone (#6934) ... Signed-off-by: Owen Diehl <ow.diehl@gmail.com>	3 years ago
Michel Hollands	25e8365d05	Improve error message (#6913) ... Signed-off-by: Michel Hollands <michel.hollands@grafana.com> Signed-off-by: Michel Hollands <michel.hollands@grafana.com>	3 years ago
Robert Jacob	2911a74175	operator: Reconcile owner reference for existing objects (#6923)	3 years ago
Travis Patterson	965650740c	Disable distributed cache for the default loki local config (#6925)	3 years ago
Susana Ferreira	87d51bd8e7	Update documentation deletion_mode and allow_deleted configuration values (#6921)	3 years ago
Owen Diehl	f470943433	reintroduces now deprecated+noop deletion configs for compatibility (#6917) ... * reintroduces now deprecated+noop deletion configs for compatibility Signed-off-by: Owen Diehl <ow.diehl@gmail.com> * better deprecation logging Signed-off-by: Owen Diehl <ow.diehl@gmail.com> * fixes import Signed-off-by: Owen Diehl <ow.diehl@gmail.com>	3 years ago
Christian Simon	953c36e3ed	Remove unnecessary spin up of mock (#6911) ... This removes the unnecessary spinning up of a mock for TestDoubleRegistration. The client itself is not used at all and that causes it to be stopped before it is actually started. Example flake: https://drone.grafana.net/grafana/loki/14724/2/5	3 years ago
Callum Styan	5919a29800	Previous PR #6888 broke some additional jsonnet, use {} instead of null (#6918) ... Signed-off-by: Callum Styan <callumstyan@gmail.com> Signed-off-by: Callum Styan <callumstyan@gmail.com>	3 years ago
irizzant	e9f5de48bc	fix(consul): allow consul deployment (#6888) ... Signed-off-by: irizzant <i.rizzante@gmail.com> Signed-off-by: irizzant <i.rizzante@gmail.com>	3 years ago
Owen Diehl	ba5d218e3e	detects issuer via useragent in querytee reqs (#6914) ... Signed-off-by: Owen Diehl <ow.diehl@gmail.com>	3 years ago
Gerard Vanloo	3d2aa76349	Adding valid subscription annotation to metadata (#6907)	3 years ago
Christian Simon	ee2fe6514d	Propagate error detail properly in all cases (#6890) ... The error handling was missing appending the error detail properly in all cases. Fixes #6265	3 years ago
Christian Simon	88f0ad2196	Update build-image to 0.23.0 (#6899)	3 years ago
Javad Hajiani	543b462364	Add missing environment variable names on logcli flags description (#6865)	3 years ago
Karen Miller	a02ce69a98	Docs: add minimal docs explaining how to use the LogQL Analyzer (#6900)	3 years ago
Christian Simon	88e2705034	Update alpine and golang base images (#6897) ... * Upgrade base images to alpine 3.15.6 everywhere https://alpinelinux.org/posts/Alpine-3.13.12-3.14.8-3.15.6-3.16.2-released.html * Update golang images to 1.18.5	3 years ago
Christian Simon	d63add09c5	Fix free port allocation flake (#6898) ... Occassionaly we can see test flakes like that: > server: listen tcp :34473: bind: address already in use This is because we allocated ports by checking if they are free and then using them. Altough there is a chance that already allocated ports are used twice. This changes prevents this from happening by keeping track of the ports that are about to be used.	3 years ago
Kaviraj Kanagaraj	693f9d7070	fix(stats_collector): Do not collect stats for failed requests. (#6878) ... * fix(stats_collector): Do not collect stats for failed requests. When any type of query fails, do not collect statistics. Rationale being when core http handler returns error, it's reponse will be nil and there won't be any statistics to collect and log. Fixes: 6231 Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * fix comment Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com>	3 years ago
carlospeon	171a09b822	Add support for journal matches (only conjuntions) (#6656)	3 years ago
Vladyslav Diachenko	b494713106	fixed typo in API host (#6892)	3 years ago
Vladyslav Diachenko	bec014dd86	Logql analyzer page cherry picks (#6889) ... * added Logql Analyzer page into loki docs * changed template to support changes in API and supported the case when query is run against empty line * added more json examples, changed some labels and changes css for line numbers * fixed last comments from Karen and pointed to production API	3 years ago
Sandeep Sukhani	3c5fdb749a	add metric for measuring overall index query latency (#6880)	3 years ago
Owen Diehl	3cba86faf2	[bugfix] fixes issues sharding series by storageref vs fingerprint in stripeSeries (#6882) ... Signed-off-by: Owen Diehl <ow.diehl@gmail.com>	3 years ago
Travis Patterson	3170edf3e3	Add timeouts to deletions (#6879) ... * Add context to chunk iteration * Configure timeouts * add expiration timeout * update test * update test * Clarify timeout implications, log more specifically	3 years ago
Travis Patterson	afedabf0e9	Remove unused code (#6872)	3 years ago
Karen Miller	8ac08e65d9	Docs: improve wording and grammar (#6861)	3 years ago
Travis Patterson	d602c1331a	Clarify delete upgrade path (#6871) ... * Clarify delete upgrade path * more specificity	3 years ago
Travis Patterson	983ab80e7c	Add delete api validations (#6860) ... * break out the middleware * Add validation to the API - Time must be in RFC3339 or a 10-digit-unix-seconds timestamp - Start time must always exist * lint * docs * improve tests * cleanup * cleanup * access runtime config via function on validation	3 years ago
Owen Diehl	b114dc93d6	update deletion docs (#6869) ... * update deletion docs Signed-off-by: Owen Diehl <ow.diehl@gmail.com> * Update docs/sources/operations/storage/logs-deletion.md	3 years ago
Michel Hollands	7c680e4ce0	update dskit and fix (#6868) ... * update dskit and fix Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Use a separate registry to avoid duplicate registration Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Fix runtime config initialization Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * run go mod tidy Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Update vendor/modules.txt Signed-off-by: Michel Hollands <michel.hollands@grafana.com> Signed-off-by: Michel Hollands <michel.hollands@grafana.com>	3 years ago
Travis Patterson	bd3d80ad80	Add ability to limit the impact of pathological delete requests. (#6854) ... * make per-tenant override the mode rather than binary on/off * Rely on overrides to configure deletion - Remove 'deletion_mode' flag * add ability to delete requests at any time * don't load requests from disabled tenants * review comments * fix tests * loadDeleteRequestsToProcess cleanup * lint * docs * review feedback * lint	3 years ago
Trevor Whitney	f4478490db	Existing limits were low for large envs (#6846) ... * Existing limits were low for large envs * fix distributor libsonnet syntax	3 years ago