849 Commits (ee0f46beed12f482032649966ece7fd1df7abdb5)

Author	SHA1	Message	Date
Jeremy Wu	161fcac646	support custom label selector in kubernetes_sd_configs (#5920) ... make promtail scrape logs of pod with specific field or label selector, so that it will watch less logs and make fewer calculations and get a better performance in some scenarios. Signed-off-by: Zhimin Wu <hoozecn@gmail.com> Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com>	3 years ago
Salva Corts	df4461d712	Evenly spread memcached across nodes (#7183) ... <!-- Thanks for sending a pull request! Before submitting: 1. Read our CONTRIBUTING.md guide 2. Name your PR as `<Feature Area>: Describe your change`. a. Do not end the title with punctuation. It will be added in the changelog. b. Start with an imperative verb. Example: Fix the latency between System A and System B. c. Use sentence case, not title case. d. Use a complete phrase or sentence. The PR title will appear in a changelog, so help other people understand what your change will be. 3. Rebase your PR if it gets out of sync with main --> **What this PR does / why we need it**: This PR enables the [TopologySpreadConstraints](https://kubernetes.io/docs/concepts/scheduling-eviction/topology-spread-constraints/) for Memcached chunks cache introduced at https://github.com/grafana/jsonnet-libs/pull/832. Now the chunks cache will be evenly spread across nodes but more than one cache can run on the same node. **Which issue(s) this PR fixes**: Fixes #<issue number> **Special notes for your reviewer**: <!-- Note about CHANGELOG entries, if a change adds: * an important feature * fixes an issue present in a previous release, * causes a change in operation that would be useful for an operator of Loki to know then please add a CHANGELOG entry. For documentation changes, build changes, simple fixes etc please skip this step. We are attempting to curate a changelog of the most relevant and important changes to be easier to ingest by end users of Loki. Note about the upgrade guide, if this changes: * default configuration values * metric names or label names * changes existing log lines such as the metrics.go query output line * configuration parameters * anything to do with any API * any other change that would require special attention or extra steps to upgrade Please document clearly what changed AND what needs to be done in the upgrade guide. --> **Checklist** - [ ] Documentation added - [ ] Tests updated - [ ] Is this an important fix or new feature? Add an entry in the `CHANGELOG.md`. - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	3 years ago
Brian Evans	e7acb5356b	Docs: Fix template function examples (#7148) ... Some examples only had a single open brace, but require two. I added the missing braces. **What this PR does / why we need it**: Improves documentation, helps users who are reading docs about template functions.	3 years ago
Travis Patterson	ef47e0ee8a	Fix errors loading deletes after force-removing partially-completed delete (#7125) ... When sharded deletes that are partially completed are force-removed, everything is removed except the shards that are currently being deleted. The shards being deleted aren't necessarily in numerical order so there's no way to know assume which shards have and haven't been deleted. The Delete store assumed it could look up requests in sequential order and returned empty requests where there were gaps in the stored data. This change takes the `sequence number` from the stored key of the delete requests it finds and uses that to look up details.	3 years ago
Michel Hollands	ebdb64b3f5	Small updates to the deletion documentation (#7112) ... **What this PR does / why we need it**: - Fix a typo - Fix the links in for the deletion API **Checklist** - [X] Documentation added - [ ] Tests updated - [ ] Is this an important fix or new feature? Add an entry in the `CHANGELOG.md`. - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` Signed-off-by: Michel Hollands <michel.hollands@grafana.com>	3 years ago
Kaviraj Kanagaraj	2c9fa05c29	feat(loki-canary): Add support to push logs directly to Loki. (#7063) ... Add additional `push` mode to loki-canary, which pushes the logs directly to given Loki URL as it generates logs. The real function of Loki Canary is to act like a tenant and help us know the whether Loki installation is working as perceived by a real tenant. Main rationale for this additional push mode is to make canary more standalone without needing for `promtail` (or `grafana-agent`) to scrape it's logs and send to loki, with this change, Loki canary happily tests Loki behavior without needing any other dependencies. **NOTES**: 1. If you run Loki behind any proxy that has different authorization policies to READ or WRITE to Loki, then important change that canary operator need to be aware of it, now the user credentials they pass via `-user` and `-pass` to access loki endpoints need to have both `read' and `write` permissions (previously canary just query the logs where as promtail is the one pushes the logs, so just READ permissions was sufficient). 2. There will be follow up PR(s) to cleanup, particularly `reader` and `comparitor` component in terms of logging with proper logger. Rationale is to keep the changes small per PR to make it easy to review. 3. This changes were tested it in one of the internal Loki dev cell. 4. **This PR is a no-op if this new `push` flag is disabled (it's disabled by default)** Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> Co-authored-by: Danny Kopping <danny.kopping@grafana.com>	3 years ago
Mohamed-Amine Bouqsimi	3d4788fb51	Add ability for Ruler to remote write to multiple clients (#6823)	3 years ago
Old Schepperhand	96623440c9	docs: fix alertmanager_client config settings (#6858) ... when setting ``` basic_auth: username: xxx password: yyy ``` following error gets thrown: ``` failed parsing config: /etc/loki/loki-config.yaml: yaml: unmarshal errors: line 123: field basic_auth not found in type base.NotifierConfig ``` <!-- Thanks for sending a pull request! Before submitting: 1. Read our CONTRIBUTING.md guide 2. Name your PR as `<Feature Area>: Describe your change`. a. Do not end the title with punctuation. It will be added in the changelog. b. Start with an imperative verb. Example: Fix the latency between System A and System B. c. Use sentence case, not title case. d. Use a complete phrase or sentence. The PR title will appear in a changelog, so help other people understand what your change will be. 3. Rebase your PR if it gets out of sync with main --> **What this PR does / why we need it**: **Which issue(s) this PR fixes**: Fixes #<issue number> **Special notes for your reviewer**: <!-- Note about CHANGELOG entries, if a change adds: * an important feature * fixes an issue present in a previous release, * causes a change in operation that would be useful for an operator of Loki to know then please add a CHANGELOG entry. For documentation changes, build changes, simple fixes etc please skip this step. We are attempting to curate a changelog of the most relevant and important changes to be easier to ingest by end users of Loki. Note about the upgrade guide, if this changes: * default configuration values * metric names or label names * changes existing log lines such as the metrics.go query output line * configuration parameters * anything to do with any API * any other change that would require special attention or extra steps to upgrade Please document clearly what changed AND what needs to be done in the upgrade guide. --> **Checklist** - [ ] Documentation added - [ ] Tests updated - [ ] Is this an important fix or new feature? Add an entry in the `CHANGELOG.md`. - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	3 years ago
Pablo	76c1198d58	Improvements to Promtail GCP push target (#7057) ... <!-- Thanks for sending a pull request! Before submitting: 1. Read our CONTRIBUTING.md guide 2. Name your PR as `<Feature Area>: Describe your change`. a. Do not end the title with punctuation. It will be added in the changelog. b. Start with an imperative verb. Example: Fix the latency between System A and System B. c. Use sentence case, not title case. d. Use a complete phrase or sentence. The PR title will appear in a changelog, so help other people understand what your change will be. 3. Rebase your PR if it gets out of sync with main --> **What this PR does / why we need it**: This PR includes some pending improvements to newish promtail GCP push target: - Pending low hanging optimization for replacing a regex that gets evaluated on every incoming request from GCP, for a simpler solution. [Related comments](https://github.com/grafana/loki/pull/6777#discussion_r935804383) - Some small refactors. See: [this](https://github.com/grafana/loki/pull/6777#pullrequestreview-1059064564) - Add some validations over the payload the target receives. Payload details are [here](https://cloud.google.com/pubsub/docs/push). This PR adds just validation over the log line, the incoming GCP message ID to add some traceability if needed, and the subscription name because it can be used as label. - Add `subscription` GCP payload field as label, which includes the PubSub subscription identifier. - Add `cause` label to error metrics. Since there's multiple failure point this might help debugging / notice strange behaviours. Also, by running the push target in some dev workloads, we noticed it has some labels different to the ones added in the pull target. This PR also refactors the logic to re-use the same GCP log entry parsing logic in both targets. That way the labels exposed to the user in the push target are $Labels_{pull} \cup Labels_{push}$. **Which issue(s) this PR fixes**: Related to: https://github.com/grafana/cloud-onboarding/issues/2067 **Special notes for your reviewer**: This PR is a follow up [this other one](https://github.com/grafana/loki/pull/6777). It works on some comments made after merge, and adds some changes we saw necessary upon running this target in real environments. <!-- Note about CHANGELOG entries, if a change adds: * an important feature * fixes an issue present in a previous release, * causes a change in operation that would be useful for an operator of Loki to know then please add a CHANGELOG entry. For documentation changes, build changes, simple fixes etc please skip this step. We are attempting to curate a changelog of the most relevant and important changes to be easier to ingest by end users of Loki. Note about the upgrade guide, if this changes: * default configuration values * metric names or label names * changes existing log lines such as the metrics.go query output line * configuration parameters * anything to do with any API * any other change that would require special attention or extra steps to upgrade Please document clearly what changed AND what needs to be done in the upgrade guide. --> **Checklist** - [x] Documentation added - [x] Tests updated - [ ] Is this an important fix or new feature? Add an entry in the `CHANGELOG.md`. - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` Co-authored-by: Kaviraj Kanagaraj <kavirajkanagaraj@gmail.com>	3 years ago
Kaviraj Kanagaraj	8162f0d796	doc: Clarity on querier remapping when shuffle sharding is disabled (#7026)	3 years ago
Michel Hollands	c271f7923a	Fix setting name in documentation (#7086) ... Signed-off-by: Michel Hollands <michel.hollands@grafana.com> **What this PR does / why we need it**: The delete_max_interval was previously named delete_max_query_range. This rename was not done in the documentation, **Checklist** - [X] Documentation added - [ ] Tests updated - [ ] Is this an important fix or new feature? Add an entry in the `CHANGELOG.md`. - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` Signed-off-by: Michel Hollands <michel.hollands@grafana.com>	3 years ago
Ludovic Cleroux	e2952f9ce1	Troubleshooting: instructions for loki + istio (#6205) ... <!-- Thanks for sending a pull request! Before submitting: 1. Read our CONTRIBUTING.md guide 2. Name your PR as `<Feature Area>: Describe your change`. a. Do not end the title with punctuation. It will be added in the changelog. b. Start with an imperative verb. Example: Fix the latency between System A and System B. c. Use sentence case, not title case. d. Use a complete phrase or sentence. The PR title will appear in a changelog, so help other people understand what your change will be. 3. Rebase your PR if it gets out of sync with main --> **What this PR does / why we need it**: Add troubleshooting instructions for running istio and loki. It takes a long time to figure out how to make it work, hope it will help others. **Which issue(s) this PR fixes**: Fixes #<issue number> **Special notes for your reviewer**: <!-- Note about CHANGELOG entries, if a change adds: * an important feature * fixes an issue present in a previous release, * causes a change in operation that would be useful for an operator of Loki to know then please add a CHANGELOG entry. For documentation changes, build changes, simple fixes etc please skip this step. We are attempting to curate a changelog of the most relevant and important changes to be easier to ingest by end users of Loki. Note about the upgrade guide, if this changes: * default configuration values * metric names or label names * changes existing log lines such as the metrics.go query output line * configuration parameters * anything to do with any API * any other change that would require special attention or extra steps to upgrade Please document clearly what changed AND what needs to be done in the upgrade guide. --> **Checklist** - [x] Documentation added - [ ] Tests updated - [ ] Is this an important fix or new feature? Add an entry in the `CHANGELOG.md`. - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com>	3 years ago
李国忠	93a5a71e62	[doc] logql: logql engine support exec vector(0) grama (#7044) ... <!-- Thanks for sending a pull request! Before submitting: 1. Read our CONTRIBUTING.md guide 2. Name your PR as `<Feature Area>: Describe your change`. a. Do not end the title with punctuation. It will be added in the changelog. b. Start with an imperative verb. Example: Fix the latency between System A and System B. c. Use sentence case, not title case. d. Use a complete phrase or sentence. The PR title will appear in a changelog, so help other people understand what your change will be. 3. Rebase your PR if it gets out of sync with main --> **What this PR does / why we need it**: logql engine support exec vector(0) grama. new PR of :https://github.com/grafana/loki/pull/7023 **Which issue(s) this PR fixes**: Fixes #6946 **Special notes for your reviewer**: preview  <!-- Note about CHANGELOG entries, if a change adds: * an important feature * fixes an issue present in a previous release, * causes a change in operation that would be useful for an operator of Loki to know then please add a CHANGELOG entry. For documentation changes, build changes, simple fixes etc please skip this step. We are attempting to curate a changelog of the most relevant and important changes to be easier to ingest by end users of Loki. Note about the upgrade guide, if this changes: * default configuration values * metric names or label names * changes existing log lines such as the metrics.go query output line * configuration parameters * anything to do with any API * any other change that would require special attention or extra steps to upgrade Please document clearly what changed AND what needs to be done in the upgrade guide. --> **Checklist** - [ ] Documentation added - [ ] Tests updated - [ ] Is this an important fix or new feature? Add an entry in the `CHANGELOG.md`. - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` Co-authored-by: Danny Kopping <dannykopping@gmail.com>	3 years ago
sardismart	821301b95f	azblob: Make azblob support private cloud endpoint suffix (#6842) ... <!-- Thanks for sending a pull request! Before submitting: 1. Read our CONTRIBUTING.md guide 2. Name your PR as `<Feature Area>: Describe your change`. a. Do not end the title with punctuation. It will be added in the changelog. b. Start with an imperative verb. Example: Fix the latency between System A and System B. c. Use sentence case, not title case. d. Use a complete phrase or sentence. The PR title will appear in a changelog, so help other people understand what your change will be. 3. Rebase your PR if it gets out of sync with main --> **What this PR does / why we need it**: The endpoint suffix of private cloud(azure stack hub) azblob is different from Azure. The loki configuration has already support customize endpoint suffix but not use when select blob/container URL. This PR introduce following change: * Consider endpoint suffix from configuration when select blob or container URL format **Which issue(s) this PR fixes**: Fixes #6863 **Special notes for your reviewer**: <!-- Note about CHANGELOG entries, if a change adds: * an important feature * fixes an issue present in a previous release, * causes a change in operation that would be useful for an operator of Loki to know then please add a CHANGELOG entry. For documentation changes, build changes, simple fixes etc please skip this step. We are attempting to curate a changelog of the most relevant and important changes to be easier to ingest by end users of Loki. Note about the upgrade guide, if this changes: * default configuration values * metric names or label names * changes existing log lines such as the metrics.go query output line * configuration parameters * anything to do with any API * any other change that would require special attention or extra steps to upgrade Please document clearly what changed AND what needs to be done in the upgrade guide. --> **Checklist** - [x] Documentation added - [x] Tests updated - [ ] Is this an important fix or new feature? Add an entry in the `CHANGELOG.md`. - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	3 years ago
Kaviraj Kanagaraj	5a6d09f83e	doc(community): Update community call timings. (#7002)	3 years ago
Callum Styan	da6fd01444	[docs] document logfmt pipeline stage (#6238) ... I noticed earlier while writing a design doc that the `logfmt` pipeline stage is not documented. It was originally added here: https://github.com/grafana/loki/pull/4346 I've also fixed a minor typo in the json pipeline stage docs. Signed-off-by: Callum Styan <callumstyan@gmail.com> Signed-off-by: Edward Welch <edward.welch@grafana.com> Co-authored-by: Ed Welch <ed@oqqer.com> Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> Co-authored-by: Edward Welch <edward.welch@grafana.com>	3 years ago
Dylan Guedes	c6dc0d2649	Docs: Fix default max_chunk_age (#6996) ... **What this PR does / why we need it**: Fix a doc stating a wrong `max_chunk_age` value. **Which issue(s) this PR fixes**: Fixes #6919	3 years ago
Ed Welch	bf5a25dd5b	Docs: update `split_queries_by_interval` documentation to have the correct explanation for how to disable. (#6715) ... <!-- Thanks for sending a pull request! Before submitting: 1. Read our CONTRIBUTING.md guide 2. Name your PR as `<Feature Area>: Describe your change`. a. Do not end the title with punctuation. It will be added in the changelog. b. Start with an imperative verb. Example: Fix the latency between System A and System B. c. Use sentence case, not title case. d. Use a complete phrase or sentence. The PR title will appear in a changelog, so help other people understand what your change will be. 3. Rebase your PR if it gets out of sync with main --> **What this PR does / why we need it**: I noticed when responding to #5123 the docs did not correctly explain how to disable splitting queries by time. I searched through the code and confirmed `0` is the correct value to disable this feature **Which issue(s) this PR fixes**: Fixes #<issue number> **Special notes for your reviewer**: <!-- Note about CHANGELOG entries, if a change adds: * an important feature * fixes an issue present in a previous release, * causes a change in operation that would be useful for an operator of Loki to know then please add a CHANGELOG entry. For documentation changes, build changes, simple fixes etc please skip this step. We are attempting to curate a changelog of the most relevant and important changes to be easier to ingest by end users of Loki. Note about the upgrade guide, if this changes: * default configuration values * metric names or label names * changes existing log lines such as the metrics.go query output line * configuration parameters * anything to do with any API * any other change that would require special attention or extra steps to upgrade Please document clearly what changed AND what needs to be done in the upgrade guide. --> **Checklist** - [ ] Documentation added - [ ] Tests updated - [ ] Is this an important fix or new feature? Add an entry in the `CHANGELOG.md`. - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` Signed-off-by: Edward Welch <edward.welch@grafana.com> Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com>	3 years ago
Ed Welch	6d54505485	Loki: add `__line__` and `__timestamp__` to label_format (#6983) ... <!-- Thanks for sending a pull request! Before submitting: 1. Read our CONTRIBUTING.md guide 2. Name your PR as `<Feature Area>: Describe your change`. a. Do not end the title with punctuation. It will be added in the changelog. b. Start with an imperative verb. Example: Fix the latency between System A and System B. c. Use sentence case, not title case. d. Use a complete phrase or sentence. The PR title will appear in a changelog, so help other people understand what your change will be. 3. Rebase your PR if it gets out of sync with main --> **What this PR does / why we need it**: We have the `__line__` and `__timestamp__` functions currently available in the `line_format` stage but not in the `label_format` stage. This PR fixes that. Specifically `__timestamp__` can be valuable in a label because you can do things like ``` label_format hour_of_day=`{{__timestamp__ | date "15"}}` ``` To be able to conditionaly select log lines based on the hour of the day they were created, this can be useful for conditional alerting as an example. **Which issue(s) this PR fixes**: Fixes #6402 **Special notes for your reviewer**: <!-- Note about CHANGELOG entries, if a change adds: * an important feature * fixes an issue present in a previous release, * causes a change in operation that would be useful for an operator of Loki to know then please add a CHANGELOG entry. For documentation changes, build changes, simple fixes etc please skip this step. We are attempting to curate a changelog of the most relevant and important changes to be easier to ingest by end users of Loki. Note about the upgrade guide, if this changes: * default configuration values * metric names or label names * changes existing log lines such as the metrics.go query output line * configuration parameters * anything to do with any API * any other change that would require special attention or extra steps to upgrade Please document clearly what changed AND what needs to be done in the upgrade guide. --> **Checklist** - [x] Documentation added - [ ] Tests updated - [x] Is this an important fix or new feature? Add an entry in the `CHANGELOG.md`. - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` Signed-off-by: Edward Welch <edward.welch@grafana.com>	3 years ago
Juha Tiensyrjä	073c62051b	lambda-promtail: Add support for Kinesis data stream events (#5977) ... **What this PR does / why we need it**: This PR adds support for sending Kinesis data stream events to lambda-promtail. One use case would be e.g. to send CloudFront realtime logs to Loki. **Which issue(s) this PR fixes**: Fixes #5978 **Special notes for your reviewer**: n/a **Checklist** - [x] Documentation added - [x] Tests updated - [x] Is this an important fix or new feature? Add an entry in the `CHANGELOG.md`. - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	3 years ago
Kim Nylander	a12b0803f0	[Doc] Fixed comma and missing "of" in observability.md (#6976) ... **What this PR does / why we need it**: Correcting two mistakes that @osg-grafana found in PR#6971. **Checklist** - [X] Documentation added - [ ] Tests updated - [ ] Is this an important fix or new feature? Add an entry in the `CHANGELOG.md`. - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	3 years ago
afayngelerindbx	4b5b16569d	compactor: adds tunable to control upload parallelism (#6817) ... **What this PR does / why we need it**: This can help with compaction latency. I patched the compactor to set this to 100. **Which issue(s) this PR fixes**: Helps with #6775 **Checklist** - [x] Documentation added	3 years ago
Ursula Kallio	8da290cd2d	Use backticks for code syntax. (#6967) ... * Use backticks for code syntax. * Fix for real. :)	3 years ago
Michel Hollands	3839522eb7	Add deletion metrics to the documentation (#6971) ... * Add deletion metrics Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Fix metric types for some entries Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Add missing dot at end of line Signed-off-by: Michel Hollands <michel.hollands@grafana.com> Signed-off-by: Michel Hollands <michel.hollands@grafana.com>	3 years ago
afayngelerindbx	1e1c2470b5	compactor: prioritize newer tables when compacting. add operator flags to compac… (#6804) ... * prioritize newer tables when compacting. add operator flags to compact time ranges of tables Co-authored-by: Sandeep Sukhani <sandeep.d.sukhani@gmail.com> * reroll doc and PR changes after rebase Co-authored-by: Sandeep Sukhani <sandeep.d.sukhani@gmail.com>	3 years ago
Dylan Guedes	5d8c48bd10	Loki: Implement stream sharding (#6952) ... * introduce 'StreamSharder' interface to decouple sharding from managing shards * make the streamsharder take a whole stream instead of just and ID * Implement stream sharding usage. * move streamShard to distributor * Implement the stream sharder and shard iter (#6941) * Modify sharding to not duplicate all entries for all shards (#6942) * Modify sharding to not duplicate all entries for all shards. * Rename ShardIter and get rid of NextShardId method. * Get rid of ShardIter and ShardStats. * Implement a StreamSharder mock and use it in tests. * Rewrite stubbing and remove flags. * Add auto-splitting. * Only shard stream when approppriate. * Make sharding mode an enum. * Bench shardStream function. * Modify shard comparison to be <= 1. * Add a guard clause for when lowerbound>upperbound. * Apply suggestions from code review Avoid using `fmt.Sprintf`. Co-authored-by: Danny Kopping <danny.kopping@grafana.com> * Update pkg/distributor/distributor.go Rename metric name. Co-authored-by: Danny Kopping <danny.kopping@grafana.com> * Avoid double-negative by Danny's suggestion. * Move TODO to its right place. * Log scenario where upperbound < lowerbound. * Reuse compiled regex. * Add docstring explaining why we use max(shards*2, 2). * make sharder use a RWMutex * simplify sharding config * replace doc string * Revert "Add auto-splitting" for standalone PR. This reverts commit e3886a6210. * shard loop refactor * move shard creation into it's own function * fix after refactor * make config more explicit and document it * reduce allocations * review feedback Co-authored-by: Travis Patterson <travis.patterson@grafana.com> Co-authored-by: Danny Kopping <danny.kopping@grafana.com>	3 years ago
Dylan Guedes	15f8f42295	Adapt queryTimeout to be a per-tenant configuration (#6835) ... What this PR does / why we need it: * Add a new per-tenant query timeout * Adds new middleware to query calls. This new middleware will timeout requests based on the new per-tenant query timeout * Add span logs to query calls missing it * Deprecate `engine.timeout` configuration The motivation to change this configuration to be per-tenant instead of global is that it makes sense to alleviate the timeout for particular tenants or make it more strict for others. Especially useful for scenarios where the Loki client doesn't handle context cancellations correctly; in such scenarios, since Loki would still process expensive queries that were canceled, this allows one to have small timeouts for most tenants, which will help mitigate unnecessary work without having a timeout that is too short for important tenants.	3 years ago
Travis Patterson	e15da437f0	Shard deletes requests based on time ranges (#6903) ... * Refactor the store API to be in terms of DeleteRequest * backfill remove tests * backfill GetAll tests * Make the delete request store deal in groups of requests instead of single ones * Make cancelation aware of request groups * remove cancelation deadline * shard deletes based on a query parameter * merge request group on gets * global configuration for max query range * sort delete requests by start time so batches are grouped to similar chunks * Fix status when some deletes haven't happened yet * fix tests * Review feedback * review feedback * make max_interval > deletion interval a validation error * list partially completed deletes * list partially completed deletes * fix flaky test * clarify naming * disallow cancelation of partial deletes unless the 'force' flag is passed * more comment specificity for the delete request hash	3 years ago
Salva Corts	7961ef6a4e	Evenly spread distributors across available nodes (#6964) ... * Evenly spread distributors across available nodes * Add note about reliability implications	3 years ago
Alex Close	288e644894	docs: Fixed examples within Loki and Promtail docs (#6591) ... * Updated the promtail install example to both fix the Daemonset yaml file and provide a more concrete example for deploying to kubernetes * Removed the sidecar documentation for promtail install as it was both broken and didn't do what it said it did * Updated the simple scalable chart docs to include an example of using a overrides values file and to point to a working example of deploying promtail in a k8s cluster * Updated promtail and loki SSD install docs based on PR feedback * Update docs/sources/clients/promtail/installation.md Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> * Update docs/sources/installation/simple-scalable-helm.md Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> * Update docs/sources/installation/simple-scalable-helm.md Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> * Update docs/sources/installation/simple-scalable-helm.md Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> * Update docs/sources/installation/simple-scalable-helm.md Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com>	3 years ago
Ed Welch	ded2614823	Remove comparisons page, it's not very acurate and the information is outdated. (#6938) ... Signed-off-by: Edward Welch <edward.welch@grafana.com> Signed-off-by: Edward Welch <edward.welch@grafana.com>	3 years ago
Salva Corts	83081653d1	Add operational guide for autoscaling (#6801) ... * Add operational guide for autoscaling * Add promql tags to code snippets * Apply feedback from Karen * Update docs/sources/operations/autoscaling_queriers.md Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> * Update docs/sources/operations/autoscaling_queriers.md Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> * Update docs/sources/operations/autoscaling_queriers.md Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> * Update docs/sources/operations/autoscaling_queriers.md Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> * Update docs/sources/operations/autoscaling_queriers.md Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> * Update docs/sources/operations/autoscaling_queriers.md Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> * Update docs/sources/operations/autoscaling_queriers.md Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> * CR feedback * Add note about needing to use the query-scheduler. * Update docs/sources/operations/autoscaling_queriers.md Co-authored-by: Christopher Moyer <35463610+chri2547@users.noreply.github.com> * Update docs/sources/operations/autoscaling_queriers.md Co-authored-by: Christopher Moyer <35463610+chri2547@users.noreply.github.com> * Update docs/sources/operations/autoscaling_queriers.md Co-authored-by: Christopher Moyer <35463610+chri2547@users.noreply.github.com> * Update docs/sources/operations/autoscaling_queriers.md Co-authored-by: Christopher Moyer <35463610+chri2547@users.noreply.github.com> * Update docs/sources/operations/autoscaling_queriers.md Co-authored-by: Christopher Moyer <35463610+chri2547@users.noreply.github.com> * Update docs/sources/operations/autoscaling_queriers.md Co-authored-by: Christopher Moyer <35463610+chri2547@users.noreply.github.com> * Update docs/sources/operations/autoscaling_queriers.md Co-authored-by: Christopher Moyer <35463610+chri2547@users.noreply.github.com> * Update docs/sources/operations/autoscaling_queriers.md Co-authored-by: Christopher Moyer <35463610+chri2547@users.noreply.github.com> * Update docs/sources/operations/autoscaling_queriers.md Co-authored-by: Christopher Moyer <35463610+chri2547@users.noreply.github.com> * Update docs/sources/operations/autoscaling_queriers.md Co-authored-by: Christopher Moyer <35463610+chri2547@users.noreply.github.com> * Update docs/sources/operations/autoscaling_queriers.md Co-authored-by: Christopher Moyer <35463610+chri2547@users.noreply.github.com> * Update docs/sources/operations/autoscaling_queriers.md Co-authored-by: Christopher Moyer <35463610+chri2547@users.noreply.github.com> * Update docs/sources/operations/autoscaling_queriers.md Co-authored-by: Christopher Moyer <35463610+chri2547@users.noreply.github.com> * Update docs/sources/operations/autoscaling_queriers.md Co-authored-by: Christopher Moyer <35463610+chri2547@users.noreply.github.com> * CR changes * Apply suggestions from code review Co-authored-by: Christopher Moyer <35463610+chri2547@users.noreply.github.com> * Update docs/sources/operations/autoscaling_queriers.md Co-authored-by: Christopher Moyer <35463610+chri2547@users.noreply.github.com> * CR feedback Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> Co-authored-by: Christopher Moyer <35463610+chri2547@users.noreply.github.com>	3 years ago
André Tadeu de Carvalho	9694776537	docs:Add documentation for TLS configuration parameters for HTTP and gRPC (#6933) ... * Add documentation for TLS configuration parameters for HTTP and gRPC. * Removed possible values from client-auth parameters.	3 years ago
Susana Ferreira	87d51bd8e7	Update documentation deletion_mode and allow_deleted configuration values (#6921)	3 years ago
Karen Miller	a02ce69a98	Docs: add minimal docs explaining how to use the LogQL Analyzer (#6900)	3 years ago
carlospeon	171a09b822	Add support for journal matches (only conjuntions) (#6656)	3 years ago
Vladyslav Diachenko	b494713106	fixed typo in API host (#6892)	3 years ago
Vladyslav Diachenko	bec014dd86	Logql analyzer page cherry picks (#6889) ... * added Logql Analyzer page into loki docs * changed template to support changes in API and supported the case when query is run against empty line * added more json examples, changed some labels and changes css for line numbers * fixed last comments from Karen and pointed to production API	3 years ago
Travis Patterson	3170edf3e3	Add timeouts to deletions (#6879) ... * Add context to chunk iteration * Configure timeouts * add expiration timeout * update test * update test * Clarify timeout implications, log more specifically	3 years ago
Karen Miller	8ac08e65d9	Docs: improve wording and grammar (#6861)	3 years ago
Travis Patterson	d602c1331a	Clarify delete upgrade path (#6871) ... * Clarify delete upgrade path * more specificity	3 years ago
Travis Patterson	983ab80e7c	Add delete api validations (#6860) ... * break out the middleware * Add validation to the API - Time must be in RFC3339 or a 10-digit-unix-seconds timestamp - Start time must always exist * lint * docs * improve tests * cleanup * cleanup * access runtime config via function on validation	3 years ago
Owen Diehl	b114dc93d6	update deletion docs (#6869) ... * update deletion docs Signed-off-by: Owen Diehl <ow.diehl@gmail.com> * Update docs/sources/operations/storage/logs-deletion.md	3 years ago
Travis Patterson	bd3d80ad80	Add ability to limit the impact of pathological delete requests. (#6854) ... * make per-tenant override the mode rather than binary on/off * Rely on overrides to configure deletion - Remove 'deletion_mode' flag * add ability to delete requests at any time * don't load requests from disabled tenants * review comments * fix tests * loadDeleteRequestsToProcess cleanup * lint * docs * review feedback * lint	3 years ago
Kaviraj Kanagaraj	7e3d2433ed	embedded-cache: Bring `fifocache` and `groupcache` into single tent. (#6821) ... * chore(groupcache): Rename groupcache into memcache{distributed: true} 1. Introduced new config called memorycache config. 2. It has flag `distributed`. By enabling it, we use groupcache Going forward. We will bring fifocache into memcache config(with distributed:false) This PR is just a POC. To show how we can simplify in-built memory cache config and bring it into single config. Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Memorycache -> EmbeddedCache * Bring fifocache into embeddedcache tent Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Fix edge case with enabling fifocache Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Fix missing flags for embedded cache Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Add changelog Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Warning message for fifocache config Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Make linter happy Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Fix listenport issue Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * idk. review remarks Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * PR remarks 1. update changelog and upgrade guide 2. s/Embeddedcache/EmbeddedCache/g 3. /groupcache/ handler -> /embedded-cache/ 4. some typos 5. Fix some cache prefixes Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Fixing corresponding tests Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Support extra timeouts on embedded-cache Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Remove unwanted single struct field Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * fix(stats-collector): Let cache.New() wrap stats collector for distributed cache Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * PR remarks Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Deprecate max_size_items from fifocache Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * PR remarks about the flag description Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Update docs/sources/upgrading/_index.md Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com>	3 years ago
Alexandre de Sá	6b47aab2de	promtail/client: add bot fields (#6828) ... * chore: add bot fields * Update CHANGELOG.md * chore: update cloudflare configuration section URI * docs: include BotScore and BotScoreSrc to all set of fields * Update CHANGELOG.md	3 years ago
Pablo	c9ef0621d1	Promtail: Add new target for receiving GCP PubSub Push messages (#6777)	3 years ago
Danny Kopping	66fdb3c572	Log stream flushes to see how many chunks we flush at a time (#6819) ... * Log stream flushes to see how many chunks we flush at a time Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Smuggling in an error wrapping which aides debugging Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Update docs/sources/upgrading/_index.md Co-authored-by: Ed Welch <ed@oqqer.com> * Removing unnecessary label Signed-off-by: Danny Kopping <danny.kopping@grafana.com> Co-authored-by: Ed Welch <ed@oqqer.com>	3 years ago
Kaviraj Kanagaraj	1c6a2410d6	doc(retention): Add clarity on default behavior. (#6815) ... * doc(retention): Add clarity on default behavior. Currently if you read through the entire doc, it's not super clear what happens if I don't enable retention. Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * update Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com>	3 years ago
Danny Kopping	c5daa31b65	Remove reference to non-existent config field (#6781) ... Signed-off-by: Danny Kopping <danny.kopping@grafana.com>	3 years ago