apitech/loki - loki - Gitea: Git with a cup of tea

Commit Graph

Author	SHA1	Message	Date
Jan Jansen	8abe080fd6	Promtail: improve behavior of partial lines (#9508 ) What this PR does / why we need it: I started to see OOM after upgrading to a promtail version with #8497. So played around to make partial line behavior more stable. You can now truncate partial lines and configure max partial lines buffer. Which issue(s) this PR fixes: Fixes #<issue number> Special notes for your reviewer: Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [x] Documentation added - [x] Tests updated - [x] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` Signed-off-by: Jan Jansen <jan.jansen@gdata.de> Co-authored-by: Michel Hollands <42814411+MichelHollands@users.noreply.github.com>	3 years ago
Jan-Otto Kröpke	14f96c0c7b	Add loki.memberlistConfig to fully control the memberlist configuration (#9646 ) What this PR does / why we need it: Add loki.memberlistConfig to fully control the memberlist configuration On singleBinary with replica 1, memberlist isn't need and its counter-productive since it delays the start of the loki binary. At the moment, it isn't possible to remove the memberlist defaults. Which issue(s) this PR fixes: Fixes #<issue number> Special notes for your reviewer: Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [x] Documentation added - [ ] Tests updated - [x] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [x] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`)	3 years ago
Jack Baldry	8391e557b1	Update doc-validator to v3.0.0 (#9666 ) ## 3.0.0 ### Added - Structured output for use with [`reviewdog`](https://github.com/reviewdog/reviewdog). You can achieve the original error output by piping the output to the following `jq` expression: `jq -r '"ERROR: \(.location.path):\(.location.range.start.line):\(.location.range.start.column): \(.message)"'`. - Suggestions for simple link fixes. In GitHub, [`reviewdog`](https://github.com/reviewdog/reviewdog) comments these suggestions for convenient replacement. - Support for anchors referring to repeated headings. In the case that a page has multiple headings that share the same text, the renderer appends a zero indexed, numbered suffix to the identifier. For example, when there are two headings that are both "Heading text", the first anchor identifier is `heading-text` and the second is `heading-text-1`. - Error when running `doc-validator` on no files. Notes for reviewer: This also updates the `make-docs` procedure. Please ensure you can run `make docs` from the `docs/` directory before approving and merging this PR. --------- Signed-off-by: Jack Baldry <jack.baldry@grafana.com>	3 years ago
Terje Sannum	986d4ca756	Helm: add support for additional labels on loki-canary pods (#9673 ) Add support for setting additional labels on the loki-canary pods --------- Signed-off-by: Terje Sannum <terje.sannum@nav.no>	3 years ago
Deepansh Mathur	50126731f8	Update README.md (#9698 ) Backticks were missing for a code block What this PR does / why we need it: Which issue(s) this PR fixes: Fixes #<issue number> Special notes for your reviewer: Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`)	3 years ago
Mohamed-Amine Bouqsimi	7069b897e5	Fix Ruler remote write panic (#9662 ) What this PR does / why we need it: This PR fixes the Ruler remote write panic, caused by the no-op scrape manager not being correctly configured.	3 years ago
Robert Jacob	74b16fb71d	operator: Fix availability of demo LokiStack size (#9689 )	3 years ago
Trevor Whitney	4a56445686	Upgrade `golangci-lint` and fix linting errors (#9601 ) What this PR does / why we need it: Upgrade `golangci-lint` and fixes all the errors. The upgrade includes some stricter linting.	3 years ago
Travis Patterson	065bee7e72	Label Volume Endpoint (#9588 ) For a given set of matchers, returns the top N associated label/value pairs by volume. A query for `{cluster=prod}` will return ``` cluster=prod: size (total logs matching this matcher) . . . nth-label=nth-value ``` This is to service use cases where users want to understand where their log volume has come from by label without making multiple requests to the stats endpoint. Note: This PR is a monster but it's mostly plumbing. I've pointed out the most interesting bits that actually get the volumes from ingesters/indexs	3 years ago
Piotr	4d997a5a10	Fix promtail cluster template not finding all clusters. (#9684 ) What this PR does / why we need it: In promtail-mixin, the dropdown template for clusters would only include clusters that run loki. So if a cluster only run promtail and not loki, it doesn't appear.	3 years ago
Kaviraj Kanagaraj	57f9452231	Revert 9217 chaudum/tsdb chunkrefs pool (#9685 ) Revert #9217 (potential bug in query result) --------- Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com>	3 years ago
Salva Corts	73ac208981	Improve docs for empty value in cache compression config (#9649 ) What this PR does / why we need it: Follow up PR for https://github.com/grafana/loki/pull/9535#discussion_r1218167670 Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [x] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`)	3 years ago
Christophe Collot	f2394355d8	feat(lambda-promtail): add cloudfront log file ingestion support (#9573 ) What this PR does / why we need it: This PR enables ingesting logs from Cloudfront log files stored in s3 (batch). The current setup only supports streaming Cloudfront logs through AWS Kinesis, whereas this PR implements the same flow as for VPC Flow logs, Load Balancer logs, and Cloudtrail logs (s3 --> SQS (optional) --> Lambda Promtail --> Loki) Special notes for your reviewer: + The Cloudfront log file format is different from the already implemented services, meaning we had to build yet another regex. AWS never bothered making all services follow the same log file naming convention but the "good" thing is that it's now very unlikely they will change it in the future. + The Cloudfront file name does not have any mention of the AWS account or the time of log it contains, it means we have to infer the log type from the filename format instead of finding the exact string "cloudfront" in the filename. This is why in `getLabels`, if no `type` parameter is found in the regex, we use the key corresponding to the name of the matching parser. + I introduced a new `parser` struct to group together several parameters specific to a type of log (and avoid relying too much on map key string matching and / or if statements for specific use cases) + I've been successfully running this code in several AWS environments for days. + I corrected a typo from my previous PR #9497 (wrong PR number in Changelog.md) Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [x] Documentation added - [x] Tests updated - [x] `CHANGELOG.md` updated - [x] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [x] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) --------- Co-authored-by: Michel Hollands <42814411+MichelHollands@users.noreply.github.com>	3 years ago
Salva Corts	c6fbff26e1	Add config to avoid caching stats for recent data (#9537 ) What this PR does / why we need it: When we query the stats for recent data, we query both the ingesters and the index gateways for the stats. `ebdb2b1800/pkg/storage/async_store.go (L112-L114)` `ebdb2b1800/pkg/storage/async_store.go (L126-L127)` Then we merge all the responses, which means summing up all the stats `ebdb2b1800/pkg/storage/async_store.go (L157-L158)` `ebdb2b1800/pkg/storage/stores/index/stats/stats.go (L23-L26)` Because we have a replication factor of 3, this means that we will get the stats from the ingesters repeated up to 3 times, hence inflating the stats. In the stats cache, we store the stats for a given matcher set for the whole day, then we extract the stats from the cache by the factor of time from the request that is stored in the cache: `336283acad/pkg/querier/queryrange/index_stats_cache.go (L33)` `336283acad/pkg/querier/queryrange/index_stats_cache.go (L40)` Inflated stats for recent data will be cached, so subsequent stats extracted from the cache will be inflated regardless of the time. This PR adds a new per-tenant limit `max_stats_cache_freshness` to not cache requests with an end time that falls within Now minus this duration. Here's a scenario illustrating this. The graphs below show the bytes stats queried in the sharding middleware. We are running a log filter query that won't match any log, every 5 seconds with a length of 3h. ![image](https://github.com/grafana/loki/assets/8354290/45c2e6e9-185c-4a18-b290-47da27fc3e39) As can be seen, after enabling the stats cache and configuring`do_not_cache_request_within` to not cache stats for requests within 30m, the bytes stats used in the sharding middleware stopped increasing. In both cases the stats cache hit ration was 100%. ![image](https://github.com/grafana/loki/assets/8354290/cd35bcb8-0c77-4693-a06b-502741fd6e23) Special notes for your reviewer: - Blocked by https://github.com/grafana/loki/pull/9535 - Note that this PR doesn't fix the root issue of inflated stats form the ingesters, but rather buys us some time to work on that. Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [x] Documentation added - [x] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`)	3 years ago
Michel Hollands	22779e13c5	Fix date template function with epoch times (#8886 ) What this PR does / why we need it: Adds new toUnixEpoch... functions to convert from a string with a Unix/Epoch time to an integer that can be used in the existing `toDate` function. Note that these are the opposites of some of the functions introduced in https://github.com/grafana/loki/pull/8774. Which issue(s) this PR fixes: Fixes https://github.com/grafana/loki/issues/8624. Special notes for your reviewer: Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [X] Documentation added - [X] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` --------- Signed-off-by: Michel Hollands <michel.hollands@grafana.com>	3 years ago
Bruno FERNANDO	1b410dbc01	feat(promtail): add CF ClientRequestSource field (#9669 ) What this PR does / why we need it: Hey folks 👋 Little contribution here to add a useful log field for cloudflare users. Indeed I add the [ClientRequestSource field](https://developers.cloudflare.com/logs/reference/clientrequestsource/ ) which is pretty useful when debugging some specific traffic handled by cloudflare Extra: Since I was on the documentation I fixed an indentation issue that I spotted Don't hesitate to reach me if you have any questions Cheers 😉 Which issue(s) this PR fixes: Fixes #<issue number> Special notes for your reviewer: Loki rocks 🚀 Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [x] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`)	3 years ago
Gregor Zeitlinger	b1917a6178	add "alignLeft" and "alignRight" functions (#9672 ) Fixes https://github.com/grafana/loki/issues/9667	3 years ago
Ashwanth	98d1307c7d	config: ensure storage config defaults apply to named stores (#9650 ) What this PR does / why we need it: Since named store config does not register any flags, storage configs defined under it do not get the defaults. For example [aws_storage_config](https://grafana.com/docs/loki/latest/configuration/#aws_storage_config) sets the default `storage_class` to `STANDARD`, but the same doesn't get applied by default when using named stores. This PR ensures that named storage configs are always assigned default values when they are unmarshalled by implementing `yaml.Unmarshaler` interface	3 years ago
Pepe Cano	4cebc2d85c	Docs: replace `k6 Cloud` mention (#9599 ) k6 is now available as a managed service on Grafana Cloud. This is a small doc changes to remove the mention of `k6 Cloud`. --------- Co-authored-by: J Stickler <julie.stickler@grafana.com>	3 years ago
Danny Kopping	1db560fdc1	Adding background cache (en\|de)queue counters (#9665 ) What this PR does / why we need it: The background writeback cache exposes gauge metric currently for the current queue size. Gauges can be useful, but they are susceptible to sample errors because they only represent the point in time as the time of the scrape. Exposing counters for the bytes (en\|de)queued to/from the cache will be more useful because they can be aggregated. Signed-off-by: Danny Kopping <danny.kopping@grafana.com>	3 years ago
Dylan Guedes	609bc22933	Distributor: Make key configurable when logging failures (#9659 ) What this PR does / why we need it: Make appending `insight=true` key-value pair to log failures configurable. Which issue(s) this PR fixes: N/A	3 years ago
Nils Griebner	d58125885b	Make table manager retention options configurable in helm chart values (#9647 ) What this PR does / why we need it: Configuration options for table manager retention are hard-coded in the helm chart values at the moment so that it's not possible to enable retention deletes. Which issue(s) this PR fixes: Fixes #8676 Special notes for your reviewer: - Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [x] Documentation added - [x] Tests updated - [x] `CHANGELOG.md` updated - [x] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [x] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) Co-authored-by: Michel Hollands <42814411+MichelHollands@users.noreply.github.com>	3 years ago
Ed Welch	79c5fdd9c0	Canary: Refactor how the direct Canary push works (#9569 ) What this PR does / why we need it: When doing some testing a while back I ran into several cases were we didn't handle errors/retries properly in the code we added for the canaries to send directly to Loki. The changes here effectively duplicate the way promtail's client does sends to Loki which is code we've been using/relying on for years now. I also had a hard time following the previous implementation which used the io.Writer interface for swapping out writing to stdout vs pushing to Loki and refactored this to use a new interface `EntryWriter` which has implementations that write to stdout and to Loki Unfortunately the changes in the diff don't read super well with this refactoring so hopefully the explanation helps. Which issue(s) this PR fixes: Fixes #<issue number> Special notes for your reviewer: Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) --------- Signed-off-by: Edward Welch <edward.welch@grafana.com> Co-authored-by: Paul Rogers <129207811+paul1r@users.noreply.github.com>	3 years ago
Ivana Huckova	eb7dae4583	Loki: Improve error message when step too low (#9641 ) What this PR does / why we need it: In https://github.com/grafana/grafana/pull/69648 we are in Grafana introducing a step editor in Loki. Unfortunately, the error message when user sets too low step parameter is hard to understand, so I am proposing following change to make it more understandable and actionable. Let me know what do you think. --------- Co-authored-by: J Stickler <julie.stickler@grafana.com>	3 years ago
Christopher Piggott	415a997fb9	Update CONTRIBUTING.md to point to the new location of the contributi… (#9654 ) What this PR does / why we need it: Updating CONTRIBUTING document to point to the location of the LID template, which has moved Which issue(s) this PR fixes: none Special notes for your reviewer: doc change only Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`)	3 years ago
Alexandre	806cd73e53	Fix configmap checksum in read sts (#9642 ) What this PR does / why we need it: @NilsGriebner : complement to #9579 Which issue(s) this PR fixes: Fixes #<issue number> Special notes for your reviewer: Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [x] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) --------- Co-authored-by: Michel Hollands <42814411+MichelHollands@users.noreply.github.com>	3 years ago
Salva Corts	af287ac3eb	Add summary stats and metrics for stats cache (#9536 ) What this PR does / why we need it: When a query finishes, we return (and log) the following stats: ```go Cache.Chunk.Requests 0 Cache.Chunk.EntriesRequested 0 Cache.Chunk.EntriesFound 0 Cache.Chunk.EntriesStored 0 Cache.Chunk.BytesSent 0 B Cache.Chunk.BytesReceived 0 B Cache.Chunk.DownloadTime 0s Cache.Index.Requests 0 Cache.Index.EntriesRequested 0 Cache.Index.EntriesFound 0 Cache.Index.EntriesStored 0 Cache.Index.BytesSent 0 B Cache.Index.BytesReceived 0 B Cache.Index.DownloadTime 0s Cache.Result.Requests 13 Cache.Result.EntriesRequested 13 Cache.Result.EntriesFound 13 Cache.Result.EntriesStored 0 Cache.Result.BytesSent 0 B Cache.Result.BytesReceived 2.5 kB Cache.Result.DownloadTime 4.600266ms ``` In addition to that, we log the following in metrics.go: ``` level=info ts=2023-05-29T09:17:10.93029945Z caller=metrics.go:152 component=frontend org_id=145265 traceID=52d59b78fe6b9221 sampled=true latency=fast query="{cluster=\"dev-us-central-0\", namespace=~\"loki.\", container=~\"distributor\|ingester \|promtail\|index-gateway\|compactor\"} \|= \"thislinewillnotexist\"" query_hash=1194136170 query_type=filter range_type=range length=3h0m0s start_delta=165h37m24.930289434s end_delta=162h37m24.930289612s step=43s duration=2.473055ms status=200 lim it=30 returned_lines=0 throughput=0B total_bytes=0B lines_per_second=0 total_lines=0 total_entries=0 store_chunks_download_time=0s queue_time=0s splits=13 shards=0 cache_chunk_req=0 cache_chunk_hit=0 cache_chunk_bytes_stored=0 cache_chunk_bytes _fetched=0 cache_chunk_download_time=0s cache_index_req=0 cache_index_hit=0 cache_index_download_time=0s cache_result_req=13 cache_result_hit=13 cache_result_download_time=4.600266ms ``` With the goal of being able to better monitor how the stats cache is performing; this PR adds stats for the index stats cache, similarly to how it's done for the results cache. Here's an example of the new stats being returned and printed: ```go ... Cache.StatsResult.Requests 180 Cache.StatsResult.EntriesRequested 129 Cache.StatsResult.EntriesFound 129 Cache.StatsResult.EntriesStored 51 Cache.StatsResult.BytesSent 0 B Cache.StatsResult.BytesReceived 75 kB ... ``` And the new stats from metrics.go ``` ... caller=metrics.go:155 ... cache_stats_results_req=129 cache_stats_results_hit=129 cache_stats_results_download_ti me=156.864429ms ... ``` Special notes for your reviewer: - Blocked by https://github.com/grafana/loki/pull/9535 - Note the new`stats.GetOrCreateContext` func. It's used inside the `query.Exec` method so we don't overwrite the stats added in the stats middleware. Checklist* - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [x] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`)	3 years ago
Julien Pinsonneau	e6802394da	operator: Expose per_stream_rate_limit & burst (#9630 ) Co-authored-by: Periklis Tsirakidis <periklis@redhat.com>	3 years ago
Salva Corts	1694ad0f9b	Stats cache can be configured independently (#9535 ) What this PR does / why we need it: Before this PR, the index stats cache would use the same config as the query results cache. This was a limitation since: 1. We would not be able to point to a different cache for storing the index stats if needed. 2. We would not be able to add specific settings for this cache, without adding it to the results cache. In this PR, we refactor the index stats cache config to be independently configurable. Note that if it's not configured, it will try to use the results cache settings. Which issue(s) this PR fixes: This is needed for: - https://github.com/grafana/loki/pull/9537 - https://github.com/grafana/loki/pull/9536 Special notes for your reviewer: - This PR also refactors all the tripperwares in rountrip.go to reuse the same stats tripperware instead of each one creating their own. - Configuring a new cache in rountrip.go is a requirement for https://github.com/grafana/loki/pull/9536 so the stats summary can distinguish before the stats cache and the results cache. Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [x] Documentation added - [x] Tests updated - [x] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`)	3 years ago
Vladyslav Diachenko	3c52034d63	[logcli] fixed query command that is run against user's bucket with TSDB index enabled (#9597 ) What this PR does / why we need it: set TSDB shipper mode to ReadOnly and disabled indexGatewayClient during local query run. Also, I increase index downloading timeout to 1m because the users run the query from local machines that not always have more than 100Mbps Which issue(s) this PR fixes: Fixes #9555 Special notes for your reviewer: Unfortunately, we do not have similar tests and test architecture to cover it with unit tests properly. Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [x] `CHANGELOG.md` updated - [x] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [x] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`)	3 years ago
Periklis Tsirakidis	f43dd58459	operator: Fix timeout config constructor when only tenants limits (#9623 )	3 years ago
Nils Griebner	2c7309e0f7	Fix configmap checksum in table manager deployment template (#9579 ) What this PR does / why we need it: This PR fixes the path to the configmap template in table-manager deployment file Which issue(s) this PR fixes: Fixes #9578 Special notes for your reviewer: This is probably the same issue as in #9546. Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [x] Documentation added - [x] Tests updated - [x] `CHANGELOG.md` updated - [x] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [x] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`)	3 years ago
Sandeep Sukhani	0a5e149ea5	query-scheduler: fix query distribution in SSD mode (#9471 ) What this PR does / why we need it: When we run the `query-scheduler` in `ring` mode, `queriers` and `query-frontend` discover the available `query-scheduler` instances using the ring. However, we have a problem when `query-schedulers` are not running in the same process as queriers and query-frontend since [we try to get the ring client interface from the scheduler instance](`abd6131bba/pkg/loki/modules.go (L358)`). This causes queries not to be spread across all the available queriers when running in SSD mode because [we point querier workers to query frontend when there is no ring client and scheduler address configured](`b05f4fced3/pkg/querier/worker_service.go (L115)`). I have fixed this issue by adding a new hidden target to initialize the ring client in `reader`/`member` mode based on which service is initializing it. `reader` mode will be used by `queriers` and `query-frontend` for discovering `query-scheduler` instances from the ring. `member` mode will be used by `query-schedulers` for registering themselves in the ring. I have also made a couple of changes not directly related to the issue but it fixes some problems: * [reset metric registry for each integration test](`18c4fe5907`) - Previously we were reusing the same registry for all the tests and just [ignored the attempts to register same metrics](`01f0ded7fc/integration/cluster/cluster.go (L113)`). This causes the registry to have metrics registered only from the first test so any updates from subsequent tests won't reflect in the metrics. metrics was the only reliable way for me to verify that `query-schedulers` were connected to `queriers` and `query-frontend` when running in ring mode in the integration test that I added to test my changes. This should also help with other tests where earlier it was hard to reliably check the metrics. * [load config from cli as well before applying dynamic config](`f9e2448fc7`) - Previously we were applying dynamic config considering just the config from config file. This results in unexpected config changes, for example, [this config change](`4148dd2c51/integration/loki_micro_services_test.go (L66)`) was getting ignored and [dynamic config tuning was unexpectedly turning on ring mode](`52cd0a39b8/pkg/loki/config_wrapper.go (L94)`) in the config. It is better to do any config tuning based on both file and cli args configs. Which issue(s) this PR fixes: Fixes #9195	3 years ago
dependabot[bot]	90ed037a78	Bump golangci/golangci-lint-action from 3.4.0 to 3.5.0 (#9627 ) Bumps [golangci/golangci-lint-action](https://github.com/golangci/golangci-lint-action) from 3.4.0 to 3.5.0. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/golangci/golangci-lint-action/releases">golangci/golangci-lint-action's releases</a>.</em></p> <blockquote> <h2>v3.5.0</h2> <h2>What's Changed</h2> <ul> <li>build(deps-dev): bump eslint from 8.32.0 to 8.33.0 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/golangci/golangci-lint-action/pull/659">golangci/golangci-lint-action#659</a></li> <li>build(deps-dev): bump <code>@typescript-eslint/eslint-plugin</code> from 5.48.2 to 5.49.0 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/golangci/golangci-lint-action/pull/661">golangci/golangci-lint-action#661</a></li> <li>build(deps-dev): bump eslint-plugin-simple-import-sort from 9.0.0 to 10.0.0 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/golangci/golangci-lint-action/pull/662">golangci/golangci-lint-action#662</a></li> <li>build(deps-dev): bump <code>@vercel/ncc</code> from 0.36.0 to 0.36.1 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/golangci/golangci-lint-action/pull/660">golangci/golangci-lint-action#660</a></li> <li>build(deps-dev): bump <code>@typescript-eslint/parser</code> from 5.48.2 to 5.49.0 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/golangci/golangci-lint-action/pull/663">golangci/golangci-lint-action#663</a></li> <li>build(deps-dev): bump <code>@typescript-eslint/eslint-plugin</code> from 5.49.0 to 5.50.0 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/golangci/golangci-lint-action/pull/665">golangci/golangci-lint-action#665</a></li> <li>build(deps-dev): bump <code>@typescript-eslint/parser</code> from 5.49.0 to 5.50.0 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/golangci/golangci-lint-action/pull/666">golangci/golangci-lint-action#666</a></li> <li>build(deps-dev): bump typescript from 4.9.4 to 4.9.5 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/golangci/golangci-lint-action/pull/667">golangci/golangci-lint-action#667</a></li> <li>build(deps): bump <code>@types/node</code> from 18.11.18 to 18.11.19 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/golangci/golangci-lint-action/pull/668">golangci/golangci-lint-action#668</a></li> <li>doc: add quote aroung go version by <a href="https://github.com/vaughany"><code>@vaughany</code></a> in <a href="https://redirect.github.com/golangci/golangci-lint-action/pull/670">golangci/golangci-lint-action#670</a></li> <li>build(deps-dev): bump <code>@typescript-eslint/parser</code> from 5.50.0 to 5.51.0 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/golangci/golangci-lint-action/pull/671">golangci/golangci-lint-action#671</a></li> <li>build(deps-dev): bump prettier from 2.8.3 to 2.8.4 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/golangci/golangci-lint-action/pull/673">golangci/golangci-lint-action#673</a></li> <li>build(deps): bump <code>@types/node</code> from 18.11.19 to 18.13.0 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/golangci/golangci-lint-action/pull/674">golangci/golangci-lint-action#674</a></li> <li>build(deps-dev): bump <code>@typescript-eslint/eslint-plugin</code> from 5.50.0 to 5.51.0 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/golangci/golangci-lint-action/pull/675">golangci/golangci-lint-action#675</a></li> <li>build(deps): bump <code>@actions/cache</code> from 3.1.2 to 3.1.3 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/golangci/golangci-lint-action/pull/672">golangci/golangci-lint-action#672</a></li> <li>build(deps-dev): bump <code>@typescript-eslint/parser</code> from 5.51.0 to 5.52.0 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/golangci/golangci-lint-action/pull/678">golangci/golangci-lint-action#678</a></li> <li>build(deps): bump <code>@types/node</code> from 18.13.0 to 18.14.0 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/golangci/golangci-lint-action/pull/679">golangci/golangci-lint-action#679</a></li> <li>build(deps-dev): bump eslint from 8.33.0 to 8.34.0 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/golangci/golangci-lint-action/pull/680">golangci/golangci-lint-action#680</a></li> <li>build(deps-dev): bump <code>@typescript-eslint/eslint-plugin</code> from 5.51.0 to 5.52.0 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/golangci/golangci-lint-action/pull/681">golangci/golangci-lint-action#681</a></li> <li>build(deps-dev): bump <code>@typescript-eslint/eslint-plugin</code> from 5.52.0 to 5.53.0 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/golangci/golangci-lint-action/pull/684">golangci/golangci-lint-action#684</a></li> <li>build(deps-dev): bump eslint from 8.34.0 to 8.35.0 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/golangci/golangci-lint-action/pull/685">golangci/golangci-lint-action#685</a></li> <li>build(deps): bump <code>@types/node</code> from 18.14.0 to 18.14.2 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/golangci/golangci-lint-action/pull/688">golangci/golangci-lint-action#688</a></li> <li>build(deps-dev): bump <code>@typescript-eslint/parser</code> from 5.52.0 to 5.53.0 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/golangci/golangci-lint-action/pull/687">golangci/golangci-lint-action#687</a></li> <li>build(deps): bump <code>@actions/cache</code> from 3.1.3 to 3.1.4 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/golangci/golangci-lint-action/pull/686">golangci/golangci-lint-action#686</a></li> <li>build(deps): bump <code>@types/node</code> from 18.14.2 to 18.14.6 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/golangci/golangci-lint-action/pull/691">golangci/golangci-lint-action#691</a></li> <li>build(deps-dev): bump <code>@typescript-eslint/parser</code> from 5.53.0 to 5.54.0 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/golangci/golangci-lint-action/pull/692">golangci/golangci-lint-action#692</a></li> <li>build(deps-dev): bump <code>@typescript-eslint/eslint-plugin</code> from 5.53.0 to 5.54.0 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/golangci/golangci-lint-action/pull/693">golangci/golangci-lint-action#693</a></li> <li>build(deps-dev): bump eslint-config-prettier from 8.6.0 to 8.7.0 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/golangci/golangci-lint-action/pull/694">golangci/golangci-lint-action#694</a></li> <li>build(deps-dev): bump eslint from 8.35.0 to 8.36.0 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/golangci/golangci-lint-action/pull/699">golangci/golangci-lint-action#699</a></li> <li>build(deps-dev): bump <code>@typescript-eslint/eslint-plugin</code> from 5.54.0 to 5.54.1 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/golangci/golangci-lint-action/pull/700">golangci/golangci-lint-action#700</a></li> <li>build(deps): bump <code>@types/node</code> from 18.14.6 to 18.15.1 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/golangci/golangci-lint-action/pull/701">golangci/golangci-lint-action#701</a></li> <li>docs/build: update to setup-go@v4 by <a href="https://github.com/caarlos0"><code>@caarlos0</code></a> in <a href="https://redirect.github.com/golangci/golangci-lint-action/pull/704">golangci/golangci-lint-action#704</a></li> <li>build(deps-dev): bump typescript from 4.9.5 to 5.0.2 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/golangci/golangci-lint-action/pull/705">golangci/golangci-lint-action#705</a></li> <li>build(deps): bump <code>@types/node</code> from 18.15.1 to 18.15.3 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/golangci/golangci-lint-action/pull/706">golangci/golangci-lint-action#706</a></li> <li>build(deps): bump <code>@actions/http-client</code> from 2.0.1 to 2.1.0 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/golangci/golangci-lint-action/pull/697">golangci/golangci-lint-action#697</a></li> <li>build(deps-dev): bump prettier from 2.8.4 to 2.8.5 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/golangci/golangci-lint-action/pull/707">golangci/golangci-lint-action#707</a></li> <li>build(deps): bump <code>@actions/cache</code> from 3.1.4 to 3.2.1 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/golangci/golangci-lint-action/pull/698">golangci/golangci-lint-action#698</a></li> <li>build(deps-dev): bump eslint-config-prettier from 8.7.0 to 8.8.0 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/golangci/golangci-lint-action/pull/709">golangci/golangci-lint-action#709</a></li> <li>build(deps): bump <code>@types/node</code> from 18.15.3 to 18.15.10 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/golangci/golangci-lint-action/pull/710">golangci/golangci-lint-action#710</a></li> <li>build(deps-dev): bump prettier from 2.8.5 to 2.8.7 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/golangci/golangci-lint-action/pull/711">golangci/golangci-lint-action#711</a></li> <li>build(deps-dev): bump <code>@typescript-eslint/eslint-plugin</code> from 5.54.1 to 5.56.0 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/golangci/golangci-lint-action/pull/712">golangci/golangci-lint-action#712</a></li> <li>build(deps-dev): bump <code>@typescript-eslint/parser</code> from 5.54.0 to 5.56.0 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/golangci/golangci-lint-action/pull/713">golangci/golangci-lint-action#713</a></li> <li>build(deps-dev): bump <code>@typescript-eslint/eslint-plugin</code> from 5.56.0 to 5.57.0 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/golangci/golangci-lint-action/pull/718">golangci/golangci-lint-action#718</a></li> <li>build(deps-dev): bump typescript from 5.0.2 to 5.0.3 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/golangci/golangci-lint-action/pull/717">golangci/golangci-lint-action#717</a></li> <li>build(deps-dev): bump eslint from 8.36.0 to 8.37.0 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/golangci/golangci-lint-action/pull/719">golangci/golangci-lint-action#719</a></li> <li>build(deps-dev): bump <code>@typescript-eslint/parser</code> from 5.56.0 to 5.57.0 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/golangci/golangci-lint-action/pull/720">golangci/golangci-lint-action#720</a></li> <li>build(deps): bump <code>@types/node</code> from 18.15.10 to 18.15.11 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/golangci/golangci-lint-action/pull/721">golangci/golangci-lint-action#721</a></li> <li>build(deps-dev): bump <code>@typescript-eslint/eslint-plugin</code> from 5.57.0 to 5.57.1 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/golangci/golangci-lint-action/pull/722">golangci/golangci-lint-action#722</a></li> </ul> <!-- raw HTML omitted --> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Commits</summary> <ul> <li><a href="`5f1fec7010`"><code>5f1fec7</code></a> build(deps-dev): bump <code>@typescript-eslint/parser</code> from 5.59.6 to 5.59.7 (<a href="https://redirect.github.com/golangci/golangci-lint-action/issues/758">#758</a>)</li> <li><a href="`601007b788`"><code>601007b</code></a> build(deps): bump <code>@types/node</code> from 20.2.3 to 20.2.5 (<a href="https://redirect.github.com/golangci/golangci-lint-action/issues/756">#756</a>)</li> <li><a href="`d2a913e97b`"><code>d2a913e</code></a> build(deps-dev): bump <code>@typescript-eslint/eslint-plugin</code> from 5.59.6 to 5.59.7 ...</li> <li><a href="`7233bd71cb`"><code>7233bd7</code></a> build(deps-dev): bump <code>@typescript-eslint/eslint-plugin</code> from 5.59.5 to 5.59.6 ...</li> <li><a href="`687f029324`"><code>687f029</code></a> build(deps): bump <code>@types/node</code> from 20.1.4 to 20.2.3 (<a href="https://redirect.github.com/golangci/golangci-lint-action/issues/755">#755</a>)</li> <li><a href="`f9990cd216`"><code>f9990cd</code></a> build(deps-dev): bump <code>@typescript-eslint/parser</code> from 5.59.5 to 5.59.6 (<a href="https://redirect.github.com/golangci/golangci-lint-action/issues/754">#754</a>)</li> <li><a href="`f30aa514f9`"><code>f30aa51</code></a> build(deps-dev): bump eslint from 8.40.0 to 8.41.0 (<a href="https://redirect.github.com/golangci/golangci-lint-action/issues/753">#753</a>)</li> <li><a href="`6b21f586ed`"><code>6b21f58</code></a> build(deps-dev): bump <code>@typescript-eslint/eslint-plugin</code> from 5.59.2 to 5.59.5 ...</li> <li><a href="`535ed3a04b`"><code>535ed3a</code></a> build(deps): bump <code>@types/semver</code> from 7.3.13 to 7.5.0 (<a href="https://redirect.github.com/golangci/golangci-lint-action/issues/748">#748</a>)</li> <li><a href="`0078ef00ab`"><code>0078ef0</code></a> build(deps-dev): bump <code>@typescript-eslint/parser</code> from 5.59.1 to 5.59.5 (<a href="https://redirect.github.com/golangci/golangci-lint-action/issues/750">#750</a>)</li> <li>Additional commits viewable in <a href="https://github.com/golangci/golangci-lint-action/compare/v3.4.0...v3.5.0">compare view</a></li> </ul> </details> <br /> [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=golangci/golangci-lint-action&package-manager=github_actions&previous-version=3.4.0&new-version=3.5.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	3 years ago
Periklis Tsirakidis	69392de8b6	Fix duplicate label values from ingester streams (#9629 )	3 years ago
Eric	c6f809a738	Fix missing ~ in the regexp match for namespace (#9618 ) What this PR does / why we need it: Fix documentation which has a missing `~` for the regexp match. I was following the documentation and found it odd that this query didn't follow [LogQL](https://grafana.com/docs/loki/latest/logql/log_queries/#log-queries), but it actually does and was just a typo in the docs. I've tested this with my own Loki 2.8 release and with the `~` this works great! I also added a similar description block to the above example explaining the pipeline, but if this is not desired, I'm happy to remove it. Which issue(s) this PR fixes: NA, trivial issue, just a small fix in the docs. Special notes for your reviewer: Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [x] Documentation added - [x] Tests updated - [x] `CHANGELOG.md` updated - [x] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [x] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`)	3 years ago
W.T. Chang	abb70456e6	Fix comment and indentation in gateway config of SSD helm chart (#9622 ) What this PR does / why we need it: The gateway config (nginx config file) incorrectly states 10 minutes (as in 600 seconds) as 6 minutes. This small PR corrects that. Which issue(s) this PR fixes: N/A Special notes for your reviewer: Although this indeed modifies the chart, I don't think it's worth a version bump. Please let me know if you have other ideas. Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`)	3 years ago
Travis Patterson	6d8a184334	Revert "revert count unflused chunks (#9504 )" (#9615 ) Now that the bug where huge values is fixed and known not to be this. Let's readd it. Original: https://github.com/grafana/loki/pull/9479	3 years ago
Travis Patterson	1f119e1356	Prevent GetFactorOfTime from returning NaN (#9602 ) `GetFactorOfTime` is used to calculate overlaps with chunk time ranges -- particularly calls to `stats`. When a chunk has only one entry, it's `start time == end time` and this `GetFactorOfTime` returns `NaN`. The stats calculation is `uint64(total chunk bytes * factor)`. When `factor` is `NaN`: `uint64(total chunk bytes * factor) == uint64(math.Nan()) == 9223372036854775808` `uint64(math.Nan()) == 9223372036854775808` is an implementation detail of golang.	3 years ago
Bryan Boreham	f5872ec4d3	marshal tests: make random quantities more obvious (#9612 ) What this PR does / why we need it: Computing a new random number on every iteration of the loop is hard to reason about, and probably not what was meant. This PR will mean that much more data is generated than before; I computed the expected value before as 11.2, whereas now it is 49.5. Consider reducing the `100` if that is undesired. Checklist - NA Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - NA Documentation added - [x] Tests updated - NA `CHANGELOG.md` updated - NA Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - NA For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`)	3 years ago
Juhyeong Ko	a21842d740	Fix typo in log_queries docs (#9506 )	3 years ago
Salva Corts	87a659a6db	Add span events for index stats and result cache (#9552 ) What this PR does / why we need it: This PR adds events to the traces to have some extra observability for how we compute the index stats. We also add some trace events to the results cache. ![image](https://github.com/grafana/loki/assets/8354290/7566b755-8193-4e46-ba10-37d3377ea31a) ![image](https://github.com/grafana/loki/assets/8354290/d1990150-84b1-4522-9898-6e37c2782c5b) ![image](https://github.com/grafana/loki/assets/8354290/a8c23e7f-a06d-4a47-8cd4-e900fce01e80) ![image](https://github.com/grafana/loki/assets/8354290/d1e15fb6-fb6c-4fe1-9c5f-f1c8164889de) ![image](https://github.com/grafana/loki/assets/8354290/0c0d001e-7083-488c-8809-0446b4b7c852) Special notes for your reviewer: Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`)	3 years ago
Danny Kopping	a248e3e970	Querier: configurable writeback queue bytes size (#9604 ) What this PR does / why we need it: The background writeback process is used to write chunks that have been fetched from the store to cache. Chunks have a target size, but not all chunks will necessarily be this size, so it is tricky to find the correct value for `writeback_buffer` which specifies how many chunks will be held in the queue at any given time. For example: setting `writeback_buffer=10000` (the default) with all chunks being the default target size (`chunk_target_size=1.5MiB`), in the worst case could result in 15GiB of memory being used for this buffer. Practically speaking, the queue never grows so deep, but if the cache Loki is writing to becomes overwhelmed this could happen. Setting a value too low would result in many chunks not being written back to cache, which defeats the purpose. This PR introduces a configurable _byte size_ limit to the writeback cache, which defaults to 1GB in size. With this setting `writeback_size_limit` in place, we get the best of both worlds: a deep queue which can hold many small chunks or a shallow queue which can hold a few large ones without using too much memory so as to overwhelm the querier. If either the `writeback_buffer` or `writeback_size_limit` reaches capacity, any new chunks which are attempted to be added to the queue will be dropped. Two new metrics are added by this PR: - `loki_cache_background_queue_bytes`: gauge, measures the size of the queue in bytes - `loki_cache_dropped_background_writes_bytes_total`: counter, measures the volume of bytes dropped	3 years ago
Salva Corts	4d00e91caf	Fix cacheGen for index stats reqs (#9529 ) What this PR does / why we need it: When deletes are enabled, we track the cache generation number to know when to invalidate the cache. The queriers add this cache generation number to the response, and in the frontend, we check if we should cache the response or not based on such generation number. Before this PR, the queriers didn't add the cache generation number to the index stats responses, therefore, when deletes were enabled, we could see the following error message in the frontend when checking if we should store a new response in the cache: ``` we found results cache gen number %s set in store but none in headers ``` This PR fixes this by adding the middleware to add the generation number to the index stats handler in the querier. As seen in the following screenshot, once we rolled out the fixes in this PR (at 10:12), the number of failed cache generation comparisons decreased to 0. At the same time, the number of version comparisons also decreased since we could store results in the cache and therefore the cache effectiveness improved. ([source][1]) ![image](https://github.com/grafana/loki/assets/8354290/752fbc66-0005-437a-9469-bc25405198b9) Which issue(s) this PR fixes: Fixes #<issue number> Special notes for your reviewer: Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) [1]: https://admin-dev-us-central-0.grafana-dev.net/grafana/explore?orgId=1&left=%7B%22datasource%22:%22P09658BD21492D183%22,%22queries%22:%5B%7B%22refId%22:%22G%22,%22expr%22:%22sum%28%5Cn%20%20%20%20rate%28%5Cn%20%20%20%20%20%20%20%20loki_results_cache_version_comparisons_failed%7Bcluster%3D%5C%22dev-us-central-0%5C%22,%20namespace%3D%5C%22loki-dev-005%5C%22%7D%5Cn%20%20%20%20%20%20%20%20%5B1m%5D%5Cn%20%20%20%20%29%5Cn%29%22,%22range%22:true,%22datasource%22:%7B%22type%22:%22prometheus%22,%22uid%22:%22P09658BD21492D183%22%7D,%22editorMode%22:%22code%22,%22legendFormat%22:%22Gen%20cmp%20fail%22%7D,%7B%22refId%22:%22A%22,%22expr%22:%22sum%28%5Cn%20%20%20%20rate%28%5Cn%20%20%20%20%20%20%20%20loki_results_cache_version_comparisons_total%7Bcluster%3D%5C%22dev-us-central-0%5C%22,%20namespace%3D%5C%22loki-dev-005%5C%22%7D%5Cn%20%20%20%20%20%20%20%20%5B1m%5D%5Cn%20%20%20%20%29%5Cn%29%22,%22range%22:true,%22instant%22:true,%22datasource%22:%7B%22type%22:%22prometheus%22,%22uid%22:%22P09658BD21492D183%22%7D,%22editorMode%22:%22code%22,%22legendFormat%22:%22Gen%20cpm%20total%22%7D%5D,%22range%22:%7B%22from%22:%221685094941101%22,%22to%22:%221685096865685%22%7D%7D	3 years ago
FL42	3fc9328862	Fix: HPA for single binary deployment (#9585 ) What this PR does / why we need it: Add HPA for single binary deployment. Off by default. Which issue(s) this PR fixes: Fixes #7097 Special notes for your reviewer: Checklist - [X] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [N/A] Documentation added - [N/A] Tests updated - [X] `CHANGELOG.md` updated - [N/A] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [X] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) --------- Co-authored-by: Michel Hollands <42814411+MichelHollands@users.noreply.github.com>	3 years ago
Periklis Tsirakidis	c4d2966fdf	operator: Remove general production use warning (#9598 )	3 years ago
Gerard Vanloo	7b706eea8c	operator: Set seccomp profile to runtime default for all variants (#9457 )	3 years ago
Tomasz Abramczyk	1b3f97fde4	Loki helm: Add Ingress labels for Loki gateway helm chart (#9395 ) Hello! I have small change suggestion. I think its too small to add documentation/changelog note, but if I should then just say and I can add. :) What this PR does / why we need it: it adds possibility to set labels for Loki gateway Ingress. Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [x] Documentation added - [ ] Tests updated - [x] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	3 years ago
Dylan Guedes	7d67b6306f	Loki: Implement initial phase of `limited_log_push_errors` (#9556 ) What this PR does / why we need it: Adds an initial implementation of `limited_log_push_errors`. This initial implementation encompass: - Runtime per-tenant configuration - Simple per-tenant rate-limiting based on error message size Notable features that will be added in future phases: - Instead of a single error with the final string, give to the manager the list of all entries - Once it supports per-entry error, make the rate-limiting separated per-reason - Hash the entry error and avoid repeating errors by caching seen errors in memory	3 years ago
Karsten Jeschkies	c9a3ff53e4	Set unique timestamp in lambda-promtail. (#9560 )	3 years ago

1 2 3 4 5 ...

5180 Commits (8abe080fd6d04dc5efb074ee0764a9bfd39c82d2) All Branches Search

5180 Commits (8abe080fd6d04dc5efb074ee0764a9bfd39c82d2)

All Branches