apitech/loki - loki - Gitea: Git with a cup of tea

Commit Graph

Author	SHA1	Message	Date
Pepe Cano	4cebc2d85c	Docs: replace `k6 Cloud` mention (#9599 ) k6 is now available as a managed service on Grafana Cloud. This is a small doc changes to remove the mention of `k6 Cloud`. --------- Co-authored-by: J Stickler <julie.stickler@grafana.com>	2 years ago
Danny Kopping	1db560fdc1	Adding background cache (en\|de)queue counters (#9665 ) What this PR does / why we need it: The background writeback cache exposes gauge metric currently for the current queue size. Gauges can be useful, but they are susceptible to sample errors because they only represent the point in time as the time of the scrape. Exposing counters for the bytes (en\|de)queued to/from the cache will be more useful because they can be aggregated. Signed-off-by: Danny Kopping <danny.kopping@grafana.com>	2 years ago
Dylan Guedes	609bc22933	Distributor: Make key configurable when logging failures (#9659 ) What this PR does / why we need it: Make appending `insight=true` key-value pair to log failures configurable. Which issue(s) this PR fixes: N/A	2 years ago
Nils Griebner	d58125885b	Make table manager retention options configurable in helm chart values (#9647 ) What this PR does / why we need it: Configuration options for table manager retention are hard-coded in the helm chart values at the moment so that it's not possible to enable retention deletes. Which issue(s) this PR fixes: Fixes #8676 Special notes for your reviewer: - Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [x] Documentation added - [x] Tests updated - [x] `CHANGELOG.md` updated - [x] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [x] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) Co-authored-by: Michel Hollands <42814411+MichelHollands@users.noreply.github.com>	2 years ago
Ed Welch	79c5fdd9c0	Canary: Refactor how the direct Canary push works (#9569 ) What this PR does / why we need it: When doing some testing a while back I ran into several cases were we didn't handle errors/retries properly in the code we added for the canaries to send directly to Loki. The changes here effectively duplicate the way promtail's client does sends to Loki which is code we've been using/relying on for years now. I also had a hard time following the previous implementation which used the io.Writer interface for swapping out writing to stdout vs pushing to Loki and refactored this to use a new interface `EntryWriter` which has implementations that write to stdout and to Loki Unfortunately the changes in the diff don't read super well with this refactoring so hopefully the explanation helps. Which issue(s) this PR fixes: Fixes #<issue number> Special notes for your reviewer: Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) --------- Signed-off-by: Edward Welch <edward.welch@grafana.com> Co-authored-by: Paul Rogers <129207811+paul1r@users.noreply.github.com>	2 years ago
Ivana Huckova	eb7dae4583	Loki: Improve error message when step too low (#9641 ) What this PR does / why we need it: In https://github.com/grafana/grafana/pull/69648 we are in Grafana introducing a step editor in Loki. Unfortunately, the error message when user sets too low step parameter is hard to understand, so I am proposing following change to make it more understandable and actionable. Let me know what do you think. --------- Co-authored-by: J Stickler <julie.stickler@grafana.com>	2 years ago
Christopher Piggott	415a997fb9	Update CONTRIBUTING.md to point to the new location of the contributi… (#9654 ) What this PR does / why we need it: Updating CONTRIBUTING document to point to the location of the LID template, which has moved Which issue(s) this PR fixes: none Special notes for your reviewer: doc change only Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`)	2 years ago
Alexandre	806cd73e53	Fix configmap checksum in read sts (#9642 ) What this PR does / why we need it: @NilsGriebner : complement to #9579 Which issue(s) this PR fixes: Fixes #<issue number> Special notes for your reviewer: Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [x] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) --------- Co-authored-by: Michel Hollands <42814411+MichelHollands@users.noreply.github.com>	2 years ago
Salva Corts	af287ac3eb	Add summary stats and metrics for stats cache (#9536 ) What this PR does / why we need it: When a query finishes, we return (and log) the following stats: ```go Cache.Chunk.Requests 0 Cache.Chunk.EntriesRequested 0 Cache.Chunk.EntriesFound 0 Cache.Chunk.EntriesStored 0 Cache.Chunk.BytesSent 0 B Cache.Chunk.BytesReceived 0 B Cache.Chunk.DownloadTime 0s Cache.Index.Requests 0 Cache.Index.EntriesRequested 0 Cache.Index.EntriesFound 0 Cache.Index.EntriesStored 0 Cache.Index.BytesSent 0 B Cache.Index.BytesReceived 0 B Cache.Index.DownloadTime 0s Cache.Result.Requests 13 Cache.Result.EntriesRequested 13 Cache.Result.EntriesFound 13 Cache.Result.EntriesStored 0 Cache.Result.BytesSent 0 B Cache.Result.BytesReceived 2.5 kB Cache.Result.DownloadTime 4.600266ms ``` In addition to that, we log the following in metrics.go: ``` level=info ts=2023-05-29T09:17:10.93029945Z caller=metrics.go:152 component=frontend org_id=145265 traceID=52d59b78fe6b9221 sampled=true latency=fast query="{cluster=\"dev-us-central-0\", namespace=~\"loki.\", container=~\"distributor\|ingester \|promtail\|index-gateway\|compactor\"} \|= \"thislinewillnotexist\"" query_hash=1194136170 query_type=filter range_type=range length=3h0m0s start_delta=165h37m24.930289434s end_delta=162h37m24.930289612s step=43s duration=2.473055ms status=200 lim it=30 returned_lines=0 throughput=0B total_bytes=0B lines_per_second=0 total_lines=0 total_entries=0 store_chunks_download_time=0s queue_time=0s splits=13 shards=0 cache_chunk_req=0 cache_chunk_hit=0 cache_chunk_bytes_stored=0 cache_chunk_bytes _fetched=0 cache_chunk_download_time=0s cache_index_req=0 cache_index_hit=0 cache_index_download_time=0s cache_result_req=13 cache_result_hit=13 cache_result_download_time=4.600266ms ``` With the goal of being able to better monitor how the stats cache is performing; this PR adds stats for the index stats cache, similarly to how it's done for the results cache. Here's an example of the new stats being returned and printed: ```go ... Cache.StatsResult.Requests 180 Cache.StatsResult.EntriesRequested 129 Cache.StatsResult.EntriesFound 129 Cache.StatsResult.EntriesStored 51 Cache.StatsResult.BytesSent 0 B Cache.StatsResult.BytesReceived 75 kB ... ``` And the new stats from metrics.go ``` ... caller=metrics.go:155 ... cache_stats_results_req=129 cache_stats_results_hit=129 cache_stats_results_download_ti me=156.864429ms ... ``` Special notes for your reviewer: - Blocked by https://github.com/grafana/loki/pull/9535 - Note the new`stats.GetOrCreateContext` func. It's used inside the `query.Exec` method so we don't overwrite the stats added in the stats middleware. Checklist* - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [x] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`)	2 years ago
Julien Pinsonneau	e6802394da	operator: Expose per_stream_rate_limit & burst (#9630 ) Co-authored-by: Periklis Tsirakidis <periklis@redhat.com>	2 years ago
Salva Corts	1694ad0f9b	Stats cache can be configured independently (#9535 ) What this PR does / why we need it: Before this PR, the index stats cache would use the same config as the query results cache. This was a limitation since: 1. We would not be able to point to a different cache for storing the index stats if needed. 2. We would not be able to add specific settings for this cache, without adding it to the results cache. In this PR, we refactor the index stats cache config to be independently configurable. Note that if it's not configured, it will try to use the results cache settings. Which issue(s) this PR fixes: This is needed for: - https://github.com/grafana/loki/pull/9537 - https://github.com/grafana/loki/pull/9536 Special notes for your reviewer: - This PR also refactors all the tripperwares in rountrip.go to reuse the same stats tripperware instead of each one creating their own. - Configuring a new cache in rountrip.go is a requirement for https://github.com/grafana/loki/pull/9536 so the stats summary can distinguish before the stats cache and the results cache. Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [x] Documentation added - [x] Tests updated - [x] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`)	2 years ago
Vladyslav Diachenko	3c52034d63	[logcli] fixed query command that is run against user's bucket with TSDB index enabled (#9597 ) What this PR does / why we need it: set TSDB shipper mode to ReadOnly and disabled indexGatewayClient during local query run. Also, I increase index downloading timeout to 1m because the users run the query from local machines that not always have more than 100Mbps Which issue(s) this PR fixes: Fixes #9555 Special notes for your reviewer: Unfortunately, we do not have similar tests and test architecture to cover it with unit tests properly. Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [x] `CHANGELOG.md` updated - [x] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [x] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`)	2 years ago
Periklis Tsirakidis	f43dd58459	operator: Fix timeout config constructor when only tenants limits (#9623 )	2 years ago
Nils Griebner	2c7309e0f7	Fix configmap checksum in table manager deployment template (#9579 ) What this PR does / why we need it: This PR fixes the path to the configmap template in table-manager deployment file Which issue(s) this PR fixes: Fixes #9578 Special notes for your reviewer: This is probably the same issue as in #9546. Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [x] Documentation added - [x] Tests updated - [x] `CHANGELOG.md` updated - [x] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [x] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`)	2 years ago
Sandeep Sukhani	0a5e149ea5	query-scheduler: fix query distribution in SSD mode (#9471 ) What this PR does / why we need it: When we run the `query-scheduler` in `ring` mode, `queriers` and `query-frontend` discover the available `query-scheduler` instances using the ring. However, we have a problem when `query-schedulers` are not running in the same process as queriers and query-frontend since [we try to get the ring client interface from the scheduler instance](`abd6131bba/pkg/loki/modules.go (L358)`). This causes queries not to be spread across all the available queriers when running in SSD mode because [we point querier workers to query frontend when there is no ring client and scheduler address configured](`b05f4fced3/pkg/querier/worker_service.go (L115)`). I have fixed this issue by adding a new hidden target to initialize the ring client in `reader`/`member` mode based on which service is initializing it. `reader` mode will be used by `queriers` and `query-frontend` for discovering `query-scheduler` instances from the ring. `member` mode will be used by `query-schedulers` for registering themselves in the ring. I have also made a couple of changes not directly related to the issue but it fixes some problems: * [reset metric registry for each integration test](`18c4fe5907`) - Previously we were reusing the same registry for all the tests and just [ignored the attempts to register same metrics](`01f0ded7fc/integration/cluster/cluster.go (L113)`). This causes the registry to have metrics registered only from the first test so any updates from subsequent tests won't reflect in the metrics. metrics was the only reliable way for me to verify that `query-schedulers` were connected to `queriers` and `query-frontend` when running in ring mode in the integration test that I added to test my changes. This should also help with other tests where earlier it was hard to reliably check the metrics. * [load config from cli as well before applying dynamic config](`f9e2448fc7`) - Previously we were applying dynamic config considering just the config from config file. This results in unexpected config changes, for example, [this config change](`4148dd2c51/integration/loki_micro_services_test.go (L66)`) was getting ignored and [dynamic config tuning was unexpectedly turning on ring mode](`52cd0a39b8/pkg/loki/config_wrapper.go (L94)`) in the config. It is better to do any config tuning based on both file and cli args configs. Which issue(s) this PR fixes: Fixes #9195	2 years ago
dependabot[bot]	90ed037a78	Bump golangci/golangci-lint-action from 3.4.0 to 3.5.0 (#9627 ) Bumps [golangci/golangci-lint-action](https://github.com/golangci/golangci-lint-action) from 3.4.0 to 3.5.0. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/golangci/golangci-lint-action/releases">golangci/golangci-lint-action's releases</a>.</em></p> <blockquote> <h2>v3.5.0</h2> <h2>What's Changed</h2> <ul> <li>build(deps-dev): bump eslint from 8.32.0 to 8.33.0 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/golangci/golangci-lint-action/pull/659">golangci/golangci-lint-action#659</a></li> <li>build(deps-dev): bump <code>@typescript-eslint/eslint-plugin</code> from 5.48.2 to 5.49.0 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/golangci/golangci-lint-action/pull/661">golangci/golangci-lint-action#661</a></li> <li>build(deps-dev): bump eslint-plugin-simple-import-sort from 9.0.0 to 10.0.0 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/golangci/golangci-lint-action/pull/662">golangci/golangci-lint-action#662</a></li> <li>build(deps-dev): bump <code>@vercel/ncc</code> from 0.36.0 to 0.36.1 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/golangci/golangci-lint-action/pull/660">golangci/golangci-lint-action#660</a></li> <li>build(deps-dev): bump <code>@typescript-eslint/parser</code> from 5.48.2 to 5.49.0 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/golangci/golangci-lint-action/pull/663">golangci/golangci-lint-action#663</a></li> <li>build(deps-dev): bump <code>@typescript-eslint/eslint-plugin</code> from 5.49.0 to 5.50.0 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/golangci/golangci-lint-action/pull/665">golangci/golangci-lint-action#665</a></li> <li>build(deps-dev): bump <code>@typescript-eslint/parser</code> from 5.49.0 to 5.50.0 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/golangci/golangci-lint-action/pull/666">golangci/golangci-lint-action#666</a></li> <li>build(deps-dev): bump typescript from 4.9.4 to 4.9.5 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/golangci/golangci-lint-action/pull/667">golangci/golangci-lint-action#667</a></li> <li>build(deps): bump <code>@types/node</code> from 18.11.18 to 18.11.19 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/golangci/golangci-lint-action/pull/668">golangci/golangci-lint-action#668</a></li> <li>doc: add quote aroung go version by <a href="https://github.com/vaughany"><code>@vaughany</code></a> in <a href="https://redirect.github.com/golangci/golangci-lint-action/pull/670">golangci/golangci-lint-action#670</a></li> <li>build(deps-dev): bump <code>@typescript-eslint/parser</code> from 5.50.0 to 5.51.0 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/golangci/golangci-lint-action/pull/671">golangci/golangci-lint-action#671</a></li> <li>build(deps-dev): bump prettier from 2.8.3 to 2.8.4 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/golangci/golangci-lint-action/pull/673">golangci/golangci-lint-action#673</a></li> <li>build(deps): bump <code>@types/node</code> from 18.11.19 to 18.13.0 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/golangci/golangci-lint-action/pull/674">golangci/golangci-lint-action#674</a></li> <li>build(deps-dev): bump <code>@typescript-eslint/eslint-plugin</code> from 5.50.0 to 5.51.0 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/golangci/golangci-lint-action/pull/675">golangci/golangci-lint-action#675</a></li> <li>build(deps): bump <code>@actions/cache</code> from 3.1.2 to 3.1.3 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/golangci/golangci-lint-action/pull/672">golangci/golangci-lint-action#672</a></li> <li>build(deps-dev): bump <code>@typescript-eslint/parser</code> from 5.51.0 to 5.52.0 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/golangci/golangci-lint-action/pull/678">golangci/golangci-lint-action#678</a></li> <li>build(deps): bump <code>@types/node</code> from 18.13.0 to 18.14.0 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/golangci/golangci-lint-action/pull/679">golangci/golangci-lint-action#679</a></li> <li>build(deps-dev): bump eslint from 8.33.0 to 8.34.0 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/golangci/golangci-lint-action/pull/680">golangci/golangci-lint-action#680</a></li> <li>build(deps-dev): bump <code>@typescript-eslint/eslint-plugin</code> from 5.51.0 to 5.52.0 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/golangci/golangci-lint-action/pull/681">golangci/golangci-lint-action#681</a></li> <li>build(deps-dev): bump <code>@typescript-eslint/eslint-plugin</code> from 5.52.0 to 5.53.0 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/golangci/golangci-lint-action/pull/684">golangci/golangci-lint-action#684</a></li> <li>build(deps-dev): bump eslint from 8.34.0 to 8.35.0 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/golangci/golangci-lint-action/pull/685">golangci/golangci-lint-action#685</a></li> <li>build(deps): bump <code>@types/node</code> from 18.14.0 to 18.14.2 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/golangci/golangci-lint-action/pull/688">golangci/golangci-lint-action#688</a></li> <li>build(deps-dev): bump <code>@typescript-eslint/parser</code> from 5.52.0 to 5.53.0 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/golangci/golangci-lint-action/pull/687">golangci/golangci-lint-action#687</a></li> <li>build(deps): bump <code>@actions/cache</code> from 3.1.3 to 3.1.4 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/golangci/golangci-lint-action/pull/686">golangci/golangci-lint-action#686</a></li> <li>build(deps): bump <code>@types/node</code> from 18.14.2 to 18.14.6 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/golangci/golangci-lint-action/pull/691">golangci/golangci-lint-action#691</a></li> <li>build(deps-dev): bump <code>@typescript-eslint/parser</code> from 5.53.0 to 5.54.0 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/golangci/golangci-lint-action/pull/692">golangci/golangci-lint-action#692</a></li> <li>build(deps-dev): bump <code>@typescript-eslint/eslint-plugin</code> from 5.53.0 to 5.54.0 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/golangci/golangci-lint-action/pull/693">golangci/golangci-lint-action#693</a></li> <li>build(deps-dev): bump eslint-config-prettier from 8.6.0 to 8.7.0 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/golangci/golangci-lint-action/pull/694">golangci/golangci-lint-action#694</a></li> <li>build(deps-dev): bump eslint from 8.35.0 to 8.36.0 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/golangci/golangci-lint-action/pull/699">golangci/golangci-lint-action#699</a></li> <li>build(deps-dev): bump <code>@typescript-eslint/eslint-plugin</code> from 5.54.0 to 5.54.1 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/golangci/golangci-lint-action/pull/700">golangci/golangci-lint-action#700</a></li> <li>build(deps): bump <code>@types/node</code> from 18.14.6 to 18.15.1 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/golangci/golangci-lint-action/pull/701">golangci/golangci-lint-action#701</a></li> <li>docs/build: update to setup-go@v4 by <a href="https://github.com/caarlos0"><code>@caarlos0</code></a> in <a href="https://redirect.github.com/golangci/golangci-lint-action/pull/704">golangci/golangci-lint-action#704</a></li> <li>build(deps-dev): bump typescript from 4.9.5 to 5.0.2 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/golangci/golangci-lint-action/pull/705">golangci/golangci-lint-action#705</a></li> <li>build(deps): bump <code>@types/node</code> from 18.15.1 to 18.15.3 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/golangci/golangci-lint-action/pull/706">golangci/golangci-lint-action#706</a></li> <li>build(deps): bump <code>@actions/http-client</code> from 2.0.1 to 2.1.0 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/golangci/golangci-lint-action/pull/697">golangci/golangci-lint-action#697</a></li> <li>build(deps-dev): bump prettier from 2.8.4 to 2.8.5 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/golangci/golangci-lint-action/pull/707">golangci/golangci-lint-action#707</a></li> <li>build(deps): bump <code>@actions/cache</code> from 3.1.4 to 3.2.1 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/golangci/golangci-lint-action/pull/698">golangci/golangci-lint-action#698</a></li> <li>build(deps-dev): bump eslint-config-prettier from 8.7.0 to 8.8.0 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/golangci/golangci-lint-action/pull/709">golangci/golangci-lint-action#709</a></li> <li>build(deps): bump <code>@types/node</code> from 18.15.3 to 18.15.10 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/golangci/golangci-lint-action/pull/710">golangci/golangci-lint-action#710</a></li> <li>build(deps-dev): bump prettier from 2.8.5 to 2.8.7 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/golangci/golangci-lint-action/pull/711">golangci/golangci-lint-action#711</a></li> <li>build(deps-dev): bump <code>@typescript-eslint/eslint-plugin</code> from 5.54.1 to 5.56.0 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/golangci/golangci-lint-action/pull/712">golangci/golangci-lint-action#712</a></li> <li>build(deps-dev): bump <code>@typescript-eslint/parser</code> from 5.54.0 to 5.56.0 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/golangci/golangci-lint-action/pull/713">golangci/golangci-lint-action#713</a></li> <li>build(deps-dev): bump <code>@typescript-eslint/eslint-plugin</code> from 5.56.0 to 5.57.0 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/golangci/golangci-lint-action/pull/718">golangci/golangci-lint-action#718</a></li> <li>build(deps-dev): bump typescript from 5.0.2 to 5.0.3 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/golangci/golangci-lint-action/pull/717">golangci/golangci-lint-action#717</a></li> <li>build(deps-dev): bump eslint from 8.36.0 to 8.37.0 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/golangci/golangci-lint-action/pull/719">golangci/golangci-lint-action#719</a></li> <li>build(deps-dev): bump <code>@typescript-eslint/parser</code> from 5.56.0 to 5.57.0 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/golangci/golangci-lint-action/pull/720">golangci/golangci-lint-action#720</a></li> <li>build(deps): bump <code>@types/node</code> from 18.15.10 to 18.15.11 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/golangci/golangci-lint-action/pull/721">golangci/golangci-lint-action#721</a></li> <li>build(deps-dev): bump <code>@typescript-eslint/eslint-plugin</code> from 5.57.0 to 5.57.1 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/golangci/golangci-lint-action/pull/722">golangci/golangci-lint-action#722</a></li> </ul> <!-- raw HTML omitted --> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Commits</summary> <ul> <li><a href="`5f1fec7010`"><code>5f1fec7</code></a> build(deps-dev): bump <code>@typescript-eslint/parser</code> from 5.59.6 to 5.59.7 (<a href="https://redirect.github.com/golangci/golangci-lint-action/issues/758">#758</a>)</li> <li><a href="`601007b788`"><code>601007b</code></a> build(deps): bump <code>@types/node</code> from 20.2.3 to 20.2.5 (<a href="https://redirect.github.com/golangci/golangci-lint-action/issues/756">#756</a>)</li> <li><a href="`d2a913e97b`"><code>d2a913e</code></a> build(deps-dev): bump <code>@typescript-eslint/eslint-plugin</code> from 5.59.6 to 5.59.7 ...</li> <li><a href="`7233bd71cb`"><code>7233bd7</code></a> build(deps-dev): bump <code>@typescript-eslint/eslint-plugin</code> from 5.59.5 to 5.59.6 ...</li> <li><a href="`687f029324`"><code>687f029</code></a> build(deps): bump <code>@types/node</code> from 20.1.4 to 20.2.3 (<a href="https://redirect.github.com/golangci/golangci-lint-action/issues/755">#755</a>)</li> <li><a href="`f9990cd216`"><code>f9990cd</code></a> build(deps-dev): bump <code>@typescript-eslint/parser</code> from 5.59.5 to 5.59.6 (<a href="https://redirect.github.com/golangci/golangci-lint-action/issues/754">#754</a>)</li> <li><a href="`f30aa514f9`"><code>f30aa51</code></a> build(deps-dev): bump eslint from 8.40.0 to 8.41.0 (<a href="https://redirect.github.com/golangci/golangci-lint-action/issues/753">#753</a>)</li> <li><a href="`6b21f586ed`"><code>6b21f58</code></a> build(deps-dev): bump <code>@typescript-eslint/eslint-plugin</code> from 5.59.2 to 5.59.5 ...</li> <li><a href="`535ed3a04b`"><code>535ed3a</code></a> build(deps): bump <code>@types/semver</code> from 7.3.13 to 7.5.0 (<a href="https://redirect.github.com/golangci/golangci-lint-action/issues/748">#748</a>)</li> <li><a href="`0078ef00ab`"><code>0078ef0</code></a> build(deps-dev): bump <code>@typescript-eslint/parser</code> from 5.59.1 to 5.59.5 (<a href="https://redirect.github.com/golangci/golangci-lint-action/issues/750">#750</a>)</li> <li>Additional commits viewable in <a href="https://github.com/golangci/golangci-lint-action/compare/v3.4.0...v3.5.0">compare view</a></li> </ul> </details> <br /> [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=golangci/golangci-lint-action&package-manager=github_actions&previous-version=3.4.0&new-version=3.5.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	2 years ago
Periklis Tsirakidis	69392de8b6	Fix duplicate label values from ingester streams (#9629 )	2 years ago
Eric	c6f809a738	Fix missing ~ in the regexp match for namespace (#9618 ) What this PR does / why we need it: Fix documentation which has a missing `~` for the regexp match. I was following the documentation and found it odd that this query didn't follow [LogQL](https://grafana.com/docs/loki/latest/logql/log_queries/#log-queries), but it actually does and was just a typo in the docs. I've tested this with my own Loki 2.8 release and with the `~` this works great! I also added a similar description block to the above example explaining the pipeline, but if this is not desired, I'm happy to remove it. Which issue(s) this PR fixes: NA, trivial issue, just a small fix in the docs. Special notes for your reviewer: Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [x] Documentation added - [x] Tests updated - [x] `CHANGELOG.md` updated - [x] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [x] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`)	2 years ago
W.T. Chang	abb70456e6	Fix comment and indentation in gateway config of SSD helm chart (#9622 ) What this PR does / why we need it: The gateway config (nginx config file) incorrectly states 10 minutes (as in 600 seconds) as 6 minutes. This small PR corrects that. Which issue(s) this PR fixes: N/A Special notes for your reviewer: Although this indeed modifies the chart, I don't think it's worth a version bump. Please let me know if you have other ideas. Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`)	2 years ago
Travis Patterson	6d8a184334	Revert "revert count unflused chunks (#9504 )" (#9615 ) Now that the bug where huge values is fixed and known not to be this. Let's readd it. Original: https://github.com/grafana/loki/pull/9479	2 years ago
Travis Patterson	1f119e1356	Prevent GetFactorOfTime from returning NaN (#9602 ) `GetFactorOfTime` is used to calculate overlaps with chunk time ranges -- particularly calls to `stats`. When a chunk has only one entry, it's `start time == end time` and this `GetFactorOfTime` returns `NaN`. The stats calculation is `uint64(total chunk bytes * factor)`. When `factor` is `NaN`: `uint64(total chunk bytes * factor) == uint64(math.Nan()) == 9223372036854775808` `uint64(math.Nan()) == 9223372036854775808` is an implementation detail of golang.	2 years ago
Bryan Boreham	f5872ec4d3	marshal tests: make random quantities more obvious (#9612 ) What this PR does / why we need it: Computing a new random number on every iteration of the loop is hard to reason about, and probably not what was meant. This PR will mean that much more data is generated than before; I computed the expected value before as 11.2, whereas now it is 49.5. Consider reducing the `100` if that is undesired. Checklist - NA Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - NA Documentation added - [x] Tests updated - NA `CHANGELOG.md` updated - NA Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - NA For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`)	2 years ago
Juhyeong Ko	a21842d740	Fix typo in log_queries docs (#9506 )	2 years ago
Salva Corts	87a659a6db	Add span events for index stats and result cache (#9552 ) What this PR does / why we need it: This PR adds events to the traces to have some extra observability for how we compute the index stats. We also add some trace events to the results cache. ![image](https://github.com/grafana/loki/assets/8354290/7566b755-8193-4e46-ba10-37d3377ea31a) ![image](https://github.com/grafana/loki/assets/8354290/d1990150-84b1-4522-9898-6e37c2782c5b) ![image](https://github.com/grafana/loki/assets/8354290/a8c23e7f-a06d-4a47-8cd4-e900fce01e80) ![image](https://github.com/grafana/loki/assets/8354290/d1e15fb6-fb6c-4fe1-9c5f-f1c8164889de) ![image](https://github.com/grafana/loki/assets/8354290/0c0d001e-7083-488c-8809-0446b4b7c852) Special notes for your reviewer: Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`)	2 years ago
Danny Kopping	a248e3e970	Querier: configurable writeback queue bytes size (#9604 ) What this PR does / why we need it: The background writeback process is used to write chunks that have been fetched from the store to cache. Chunks have a target size, but not all chunks will necessarily be this size, so it is tricky to find the correct value for `writeback_buffer` which specifies how many chunks will be held in the queue at any given time. For example: setting `writeback_buffer=10000` (the default) with all chunks being the default target size (`chunk_target_size=1.5MiB`), in the worst case could result in 15GiB of memory being used for this buffer. Practically speaking, the queue never grows so deep, but if the cache Loki is writing to becomes overwhelmed this could happen. Setting a value too low would result in many chunks not being written back to cache, which defeats the purpose. This PR introduces a configurable _byte size_ limit to the writeback cache, which defaults to 1GB in size. With this setting `writeback_size_limit` in place, we get the best of both worlds: a deep queue which can hold many small chunks or a shallow queue which can hold a few large ones without using too much memory so as to overwhelm the querier. If either the `writeback_buffer` or `writeback_size_limit` reaches capacity, any new chunks which are attempted to be added to the queue will be dropped. Two new metrics are added by this PR: - `loki_cache_background_queue_bytes`: gauge, measures the size of the queue in bytes - `loki_cache_dropped_background_writes_bytes_total`: counter, measures the volume of bytes dropped	2 years ago
Salva Corts	4d00e91caf	Fix cacheGen for index stats reqs (#9529 ) What this PR does / why we need it: When deletes are enabled, we track the cache generation number to know when to invalidate the cache. The queriers add this cache generation number to the response, and in the frontend, we check if we should cache the response or not based on such generation number. Before this PR, the queriers didn't add the cache generation number to the index stats responses, therefore, when deletes were enabled, we could see the following error message in the frontend when checking if we should store a new response in the cache: ``` we found results cache gen number %s set in store but none in headers ``` This PR fixes this by adding the middleware to add the generation number to the index stats handler in the querier. As seen in the following screenshot, once we rolled out the fixes in this PR (at 10:12), the number of failed cache generation comparisons decreased to 0. At the same time, the number of version comparisons also decreased since we could store results in the cache and therefore the cache effectiveness improved. ([source][1]) ![image](https://github.com/grafana/loki/assets/8354290/752fbc66-0005-437a-9469-bc25405198b9) Which issue(s) this PR fixes: Fixes #<issue number> Special notes for your reviewer: Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) [1]: https://admin-dev-us-central-0.grafana-dev.net/grafana/explore?orgId=1&left=%7B%22datasource%22:%22P09658BD21492D183%22,%22queries%22:%5B%7B%22refId%22:%22G%22,%22expr%22:%22sum%28%5Cn%20%20%20%20rate%28%5Cn%20%20%20%20%20%20%20%20loki_results_cache_version_comparisons_failed%7Bcluster%3D%5C%22dev-us-central-0%5C%22,%20namespace%3D%5C%22loki-dev-005%5C%22%7D%5Cn%20%20%20%20%20%20%20%20%5B1m%5D%5Cn%20%20%20%20%29%5Cn%29%22,%22range%22:true,%22datasource%22:%7B%22type%22:%22prometheus%22,%22uid%22:%22P09658BD21492D183%22%7D,%22editorMode%22:%22code%22,%22legendFormat%22:%22Gen%20cmp%20fail%22%7D,%7B%22refId%22:%22A%22,%22expr%22:%22sum%28%5Cn%20%20%20%20rate%28%5Cn%20%20%20%20%20%20%20%20loki_results_cache_version_comparisons_total%7Bcluster%3D%5C%22dev-us-central-0%5C%22,%20namespace%3D%5C%22loki-dev-005%5C%22%7D%5Cn%20%20%20%20%20%20%20%20%5B1m%5D%5Cn%20%20%20%20%29%5Cn%29%22,%22range%22:true,%22instant%22:true,%22datasource%22:%7B%22type%22:%22prometheus%22,%22uid%22:%22P09658BD21492D183%22%7D,%22editorMode%22:%22code%22,%22legendFormat%22:%22Gen%20cpm%20total%22%7D%5D,%22range%22:%7B%22from%22:%221685094941101%22,%22to%22:%221685096865685%22%7D%7D	2 years ago
FL42	3fc9328862	Fix: HPA for single binary deployment (#9585 ) What this PR does / why we need it: Add HPA for single binary deployment. Off by default. Which issue(s) this PR fixes: Fixes #7097 Special notes for your reviewer: Checklist - [X] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [N/A] Documentation added - [N/A] Tests updated - [X] `CHANGELOG.md` updated - [N/A] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [X] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) --------- Co-authored-by: Michel Hollands <42814411+MichelHollands@users.noreply.github.com>	2 years ago
Periklis Tsirakidis	c4d2966fdf	operator: Remove general production use warning (#9598 )	2 years ago
Gerard Vanloo	7b706eea8c	operator: Set seccomp profile to runtime default for all variants (#9457 )	2 years ago
Tomasz Abramczyk	1b3f97fde4	Loki helm: Add Ingress labels for Loki gateway helm chart (#9395 ) Hello! I have small change suggestion. I think its too small to add documentation/changelog note, but if I should then just say and I can add. :) What this PR does / why we need it: it adds possibility to set labels for Loki gateway Ingress. Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [x] Documentation added - [ ] Tests updated - [x] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	2 years ago
Dylan Guedes	7d67b6306f	Loki: Implement initial phase of `limited_log_push_errors` (#9556 ) What this PR does / why we need it: Adds an initial implementation of `limited_log_push_errors`. This initial implementation encompass: - Runtime per-tenant configuration - Simple per-tenant rate-limiting based on error message size Notable features that will be added in future phases: - Instead of a single error with the final string, give to the manager the list of all entries - Once it supports per-entry error, make the rate-limiting separated per-reason - Hash the entry error and avoid repeating errors by caching seen errors in memory	2 years ago
Karsten Jeschkies	c9a3ff53e4	Set unique timestamp in lambda-promtail. (#9560 )	2 years ago
Travis Patterson	fd0efe0799	Regex line/label filters return original regex when `String` is called (#9584 ) To optimize regex line and label filters, Loki rewrites them to be non-greedy. This can cause extra characters to appear in the regex when `String()` is called. In addition to confusing logging, this can also cause internal panics when a previously-parsable query becomes too long. This PR stores the original regex in the label/line filter and uses that in calls to `String`. This PR also removes regex optimizations from label selectors. Making the optimization at parse time caused the original regex to be lost.	2 years ago
Philip Gough	20752d20d0	operator: Update snippet to generate Secret for openshift-logging (#9586 ) Signed-off-by: Philip Gough <philip.p.gough@gmail.com>	2 years ago
Joshua Hesketh	033b504aa7	Fix marshalling ByteSize (#9570 ) When marshalling to yaml or json, expect the value not a pointer. Co-authored-by: Michel Hollands <42814411+MichelHollands@users.noreply.github.com>	2 years ago
Michel Hollands	8a9f519d82	Fix version number of helm chart (#9576 ) What this PR does / why we need it: There were 2 commits for the helm chart with the same version number. In this case the build scripts only releases 1 new helm chart, containing the first commit. This PR adds a new version so a helm chart will be created that contains the later PR. Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [X] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) Signed-off-by: Michel Hollands <michel.hollands@grafana.com>	2 years ago
Dylan Guedes	3941767c6f	Loki: Introduce new `limited_log_push_errors`. (#9568 ) What this PR does / why we need it: Introduce a new configuration named `limited_log_push_errors` that will be implemented. The configuration is being introduced on a separated PR to facilitate future rollbacks. Which issue(s) this PR fixes: N/A	2 years ago
Bryan Boreham	0a2f58891c	logql tests: use abstractions over Labels (#9533 ) What this PR does / why we need it: Call functions like `FromStrings`, so we don't have to know how the data structure is laid out. Special notes for your reviewer: This PR only touches tests. Upstream Prometheus is using a different implementation of `labels.Labels`, so I plan to create a series of PRs which make Loki compatible with that. #9532 was the first. Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - NA Documentation added - [x] Tests updated - NA `CHANGELOG.md` updated - NA Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - NA For Helm chart changes bump the Helm chart version. --------- Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2 years ago
Bryan Boreham	ee74fbdf3a	logql/log tests: use abstractions over labels (#9532 ) What this PR does / why we need it: Call functions like `FromStrings`, so we don't have to know how the data structure is laid out. `FromStrings` sorts labels, so we don't have to call `Sort()`. Special notes for your reviewer: This PR only touches tests. Upstream Prometheus is using a different implementation of `labels.Labels`, so I plan to create a series of PRs which make Loki compatible with that. Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - NA Documentation added - [x] Tests updated - NA `CHANGELOG.md` updated - NA Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - NA For Helm chart changes bump the Helm chart version. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2 years ago
Andre Bossi	aad436b799	Helm: Fix single binary checksum annotation for config.yaml (#9546 ) What this PR does / why we need it: This PR fix name to config checksum annotation Which issue(s) this PR fixes: Fixes #9545 Special notes for your reviewer: Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [x] Documentation added - [x] Tests updated - [x] `CHANGELOG.md` updated - [x] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [x] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`)	2 years ago
oleksii-boiko-ua	558cebf7a2	helm: fix backend hpa missing parameter (#9548 ) What this PR does / why we need it: Which issue(s) this PR fixes: Fixes #9547 Special notes for your reviewer: Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [x] Documentation added - [ ] Tests updated - [x] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [x] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) Co-authored-by: Michel Hollands <42814411+MichelHollands@users.noreply.github.com>	2 years ago
Christophe Collot	b709b32d6a	feat(lambda-promtail): add cloudtrail log ingestion support (#9497 ) What this PR does / why we need it: ### Add support for AWS CloudTrail audit logs ingestion using lambda-promtail Calls to AWS APIs are logged in AWS Cloudtrail and are helpful for security and debugging purposes. However, I've experienced difficulties with it: + The AWS CloudTrail service is not well integrated with Prometheus (no metrics, no alerts) and I don't want to manage alerts in CloudWatch Alerts + The search experience is painful with CloudTrail via the AWS Console (I will not elaborate 😅). This PR allows ingesting CloudTrail audit logs sent to an S3 bucket using the same approach as VPC flow logs or Load Balancer logs. Special notes for your reviewer: + Because the Cloudtrail file format is not text but json, we stream the json CloudTrail records instead of using the already existing scanner. + Because the Cloudtrail filename format is not the same as for the Flow log or the Load balancer log files, we need to split the regexes by service (although many AWS services seem to share the same `defaultFilenameRegex`). + In the `getLabels` function, we expect the `type` parameter to be found in the filename using the Regex. For some log files (ex: Cloudfront log files). The file name has no reference to the service name. This is why, as a default, when no type is found in the name of the file, I set it to use the key of the matching Regex expression. Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [x] Documentation added - [x] Tests updated - [x] `CHANGELOG.md` updated - [x] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	2 years ago
Irapuan Bottosso	a7976b584f	Fix link to API docs (#9517 ) What this PR does / why we need it: Fix the link to the API docs. Special notes for your reviewer: Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [x] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) Co-authored-by: Irapuan Bottosso <irapuan@Irapuans-MacBook-Pro-2.local>	2 years ago
Karsten Jeschkies	2830ec3791	Reuse allocated buffer space in merge iterator. (#9538 ) What this PR does / why we need it: We saw a performance decrease when running the merge iterator with a very small pool of ingesters. The reason was that we would allocate a new buffer with each next call. Special notes for your reviewer: `main` is missing `merge_sort_dedupe-16` so I ran it once with a small patch to generate the data. ```goos: linux goarch: amd64 pkg: github.com/grafana/loki/pkg/iter cpu: AMD Ryzen 7 3700X 8-Core Processor │ mergeiter-old.txt │ mergeiter-new.txt │ │ sec/op │ sec/op vs base │ SortIterator/merge_sort-16 1.350m ± 4% 1.368m ± 4% ~ (p=0.579 n=10) SortIterator/merge_sort_dedupe-16 2.566m ± 4% 2.462m ± 5% ~ (p=0.105 n=10) SortIterator/sort-16 1.034m ± 6% 1.029m ± 6% ~ (p=0.684 n=10) SortSampleIterator/merge-16 1.877m ± 2% 1.857m ± 2% ~ (p=0.075 n=10) SortSampleIterator/sort-16 432.7µ ± 6% 436.0µ ± 4% ~ (p=0.579 n=10) geomean 1.238m 1.229m -0.72% │ mergeiter-old.txt │ mergeiter-new.txt │ │ B/op │ B/op vs base │ SortIterator/merge_sort-16 21.22Ki ± 0% 21.22Ki ± 0% ~ (p=1.000 n=10) SortIterator/merge_sort_dedupe-16 41.47Ki ± 0% 41.47Ki ± 0% ~ (p=1.000 n=10) ¹ SortIterator/sort-16 13.94Ki ± 0% 13.94Ki ± 0% ~ (p=1.000 n=10) ¹ geomean 23.06Ki 23.06Ki +0.00% ¹ all samples are equal │ mergeiter-old.txt │ mergeiter-new.txt │ │ allocs/op │ allocs/op vs base │ SortIterator/merge_sort-16 7.000 ± 0% 7.000 ± 0% ~ (p=1.000 n=10) ¹ SortIterator/merge_sort_dedupe-16 7.000 ± 0% 7.000 ± 0% ~ (p=1.000 n=10) ¹ SortIterator/sort-16 5.000 ± 0% 5.000 ± 0% ~ (p=1.000 n=10) ¹ geomean 6.257 6.257 +0.00% ¹ all samples are equal ``` Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`)	2 years ago
risson	d1fd6b7883	helm: gateway: also listen on IPv6 (#9510 ) What this PR does / why we need it: Currently the gateway only listens on IPv4. On IPv6-only clusters, this causes the readiness probe to fail, rendering the gateway unusable. Which issue(s) this PR fixes: No issue created Special notes for your reviewer: Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [x] `CHANGELOG.md` updated - [x] N/A Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>	2 years ago
Ryan Garcia	028c12f624	[Helm] configStorageType value support (#8526 ) What this PR does / why we need it: Adds `loki.configStorageType` & `loki.externalConfigSecretName` values and associated helpers and templates to support choosing either `ConfigMap` or `Secret` as config medium types. Which issue(s) this PR fixes: Fixes #8268 Special notes for your reviewer: Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [x] Documentation added - [ ] Tests updated - [x] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` --------- Signed-off-by: garcia.ryan <garcia.ryan@solute.us> Co-authored-by: Stefan Büringer <4662360+sbueringer@users.noreply.github.com> Co-authored-by: Sandeep Sukhani <sandeep.d.sukhani@gmail.com>	2 years ago
risson	41c19ec747	helm: support of annotations to deployments and statefulsets (#9509 ) What this PR does / why we need it: Allows annotating loki components with custom annotations. For instance: ```yaml loki: annotations: secret.reloader.stakater.com/reload: loki-aws-credentials ``` Which issue(s) this PR fixes: No issue Special notes for your reviewer: Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - Helm chart section is empty. - [x] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [x] N/A Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>	2 years ago
Bayan Taani	bb96326953	operator: Include runtime-config in compiling the SHA1 checksum (#9448 )	2 years ago
Michel Hollands	a70d092df6	Add help target to Makefile (#9531 ) What this PR does / why we need it: Add a help target to the Makefile. Only the main targets were added. ``` % make help Usage: make <target> Targets: help Display this help and any documented user-facing targets. Other undocumented targets may be present in the Makefile. all build all executables (loki, logcli, promtail, loki-canary) logcli build logcli executable logcli-debug build debug logcli executable logcli-image build logcli docker image loki build loki executable loki-debug build loki debug executable loki-canary build loki-canary executable helm-test run helm tests helm-lint run helm linter loki-querytee build loki-querytee executable promtail build promtail executable promtail-debug build debug promtail executable loki-mixin compile the loki mixin loki-mixin-check check the loki mixin is up to date lint run linters test run the unit tests clean clean the generated files docker-driver build the docker-driver executable fluent-bit-plugin build the fluent-bit plugin fluent-bit-image build the fluent-bit plugin docker image fluent-bit-push push the fluent-bit plugin docker image fluentd-plugin build the fluentd plugin fluentd-plugin-push push the fluentd plugin fluentd-image build the fluentd docker image fluentd-image-push push the fluentd docker image logstash-image build the logstash image logstash-push push the logstash image promtail-image build the promtail docker image promtail-debug-image build the promtail debug docker image loki-image build the loki docker image loki-debug-image build the debug loki docker image loki-canary-image build the loki canary docker image helm-test-image build the helm test image helm-test-push push the helm test image logql-analyzer-image build the LogQL Analyzer image logql-analyzer-push push the LogQL Analyzer image build-image build the docker build image build-image-push push the docker build image doc Generates the config file documentation check-doc Check the documentation files are up to date ``` Checklist - [X] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) Signed-off-by: Michel Hollands <michel.hollands@grafana.com>	2 years ago
BlakeFernandez	905d14bee7	Add Prometheus Link for scrape_configs (#9270 ) What this PR does / why we need it: Adds some context to the scrape configs section in Promtail Config Docs to ensure clarity around the connection to Prometheus Here is a link to a slack thread where the customer was confused on this: https://raintank-corp.slack.com/archives/C04KY6NJSCE/p1681761335754199 --------- Co-authored-by: Michel Hollands <42814411+MichelHollands@users.noreply.github.com> Co-authored-by: J Stickler <julie.stickler@grafana.com>	2 years ago

1 2 3 4 5 ...

5162 Commits (4cebc2d85cc7b22526f4ba5f5364a2a9ee15aba2) All Branches Search

5162 Commits (4cebc2d85cc7b22526f4ba5f5364a2a9ee15aba2)

All Branches