apitech/loki - loki - Gitea: Git with a cup of tea

Commit Graph

Author	SHA1	Message	Date
Travis Patterson	8c84ee8b7a	Fix stream sharding (#7393 ) Promtail doesn't add a stream hash to push requests. As a result, we weren't accounting for the current rate in our sharding calculation and only sharding on individual large pushes. This PR ensures that streams have a hash as they're ingested. It also add a couple of metrics that were useful in debugging this issue.	3 years ago
Danny Kopping	33bf36aa21	Promtail: flush logs on exit (#7400 ) Promtail is using the line-buffered logger (https://github.com/grafana/loki/pull/6954), and log messages were not being printed on exit	3 years ago
Vladyslav Diachenko	57c0c2cde2	[docs: logql-analyzer] changed url for main branch (#7378 ) pointed unreleased docs(`next`) to logql-analyzer that is built from main branch	3 years ago
MarNicGit	82c1d1276c	Update installation.md (#7388 ) What this PR does / why we need it: This PR attempts to clarify where someone can find the latest binary release of Promtail. I spent a good hour trying to find a download for Promtail before realizing it is included in the assets of each Loki release. Opening the releases page barely shows any mention of Promtail until you fold out the assets list. Special notes for your reviewer: This is a documentation change. Checklist - [x] Reviewed the `CONTRIBUTING.md` guide - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	3 years ago
Hervé Nicol	91299ab729	[helm loki] add support for custom analytics config (#7262 ) What this PR does / why we need it: I want to disable usage reporting to Grafana. On clusters that don't have access to Internet it generates error logs. And without this patch, the only way is to have the whole `loki.config` section in my custom values file. Checklist - [x] Reviewed the `CONTRIBUTING.md` guide - [x] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` Co-authored-by: Herve Nicol <12008875+hervenicol@users.noreply.github.com>	3 years ago
Travis Patterson	543ea78b7a	Add metrics to the distributor about stream sharding (#7390 ) This PR adds some more metrics to the steam sharding interactions from the Distributor. 1. `rate_store_stream_count`: The total number of streams 2. `rate_store_max_stream_shard_count`: The stream with the largest number of shards 3. `rate_store_max_stream_rate_bytes`: The largest stream seen by a distributor 4. `rate_store_refresh_duration_seconds`: A histogram of the latency to query all the ingesters and process their rates	3 years ago
Travis Patterson	5dc27e8064	Fix Flaky Distributor Test (#7391 ) Use eventually for async tests	3 years ago
Travis Patterson	3e7adb2c34	Measure Per-Stream rates (#7370 ) This PR implements the ability for ingesters to measure per-stream rates at 1/s resolution. It also fixes bugs in the kind of Limits used in the RateStore and the Ingester API	3 years ago
Dennis Szczepanski	4e61c60296	Loki: Renames metric 'loki_log_messages_total' to 'loki_internal_log_messages_total' (#7361 ) What this PR does / why we need it: Renames the metric 'loki_log_messages_total' to a more descriptive name. Keeps and deprecates the previously existing metric to not break functionality. Which issue(s) this PR fixes: Fixes #7088 Special notes for your reviewer: Checklist - [x] Reviewed the `CONTRIBUTING.md` guide - [x] Documentation added - [x] Tests updated - [x] `CHANGELOG.md` updated - [x] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	3 years ago
李国忠	fdbb06d123	add promtail reload changelog and doc (#7386 )	3 years ago
Bryan Boreham	74d206bf28	chunks: improve readability of compression benchmarks (#7246 ) What this PR does / why we need it: ### BenchmarkWrite Use Go runtime facilities to report MB/s for each operation and compression ratio. Before: ``` BenchmarkWrite/ordered-none-4 1018 1104885 ns/op 1431962 B/op 33 allocs/op BenchmarkWrite/ordered-gzip-4 12 87333201 ns/op 3516699 B/op 696 allocs/op BenchmarkWrite/ordered-lz4-64k-4 51 21483048 ns/op 3095117 B/op 649 allocs/op ``` After: ``` BenchmarkWrite/ordered-none-4 841 1202007 ns/op 1054353.53 MB/s 101.9 %compressed 1569953 B/op 34 allocs/op BenchmarkWrite/ordered-gzip-4 13 86299699 ns/op 3357.25 MB/s 6.891 %compressed 3778896 B/op 702 allocs/op BenchmarkWrite/ordered-lz4-64k-4 49 22332214 ns/op 34107.61 MB/s 9.880 %compressed 3407522 B/op 661 allocs/op ``` Stop collecting all compressed chunks, as this blows up the memory of the benchmark and creates unrealistic test conditions for the later encoding. Before: `4724260 maxresident KB`; after: `375804 maxresident KB` ### BenchmarkRead Replace reporting of MB/s which gave several values for each encoding as Go hunted for the right number of benchmark iterations. Add stats to the context for decoding, otherwise a new stats object is created each time round the loop. Re-order so all unsampled results come before all sampled results. Before: ``` BenchmarkRead/none_66_kB-4 278 4555157 ns/op 131744 B/op 717 allocs/op BenchmarkRead/sample_none_66_kB-4 142 8238861 ns/op 129771 B/op 717 allocs/op BenchmarkRead/gzip_66_kB-4 6 179445187 ns/op 2100498 B/op 15010 allocs/op BenchmarkRead/sample_gzip_66_kB-4 4 261040057 ns/op 2076030 B/op 15028 allocs/op BenchmarkRead/lz4-64k_66_kB-4 19 62855240 ns/op 1722979 B/op 14219 allocs/op BenchmarkRead/sample_lz4-64k_66_kB-4 10 115577796 ns/op 1706040 B/op 14220 allocs/op BenchmarkRead/lz4-256k_66_kB-4 18 66144317 ns/op 1880851 B/op 15078 allocs/op BenchmarkRead/sample_lz4-256k_66_kB-4 9 118405993 ns/op 1800396 B/op 15077 allocs/op BenchmarkRead/lz4-1M_66_kB-4 18 67832189 ns/op 1821806 B/op 15076 allocs/op ... none_524 kB: 1828.92 MB/s none_524 kB: 1809.33 MB/s none_262 kB: 1790.79 MB/s none_66 kB: 1673.84 MB/s ... gzip_66 kB: 647.35 MB/s ... ``` After: ``` BenchmarkRead/none_66_kB-4 286 4377585 ns/op 1725.23 MB/s 41896 B/op 600 allocs/op BenchmarkRead/gzip_66_kB-4 6 180770638 ns/op 655.82 MB/s 723426 B/op 13265 allocs/op BenchmarkRead/lz4-64k_66_kB-4 19 65380524 ns/op 1296.10 MB/s 727571 B/op 12925 allocs/op BenchmarkRead/lz4-256k_66_kB-4 16 63737292 ns/op 1409.75 MB/s 786994 B/op 13707 allocs/op BenchmarkRead/lz4-1M_66_kB-4 18 65963825 ns/op 1362.17 MB/s 770441 B/op 13707 allocs/op BenchmarkRead/lz4_66_kB-4 18 64955415 ns/op 1383.31 MB/s 1003277 B/op 13707 allocs/op BenchmarkRead/snappy_66_kB-4 20 51588959 ns/op 1273.38 MB/s 361872 B/op 5020 allocs/op BenchmarkRead/flate_66_kB-4 6 172147502 ns/op 691.74 MB/s 715706 B/op 13242 allocs/op BenchmarkRead/zstd_66_kB-4 3 359473273 ns/op 390.07 MB/s 427247248 B/op 18551 allocs/op ... BenchmarkRead/sample_none_66_kB-4 135 8533283 ns/op 885.04 MB/s 39866 B/op 600 allocs/op BenchmarkRead/sample_gzip_66_kB-4 5 243016949 ns/op 487.84 MB/s 686358 B/op 13212 allocs/op BenchmarkRead/sample_lz4-64k_66_kB-4 10 114330032 ns/op 741.18 MB/s 707104 B/op 12926 allocs/op BenchmarkRead/sample_lz4-256k_66_kB-4 9 117294928 ns/op 766.05 MB/s 778461 B/op 13709 allocs/op ... ``` Note "MB" in previous code was 2^^20 while Go uses 10^6. Checklist - [x] Reviewed the `CONTRIBUTING.md` guide - NA Documentation added - [x] Tests updated - NA `CHANGELOG.md` updated - not user-facing - NA Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	3 years ago
李国忠	fb26baa5b1	[new feature] promtail: Add config reload endoint / signal to promtail (#7247 )	3 years ago
Brandon W Maister	381735d3d4	logging: Change Docker target discovery log level from Error to Info (#7312 ) What this PR does / why we need it: Discovering a new node is not an error, so having it send errors to dashboards is confusing. Special notes for your reviewer: Checklist - [x] Reviewed the `CONTRIBUTING.md` guide - [x] Documentation added - [ ] Tests updated - [x] `CHANGELOG.md` updated - [x] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	3 years ago
Paul Tobias	fe17feeda5	promtail/scraping.md: Fix 404 (#7317 ) What this PR does / why we need it: Currently the `Cloudflare` link refers to: https://grafana.com/docs/loki/latest/clients/configuration/#cloudflare But it should be: https://grafana.com/docs/loki/latest/clients/promtail/configuration/#cloudflare Checklist - [X] Reviewed the `CONTRIBUTING.md` guide - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	3 years ago
dependabot[bot]	c8632e6329	Bump actions/checkout from 2.4.0 to 3.1.0 (#7376 ) Bumps [actions/checkout](https://github.com/actions/checkout) from 2.4.0 to 3.1.0. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/actions/checkout/releases">actions/checkout's releases</a>.</em></p> <blockquote> <h2>v3.1.0</h2> <h2>What's Changed</h2> <ul> <li>Inject GitHub host to be able to clone from another GitHub instance by <a href="https://github.com/peter-murray"><code>@peter-murray</code></a> in <a href="https://github-redirect.dependabot.com/actions/checkout/pull/922">actions/checkout#922</a></li> <li>Bump <code>@actions/core</code> to 1.10.0 by <a href="https://github.com/rentziass"><code>@rentziass</code></a> in <a href="https://github-redirect.dependabot.com/actions/checkout/pull/939">actions/checkout#939</a></li> </ul> <h2>New Contributors</h2> <ul> <li><a href="https://github.com/peter-murray"><code>@peter-murray</code></a> made their first contribution in <a href="https://github-redirect.dependabot.com/actions/checkout/pull/922">actions/checkout#922</a></li> <li><a href="https://github.com/rentziass"><code>@rentziass</code></a> made their first contribution in <a href="https://github-redirect.dependabot.com/actions/checkout/pull/939">actions/checkout#939</a></li> </ul> <p><strong>Full Changelog</strong>: <a href="https://github.com/actions/checkout/compare/v3.0.2...v3.1.0">https://github.com/actions/checkout/compare/v3.0.2...v3.1.0</a></p> <h2>v3.0.2</h2> <h2>What's Changed</h2> <ul> <li>Add set-safe-directory input to allow customers to take control. by <a href="https://github.com/TingluoHuang"><code>@TingluoHuang</code></a> in <a href="https://github-redirect.dependabot.com/actions/checkout/pull/770">actions/checkout#770</a></li> <li>Prepare changelog for v3.0.2. by <a href="https://github.com/TingluoHuang"><code>@TingluoHuang</code></a> in <a href="https://github-redirect.dependabot.com/actions/checkout/pull/777">actions/checkout#777</a></li> </ul> <p><strong>Full Changelog</strong>: <a href="https://github.com/actions/checkout/compare/v3...v3.0.2">https://github.com/actions/checkout/compare/v3...v3.0.2</a></p> <h2>v3.0.1</h2> <ul> <li><a href="https://github-redirect.dependabot.com/actions/checkout/pull/762">Fixed an issue where checkout failed to run in container jobs due to the new git setting <code>safe.directory</code></a></li> <li><a href="https://github-redirect.dependabot.com/actions/checkout/pull/744">Bumped various npm package versions</a></li> </ul> <h2>v3.0.0</h2> <ul> <li>Updated to the node16 runtime by default <ul> <li>This requires a minimum <a href="https://github.com/actions/runner/releases/tag/v2.285.0">Actions Runner</a> version of v2.285.0 to run, which is by default available in GHES 3.4 or later.</li> </ul> </li> </ul> <h2>v2.4.2</h2> <h2>What's Changed</h2> <ul> <li>Add set-safe-directory input to allow customers to take control. (<a href="https://github-redirect.dependabot.com/actions/checkout/issues/770">#770</a>) by <a href="https://github.com/TingluoHuang"><code>@TingluoHuang</code></a> in <a href="https://github-redirect.dependabot.com/actions/checkout/pull/776">actions/checkout#776</a></li> <li>Prepare changelog for v2.4.2. by <a href="https://github.com/TingluoHuang"><code>@TingluoHuang</code></a> in <a href="https://github-redirect.dependabot.com/actions/checkout/pull/778">actions/checkout#778</a></li> </ul> <p><strong>Full Changelog</strong>: <a href="https://github.com/actions/checkout/compare/v2...v2.4.2">https://github.com/actions/checkout/compare/v2...v2.4.2</a></p> <h2>v2.4.1</h2> <ul> <li>Fixed an issue where checkout failed to run in container jobs due to the new git setting <code>safe.directory</code></li> </ul> </blockquote> </details> <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/actions/checkout/blob/main/CHANGELOG.md">actions/checkout's changelog</a>.</em></p> <blockquote> <h2>v3.1.0</h2> <ul> <li><a href="https://github-redirect.dependabot.com/actions/checkout/pull/939">Use <code>@actions/core</code> <code>saveState</code> and <code>getState</code></a></li> <li><a href="https://github-redirect.dependabot.com/actions/checkout/pull/922">Add <code>github-server-url</code> input</a></li> </ul> <h2>v3.0.2</h2> <ul> <li><a href="https://github-redirect.dependabot.com/actions/checkout/pull/770">Add input <code>set-safe-directory</code></a></li> </ul> <h2>v3.0.1</h2> <ul> <li><a href="https://github-redirect.dependabot.com/actions/checkout/pull/762">Fixed an issue where checkout failed to run in container jobs due to the new git setting <code>safe.directory</code></a></li> <li><a href="https://github-redirect.dependabot.com/actions/checkout/pull/744">Bumped various npm package versions</a></li> </ul> <h2>v3.0.0</h2> <ul> <li><a href="https://github-redirect.dependabot.com/actions/checkout/pull/689">Update to node 16</a></li> </ul> <h2>v2.3.1</h2> <ul> <li><a href="https://github-redirect.dependabot.com/actions/checkout/pull/284">Fix default branch resolution for .wiki and when using SSH</a></li> </ul> <h2>v2.3.0</h2> <ul> <li><a href="https://github-redirect.dependabot.com/actions/checkout/pull/278">Fallback to the default branch</a></li> </ul> <h2>v2.2.0</h2> <ul> <li><a href="https://github-redirect.dependabot.com/actions/checkout/pull/258">Fetch all history for all tags and branches when fetch-depth=0</a></li> </ul> <h2>v2.1.1</h2> <ul> <li>Changes to support GHES (<a href="https://github-redirect.dependabot.com/actions/checkout/pull/236">here</a> and <a href="https://github-redirect.dependabot.com/actions/checkout/pull/248">here</a>)</li> </ul> <h2>v2.1.0</h2> <ul> <li><a href="https://github-redirect.dependabot.com/actions/checkout/pull/191">Group output</a></li> <li><a href="https://github-redirect.dependabot.com/actions/checkout/pull/199">Changes to support GHES alpha release</a></li> <li><a href="https://github-redirect.dependabot.com/actions/checkout/pull/184">Persist core.sshCommand for submodules</a></li> <li><a href="https://github-redirect.dependabot.com/actions/checkout/pull/163">Add support ssh</a></li> <li><a href="https://github-redirect.dependabot.com/actions/checkout/pull/179">Convert submodule SSH URL to HTTPS, when not using SSH</a></li> <li><a href="https://github-redirect.dependabot.com/actions/checkout/pull/157">Add submodule support</a></li> <li><a href="https://github-redirect.dependabot.com/actions/checkout/pull/144">Follow proxy settings</a></li> <li><a href="https://github-redirect.dependabot.com/actions/checkout/pull/141">Fix ref for pr closed event when a pr is merged</a></li> <li><a href="https://github-redirect.dependabot.com/actions/checkout/pull/128">Fix issue checking detached when git less than 2.22</a></li> </ul> <h2>v2.0.0</h2> <ul> <li><a href="https://github-redirect.dependabot.com/actions/checkout/pull/108">Do not pass cred on command line</a></li> <li><a href="https://github-redirect.dependabot.com/actions/checkout/pull/107">Add input persist-credentials</a></li> <li><a href="https://github-redirect.dependabot.com/actions/checkout/pull/104">Fallback to REST API to download repo</a></li> </ul> <h2>v2 (beta)</h2> <!-- raw HTML omitted --> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Commits</summary> <ul> <li><a href="`93ea575cb5`"><code>93ea575</code></a> Prepare release v3.1.0 (<a href="https://github-redirect.dependabot.com/actions/checkout/issues/940">#940</a>)</li> <li><a href="`6a84743051`"><code>6a84743</code></a> Bump <code>@actions/core</code> to 1.10.0 (<a href="https://github-redirect.dependabot.com/actions/checkout/issues/939">#939</a>)</li> <li><a href="`e6d535c99c`"><code>e6d535c</code></a> Inject GitHub host to be able to clone from another GitHub instance (<a href="https://github-redirect.dependabot.com/actions/checkout/issues/922">#922</a>)</li> <li><a href="`2541b1294d`"><code>2541b12</code></a> Prepare changelog for v3.0.2. (<a href="https://github-redirect.dependabot.com/actions/checkout/issues/777">#777</a>)</li> <li><a href="`0ffe6f9c55`"><code>0ffe6f9</code></a> Add set-safe-directory input to allow customers to take control. (<a href="https://github-redirect.dependabot.com/actions/checkout/issues/770">#770</a>)</li> <li><a href="`dcd71f6466`"><code>dcd71f6</code></a> Enforce safe directory (<a href="https://github-redirect.dependabot.com/actions/checkout/issues/762">#762</a>)</li> <li><a href="`add3486cc3`"><code>add3486</code></a> Patch to fix the dependbot alert. (<a href="https://github-redirect.dependabot.com/actions/checkout/issues/744">#744</a>)</li> <li><a href="`5126516654`"><code>5126516</code></a> Bump minimist from 1.2.5 to 1.2.6 (<a href="https://github-redirect.dependabot.com/actions/checkout/issues/741">#741</a>)</li> <li><a href="`d50f8ea767`"><code>d50f8ea</code></a> Add v3.0 release information to changelog (<a href="https://github-redirect.dependabot.com/actions/checkout/issues/740">#740</a>)</li> <li><a href="`2d1c1198e7`"><code>2d1c119</code></a> update test workflows to checkout v3 (<a href="https://github-redirect.dependabot.com/actions/checkout/issues/709">#709</a>)</li> <li>Additional commits viewable in <a href="https://github.com/actions/checkout/compare/v2.4.0...v3.1.0">compare view</a></li> </ul> </details> <br /> [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=actions/checkout&package-manager=github_actions&previous-version=2.4.0&new-version=3.1.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	3 years ago
Sandeep Sukhani	18da26e668	use a channel to signal completion of index read operation for tsdb and boltdb shipper (#7381 ) What this PR does / why we need it: In PR #7367, I was relying on cancellation of context to detect if the index read operation has finished or not, but it is error prone since a cancelled or timed-out query could cancel the context while we are still reading the index. Since `tsdb` doesn't can't check the context for each read operation, we could still panic in this scenario. I am adding a channel where the consumer of the index would explicitly signal the completion of the read operation to the `indexshipper` Checklist - [x] Tests updated	3 years ago
Herve Nicol	8d9f162460	CONTRIBUTING: fix layout	3 years ago
Hemanth Krishna	0e54477309	area(promtail): add -check-syntax functionality (#7363 ) Adds `-check-syntax` to promtail which will exit just by validating the config file. Fixes #6862. Signed-off-by: Hemanth Krishna <hkpdev008@gmail.com>	3 years ago
Karsten Jeschkies	30caf142b2	Provision alerts for Loki Canary in Helm Chart. (#7286 ) What this PR does / why we need it: The Helm Charts install the Loki canary. It should also install the alerts for the canary so the users get the full value. Which issue(s) this PR fixes: Fixes #7284 Special notes for your reviewer: Checklist - [ ] Reviewed the `CONTRIBUTING.md` guide - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	3 years ago
Karsten Jeschkies	6d495a393e	Import Canary dashboard into Loki mixin. (#7304 ) Summary: This imports our canary into the Loki mixin so it can be used by the Helm chart. The dashboard is disabled by default. It can be enabled by setting ```jsonnet { _config+:: { canary+: { enabled: true, }, }, } ``` Co-authored-by: Vladyslav Diachenko <82767850+vlad-diachenko@users.noreply.github.com>	3 years ago
ebCrypto	a7c7f075e2	documentation: correct yaml file (#7375 ) What this PR does / why we need it: correct indentation of `relabel_configs` to make it valid and consistent with https://grafana.com/docs/loki/latest/clients/promtail/scraping/#syslog-receiver Which issue(s) this PR fixes: Fixes #<issue number> Special notes for your reviewer: Checklist - [x] Reviewed the `CONTRIBUTING.md` guide - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	3 years ago
Owen Diehl	8331785a23	Revert "Update Thanos dependency to latest version" (#7371 ) Reverts grafana/loki#7348 I expect we'll merge this again in the future, but we'll need some more time for downstream projects to be ready for this change.	3 years ago
Vladyslav Diachenko	545668c5a3	[logql-analyzer] auto-deploy on Tag or Main (#7366 ) migrated step `deploy` to `updater` plugin from the deprecated `deploy-image` plugin and added `logql-analyzer` to updater-config. It's needed to automate adding a new version to logql-analyzer namespace once the new version is released or once the new version is published on docker hub.	3 years ago
Sandeep Sukhani	bbe68802d1	lock boltdb and tsdb index while queries are in process (#7367 ) What this PR does / why we need it: We noticed ingesters and index-gateways panicking when using `tsdb` as an index store with the latest build. This is caused by closing `mmap` while the index queries are still being processed. This PR takes care of it by locking the index until we are done querying the index. Checklist - [x] Tests updated	3 years ago
Owen Diehl	f4e2cf1993	TSDB explicit fingerprint (#7362 ) This PR does two things: * Includes the series fingerprint in TSDB WALs. Doing this ensures we recover fingerprint used in chunk addresses after restart even if loki is upgraded and the label hashing algorithm changes. * Removes much of the extra hashing in the TSDB write path + manager. Occasionally, the fingerprint may be different than the hash of labels. Historically this has happened in a few places: * When the `__name__="logs"` label was injected for non-tsdb indices (not applicable to TSDB) * When building a multitenant tsdb index and adding a synthetic `__loki_tenant__` label * In a previous bug(https://github.com/grafana/loki/pull/7355) when empty label values altered calculated fingerprints. Explicitly storing/retrieving the desired fingerprint throughout helps avoid calculating it in error.	3 years ago
Gerard Vanloo	bc7f2f5adb	operator: Add TLS profile support for Loki server and client HTTP and GRPC TLS options (#7322 )	3 years ago
Travis Patterson	7979cfbe07	Implement Distributor Rate Store (#7349 ) The `RateStore` concurrently queries all ingester `GetStreamRate` APIs and stores the results. `RateStore` deduplicates replicas and combines sharded streams so a caller of `RateFor` gets an unsharded view of a stream's rate.	3 years ago
Callum Styan	a7e7a8e013	[dashboards] update dashboards for zone aware ingesters (#7313 ) Mostly related to the job selector; I deployed these against a local grafana instance along the way and I think I got everything correct but we might still need to update some of these again. Signed-off-by: Callum Styan <callumstyan@gmail.com> Signed-off-by: Callum Styan <callumstyan@gmail.com>	3 years ago
Travis Patterson	e89c30ce6b	Fix Flaky Integration Tests (#7360 ) Allow the OS kernel to decide what's an open port instead of trying to guess	3 years ago
Owen Diehl	289190ee26	removes unused tsdb files (#7353 ) These were introduced in an earlier revision of the TSDB work, but are unused now.	3 years ago
Owen Diehl	eaa0919e53	Use labelbuilder in syntax.ParseLabels to remove empty label values (#7355 ) There's a bug in TSDB where empty label values alter the series fingerprints used in chunk addresses, but are stripped out and create different fingerprints in our TSDB based index. I plan to open a PR to Prometheus, but I also prefer normalizing labels here by removing empty label values. According to [Prometheus](https://prometheus.io/docs/concepts/data_model/), > A label with an empty label value is considered equivalent to a label that does not exist. Our bug comes from the Prometheus model `LabelBuilder` stripping out empty label values, [here](https://github.com/grafana/loki/blob/main/vendor/github.com/prometheus/prometheus/model/labels/labels.go#L419-L423). This means that `Labels.NewBuilder({job="foo",bar=""}) => {job="foo"}`. The [labels.Hash()](https://github.com/grafana/loki/blob/main/vendor/github.com/prometheus/prometheus/model/labels/labels.go#L136-L182) function does not skip empty label values, meaning it's easy to end up with different fingerprints for effectively the same series (an empty label value is supposed to be equivalent to a missing label in prometheus, but they generate different label hashes).	3 years ago
Robert Fratto	c657857e95	Update Thanos dependency to latest version (#7348 )	3 years ago
Pablo	cd919f70eb	Adding Promtail and Heroku blogpost files to `examples/` (#7347 )	3 years ago
Dylan Guedes	b1d4efab12	Loki: Per-tenant stream sharding (#7311 ) What this PR does / why we need it: - Move stream sharding configuration to its own package to avoid cyclic imports - Change stream sharding to be a per-tenant configuration - Change ingesters to reject whole streams due to rate-limit based on per-tenant stream sharding - Change stream sharding flags prefix from `distributor.shard-stream` to `shard-stream`	3 years ago
Periklis Tsirakidis	f5501469ab	Upgrade operator gh workflows to go1.19 (#7341 )	3 years ago
Mostafa Moradian	f07e562a9d	Clarify how and where to download the Loki config file from (#7346 ) What this PR does / why we need it: The config file URL mentioned in the docs contain config options that doesn't match the latest version of Loki, which, as of now, is v2.6.1. Which issue(s) this PR fixes: Fixes #6915 Special notes for your reviewer: Disclaimer: I am a Grafanista from k6. Checklist - [x] Reviewed the `CONTRIBUTING.md` guide - [ ] Documentation added - [ ] Tests updated - [x] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	3 years ago
Travis Patterson	e3e4e27c8c	Implement ingester API to return stream rates (#7176 ) This PR introduces an API on the Ingester that reports all current streams in memory and their current rate in bytes/second. The reponse from the API contains a slice of `StreamRate` structs: ```go type StreamRate struct { StreamHash uint64 `protobuf:"varint,1,opt,name=streamHash,proto3" json:"streamHash,omitempty"` StreamHashNoShard uint64 `protobuf:"varint,2,opt,name=streamHashNoShard,proto3" json:"streamHashNoShard,omitempty"` Rate int64 `protobuf:"varint,3,opt,name=rate,proto3" json:"rate,omitempty"` } ``` In the interest of space on the caller and in transmission, only hashes and rates are sent. - `StreamHash` will be used to deduplicate replicas - `StreamHashNoShard` will be used to aggregate a sharded stream's total rate. Co-authored-by: Dylan Guedes <djmgguedes@gmail.com>	3 years ago
Vladyslav Diachenko	c9165fb768	[loki helm chart] fixed to pass CI (#7345 ) Bumped version of the chart and added changelog. These changes were missed in https://github.com/grafana/loki/pull/7301	3 years ago
Byron Esteban	5d441b5526	Service selector for single binary deployment fix. (#7301 ) Single binary deployment is not working when going through default service. The issue is with the labels that currently includes the canary pods. Fixes #7260	3 years ago
Dylan Guedes	da7acb49e5	Loki: Add sharding support for negative/zeroed desired rate (#7342 ) What this PR does / why we need it: Adds the handling of negative/zeroed desired rate to our distributor stream sharding. Without this, invalid desired rates results in a -Inf number of shards. This also adds a log line to report that such a thing happened. Signed-off-by: Danny Kopping <danny.kopping@grafana.com> Co-authored-by: Danny Kopping <danny.kopping@grafana.com>	3 years ago
Danny Kopping	c3e92b876c	Restoring lint checks that were previously disabled (#7340 ) I removed these checks in https://github.com/grafana/loki/pull/7337 to not make https://github.com/grafana/loki/pull/7243 overly complex to review. Refactored code to fix valid lint issues, ignored the irrelevant/dangerous ones. I also removed `unused` and `unparam` as these produce low-quality results that are largely useless.	3 years ago
Sandeep Sukhani	10d3fa0029	mmap tsdb index file while opening it for querying (#7338 ) What this PR does / why we need it: In PR #7321, we re-introduced mmap for reading tsdb index, but it does not cover all the paths where we open the index. This PR takes care of it. I have also removed the code for decompressing the index file since we always get a decompressed file from the `indexshipper`.	3 years ago
Danny Kopping	355175ff89	Upgrade to Go 1.19 (#7243 ) Upgrade Loki and all related components to Go 1.19.2.	3 years ago
Thomas Belian	b795672bba	feat(lambda-promtail): add bearer token support (#7333 ) It adds bearer token support for lambda-promtail, as it supports only http basic auth. Signed-off-by: Thomas Belian <thomas.belian@bt909.de>	3 years ago
Danny Kopping	6e2b7d2589	Patching loki-build-image with latest golangci-lint version (#7337 ) What this PR does / why we need it: Currently the `loki-build-image` has a version of golangci-lint which is based on Go 1.18. [This issue](https://drone.grafana.net/grafana/loki/16278/2/9) has arisen in our CI: https://github.com/golangci/golangci-lint/issues/3107	3 years ago
Periklis Tsirakidis	e8e2523768	operator: Use quayio v2.7.0-pre image for openshift overlay (#7329 )	3 years ago
David Birks	5465aaf3e0	docs: Document setting up command completion for logcli (#7325 ) This documents the existing command completion that kingpin provides (docs [here](https://github.com/alecthomas/kingpin#bashzsh-shell-completion) on that). Which issue(s) this PR fixes: Fixes #2949	3 years ago
Danny Kopping	c2ccb25390	Updating loki-build-image to Go 1.19 (#7334 ) What this PR does / why we need it: Updated the `loki-build-image` to Go 1.19.2 as a precursor for https://github.com/grafana/loki/pull/7243. Signed-off-by: Danny Kopping <danny.kopping@grafana.com>	3 years ago
Danny Kopping	fb6231bd6b	Add compactor_address to fix startup (#7332 ) The `production/docker` docker-compose setup was broken because of a missing configuration.	3 years ago
Periklis Tsirakidis	e9419747c9	Fix internal server bootstrap for query frontend (#7328 ) Adds a fix for setting up the internal server module	3 years ago

1 2 3 4 5 ...

4063 Commits (8c84ee8b7a4a8dedc7ab8d205d27682aa323146a) All Branches Search

4063 Commits (8c84ee8b7a4a8dedc7ab8d205d27682aa323146a)

All Branches