apitech/loki - loki - Gitea: Git with a cup of tea

Commit Graph

Author	SHA1	Message	Date
Karsten Jeschkies	8d01cbf0c5	Merge quantile sketches as they arrive (#11544 )	1 year ago
Travis Patterson	d2f43787b6	Add tenant to tee (#11573 ) It is useful for implementations of Tee to also know the tenant who sent the logs. This PR adds tenant to the tee interface.	1 year ago
Karsten Jeschkies	e9446a93de	Run checks in GitHub actions. (#11475 )	1 year ago
Christian Haudum	5c502e497f	Docs: Fix link to `ecs-task.json` file in ECS instructions (#11568 ) Also updated the Docker image to match the latest version Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	1 year ago
Christian Haudum	0a0b7c83ea	Bloomshipper: Use `model.Time` in `MetaRef` and `BlockRef` (#11566 ) The `Ref` of `MetaRef` and `BlockRef` do have start and end timestamps, which are of type `int64`. This however, can lead to the problem that different components might use different precisions (`s`, `ms`, `ns`) for these timestamps. This PR changes the current implementation of the bloom shipper to use `ms` precision timestamps (using `model.Time`) instead of `s`, to match the behaviour that we have to encode ChunkRefs to object store keys. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	1 year ago
Danny Kopping	76bf5053f2	Revendor gomemcache to import deadline fix (#11545 ) What this PR does / why we need it: See https://github.com/grafana/gomemcache/pull/16	1 year ago
Danny Kopping	18ea38883f	Do not reflect label names in request metrics' "route" label (#11551 ) Which issue(s) this PR fixes: Fixes #11549	1 year ago
Karsten Jeschkies	afe146c884	Benchmark and improve sketch join (#11534 ) What this PR does / why we need it: We saw that `JoinQuantileSketchVector` was using too much memory. This is a small step to improve the allocated memory. ``` pkg: github.com/grafana/loki/pkg/logql │ main.log │ pool.log │ │ sec/op │ sec/op vs base │ JoinQuantileSketchVector-10 3.603µ ± 4% 2.631µ ± 0% -26.99% (p=0.002 n=6) │ main.log │ pool.log │ │ B/op │ B/op vs base │ JoinQuantileSketchVector-10 8.344Ki ± 0% 2.539Ki ± 0% -69.57% (p=0.002 n=6) │ main.log │ pool.log │ │ allocs/op │ allocs/op vs base │ JoinQuantileSketchVector-10 109.0 ± 0% 104.0 ± 0% -4.59% (p=0.002 n=6) ``` Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) - [ ] If the change is deprecating or removing a configuration option, update the `deprecated-config.yaml` and `deleted-config.yaml` files respectively in the `tools/deprecated-config-checker` directory. [Example PR](`0d4416a4b0`) --------- Signed-off-by: Callum Styan <callumstyan@gmail.com> Co-authored-by: Callum Styan <callumstyan@gmail.com>	1 year ago
autoantwort	d8c881a219	Update installation.md: Install using apt or rpm (#11502 ) What this PR does / why we need it: Currently you suggest to download the files manually instead of using apt/rpm which means you don't get any updates. I thing using apt/rpm would be better. Special notes for your reviewer: Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [x] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) - [ ] If the change is deprecating or removing a configuration option, update the `deprecated-config.yaml` and `deleted-config.yaml` files respectively in the `tools/deprecated-config-checker` directory. [Example PR](`0d4416a4b0`) I admit I have not read the full https://grafana.com/docs/writers-toolkit/ guide, seems a lot of work for such a small change 😅 --------- Co-authored-by: J Stickler <julie.stickler@grafana.com>	1 year ago
Poyzan	cc450dcaf9	Add labels to bloom tokenizer metrics (#11542 ) What this PR does / why we need it: Bloom-compactor panics for duplicate metrics registration in bloom-tokenizers. This PR adds labels to bloom-tokenizer metrics, adopting same solution in https://github.com/grafana/loki/pull/11536	1 year ago
Sandeep Sukhani	0e5da205a6	structured metadata: track usage of structured metadata in queries (#11505 ) What this PR does / why we need it: Adds support for tracking queries referencing structured metadata. It tracks the following kinds of references: * Fields in structured metadata used as label filters, `drop`/`keep` stages etc. in the query. * Fields in structured metadata used in metric queries like `uwrap` and `by` or `without` clauses in aggregations. Checklist - [x] Tests updated	1 year ago
Christian Haudum	64392ba164	Fix index gateway startup if bloom gateway is enabled (#11543 ) The index gateways discovers the bloom gateways by using the bloom gateway ring. In order to initialize the ring before the index gateway, it needs to be added as dependency of the index gateway. The dependency is conditional, because we don't want the ring to be initialized even though the bloom gateway is not used. Special notes for your reviewer: This regression was introduced with https://github.com/grafana/loki/pull/11537 Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	1 year ago
Christian Haudum	55fa0d19e4	Fix panic in bloom gateway client (#11537 ) The bloom gateway client panicked, because the ring, from which it gets the subring for the tenant was never passed as argument in the constructor, even though it was part of the client struct. --------- Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	1 year ago
Joao Marcal	26432c0bf8	operator: Add serviceaccount per lokistack resource (#11533 )	1 year ago
Christian Haudum	a760cdfa3f	Bloom Compactor: Fix panic at startup if multiple TSDB period configs are defined (#11536 ) ### Bloom Compactor: Fix duplicate metrics registration If there are multiple period configs with TSDB, then each shipper instance needs to register their metrics with a unique static label set, otherwise, the bloom compactor would panic at startup due to duplicate metric registration attempt. ### Bloom compactor: Start indexshipper in RO mode We don't want/need to upload any index files in the bloom compactor, but we only download files. --------- Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	1 year ago
Joao Marcal	003a51b860	operator: Refactor ensureObjectStoreCredentials in manifests/storage package (#11531 ) Co-authored-by: Periklis Tsirakidis <periklis@redhat.com>	1 year ago
Christian Haudum	62176339c4	Bloom querying: Propagate line filters from querier to bloom filter (#11492 ) In order to query the bloom filters with the search strings from the user query, we need to propagate the line filters through the different implementations of the store, namely through the GetChunks function. In order to avoid converting data structures, the line filter from the syntax tree is used as custom protobuf type, because it is passed from querier to index gateway and from index gateway to bloom gateway. So far, the line filters request have not been tokenized to n-grams. This PR adds the ability to read the n-gram length from the block schema at query time and tokenizes the line filters from the input just ahead of searching the block. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	1 year ago
Alejandro	aa0d706db6	Fix the structured-metadata docs page (#11518 ) What this PR does / why we need it: This little typo may confuse somebody if they would copy/paste the example. Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) - [ ] If the change is deprecating or removing a configuration option, update the `deprecated-config.yaml` and `deleted-config.yaml` files respectively in the `tools/deprecated-config-checker` directory. [Example PR](`0d4416a4b0`) Co-authored-by: Michel Hollands <42814411+MichelHollands@users.noreply.github.com>	1 year ago
Christian Haudum	dabd444f30	Fix configuration for bloom filter deployment (#11530 ) What this PR does / why we need it: * The config block for storage is named `storage_config`, not `storage`. * The replication factor for the bloom gateway is part of the `ring` config. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	1 year ago
Travis Patterson	f668a1b80b	Add context and tenant to pipeline/extractor wrappers (#11528 ) Adding the context here means the wrapping pipeline can know when the query has stopped or been cancelled.	1 year ago
Dylan Guedes	31f8c91771	Jsonnet: Fix and update loki-mixin panels (#11529 ) What this PR does / why we need it: * Update our legacy panels (Loki/Logs and Loki/Operation) do use `"type": "timeseries"` instead of `"type": "graph"`. The graph type is deprecated on recent Grafana versions. I tested the changes with Grizzly and it worked fine. * Fix our p99 panel unit. I previously made it to be `s` instead of `short` but when replicating the change on our source code I put the change at the wrong identation level. This time I made sure to test the change with Grizzly.	1 year ago
dependabot[bot]	9e7c256d26	Bump golang.org/x/crypto from 0.14.0 to 0.17.0 in /operator (#11519 ) Bumps [golang.org/x/crypto](https://github.com/golang/crypto) from 0.14.0 to 0.17.0. <details> <summary>Commits</summary> <ul> <li><a href="`9d2ee975ef`"><code>9d2ee97</code></a> ssh: implement strict KEX protocol changes</li> <li><a href="`4e5a26183e`"><code>4e5a261</code></a> ssh: close net.Conn on all NewServerConn errors</li> <li><a href="`152cdb1503`"><code>152cdb1</code></a> x509roots/fallback: update bundle</li> <li><a href="`fdfe1f8531`"><code>fdfe1f8</code></a> ssh: defer channel window adjustment</li> <li><a href="`b8ffc16e10`"><code>b8ffc16</code></a> blake2b: drop Go 1.6, Go 1.8 compatibility</li> <li><a href="`7e6fbd82c8`"><code>7e6fbd8</code></a> ssh: wrap errors from client handshake</li> <li><a href="`bda2f3f5cf`"><code>bda2f3f</code></a> argon2: avoid clobbering BP</li> <li><a href="`325b735346`"><code>325b735</code></a> ssh/test: skip TestSSHCLIAuth on Windows</li> <li><a href="`1eadac50a5`"><code>1eadac5</code></a> go.mod: update golang.org/x dependencies</li> <li><a href="`b2d7c26edb`"><code>b2d7c26</code></a> ssh: add (*Client).DialContext method</li> <li>Additional commits viewable in <a href="https://github.com/golang/crypto/compare/v0.14.0...v0.17.0">compare view</a></li> </ul> </details> <br /> [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=golang.org/x/crypto&package-manager=go_modules&previous-version=0.14.0&new-version=0.17.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/grafana/loki/network/alerts). </details> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Periklis Tsirakidis <periklis@redhat.com>	1 year ago
Callum Styan	48c59ba96c	remove mistakenly committed files from 11484 (#11527 ) remove mistakenly committed files (benchmarking related) from #11484 Signed-off-by: Callum Styan <callumstyan@gmail.com>	1 year ago
Poyzan	e56e4c92bc	Run compaction over existing bloom blocks (#11486 ) What this PR does / why we need it: Upon changes to TSDB indexes, the existing blooms needs to be amended with newly added chunks, or new series should be added to blocks. This PR compares the existing meta information with the most up to date TSDB information in the job. In case of no changes to indexes, compaction is skipped; if there's any block needs amending, it does that and merges all blocks in fp range; if there's no previous compaction, it compacts from scratch. Special notes for your reviewer: This logic does not support cutting blocks upon topology changes to bloom-compactors. It can create blocks with series outside of the fp range of the compactor. Cutting blocks will be addressed in a follow-up PR.	1 year ago
Karsten Jeschkies	9aae25008d	Optimize series response format by using repeated fileds. (#11498 ) What this PR does / why we need it: The Protobuf map type is encodied as a repeated field of map entries. Decoding them to a slice is much faster than decoding them into a map. Since Loki is not using the fast key check for a map we can use the slice decoding. This change also allows us to decode the JSON directly into the right protobuf struct. This doulbes the JSON decoding speed and reduces the memory pressure by ~40%. ``` › go test -bench=. -run=^$ -count=10 ./pkg/querier/queryrange > before.log › go test -bench=. -run=^$ -count=10 ./pkg/querier/queryrange > after.log › benchstat before.log after.log goos: darwin goarch: arm64 pkg: github.com/grafana/loki/pkg/querier/queryrange │ before.log │ after.log │ │ sec/op │ sec/op vs base │ ResponseMerge/mergeStreams_unlimited-10 32.36m ± 0% 32.63m ± 2% ~ (p=0.393 n=10) ResponseMerge/mergeOrderedNonOverlappingStreams_unlimited-10 1.050m ± 1% 1.080m ± 3% +2.84% (p=0.005 n=10) ResponseMerge/mergeStreams_limited-10 33.02m ± 0% 32.60m ± 1% -1.29% (p=0.004 n=10) ResponseMerge/mergeOrderedNonOverlappingStreams_limited-10 15.11m ± 0% 15.07m ± 0% ~ (p=0.075 n=10) _CodecDecodeLogs-10 4.395m ± 1% 4.364m ± 0% -0.72% (p=0.005 n=10) _CodecDecodeSamples-10 16.97m ± 0% 16.84m ± 2% -0.77% (p=0.023 n=10) _CodecDecodeSeries/application/vnd.google.protobuf-10 745.8µ ± 8% 736.8µ ± 12% ~ (p=0.739 n=10) _CodecDecodeSeries/application/json;_charset=utf-8-10 15.37m ± 1% 10.60m ± 0% -31.03% (p=0.000 n=10) _MergeResponses-10 1186.9m ± 2% 149.8m ± 1% -87.38% (p=0.000 n=10) _UnwrapSeries-10 9.399m ± 1% 4.049m ± 0% -56.92% (p=0.000 n=10) _DecodeMergeEncodeCycle-10 666.0m ± 3% 194.6m ± 0% -70.79% (p=0.000 n=10) geomean 18.87m 12.51m -33.70% │ before.log │ after.log │ │ B/op │ B/op vs base │ _CodecDecodeLogs-10 3.649Mi ± 0% 3.649Mi ± 0% ~ (p=0.364 n=10) _CodecDecodeSamples-10 18.12Mi ± 0% 18.12Mi ± 0% ~ (p=0.926 n=10) _CodecDecodeSeries/application/vnd.google.protobuf-10 7.647Mi ± 0% 7.647Mi ± 0% ~ (p=0.587 n=10) _CodecDecodeSeries/application/json;_charset=utf-8-10 27.94Mi ± 0% 16.99Mi ± 0% -39.18% (p=0.000 n=10) _MergeResponses-10 2.362Mi ± 0% 2.408Mi ± 0% +1.98% (p=0.000 n=10) _UnwrapSeries-10 19.495Mi ± 0% 8.595Mi ± 0% -55.91% (p=0.000 n=10) _DecodeMergeEncodeCycle-10 772.3Mi ± 0% 772.3Mi ± 0% ~ (p=0.912 n=10) geomean 17.50Mi 14.54Mi -16.91% │ before.log │ after.log │ │ allocs/op │ allocs/op vs base │ _CodecDecodeLogs-10 41.10k ± 0% 41.10k ± 0% ~ (p=1.000 n=10) ¹ _CodecDecodeSamples-10 411.9k ± 0% 411.9k ± 0% ~ (p=1.000 n=10) _CodecDecodeSeries/application/vnd.google.protobuf-10 32.00 ± 0% 32.00 ± 0% ~ (p=1.000 n=10) ¹ _CodecDecodeSeries/application/json;_charset=utf-8-10 304.2k ± 0% 298.1k ± 0% -2.01% (p=0.000 n=10) _MergeResponses-10 100.1k ± 0% 100.1k ± 0% -0.00% (p=0.002 n=10) _UnwrapSeries-10 201.1k ± 0% 198.0k ± 0% -1.54% (p=0.000 n=10) _DecodeMergeEncodeCycle-10 203.1k ± 0% 203.1k ± 0% ~ (p=0.621 n=10) geomean 48.95k 48.70k -0.51% ¹ all samples are equal ``` Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [x] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) - [ ] If the change is deprecating or removing a configuration option, update the `deprecated-config.yaml` and `deleted-config.yaml` files respectively in the `tools/deprecated-config-checker` directory. [Example PR](`0d4416a4b0`)	1 year ago
Periklis Tsirakidis	4c506532f2	operator: Apply linter on all test files (#11501 )	1 year ago
Karsten Jeschkies	27d21067e1	Clone filterer of label filter (#11514 )	1 year ago
Bayan Taani	8931f4a978	operator: Add warning for old schema configuration (#11158 ) Co-authored-by: Robert Jacob <rojacob@redhat.com> Co-authored-by: Periklis Tsirakidis <periklis@nefeli.eu>	1 year ago
na--	724a841f0f	Add log duplication (tee-ing) in the Distributor (#11496 ) This allows us to inject a Tee in the Distributor that can duplicate all of the incoming log stream, before they are replicated.	1 year ago
Callum Styan	e93243ff5b	reduce cost of pipeline stages by pooling maps we create for labels (#11484 ) If we look at profiling of ingesters in some of our production deployments during which we have high ingester load from querying we see that we spend roughly 15-20% of our CPU time on GC related work. Looking at different `runtime` memory related calls, it looks like ~50% of all CPU time during Pipeline stage processing is the creation of the map for labels that is then passed to text template parsing. This change will make the pipeline processing itself ~15% faster but hopefully will also reduce (very slightly) the impact of GC in heavy query load situations. ``` goos: linux goarch: amd64 pkg: github.com/grafana/loki/pkg/logql/log cpu: AMD Ryzen 9 5950X 16-Core Processor │ base │ mapPool │ │ sec/op │ sec/op vs base │ _Pipeline/pipeline_bytes-32 7.749µ ± ∞ ¹ 6.464µ ± ∞ ¹ ~ (p=1.000 n=1) ² _Pipeline/pipeline_string-32 7.691µ ± ∞ ¹ 6.549µ ± ∞ ¹ ~ (p=1.000 n=1) ² _Pipeline/line_extractor_bytes-32 7.922µ ± ∞ ¹ 6.882µ ± ∞ ¹ ~ (p=1.000 n=1) ² _Pipeline/line_extractor_string-32 7.900µ ± ∞ ¹ 6.878µ ± ∞ ¹ ~ (p=1.000 n=1) ² _Pipeline/label_extractor_bytes-32 7.952µ ± ∞ ¹ 6.976µ ± ∞ ¹ ~ (p=1.000 n=1) ² _Pipeline/label_extractor_string-32 7.940µ ± ∞ ¹ 7.102µ ± ∞ ¹ ~ (p=1.000 n=1) ² geomean 7.858µ 6.805µ -13.41% ¹ need >= 6 samples for confidence interval at level 0.95 ² need >= 4 samples to detect a difference at alpha level 0.05 │ base │ mapPool │ │ B/op │ B/op vs base │ _Pipeline/pipeline_bytes-32 4.963Ki ± ∞ ¹ 1.413Ki ± ∞ ¹ ~ (p=1.000 n=1) ² _Pipeline/pipeline_string-32 5.025Ki ± ∞ ¹ 1.476Ki ± ∞ ¹ ~ (p=1.000 n=1) ² _Pipeline/line_extractor_bytes-32 5.030Ki ± ∞ ¹ 1.478Ki ± ∞ ¹ ~ (p=1.000 n=1) ² _Pipeline/line_extractor_string-32 5.028Ki ± ∞ ¹ 1.478Ki ± ∞ ¹ ~ (p=1.000 n=1) ² _Pipeline/label_extractor_bytes-32 5.027Ki ± ∞ ¹ 1.478Ki ± ∞ ¹ ~ (p=1.000 n=1) ² _Pipeline/label_extractor_string-32 5.027Ki ± ∞ ¹ 1.478Ki ± ∞ ¹ ~ (p=1.000 n=1) ² geomean 5.017Ki 1.466Ki -70.77% ¹ need >= 6 samples for confidence interval at level 0.95 ² need >= 4 samples to detect a difference at alpha level 0.05 │ base │ mapPool │ │ allocs/op │ allocs/op vs base │ _Pipeline/pipeline_bytes-32 37.00 ± ∞ ¹ 35.00 ± ∞ ¹ ~ (p=1.000 n=1) ² _Pipeline/pipeline_string-32 38.00 ± ∞ ¹ 36.00 ± ∞ ¹ ~ (p=1.000 n=1) ² _Pipeline/line_extractor_bytes-32 37.00 ± ∞ ¹ 35.00 ± ∞ ¹ ~ (p=1.000 n=1) ² _Pipeline/line_extractor_string-32 37.00 ± ∞ ¹ 35.00 ± ∞ ¹ ~ (p=1.000 n=1) ² _Pipeline/label_extractor_bytes-32 37.00 ± ∞ ¹ 35.00 ± ∞ ¹ ~ (p=1.000 n=1) ² _Pipeline/label_extractor_string-32 37.00 ± ∞ ¹ 35.00 ± ∞ ¹ ~ (p=1.000 n=1) ² geomean 37.16 35.16 -5.38% ¹ need >= 6 samples for confidence interval at level 0.95 ² need >= 4 samples to detect a difference at alpha level 0.05 ``` --------- Signed-off-by: Callum Styan <callumstyan@gmail.com>	1 year ago
Travis Patterson	be71a80b15	Implement hooks to instrument query pipelines (#11493 ) This PR implements interfaces to inject new pipelines/extractors into the query path of queriers and ingesters.	1 year ago
Andres Perez	b51b7d7b55	Helm: Use `/ingester/shutdown` endpoint in write pod for `preStop` hook (#11490 ) What this PR does / why we need it: Which issue(s) this PR fixes: Fixes #<issue number> Special notes for your reviewer: Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [x] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [x] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) - [ ] If the change is deprecating or removing a configuration option, update the `deprecated-config.yaml` and `deleted-config.yaml` files respectively in the `tools/deprecated-config-checker` directory. [Example PR](`0d4416a4b0`)	1 year ago
Ryan Lonergan	a192fc2a1e	Helm: add support for setting an s3 backoff_config via the loki helm chart (#11458 ) What this PR does / why we need it: Currently the Loki helm chart does not support defining s3 backoff config settings. This PR adds support to the loki.commonStorageConfig helper to add backoff config options when they're specified in the helm values Which issue(s) this PR fixes: Fixes #<issue number> Special notes for your reviewer: Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [x] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) - [ ] If the change is deprecating or removing a configuration option, update the `deprecated-config.yaml` and `deleted-config.yaml` files respectively in the `tools/deprecated-config-checker` directory. [Example PR](`0d4416a4b0`) --------- Co-authored-by: Michel Hollands <42814411+MichelHollands@users.noreply.github.com>	1 year ago
Karsten Jeschkies	d048e93f1a	Always set matcher in label filter deserialization. (#11497 ) What this PR does / why we need it: Originally we would not set the matcher of a label filter if the name or value was empty. However , `post != ""` is a valid filter expression with an empty value. Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [x] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) - [ ] If the change is deprecating or removing a configuration option, update the `deprecated-config.yaml` and `deleted-config.yaml` files respectively in the `tools/deprecated-config-checker` directory. [Example PR](`0d4416a4b0`)	1 year ago
Christian Haudum	797720d3b2	Add jsonnet lib for deploying Bloom Compactor and Bloom Gateway (#11397 ) This PR adds the deployment of the bloom-compactor and the bloom-gateway to the Loki jsonnet lib. Both components are deployed as StatefulSets, using disks backed by PersistentVolumeClaims. Additionally, the bloom-gateways can also be configured with local SSDs when available. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	1 year ago
na--	71e9fcaab7	Refactor Distributor.Push() to be able to Tee log streams (#11488 ) This allows us to duplicate (i.e. "tee") some of the log streams from the `Distributor` by introducing an optional interface in a follow-up PR.	1 year ago
Matthew Penner	a91f3f11b3	frontend: Use `net.JoinHostPort` to support IPv6 addresses (#10650 ) Signed-off-by: Matthew Penner <me@matthewp.io> Co-authored-by: Periklis Tsirakidis <periklis@redhat.com> Co-authored-by: J Stickler <julie.stickler@grafana.com>	1 year ago
Christian Haudum	cd74ddf4b4	Implement default DFS traversal visitor for LogQL expressions (#11489 ) This can be used as a default visitor for use cases where you only want to visit specific expression without the needing to implement the traversal logic of each and every expression, see implementation in test case. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	1 year ago
Michel Hollands	37a0ed60a0	Support GET for /ingester/shutdown (#11477 ) Which issue(s) this PR fixes: Fixes #11431 Checklist - [X] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [X] Documentation added - [ ] Tests updated - [x] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) - [ ] If the change is deprecating or removing a configuration option, update the `deprecated-config.yaml` and `deleted-config.yaml` files respectively in the `tools/deprecated-config-checker` directory. [Example PR](`0d4416a4b0`) --------- Signed-off-by: Michel Hollands <michel.hollands@gmail.com>	1 year ago
Karsten Jeschkies	8822ae660d	Use correct error write function (#11487 ) What this PR does / why we need it: This fixes the last known regression for `protobuf` encoding. Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [x] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) - [ ] If the change is deprecating or removing a configuration option, update the `deprecated-config.yaml` and `deleted-config.yaml` files respectively in the `tools/deprecated-config-checker` directory. [Example PR](`0d4416a4b0`)	1 year ago
Trevor Whitney	60340639cc	Add trivy filesystem scanning (#11462 ) What this PR does / why we need it: Adds trivy filesystem scanning to our Makefile and PR pipelines. This will scan the `go.mod` file for vulnerabilities in dependencies, which are sometimes missed by just scanning the image. PS @dannykopping this also adds the changes to the vulnerabilities output we talked about that should make it easier to identify a) where the vulnerability is (ie. docker image / os dep vs go.mod dep), and b) what version the dependency needs to be updated to in order to fix it.	1 year ago
Paul Rogers	3da8a29b50	Chore: Spelling fix for linter (#11482 ) What this PR does / why we need it: Just a minor spelling fix in a comment, no functional change. Which issue(s) this PR fixes: Fixes #<issue number> Special notes for your reviewer: Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) - [ ] If the change is deprecating or removing a configuration option, update the `deprecated-config.yaml` and `deleted-config.yaml` files respectively in the `tools/deprecated-config-checker` directory. [Example PR](`0d4416a4b0`)	1 year ago
J Stickler	6b4d27c955	fix image links in getting started (#11478 ) Fixes two image links as images aren't showing up from previous PR (#11350). https://grafana.com/docs/loki/next/get-started/quick-start/ should display a graphic and a screenshot of Grafana Explore, but only the captions are visible.	1 year ago
Paul Rogers	23780f0cd4	Compress bloom data before uploading it to object storage (#11460 ) What this PR does / why we need it: Currently, the block_downloader.go file is preparing to use the UnTarGz function when retrieving a bloom block from object storage. This PR is to put the corresponding compression in for the upload to object storage. Which issue(s) this PR fixes: Fixes #<issue number> Special notes for your reviewer: Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) - [ ] If the change is deprecating or removing a configuration option, update the `deprecated-config.yaml` and `deleted-config.yaml` files respectively in the `tools/deprecated-config-checker` directory. [Example PR](`0d4416a4b0`)	1 year ago
Kaviraj Kanagaraj	ed1fccfa12	Revert "bugfix(memcached): Make `memcached` batch fetch truely context aware" (#11479 ) Reverts grafana/loki#11363 This PR was causing some issue. Spend some time testing on some internal cell. Still couldn't track down the issue yet. Reverting to make `main` stable for now.	1 year ago
Vladyslav Diachenko	9b691904ef	BloomShipper: add cache for downloaded blocks (#11394 ) adapted embeddedcache.go to store downloaded bloom blocks on local filesystem. What this PR does / why we need it: This cache will be used by bloom-gateway to not download the blocks each time. In the cache we store the objects(`cachedBlock` struct) that contain: 1. `blockDirectory`: the `path` to the directory where the block was extracted 2. `activeQueriers`: thread-safe counter of active block users. This field is important because we do not want the directory to be removed while it's in use. 3. rest fields are needed for service needs When the downloadingWorker receives this entity, it increases the counter `activeQueriers` and creates blockQuerier wrapper that has `Close` function that will decrease the counter once blockQuerier is not needed anymore. When the cache entry is being removed from the cache, the cache calls the function `removeDirectoryAsync` which asynchronously removes the block's folder. This function checks every `Xms` if there are still active block queriers and once `activeQueriers` count is `0` the folder will be removed. Also, there is a timeout in this function, and once the timeout is reached, the folder will be force removed. Special notes for your reviewer: * If the cache is disabled, then the blocks will be downloaded each time the block is requested. * If the cache is used, the folder will be deleted by embeddedCache when it reaches memory size limit or items count limit. Otherwise, the block's folder will be deleted when `Close` function is called. Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [x] Documentation added - [x] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) - [ ] If the change is deprecating or removing a configuration option, update the `deprecated-config.yaml` and `deleted-config.yaml` files respectively in the `tools/deprecated-config-checker` directory. [Example PR](`0d4416a4b0`) --------- Signed-off-by: Vladyslav Diachenko <vlad.diachenko@grafana.com>	1 year ago
Karsten Jeschkies	b18078827e	Map rpc Status back to user error (#11449 ) What this PR does / why we need it: Errors forwarded from the querier to the frontend would be reported with the wrong error code and message. This change fixes this. Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [x] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) - [ ] If the change is deprecating or removing a configuration option, update the `deprecated-config.yaml` and `deleted-config.yaml` files respectively in the `tools/deprecated-config-checker` directory. [Example PR](`0d4416a4b0`)	1 year ago
Joao Marcal	9a3bcba174	operator: Adds structured metadata dashboards (#11473 )	1 year ago
Zirko	f0ec743ba0	Helm: add cilium networkpolicies (#11425 ) What this PR does / why we need it: This PR adds `ciliumnetworkpolicies` that are equivalent to the standard `networkpolicies` already present in the templates. As `cilium` usage as a CNI is rising and the usage of `ciliumnetworkpolicies` is more and more widespread, having the possibility to deploy those directly from a setting in the values would be a time-saving option for a lot of deployments. Special notes for your reviewer: Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [x] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [x] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) - [ ] If the change is deprecating or removing a configuration option, update the `deprecated-config.yaml` and `deleted-config.yaml` files respectively in the `tools/deprecated-config-checker` directory. [Example PR](`0d4416a4b0`) --------- Signed-off-by: QuantumEnigmaa <thibaud@giantswarm.io>	1 year ago
Periklis Tsirakidis	bfc2312427	operator: Update Loki operand to v2.9.3 (#11448 )	1 year ago

1 2 3 4 5 ...

6020 Commits (8d01cbf0c5f6f6b81522b9c2cecf0fa9b4815df3) All Branches Search

6020 Commits (8d01cbf0c5f6f6b81522b9c2cecf0fa9b4815df3)

All Branches