apitech/loki - loki - Gitea: Git with a cup of tea

Commit Graph

Author	SHA1	Message	Date
Karsten Jeschkies	c9a3ff53e4	Set unique timestamp in lambda-promtail. (#9560 )	2 years ago
Travis Patterson	fd0efe0799	Regex line/label filters return original regex when `String` is called (#9584 ) To optimize regex line and label filters, Loki rewrites them to be non-greedy. This can cause extra characters to appear in the regex when `String()` is called. In addition to confusing logging, this can also cause internal panics when a previously-parsable query becomes too long. This PR stores the original regex in the label/line filter and uses that in calls to `String`. This PR also removes regex optimizations from label selectors. Making the optimization at parse time caused the original regex to be lost.	2 years ago
Philip Gough	20752d20d0	operator: Update snippet to generate Secret for openshift-logging (#9586 ) Signed-off-by: Philip Gough <philip.p.gough@gmail.com>	2 years ago
Joshua Hesketh	033b504aa7	Fix marshalling ByteSize (#9570 ) When marshalling to yaml or json, expect the value not a pointer. Co-authored-by: Michel Hollands <42814411+MichelHollands@users.noreply.github.com>	2 years ago
Michel Hollands	8a9f519d82	Fix version number of helm chart (#9576 ) What this PR does / why we need it: There were 2 commits for the helm chart with the same version number. In this case the build scripts only releases 1 new helm chart, containing the first commit. This PR adds a new version so a helm chart will be created that contains the later PR. Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [X] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) Signed-off-by: Michel Hollands <michel.hollands@grafana.com>	2 years ago
Dylan Guedes	3941767c6f	Loki: Introduce new `limited_log_push_errors`. (#9568 ) What this PR does / why we need it: Introduce a new configuration named `limited_log_push_errors` that will be implemented. The configuration is being introduced on a separated PR to facilitate future rollbacks. Which issue(s) this PR fixes: N/A	2 years ago
Bryan Boreham	0a2f58891c	logql tests: use abstractions over Labels (#9533 ) What this PR does / why we need it: Call functions like `FromStrings`, so we don't have to know how the data structure is laid out. Special notes for your reviewer: This PR only touches tests. Upstream Prometheus is using a different implementation of `labels.Labels`, so I plan to create a series of PRs which make Loki compatible with that. #9532 was the first. Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - NA Documentation added - [x] Tests updated - NA `CHANGELOG.md` updated - NA Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - NA For Helm chart changes bump the Helm chart version. --------- Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2 years ago
Bryan Boreham	ee74fbdf3a	logql/log tests: use abstractions over labels (#9532 ) What this PR does / why we need it: Call functions like `FromStrings`, so we don't have to know how the data structure is laid out. `FromStrings` sorts labels, so we don't have to call `Sort()`. Special notes for your reviewer: This PR only touches tests. Upstream Prometheus is using a different implementation of `labels.Labels`, so I plan to create a series of PRs which make Loki compatible with that. Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - NA Documentation added - [x] Tests updated - NA `CHANGELOG.md` updated - NA Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - NA For Helm chart changes bump the Helm chart version. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2 years ago
Andre Bossi	aad436b799	Helm: Fix single binary checksum annotation for config.yaml (#9546 ) What this PR does / why we need it: This PR fix name to config checksum annotation Which issue(s) this PR fixes: Fixes #9545 Special notes for your reviewer: Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [x] Documentation added - [x] Tests updated - [x] `CHANGELOG.md` updated - [x] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [x] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`)	2 years ago
oleksii-boiko-ua	558cebf7a2	helm: fix backend hpa missing parameter (#9548 ) What this PR does / why we need it: Which issue(s) this PR fixes: Fixes #9547 Special notes for your reviewer: Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [x] Documentation added - [ ] Tests updated - [x] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [x] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) Co-authored-by: Michel Hollands <42814411+MichelHollands@users.noreply.github.com>	2 years ago
Christophe Collot	b709b32d6a	feat(lambda-promtail): add cloudtrail log ingestion support (#9497 ) What this PR does / why we need it: ### Add support for AWS CloudTrail audit logs ingestion using lambda-promtail Calls to AWS APIs are logged in AWS Cloudtrail and are helpful for security and debugging purposes. However, I've experienced difficulties with it: + The AWS CloudTrail service is not well integrated with Prometheus (no metrics, no alerts) and I don't want to manage alerts in CloudWatch Alerts + The search experience is painful with CloudTrail via the AWS Console (I will not elaborate 😅). This PR allows ingesting CloudTrail audit logs sent to an S3 bucket using the same approach as VPC flow logs or Load Balancer logs. Special notes for your reviewer: + Because the Cloudtrail file format is not text but json, we stream the json CloudTrail records instead of using the already existing scanner. + Because the Cloudtrail filename format is not the same as for the Flow log or the Load balancer log files, we need to split the regexes by service (although many AWS services seem to share the same `defaultFilenameRegex`). + In the `getLabels` function, we expect the `type` parameter to be found in the filename using the Regex. For some log files (ex: Cloudfront log files). The file name has no reference to the service name. This is why, as a default, when no type is found in the name of the file, I set it to use the key of the matching Regex expression. Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [x] Documentation added - [x] Tests updated - [x] `CHANGELOG.md` updated - [x] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	2 years ago
Irapuan Bottosso	a7976b584f	Fix link to API docs (#9517 ) What this PR does / why we need it: Fix the link to the API docs. Special notes for your reviewer: Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [x] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) Co-authored-by: Irapuan Bottosso <irapuan@Irapuans-MacBook-Pro-2.local>	2 years ago
Karsten Jeschkies	2830ec3791	Reuse allocated buffer space in merge iterator. (#9538 ) What this PR does / why we need it: We saw a performance decrease when running the merge iterator with a very small pool of ingesters. The reason was that we would allocate a new buffer with each next call. Special notes for your reviewer: `main` is missing `merge_sort_dedupe-16` so I ran it once with a small patch to generate the data. ```goos: linux goarch: amd64 pkg: github.com/grafana/loki/pkg/iter cpu: AMD Ryzen 7 3700X 8-Core Processor │ mergeiter-old.txt │ mergeiter-new.txt │ │ sec/op │ sec/op vs base │ SortIterator/merge_sort-16 1.350m ± 4% 1.368m ± 4% ~ (p=0.579 n=10) SortIterator/merge_sort_dedupe-16 2.566m ± 4% 2.462m ± 5% ~ (p=0.105 n=10) SortIterator/sort-16 1.034m ± 6% 1.029m ± 6% ~ (p=0.684 n=10) SortSampleIterator/merge-16 1.877m ± 2% 1.857m ± 2% ~ (p=0.075 n=10) SortSampleIterator/sort-16 432.7µ ± 6% 436.0µ ± 4% ~ (p=0.579 n=10) geomean 1.238m 1.229m -0.72% │ mergeiter-old.txt │ mergeiter-new.txt │ │ B/op │ B/op vs base │ SortIterator/merge_sort-16 21.22Ki ± 0% 21.22Ki ± 0% ~ (p=1.000 n=10) SortIterator/merge_sort_dedupe-16 41.47Ki ± 0% 41.47Ki ± 0% ~ (p=1.000 n=10) ¹ SortIterator/sort-16 13.94Ki ± 0% 13.94Ki ± 0% ~ (p=1.000 n=10) ¹ geomean 23.06Ki 23.06Ki +0.00% ¹ all samples are equal │ mergeiter-old.txt │ mergeiter-new.txt │ │ allocs/op │ allocs/op vs base │ SortIterator/merge_sort-16 7.000 ± 0% 7.000 ± 0% ~ (p=1.000 n=10) ¹ SortIterator/merge_sort_dedupe-16 7.000 ± 0% 7.000 ± 0% ~ (p=1.000 n=10) ¹ SortIterator/sort-16 5.000 ± 0% 5.000 ± 0% ~ (p=1.000 n=10) ¹ geomean 6.257 6.257 +0.00% ¹ all samples are equal ``` Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`)	2 years ago
risson	d1fd6b7883	helm: gateway: also listen on IPv6 (#9510 ) What this PR does / why we need it: Currently the gateway only listens on IPv4. On IPv6-only clusters, this causes the readiness probe to fail, rendering the gateway unusable. Which issue(s) this PR fixes: No issue created Special notes for your reviewer: Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [x] `CHANGELOG.md` updated - [x] N/A Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>	2 years ago
Ryan Garcia	028c12f624	[Helm] configStorageType value support (#8526 ) What this PR does / why we need it: Adds `loki.configStorageType` & `loki.externalConfigSecretName` values and associated helpers and templates to support choosing either `ConfigMap` or `Secret` as config medium types. Which issue(s) this PR fixes: Fixes #8268 Special notes for your reviewer: Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [x] Documentation added - [ ] Tests updated - [x] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` --------- Signed-off-by: garcia.ryan <garcia.ryan@solute.us> Co-authored-by: Stefan Büringer <4662360+sbueringer@users.noreply.github.com> Co-authored-by: Sandeep Sukhani <sandeep.d.sukhani@gmail.com>	2 years ago
risson	41c19ec747	helm: support of annotations to deployments and statefulsets (#9509 ) What this PR does / why we need it: Allows annotating loki components with custom annotations. For instance: ```yaml loki: annotations: secret.reloader.stakater.com/reload: loki-aws-credentials ``` Which issue(s) this PR fixes: No issue Special notes for your reviewer: Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - Helm chart section is empty. - [x] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [x] N/A Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>	2 years ago
Bayan Taani	bb96326953	operator: Include runtime-config in compiling the SHA1 checksum (#9448 )	2 years ago
Michel Hollands	a70d092df6	Add help target to Makefile (#9531 ) What this PR does / why we need it: Add a help target to the Makefile. Only the main targets were added. ``` % make help Usage: make <target> Targets: help Display this help and any documented user-facing targets. Other undocumented targets may be present in the Makefile. all build all executables (loki, logcli, promtail, loki-canary) logcli build logcli executable logcli-debug build debug logcli executable logcli-image build logcli docker image loki build loki executable loki-debug build loki debug executable loki-canary build loki-canary executable helm-test run helm tests helm-lint run helm linter loki-querytee build loki-querytee executable promtail build promtail executable promtail-debug build debug promtail executable loki-mixin compile the loki mixin loki-mixin-check check the loki mixin is up to date lint run linters test run the unit tests clean clean the generated files docker-driver build the docker-driver executable fluent-bit-plugin build the fluent-bit plugin fluent-bit-image build the fluent-bit plugin docker image fluent-bit-push push the fluent-bit plugin docker image fluentd-plugin build the fluentd plugin fluentd-plugin-push push the fluentd plugin fluentd-image build the fluentd docker image fluentd-image-push push the fluentd docker image logstash-image build the logstash image logstash-push push the logstash image promtail-image build the promtail docker image promtail-debug-image build the promtail debug docker image loki-image build the loki docker image loki-debug-image build the debug loki docker image loki-canary-image build the loki canary docker image helm-test-image build the helm test image helm-test-push push the helm test image logql-analyzer-image build the LogQL Analyzer image logql-analyzer-push push the LogQL Analyzer image build-image build the docker build image build-image-push push the docker build image doc Generates the config file documentation check-doc Check the documentation files are up to date ``` Checklist - [X] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) Signed-off-by: Michel Hollands <michel.hollands@grafana.com>	2 years ago
BlakeFernandez	905d14bee7	Add Prometheus Link for scrape_configs (#9270 ) What this PR does / why we need it: Adds some context to the scrape configs section in Promtail Config Docs to ensure clarity around the connection to Prometheus Here is a link to a slack thread where the customer was confused on this: https://raintank-corp.slack.com/archives/C04KY6NJSCE/p1681761335754199 --------- Co-authored-by: Michel Hollands <42814411+MichelHollands@users.noreply.github.com> Co-authored-by: J Stickler <julie.stickler@grafana.com>	2 years ago
dependabot[bot]	b4dd0ca432	Bump github.com/alicebob/miniredis/v2 from 2.30.0 to 2.30.2 (#9461 ) Bumps [github.com/alicebob/miniredis/v2](https://github.com/alicebob/miniredis) from 2.30.0 to 2.30.2. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/alicebob/miniredis/releases">github.com/alicebob/miniredis/v2's releases</a>.</em></p> <blockquote> <h2>XADD minid, BLMOVE, COMMAND fix, XREAD fix</h2> <ul> <li>support MINID in XADD (thanks <a href="https://github.com/nathan-cormier"><code>@nathan-cormier</code></a>)</li> <li>support BLMOVE (thanks <a href="https://github.com/sevein"><code>@sevein</code></a>)</li> <li>fix COMMAND (thanks <a href="https://github.com/pje"><code>@pje</code></a>)</li> <li>fix 'XREAD ... $' on a non-existing stream</li> </ul> <h2>SET NX GET</h2> <ul> <li>support SET NX GET special case</li> </ul> </blockquote> </details> <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/alicebob/miniredis/blob/master/CHANGELOG.md">github.com/alicebob/miniredis/v2's changelog</a>.</em></p> <blockquote> <h3>v2.30.2</h3> <ul> <li>support MINID in XADD (thanks <a href="https://github.com/nathan-cormier"><code>@nathan-cormier</code></a>)</li> <li>support BLMOVE (thanks <a href="https://github.com/sevein"><code>@sevein</code></a>)</li> <li>fix COMMAND (thanks <a href="https://github.com/pje"><code>@pje</code></a>)</li> <li>fix 'XREAD ... $' on a non-existing stream</li> </ul> <h3>v2.30.1</h3> <ul> <li>support SET NX GET special case</li> </ul> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href="`d807499fc3`"><code>d807499</code></a> update changelog</li> <li><a href="`4be3b8260d`"><code>4be3b82</code></a> fix 'XREAD ... $' on a non-existing stream</li> <li><a href="`2f1aeb304a`"><code>2f1aeb3</code></a> Merge pull request <a href="https://redirect.github.com/alicebob/miniredis/issues/323">#323</a> from pje/pje/bring-back-273-fix</li> <li><a href="`c71c3127c1`"><code>c71c312</code></a> Bring back the COMMAND response fix introduced in <a href="https://redirect.github.com/alicebob/miniredis/issues/273">#273</a></li> <li><a href="`84b7146f54`"><code>84b7146</code></a> Merge pull request <a href="https://redirect.github.com/alicebob/miniredis/issues/321">#321</a> from sevein/dev/cmd-list-blmove</li> <li><a href="`93ce01ac66`"><code>93ce01a</code></a> Implement BLMOVE</li> <li><a href="`9077c868d4`"><code>9077c86</code></a> Merge pull request <a href="https://redirect.github.com/alicebob/miniredis/issues/318">#318</a> from nathan-cormier/minid</li> <li><a href="`a80f1409fc`"><code>a80f140</code></a> Support MINID in XADD</li> <li><a href="`e5c9dd403d`"><code>e5c9dd4</code></a> changelog</li> <li><a href="`5c16d558d3`"><code>5c16d55</code></a> support SET NX GET special case</li> <li>Additional commits viewable in <a href="https://github.com/alicebob/miniredis/compare/v2.30.0...v2.30.2">compare view</a></li> </ul> </details> <br /> [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=github.com/alicebob/miniredis/v2&package-manager=go_modules&previous-version=2.30.0&new-version=2.30.2)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	2 years ago
oleksii-boiko-ua	65b5df68ff	helm: fix breaking sts change (#9528 ) What this PR does / why we need it: It fixes the breaking change I introduced by changing sts podManagementPolicy from Parallel to OrderedReady Which issue(s) this PR fixes: Fixes #9524 Special notes for your reviewer: Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [x] Documentation added - [ ] Tests updated - [x] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [x] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) Co-authored-by: Michel Hollands <42814411+MichelHollands@users.noreply.github.com>	2 years ago
Christian Simon	9399ed930a	Fix wrong error check (#9518 ) I am afraid I broke main with #9502. It seem to have auto-merged with failing tests 😢	2 years ago
Karsten Jeschkies	5d7e0fd6c0	Pass emtpy TLS config to Loki Canary client. (#9516 )	2 years ago
AleksandrKovshariov	8fbb095aae	Update replication factor ceveat sentence (#9358 ) What this PR does / why we need it: This sentence "There’s also an edge case where we acknowledge a write if 2 of the three ingesters do which means that in the case where 2 writes succeed, we can only lose one ingester before suffering data loss." is really hard to understand. I suggest "If a write is acknowledged by 2 out of 3 ingesters, we can tolerate the loss of one ingester but not two, as this would result in data loss." Which issue(s) this PR fixes: Fixes #<issue number> Special notes for your reviewer: Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	2 years ago
Alexandre de Verteuil	e38ccb4418	Uniformize "Available Labels" headers in Promtail config reference doc (#9436 ) What this PR does / why we need it: There were various formats for the "Available Labels" headings. This PR makes them all the same heading level and spelling.	2 years ago
bluesky6529	a9cde84c95	Fix typo on lambda-promtail docs (#9472 ) What this PR does / why we need it: fix for doc typo Which issue(s) this PR fixes:N/A Special notes for your reviewer: N/A	2 years ago
Christian Simon	7c26c8c093	Fix panic in `pkg/analytics` (#9502 ) We discovered a bug in Phlare https://github.com/grafana/phlare/pull/706. There can be a race occuring between two different `New<Counter\|Int>` calls for the same `name`.	2 years ago
oleksii-boiko-ua	f9aba99455	Helm: add hpa for read, write and backend (#8684 ) What this PR does / why we need it: It adds missing HPA (HorizontalPodAutoscaler) templates to write, read, and backend components Which issue(s) this PR fixes: Fixes #7864 Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [x] Documentation added - [ ] Tests updated - [x] `CHANGELOG.md` updated - [x] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` Co-authored-by: Oleksii Boiko <oleksii_boiko@external.mckinsey.com>	2 years ago
indX	a55404766c	Logging: Add HTTP API to change the log level at runtime (#9357 ) What this PR does / why we need it: To be able to change the log level at runtime, particularly from info to debug to enable debugging information during an incident without restarting Loki. Which issue(s) this PR fixes: Fixes #6805 Special notes for your reviewer: Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [x] Documentation added - [x] Tests updated - [x] `CHANGELOG.md` updated - [x] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`: No such changes --------- Co-authored-by: J Stickler <julie.stickler@grafana.com>	2 years ago
Christian Haudum	b4d0a2cd68	Force using a buffer from the ChunkRefsPool (#9217 ) What this PR does / why we need it: Re-using a buffer from the `ChunkRefsPool` avoids unnecessary allocations in TSDB's `GetChunkRefs` call. Special notes for your reviewer: Benchmarks ``` $ gotest -v -count=1 ./pkg/storage/stores/tsdb/ -test.run="^$" -test.bench=Bench -test.benchtime=10000x ``` ```console $ benchstat old.txt new.txt name old time/op new time/op delta TenantHeads/10-8 244µs ±58% 55µs ± 4% -77.49% (p=0.000 n=10+9) TenantHeads/100-8 1.75ms ± 9% 0.58ms ±26% -66.89% (p=0.000 n=9+10) TenantHeads/1000-8 16.5ms ± 6% 6.4ms ± 2% -60.89% (p=0.000 n=9+10) IndexClient_Stats-8 25.0µs ±10% 25.1µs ±13% ~ (p=0.863 n=9+9) TSDBIndex_GetChunkRefs-8 3.34ms ±12% 1.46ms ± 2% -56.41% (p=0.000 n=10+9) name old alloc/op new alloc/op delta IndexClient_Stats-8 2.37kB ±12% 2.37kB ±12% ~ (p=0.923 n=10+10) TSDBIndex_GetChunkRefs-8 6.40MB ± 0% 1.40MB ± 0% -78.08% (p=0.000 n=9+10) name old allocs/op new allocs/op delta IndexClient_Stats-8 59.0 ± 0% 59.0 ± 0% ~ (all equal) TSDBIndex_GetChunkRefs-8 42.0 ± 0% 27.0 ± 0% -35.71% (p=0.000 n=10+10) ``` Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
Michel Hollands	478510a1bb	Fix Grouping String function and usage so it prints "without ()" (#9515 ) What this PR does / why we need it: The changes rangemapper made to LogQL expressions created invalid LogQL. It added a `without` clause instead of `without ()`. This PR changes the String function of the Grouping struct. This is then used in the rangemapper. This caused the following panic: ``` panic: error cloning query: count without (rate({namespace="apps"}[15s])): parse error at line 1, col 16: syntax error: unexpected RATE, expecting IDENTIFIER or ) ``` Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [x] Tests updated - [x] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) --------- Signed-off-by: Michel Hollands <michel.hollands@grafana.com>	2 years ago
Michel Hollands	89b786fe51	Add instructions for Helm chart updates in the pull request template (#9514 ) What this PR does / why we need it: Add additional instructions for Helm charts in the pull request template. Special notes for your reviewer: Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` --------- Signed-off-by: Michel Hollands <michel.hollands@grafana.com>	2 years ago
Graham Rounds	fce2b3cc7e	Helm: Quote tenantId value in logsInstance (#9486 ) What this PR does / why we need it: Quotes the value of `tenantId` for the cases where tenantId is made up of only numerical values (i.e. if tenantId is `0`) the motivation for this is that we have long used the tenant id `0` in our loki instance for some stuff. We probably should fix that, but it's going to be easier (at least for us) to patch this than to rename this tenant or migrate to a new tenant. Which issue(s) this PR fixes: N/A - If you'd like me to file an issue to track this there as well, let me know and i'll write one up. i figured this fix was simple enough Special notes for your reviewer: Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [x] Documentation added - [x] Tests updated - [x] `CHANGELOG.md` updated - [x] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	2 years ago
Robert Jacob	9f19c384e1	operator: Do not update status after setting degraded condition (#9511 )	2 years ago
Artem Nosulchyk	2c57d2e2b9	Helm: Support of extraVolumeClaimTemplates for writer statefulset (#9485 ) What this PR does / why we need it: This PR adds `extraVolumeClaimTemplates` value for write loki component, it allows to have more than one volume claims for persistent statefulset configuration. For example, when it's needed to have ingester's and shipper's WAL files on different persistent volumes. Special notes for your reviewer: Checklist - [X] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [X] Documentation added - [x] Tests updated - [x] `CHANGELOG.md` updated - [x] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` --------- Co-authored-by: J Stickler <julie.stickler@grafana.com>	2 years ago
Travis Patterson	a4ef06890b	revert count unflused chunks (#9504 ) revert this commit: `aeba51ab0f` We seem to be dramatically overcounting bytes now which is causing oversharding This is the same query before and after: Before: ~20 GB ![image (2)](https://github.com/grafana/loki/assets/1413241/6d0e3d4d-56ce-40aa-b57e-d2dd592eefb4) After: ~8 EB ![image (1)](https://github.com/grafana/loki/assets/1413241/f07cca10-0e32-467f-bf96-13a83eea8ce6)	2 years ago
Travis Patterson	7ff8463def	Fix query behavior in Single Binary and SSD deployments (#9476 ) To keep ingesters from calling themselves, the AsyncStore was only being used when the module was `Querier`, `Ruler`, or `Read`. This means that, for some operations, ingesters were never queries when the deployment mode was single-binary or `Backend` (where the ruler now lives). This change scopes the async store to just the querier rather than all storage access and always includes in it queriers. Because the `AsyncStore` isn't automatically instantiated, now, I've also moved `IngesterQuerier` to be a dependency to the store. That way anything that wants to make an `AsyncStore` is guaranteed to have what it needs.	2 years ago
Dylan Guedes	e81dd8aed2	Loki: Bump `dskit` to inherit new backoff configs (#9484 ) What this PR does / why we need it: Bump `dskit` to inherit https://github.com/grafana/dskit/pull/296. It will allow us to configure custom backoff timeouts. Which issue(s) this PR fixes: N/A	2 years ago
dependabot[bot]	f6dc3aff5c	Bump helm/kind-action from 1.5.0 to 1.7.0 (#9494 ) Bumps [helm/kind-action](https://github.com/helm/kind-action) from 1.5.0 to 1.7.0. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/helm/kind-action/releases">helm/kind-action's releases</a>.</em></p> <blockquote> <h2>v1.7.0</h2> <h2>What's Changed</h2> <ul> <li>default action to use kind release v0.19.0 by <a href="https://github.com/cpanato"><code>@cpanato</code></a> in <a href="https://redirect.github.com/helm/kind-action/pull/85">helm/kind-action#85</a></li> </ul> <p><strong>Full Changelog</strong>: <a href="https://github.com/helm/kind-action/compare/v1.6.0...v1.7.0">https://github.com/helm/kind-action/compare/v1.6.0...v1.7.0</a></p> <h2>v1.6.0</h2> <h2>What's Changed</h2> <ul> <li>fix: update documentation to represent actual kubectl version used by <a href="https://github.com/tomhobson"><code>@tomhobson</code></a> in <a href="https://redirect.github.com/helm/kind-action/pull/83">helm/kind-action#83</a></li> <li>update kind to v0.18.0 and kubectl to v1.26.4 by <a href="https://github.com/cpanato"><code>@cpanato</code></a> in <a href="https://redirect.github.com/helm/kind-action/pull/84">helm/kind-action#84</a></li> </ul> <h2>New Contributors</h2> <ul> <li><a href="https://github.com/tomhobson"><code>@tomhobson</code></a> made their first contribution in <a href="https://redirect.github.com/helm/kind-action/pull/83">helm/kind-action#83</a></li> </ul> <p><strong>Full Changelog</strong>: <a href="https://github.com/helm/kind-action/compare/v1.5.0...v1.6.0">https://github.com/helm/kind-action/compare/v1.5.0...v1.6.0</a></p> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href="`fa81e57adf`"><code>fa81e57</code></a> default action to use kind release v0.19.0 (<a href="https://redirect.github.com/helm/kind-action/issues/85">#85</a>)</li> <li><a href="`28ee08d267`"><code>28ee08d</code></a> update kind to v0.18.0 and kubectl to v1.26.4 (<a href="https://redirect.github.com/helm/kind-action/issues/84">#84</a>)</li> <li><a href="`2d96ac1d71`"><code>2d96ac1</code></a> fix: update documentation to represent actual kubectl version used (<a href="https://redirect.github.com/helm/kind-action/issues/83">#83</a>)</li> <li><a href="`b2013cb316`"><code>b2013cb</code></a> Bump actions/checkout from 3.5.0 to 3.5.2 (<a href="https://redirect.github.com/helm/kind-action/issues/82">#82</a>)</li> <li><a href="`716a7056e0`"><code>716a705</code></a> Bump actions/checkout from 3.4.0 to 3.5.0 (<a href="https://redirect.github.com/helm/kind-action/issues/81">#81</a>)</li> <li><a href="`52d48f105d`"><code>52d48f1</code></a> Bump actions/checkout from 3.3.0 to 3.4.0 (<a href="https://redirect.github.com/helm/kind-action/issues/80">#80</a>)</li> <li><a href="`15e2bf8beb`"><code>15e2bf8</code></a> Bump actions/checkout from 3.2.0 to 3.3.0 (<a href="https://redirect.github.com/helm/kind-action/issues/77">#77</a>)</li> <li><a href="`ab62653af4`"><code>ab62653</code></a> Bump actions/checkout from 3.1.0 to 3.2.0 (<a href="https://redirect.github.com/helm/kind-action/issues/74">#74</a>)</li> <li>See full diff in <a href="https://github.com/helm/kind-action/compare/v1.5.0...v1.7.0">compare view</a></li> </ul> </details> <br /> [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=helm/kind-action&package-manager=github_actions&previous-version=1.5.0&new-version=1.7.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Michel Hollands <42814411+MichelHollands@users.noreply.github.com>	2 years ago
Periklis Tsirakidis	1b87aeab4e	operator: Add support for configuring HTTP server timeouts (#9405 ) Co-authored-by: Robert Jacob <rojacob@redhat.com>	2 years ago
Andreas Gebhardt	a205dce836	chunks-inspect: print chunk version (format) (#9490 ) Get the version byte of the current inspecting Loki chunk printed.	2 years ago
Piotr	ac4a442291	Promtail: Fix goroutine leak in file tailer (#9495 ) What this PR does / why we need it: There's a goroutine leak in promtail file tailer, which has been encountered in https://github.com/grafana/agent/issues/3885. The issue is the following: - There are two goroutines that tailer manages: one for reading lines, the other for updating positions: https://github.com/grafana/loki/blob/main/clients/pkg/promtail/targets/file/tailer.go#L103 - The tailer can be cleanly stopped & drained via `.Stop()` https://github.com/grafana/loki/blob/main/clients/pkg/promtail/targets/file/tailer.go#L221 - But if the goroutine reading lines encounters an error from the underlying `tail`, the tailer is stopped without draining using the defer statement here: https://github.com/grafana/loki/blob/main/clients/pkg/promtail/targets/file/tailer.go#L151 - The above defer statement does not shut down the position updating goroutine and the `.Stop()` won't be called, so we end up with a leaked goroutine. Which issue(s) this PR fixes: Fixes https://github.com/grafana/agent/issues/3885 As can be seen in the [goroutine dump attached to the issue](https://gist.github.com/amseager/e37f31984facd9cbc51ce0f2153d6864), there were 933 update position goroutines, while we only had 21 files being tailed. Special notes for your reviewer: Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [x] Tests updated - [x] `CHANGELOG.md` updated	2 years ago
Sandeep Sukhani	d10549e3ec	helm: release a new helm charts to use k8s service for discover query-scheduler replicas (#9477 ) What this PR does / why we need it: Currently, we have a bug in our code when running Loki in SSD mode and using the ring for query-scheduler discovery. It causes queries to not be distributed to all the available read pods. I have explained the issue in detail in [the PR which fixes the code](https://github.com/grafana/loki/pull/9471). Since this bug causes a major query performance impact and code release might take time, in this PR we are doing a new helm release which fixes the issue by using the k8s service for discovering `query-scheduler` replicas. Which issue(s) this PR fixes: Fixes #9195	2 years ago
Travis Patterson	aeba51ab0f	Count unflushed chunks in instance stats (#9479 ) The instance wasn't counting unflushed chunks in the stats. This change checks that the chunk's `flushed time` is 0 which means it hasn't been flushed yet.	2 years ago
Owen Diehl	64c6f6b8cc	Better combines stats in ingesters (#9474 ) * Reduces double counting streams when possible * proportionally add stats based on chunk overlap range * code reuse	2 years ago
Dylan Guedes	358b8965c4	Distributor: Add org_id to log error (#9475 ) What this PR does / why we need it: Add `org_id` to push error when the error is "max active streams limit exceeded". Apparently all other push errors correctly report the `org_id`, but not this one. Which issue(s) this PR fixes: N/A	2 years ago
Pablo	306c96ce8a	Notification based Promtail WAL Watcher polling approach (#9135 ) What this PR does / why we need it: This PR implements a new mechanism for the wal Watcher in Promtail, to know there are new records to be read. It uses a combination of: - https://github.com/prometheus/prometheus/pull/11950 - https://github.com/prometheus/prometheus/pull/11949 The main idea is that the primary mechanism is a notification channel between the `wal.Writer` and `wal.Watcher`. The Watcher subscribes to write events the writer publishes, getting notified if the wal has been written. The same subscriptions design is used for cleanup events. As a backup, the watcher has a timer that implements an exponential backoff strategy, which is constrained by a minimum and maximum that the user can configure. Below the cpu difference is shown of running both main and this branch against the same scrape target. <img width="2496" alt="image" src="https://user-images.githubusercontent.com/2617411/232099483-7e5c36fa-9360-4eb9-8240-687adf46e330.png"> The yellow line is the latest main build from where this branch started, and the green line is this branch. Both promtails tailing docker logs, and using the following metrics to get cpu usage from cadvisor: ``` avg by (name) (rate(container_cpu_usage_seconds_total{job=~".+", instance=~".+", name=~"promtail-wal-test_promtail.+"}[$__rate_interval])) ``` Which issue(s) this PR fixes: Part of https://github.com/grafana/loki/issues/8197 Special notes for your reviewer: Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	2 years ago
Owen Diehl	2efd059b49	Slight improvements to `GetFactorOfTime` (#9473 ) * correctly returns zero for non-overlapping data * adds tests	2 years ago
Mohamed-Amine Bouqsimi	df77bf6024	operator: add zone aware API spec validation (#9378 )	2 years ago
Stefan Büringer	bd7fe5c3bf	Helm: use $.Release.Namespace consistently (#9330 )	2 years ago

1 2 3 4 5 ...

5131 Commits (c9a3ff53e4b60330c53fe8659d5632fd4d12d734) All Branches Search

5131 Commits (c9a3ff53e4b60330c53fe8659d5632fd4d12d734)

All Branches