apitech/loki - loki - Gitea: Git with a cup of tea

Commit Graph

Author	SHA1	Message	Date
Christian Haudum	d4a6b159fa	chore: Cleanup label extraction expression (#8596 ) The aim of this is to remove redundancy from the parser grammar and make the expression name more descriptive. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
Bryan Boreham	019ac9975c	Iterators: re-implement entrySortIterator using loserTree for performance (#8351 ) What this PR does / why we need it: [Draft PR for comment at this time] This implementation uses a data structure called Loser Tree, also known as Tournament Tree, based on Knuth, "Sorting and Searching" section 5.4.1. I couldn't find an existing Loser Tree in Go, so I wrote my own; the file is 132 lines long. It uses generics in the hope it can also be applied elsewhere. One benchmark result: ``` name old time/op new time/op delta SortIterator/sort-4 3.78ms ± 4% 2.66ms ± 3% -29.54% (p=0.008 n=5+5) name old alloc/op new alloc/op delta SortIterator/sort-4 319kB ± 0% 14kB ± 0% -95.52% (p=0.008 n=5+5) name old allocs/op new allocs/op delta SortIterator/sort-4 104 ± 0% 5 ± 0% -95.19% (p=0.008 n=5+5) ``` Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - NA Documentation added - [x] Tests updated - [ ] `CHANGELOG.md` updated - NA Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	2 years ago
Karsten Jeschkies	eeca4589fb	Define Container for Codespaces (#8561 )	2 years ago
Susana Ferreira	b8118b1e7b	Helm enterprise-logs: update kubernetes secrets creation (#8467 ) What this PR does / why we need it: Fix the Kubernetes secrets creation for the tokengen and provisioner jobs for multiple helm installations/uninstallations. The execution flow for the helm installation is the following: 1) `tokengen` job `loki` container generates a new admin token, stores it in the object storage and writes it to the `shared` mount. 2) `tokengen` job `create-secret` container creates a new Kubernetes secret for the generated admin token. 3) `provisioner` job `provisioner` container creates the new admin resources via the Admin API and writes the tokens to the `bootstrap` mount. 4) `provisioner` job `create-secret` container creates new Kubernetes secrets for each token of the additional tenants. ## Previous implementation Uninstall and re-install the enterprise-logs helm: 1) `tokengen` job `loki` container generates a new admin token, stores it in the object storage, and writes it to the `shared` mount. This is because this job uses an [emptyDir](https://kubernetes.io/docs/concepts/storage/volumes/#emptydir) mount which is deleted once the job completes. When a new job is created, the directory will be again empty, forcing the `tokengen` to generate a new token. 2) `tokengen` job `create-secret` container fails, because an admin token secret already exists: ``` Defaulted container "create-secret" out of: create-secret, loki (init) error: failed to create secret secrets "enterprise-logs-admin-token" already exists ``` 3) The helm installation fails ❌ ## Implementation in this PR 1) `tokengen` job `loki` container generates a new admin token, stores it in the object storage, and writes it to the `shared` mount. 2) `tokengen` job `create-secret` container updates the previously created Kubernetes secret for the admin token. 3) `provisioner` job `provisioner` container checks that the admin resources already exist and doesn't write the token files to the `bootstrap` mount. 4) `provisioner` job `create-secret` container checks that the token files are not present on the `bootstrap` mount, and does not create new secrets. 5) The helm installation succeeds ✅ Special notes for your reviewer: Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	2 years ago
Michel Hollands	4f8d3240f6	Use 0.28.1 build image and update go and alpine versions (#8583 ) What this PR does / why we need it: - Use the 0.28.1 build image - Update Go version to 1.20.1 - Use Alpine 3.16.4 - Fix linter issues due to updated govet in 1.20.1 - Modify images to not create `/etc/nsswitch.conf` file anymore (available by default on Alpine 3.16.4) - Remove impossible test cases for `TestMemChunk_IteratorBounds`(see https://github.com/grafana/loki/pull/8591#discussion_r1115020111 for context) Signed-off-by: Michel Hollands <michel.hollands@grafana.com> Signed-off-by: Christian Haudum <christian.haudum@gmail.com> Co-authored-by: Christian Haudum <christian.haudum@gmail.com> Co-authored-by: DylanGuedes <djmgguedes@gmail.com>	2 years ago
dependabot[bot]	48aa833229	Bump golang.org/x/text from 0.3.7 to 0.3.8 in /operator (#8597 ) Bumps [golang.org/x/text](https://github.com/golang/text) from 0.3.7 to 0.3.8. <details> <summary>Commits</summary> <ul> <li><a href="`434eadcdbc`"><code>434eadc</code></a> language: reject excessively large Accept-Language strings</li> <li><a href="`23407e72ed`"><code>23407e7</code></a> go.mod: ignore cyclic dependency for tagging</li> <li><a href="`b18d3dd8a4`"><code>b18d3dd</code></a> secure/precis: replace bytes.Compare with bytes.Equal</li> <li><a href="`795e854ff3`"><code>795e854</code></a> all: replace io/ioutil with io and os package</li> <li><a href="`b0ca10ff35`"><code>b0ca10f</code></a> internal/language: bump script types to uint16 and update registry</li> <li><a href="`ba9b0e1d4b`"><code>ba9b0e1</code></a> go.mod: update x/tools to HEAD</li> <li><a href="`d03b418000`"><code>d03b418</code></a> A+C: delete AUTHORS and CONTRIBUTORS</li> <li><a href="`b4bca84b03`"><code>b4bca84</code></a> language/display: fix Tag method comment</li> <li><a href="`ea49e3e2d5`"><code>ea49e3e</code></a> go.mod: update x/tools to HEAD</li> <li><a href="`78819d01d0`"><code>78819d0</code></a> go.mod: update to golang.org/x/text v0.1.10</li> <li>Additional commits viewable in <a href="https://github.com/golang/text/compare/v0.3.7...v0.3.8">compare view</a></li> </ul> </details> <br /> [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=golang.org/x/text&package-manager=go_modules&previous-version=0.3.7&new-version=0.3.8)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/grafana/loki/network/alerts). </details> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	2 years ago
Christian Haudum	25d266e1f4	Clean up CHANGELOG (#8595 ) Keep the changelog clean so it's easier for the release shepherd to compile the changelog and release notes for the next release. * Add newline underneath section heading * Move promtail related changelog entries into promtail section * Remove changelog entry from already released enhancement * Move changelog entry into correct position	2 years ago
Robert Jacob	93c35a7a13	operator: Refactor status update to reduce API calls (#8578 )	2 years ago
Gerard Vanloo	233997ef20	operator: Store gateway tenant information in secret instead of configmap (#8577 )	2 years ago
Owen Diehl	2836c79537	Revert "Loki: Fix memchunk headblock filter" (#8594 ) Reverts grafana/loki#8591 This altered an internal API. We execute queries that are inclusive of start and exclusive of end: `[start,end)` and shouldn't break this. We use other ways to ensure `start==end` still works, such as by using the lookback-period to alter the range of queries: https://github.com/grafana/loki/blob/main/pkg/logql/evaluator.go#L179-L181	2 years ago
Bayan Taani	68a9fd6c0a	Add logfmt selective label extraction (#6675 ) <!-- Thanks for sending a pull request! Before submitting: 1. Read our CONTRIBUTING.md guide 2. Name your PR as `<Feature Area>: Describe your change`. a. Do not end the title with punctuation. It will be added in the changelog. b. Start with an imperative verb. Example: Fix the latency between System A and System B. c. Use sentence case, not title case. d. Use a complete phrase or sentence. The PR title will appear in a changelog, so help other people understand what your change will be. 3. Rebase your PR if it gets out of sync with main --> What this PR does / why we need it: This PR introduces extracting labels from a log line in `logfmt`, with an extra option to rename them. For example, this query: ``` {app=”foo”} \| logfmt msg,status ``` will extract the labels `msg` and `status` from the following logfmt line: ``` level=error ts=2021-02-12T19:18:10.037940878Z caller=client.go:294 component=client host=observability-loki-gateway msg="final error sending batch" status=400 error="server returned HTTP status 400 Bad Request (400): entry with timestamp 2021-02-12 19:18:08.917452 +0000 UTC ignored, reason: 'entry out of order' for stream..." ``` With the following results: ``` msg="final error sending batch" status=”400” ``` -------------- Another possible scenario with label renaming: ``` {app=”foo”} \| logfmt message="msg", status ``` That produces the following results: ``` message="final error sending batch" status=”400” ``` Which issue(s) this PR fixes: Fixes #3355 Special notes for your reviewer: <!-- Note about CHANGELOG entries, if a change adds: * an important feature * fixes an issue present in a previous release, * causes a change in operation that would be useful for an operator of Loki to know then please add a CHANGELOG entry. For documentation changes, build changes, simple fixes etc please skip this step. We are attempting to curate a changelog of the most relevant and important changes to be easier to ingest by end users of Loki. Note about the upgrade guide, if this changes: * default configuration values * metric names or label names * changes existing log lines such as the metrics.go query output line * configuration parameters * anything to do with any API * any other change that would require special attention or extra steps to upgrade Please document clearly what changed AND what needs to be done in the upgrade guide. --> Checklist - [x] Documentation added - [x] Tests updated - [x] Is this an important fix or new feature? Add an entry in the `CHANGELOG.md`. - [x] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` --------- Signed-off-by: Christian Haudum <christian.haudum@gmail.com> Co-authored-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
Dylan Guedes	030d3232ff	Loki: Fix memchunk headblock filter (#8591 ) What this PR does / why we need it: Fix our memchunk headblock filter to not skip entries that have timestamp=maxT.	2 years ago
Gerard Vanloo	f6871d8f5f	operator: Remove deprecated field querier.engine.timeout (#8579 )	2 years ago
Travis Patterson	eb1b72ab80	Fix ruler deadlock (#8486 ) A circular dependency between the Prometheus rules.Manager and the MemStore causes occasional deadlock in the rulers. This change makes both the manager and the memstore rely on the `GroupLoader` interface to break the circular dependency. The manger holding the `GroupLoader` only ever adds rules to it so we add another layer to cleanup any leftover state after an update happens. This supersedes https://github.com/grafana/loki/pull/8453 --------- Co-authored-by: Michel Hollands <42814411+MichelHollands@users.noreply.github.com>	2 years ago
Michel Hollands	6dfae7c42d	Use golangci-lint v1.51.2 in build image (#8587 ) What this PR does / why we need it: Golangci-lint [1.50.0 OOMs](https://github.com/golangci/golangci-lint/issues/3538#issuecomment-1414341372) when used with go 1.20.1. This also updates the build image. Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` --------- Signed-off-by: Michel Hollands <michel.hollands@grafana.com> Signed-off-by: Christian Haudum <christian.haudum@gmail.com> Co-authored-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
李国忠	b78ed9d106	[RFR] fix vector() function produces wrong timestamp on instant query (#8370 )	2 years ago
Aditya C S	b94c8aaf16	feature: geoip stage in promtail (#3493 ) Co-authored-by: Ed Welch <edward.welch@grafana.com>	2 years ago
Michel Hollands	272ebdec3c	Update build image to go 1 20 1 and alpine 3 16 4 (#8575 ) Signed-off-by: Michel Hollands <michel.hollands@grafana.com> Signed-off-by: Christian Haudum <christian.haudum@gmail.com> Co-authored-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
Christophe Collot	a013e9f342	Lambda-promtail: Add support for processing SQS messages, add promtailClient Type, add logger, upgrade dependencies and fix unexpected flushing behaviors (#8231 )	2 years ago
Yasmine El wahbi	c6542e61fa	Promtail: Add ready endoint to loki-push-api server (#8552 ) When trying to expose loki_push_api service (in our case using GKE ingress), we need a health check target endpoint that would return a 200 response code. This PR contains a simple addition for a HTTP handler that handles the /ready path, along with the necessary tests. Fixes #8551 Co-authored-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
a5r0n	0f0553ff1c	helm: make multi tenant easy: use tenant id from nginx $remote_user (#8404 ) Fixes #8380 Co-authored-by: a5r0n <a5r0n@users.noreply.github.com>	2 years ago
Tristan Morgan	2e11ff00eb	Replace deprecated inet.af/netaddr (#8560 ) This removes the deprecated `inet.af/netaddr` that imports `go4.org/unsafe/assume-no-moving-gc` and panic when it's not updated. Fixes #8399 and replaces #8495	2 years ago
James Callahan	6d59b4144e	lambda-promtail: Various Terraform fixes (#8549 ) * lambda-promtail: omit_extra_labels_prefix is a boolean This can be changed without a migration as it hasn't been in a release yet * lambda-promtail: fix typo * lambda-promtail: declare variables as a set * lambda-promtail: add a versions file with provider requirements * lambda-promtail: remove provider block - There's no reason to specify a region here - Empty provider blocks are not needed in terraform for quite some time now (and infact they end up producing a warning about their deprecation) * lambda-promtail: bucket_names can be the empty array Without this, you get an error that `resources` cannot be the empty list * lambda-promtail: convert inline policy to a aws_iam_policy_document	2 years ago
dependabot[bot]	3d1ef31af4	Bump golang.org/x/net from 0.5.0 to 0.7.0 in /tools/lambda-promtail (#8556 ) Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Michel Hollands <42814411+MichelHollands@users.noreply.github.com>	2 years ago
Kaviraj Kanagaraj	43a21eac03	fix(promtail): Make `cri` tags streams aware. (#8497 )	2 years ago
Karsten Jeschkies	880de93093	Document memcached configuration for caching. (#8373 ) What this PR does / why we need it: Memcached is our default options for caching. However, the options for Memcached are not well documented. This change documents how Memcached should be configured and how Loki itself should be documented based on our ksonnet deployment. Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [x] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` Co-authored-by: J Stickler <julie.stickler@grafana.com>	2 years ago
Karsten Jeschkies	2f8d771c6b	Fix logcli parallel download deadlock. (#8553 )	2 years ago
justcompile	ea55f285b6	S3 Storage Class Configuration (#8532 )	2 years ago
Christian Haudum	20696522fe	lambda-promtail: Add option to omit extra labels prefix `__extra_` (#8548 ) If you want to add an extra label env, lambda-promtail would add the __extra_ prefix, so that the resulting label is __extra_env. However, if you already have the env variable in other stream and you want to have consistent label naming, you would want to omit the prefix so the resulting label is env. Closes https://github.com/grafana/loki/issues/8471 Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
Christian Haudum	ff39654a99	Fix regression in lambda-promtail tool that yields incorrect label names (#8547 ) Commit `4d5678aa17 (diff-c898e72f3b15b46b32f9405b243b7d5b56b68db7bf03541fefd587dc9e49af8eL70-R90)` introduced a regression that caused labels to have duplicate `_lb` parts in the label names, as described in https://github.com/grafana/loki/issues/8470 Fixes #8470 Also updated Loki to the latest commit on `main`. Previously the dependency pointed to a Git commit that was out of the tree of this repository.	2 years ago
Angus Dippenaar	42522d794a	logcli: Add parallel flags (#8518 )	2 years ago
Vladyslav Diachenko	4e4359e67c	Fixed XSS on LogQL Analyzer page (#8521 ) Signed-off-by: Vladyslav Diachenko <vlad.diachenko@grafana.com>	2 years ago
Karsten Jeschkies	1875a1d6d8	Fetch all entries with logcli if limit==0. (#8537 )	2 years ago
Gerard Vanloo	08ac4f689b	operator: Update Promtail and LogCLI addons to v2.7.3 (#8536 )	2 years ago
Garrett	433d5bf913	fix panics when cloning a special query (#8531 ) Signed-off-by: garrettlish <garrett.li.sh@gmail.com>	2 years ago
Periklis Tsirakidis	099ca230b1	operator: Provide community operator bundle (#7308 ) Co-authored-by: Robert Jacob <rojacob@redhat.com>	2 years ago
bcirh	0394b56368	Fix missing braces for service stanza (#8510 )	2 years ago
Christian Haudum	d960ee70c6	Helm: Remove unreleased setting from Helm chart configuration file (#8515 ) Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
Michal Vasilek	0fc6cd4b01	operator: update assume-no-moving-gc to fix go 1.20 (#8507 )	2 years ago
Hervé Nicol	b04ad2e409	helm: azure config: don't set use_federated_token if false (#8513 )	2 years ago
brettlounsbury	4c227bd268	DynamoDB: Accept a KMS key for DynamoDB tables (#8469 ) Co-authored-by: Travis Patterson <travis.patterson@grafana.com>	2 years ago
J Stickler	06a6c7c9fc	Temporarily unpublish Analyzer page. (#8503 )	2 years ago
Susana Ferreira	00d5142c40	Helm: add sleep argument to helm makefile on local development (#8502 )	2 years ago
Hervé Nicol	f4119de4df	helm: values to override read/write/backend paths in gateway config (#8490 ) What this PR does / why we need it: Add configurations to override the generated read/write/backend URLs for the gateway. Which issue(s) this PR fixes: Fixes #8488	2 years ago
Tobias Abarbanell	d1d47b2fb2	Loki Documentation: corrected config example for loki.storage (#8277 )	2 years ago
Owen Diehl	6a7403c4f5	correctly calculate max shards (#8494 )	2 years ago
Susana Ferreira	27d5b83c24	Helm: add debug option to helm command for verbose output on local development (#8479 )	2 years ago
Ed Welch	9f0834793b	Loki: set a maximum number of shards for "limited" queries instead of fixed number (#8487 ) Signed-off-by: Edward Welch <edward.welch@grafana.com>	2 years ago
Ed Welch	37169ca444	Loki: Process "Limited" queries sequentially and not in parallel (#8482 ) Signed-off-by: Edward Welch <edward.welch@grafana.com>	2 years ago
李国忠	1fc00fa02d	[new] chunk backend: Integrate Alibaba Cloud oss (#8410 )	2 years ago

1 2 3 4 5 ...

4678 Commits (d4a6b159fab621e4072a5173ce082d7f5d54ec13) All Branches Search

4678 Commits (d4a6b159fab621e4072a5173ce082d7f5d54ec13)

All Branches