apitech/loki - loki - Gitea: Git with a cup of tea

Commit Graph

Author	SHA1	Message	Date
Ed Welch	79b876b65d	feat!: update helm chart to support distributed mode and 3.0 (#12067 ) Signed-off-by: Edward Welch <edward.welch@grafana.com> Co-authored-by: Trevor Whitney <trevorjwhitney@gmail.com> Co-authored-by: Dylan Guedes <djmgguedes@gmail.com> Co-authored-by: J Stickler <julie.stickler@grafana.com>	2 years ago
Trevor Whitney	9e275792e3	Fix k3d and docker-compose local development envs (#10387 ) This PR fixes bugs in two of the local dev envs we have. * production/docker was missing a required CLI flag for the backend target * tools/dev/k3d was failing to bring up a k8s cluster due to an issue with the prometheus operator crd bundled with the kube-prometheus-stack chart.	2 years ago
Paul Rogers	6a4ba250a6	Bump grafana and grafana-enterprise versions used for GEL Helm env (#9368 ) What this PR does / why we need it: Bumps the version of Grafana/Grafana-Enterprise being used to what is currently current. Which issue(s) this PR fixes: This gets rid of a warning displayed about "The current Grafana version is not supported" in relation to the GEL plugin.	3 years ago
Trevor Whitney	594ca50f6f	Fix GF_INSTALL_PLUGINS syntax for k3d env GEL plugin (#9300 ) What this PR does / why we need it: Not sure if this is a change to Grafana, but while the old sytnax used to work, it no longer does, and needs this `;grafana-enterprise-logs-app` suffix, which appears to fix things.	3 years ago
Trevor Whitney	0f4a7f4d92	[helm] Bump GEL version, default to 3 scalable targets (#9170 ) What this PR does / why we need it: This PR bumps the GEL version to match the Loki version that was added in version 5 of the helm chart. With these new versions we can now default to 3 targets in scalable mode `read`, `write`, and `backend`. This PR also fixes 2 bugs when running in scalable mode: * the gateway should forward requests to the admin api to the `backend` target when running 3 targets * the index gateway should default to using `ring` mode, which will allow the queriers in the new `read` target to find the index gateway address for their index gateway clients via the ring.	3 years ago
Trevor Whitney	dac3b84d08	Loki cloud integration instructions (and necessary mixin changes) (#8492 ) What this PR does / why we need it: This PR is a first stab at documentation for setting up the Grafana Cloud Loki Integration to monitor a local Loki cluster, installed using the helm chart, using Grafana Cloud. In addition to instructions on how to collect the necessary kubernetes metrics, there were also some things with the Loki mixin (which provides dashboards, alerts, and rules to the integration) that needed to be fixed. For example, our dashboards rely on there being a cluster label on our recording rules, yet our mixin does not include this? Also, we are still including `cortex-gw` panels even when the mixin is compile with `internal_components: false`. Special notes for your reviewer: The changes to the mixin will not be reflected in the current version of the integration. We need to merge this PR first then update the integration. Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` --------- Co-authored-by: Michel Hollands <42814411+MichelHollands@users.noreply.github.com> Co-authored-by: J Stickler <julie.stickler@grafana.com>	3 years ago
Susana Ferreira	b8118b1e7b	Helm enterprise-logs: update kubernetes secrets creation (#8467 ) What this PR does / why we need it: Fix the Kubernetes secrets creation for the tokengen and provisioner jobs for multiple helm installations/uninstallations. The execution flow for the helm installation is the following: 1) `tokengen` job `loki` container generates a new admin token, stores it in the object storage and writes it to the `shared` mount. 2) `tokengen` job `create-secret` container creates a new Kubernetes secret for the generated admin token. 3) `provisioner` job `provisioner` container creates the new admin resources via the Admin API and writes the tokens to the `bootstrap` mount. 4) `provisioner` job `create-secret` container creates new Kubernetes secrets for each token of the additional tenants. ## Previous implementation Uninstall and re-install the enterprise-logs helm: 1) `tokengen` job `loki` container generates a new admin token, stores it in the object storage, and writes it to the `shared` mount. This is because this job uses an [emptyDir](https://kubernetes.io/docs/concepts/storage/volumes/#emptydir) mount which is deleted once the job completes. When a new job is created, the directory will be again empty, forcing the `tokengen` to generate a new token. 2) `tokengen` job `create-secret` container fails, because an admin token secret already exists: ``` Defaulted container "create-secret" out of: create-secret, loki (init) error: failed to create secret secrets "enterprise-logs-admin-token" already exists ``` 3) The helm installation fails ❌ ## Implementation in this PR 1) `tokengen` job `loki` container generates a new admin token, stores it in the object storage, and writes it to the `shared` mount. 2) `tokengen` job `create-secret` container updates the previously created Kubernetes secret for the admin token. 3) `provisioner` job `provisioner` container checks that the admin resources already exist and doesn't write the token files to the `bootstrap` mount. 4) `provisioner` job `create-secret` container checks that the token files are not present on the `bootstrap` mount, and does not create new secrets. 5) The helm installation succeeds ✅ Special notes for your reviewer: Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	3 years ago
Susana Ferreira	00d5142c40	Helm: add sleep argument to helm makefile on local development (#8502 )	3 years ago
Susana Ferreira	27d5b83c24	Helm: add debug option to helm command for verbose output on local development (#8479 )	3 years ago
Trevor Whitney	ef8b382c06	[helm] Single binary object storage (#8284 ) Co-authored-by: J Stickler <julie.stickler@grafana.com>	3 years ago
Trevor Whitney	9920a27f06	Update GEL plugin in k3d env to latest (#8378 )	3 years ago
Trevor Whitney	cc39334391	[helm] Fix helm dashboards (#8261 ) Fix the dashboards shipped with the helm chart. Co-authored-by: Karsten Jeschkies <k@jeschkies.xyz> Co-authored-by: J Stickler <julie.stickler@grafana.com>	3 years ago
Trevor Whitney	71846b0430	[helm] add local k3d loki cluster (#8218 )	3 years ago
Trevor Whitney	40ed217620	Make secret job fail when not logged in (#8215 ) Co-authored-by: Susana Ferreira <ssncferreira@gmail.com>	3 years ago
Karsten Jeschkies	44e8422dc4	Enalbe multi tenant queries on Helm test chart. (#8228 ) Co-authored-by: Christian Haudum <christian.haudum@gmail.com>	3 years ago
Trevor Whitney	b6004050a6	[helm] Fix self-monitoring tenant secret (#8198 ) What this PR does / why we need it: https://github.com/grafana/loki/pull/8081 introduced a bug that caused the `provisioner` job to create an incorrect secret for the self-monitoring tenant used by the canary. This fixes that bug, as well as adding a readme on how to use the `helm-cluster` k3d env. Special notes for your reviewer: Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [x] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	3 years ago
Trevor Whitney	eb39e2645f	[helm] allow provisioned tokens to be created in different namespaces (#8081 ) What this PR does / why we need it: _Note_: This PR bumps the chart to 4.0, so it includes a few other breaking changes we've been waiting to make. The new helm chart has a few different ways to create tokens when running in enterprise mode: * The tokengen job creates the initial admin token * The provisioner job creates the self-monitoring tenant if enabled * The provisioner job can also create additional tenants if configured The problem with these jobs is they put the generated read/write tokens for these tenants in secrets in the same namespace as the loki deploiyment. This is problematic if you need these tokens for a specific application, for example a Grafana datasource, that is deployed in a separate namespace. This PR introduces two breaking changes, one of which is breaking as it significantly changes how both the self monitoring and additional tenants are configured. First it adds the value `enterprise.adminToken.additionalNamespaces` and moves `enterprise.adminTokenSecret` to `enterprise.adminToken.secret`. If `additionalNamespaces` are included, additional secrets containing the GEL admin token are created in those namespaces as well as the release namespace. Second, it changes the structure of defining a tenant from just a string name, to an object containing a `name` and `secretNamespace` field. The `secretNamespace` is the namespace the secret containing that tenants token(s) will be placed in. In the case of the self-monitoring token, a secret will still be created in the default namespace in addition to the `secretNamespace` specified as that token is required by multiple components.	3 years ago
Trevor Whitney	712bfdd0e0	[helm] add third scalable target `backend` (#7920 ) What this PR does / why we need it: Adds the new 3rd scalable deployment mode target (backend) to the helm chart. This new target was added in #7650	3 years ago
Trevor Whitney	3f9b9cc961	[helm] fix race between minio bucket job and enterprise tokengen job (#8064 ) What this PR does / why we need it: This PR replaces https://github.com/grafana/loki/pull/7985. It solves the same problem, which is the race condition between the minio create bucket job and the enterprise tokengen job, but without removing minio.	3 years ago
Trevor Whitney	98fd01aef8	Add empty k3d cluster for testing helm chart (#7984 ) What this PR does / why we need it: Add a new k3d cluster which is a cluster that has all Loki dependencies for purposes of testing the helm charts using `helm install`. The other environments use the helm charts, but deploy them using `tanka` and therefore can't catch certain race conditions or issues that occur with helm installs.	3 years ago
Trevor Whitney	6c53113be1	[helm] Add a loki canary test to the helm chart (#7229 ) What this PR does / why we need it: Add a helm chart test that utilizes the Loki canary to confirm the Loki cluster is functioning correctly. Which issue(s) this PR fixes: Fixes #7228 Co-authored-by: Karsten Jeschkies <k@jeschkies.xyz>	3 years ago
Karsten Jeschkies	9bc6e85604	Helm: Allow users to define remote write for metrics. (#7399 )	3 years ago
Trevor Whitney	afd63c5981	[helm] Add Loki Canary to Helm Chart (#7173 ) What this PR does / why we need it: This PR adds the Loki canary as well as the GEL provisioner to the Loki helm chart. The Loki canary expands the self-monitoring functionality of the Helm chart, giving operators insight beyond whether Loki is just running, but also if it's functioning properly. The GEL provisioner was necessary for the canary to work when the enterprise option (`enterprise.enabled = true`) is enabled. Since enabling enterprise in the helm chart also enables auth/multi-tenancy, the addition of the provisioner allows the helm chart to automatically create a tenant for the canary via the GEL admin API, and to create applicable access policies and tokens. Furthermore, additional tenants can also be specified, which will be provisioned, creating a read and write token for each that will be stored in k8s secrets. Which issue(s) this PR fixes: Fixes #7019 Co-authored-by: Karsten Jeschkies <k@jeschkies.xyz> Co-authored-by: Dylan Guedes <djmgguedes@gmail.com>	3 years ago
Trevor Whitney	4b6df86333	Add local k3d environment for enterprise-logs (#7146 ) What this PR does / why we need it: This adds a local k3d based development cluster for testing out the enterprise logs side of the helm chart, which can be enabled by setting: ``` enterprise: enabled: true ``` It requires both the `gex-plugins` and `enterprise-logs` repos to be checked out locally (though the latter soon will not be needed once the provisioner is published). We need this environment when developing enterprise features for the helm chart.	3 years ago
Karsten Jeschkies	c23e541fe7	Generate Helm chart README. (#7012 ) What this PR does / why we need it: The Helm CI would break because the Helm chart README was not generated. This change * updates the README * introduces a filter that will only run the Helm CI on changes to the Helm chart directory * removes trailing whitespace * corrects the Minio chart repository	3 years ago
Karsten Jeschkies	1c3f5d0985	Helm: Upgrade Minio chart dependency to 4.0.12. (#6974 ) What this PR does / why we need it: Changes to a supported Minio chart. Minio is also enabled by default. Which issue(s) this PR fixes: Fixes #6973 Checklist - [x] Documentation added - [ ] Tests updated - [ ] Is this an important fix or new feature? Add an entry in the `CHANGELOG.md`. - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	3 years ago
Salva Corts	651ac093b6	Fix circular dependencies (#5420 )	4 years ago
Trevor Whitney	c477e29c3f	Add tooling for local k3d clusters under tools/dev (#5272 ) * add k3d folder with loki-distributed env * add readme * shellcheck create_cluster.sh * lint jsonnet * Update grafana image to latest	4 years ago

28 Commits (a2c8ec7dc11546e2876e159e76eabd727ebc1929)