[helm] Single binary object storage (#8284)

Co-authored-by: J Stickler <julie.stickler@grafana.com>

docs/sources/installation/helm/install-monolithic/index.md

@@ -12,9 +12,8 @@ keywords: []
 
 This Helm Chart installation runs the Grafana Loki *single binary* within a Kubernetes cluster.
 
-If the storage type is set to `filesystem`, this chart configures Loki to run the `all` target in a [monolithic mode]({{<relref "../../../fundamentals/architecture/deployment-modes#monolithic-mode">}}), designed to work with a filesystem storage. It will also configure meta-monitoring of metrics and logs.
-
-It is not possible to install the single binary with a different storage type.
+If you set the `singleBinary.replicas` value to 1, this chart configures Loki to run the `all` target in a [monolithic mode]({{<relref "../../../fundamentals/architecture/deployment-modes#monolithic-mode">}}), designed to work with a filesystem storage. It will also configure meta-monitoring of metrics and logs.
+If you set the `singleBinary.replicas` value to 2 or more, this chart configures Loki to run a *single binary* in a replicated, highly available mode.  When running replicas of a single binary, you must configure object storage.
 
 **Before you begin: Software Requirements**
 
@@ -35,9 +34,9 @@ It is not possible to install the single binary with a different storage type.
     helm repo update
     ```
 
-1. Configure the `filesystem` storage:
+1. Create the configuration file `values.yaml`:
 
-    - Create the configuration file `values.yaml`:
+    - If running a single replica of Loki, configure the `filesystem` storage:
 
       ```yaml
       loki:
@@ -45,6 +44,25 @@ It is not possible to install the single binary with a different storage type.
           replication_factor: 1
         storage:
           type: 'filesystem'
+      singleBinary:
+        replicas: 1
+      ```
+
+    - If running Loki with a replication factor greater than 1, set the desired number replicas and provide object storage credentials:
+
+      ```yaml
+      loki:
+        commonConfig:
+          replication_factor: 3
+        storage:
+          type: 's3'
+          s3:
+            endpoint: foo.aws.com
+            bucketnames: loki-chunks
+            secret_access_key: supersecret
+            access_key_id: secret
+      singleBinary:
+        replicas: 3
       ```
 
 1. Deploy the Loki cluster using one of these commands.
@@ -58,5 +76,5 @@ It is not possible to install the single binary with a different storage type.
     - Deploy with the defined configuration in a custom Kubernetes cluster namespace:
 
         ```bash
-        helm install --values values.yaml loki --namespace=loki grafana/loki-simple-scalable
+        helm install --values values.yaml loki --namespace=loki grafana/loki
         ```

docs/sources/installation/helm/monitor-and-alert/index.md

@@ -15,7 +15,7 @@ keywords:
 
 By default this Helm Chart configures meta-monitoring of metrics (service monitoring) and logs (self monitoring).
 
-The `ServiceMonitor` resource works with either the Prometheus Operator or the Grafana Agent Operator, and defines how Loki's metrics should be scraped. Scraping this Loki cluster using the scrape config defined in the `ServiceMonitor` resource is required for the included dashboards to work. A `MetricsInstance` can be configured to write the metrics to a remote Prometheus instance such as Grafana Cloud Metrics.
+The `ServiceMonitor` resource works with either the Prometheus Operator or the Grafana Agent Operator, and defines how Loki's metrics should be scraped. Scraping this Loki cluster using the scrape config defined in the `SerivceMonitor` resource is required for the included dashboards to work. A `MetricsInstance` can be configured to write the metrics to a remote Prometheus instance such as Grafana Cloud Metrics.
 
 _Self monitoring_ is enabled by default. This will deploy a `GrafanaAgent`, `LogsInstance`, and `PodLogs` resource which will instruct the Grafana Agent Operator (installed seperately) on how to scrape this Loki cluster's logs and send them back to itself. Scraping this Loki cluster using the scrape config defined in the `PodLogs` resource is required for the included dashboards to work.
 

docs/sources/installation/helm/reference.md

@@ -3167,7 +3167,7 @@ null
 			<td>int</td>
 			<td>Number of replicas for the single binary</td>
 			<td><pre lang="json">
-1
+0
 </pre>
 </td>
 		</tr>

production/helm/loki/CHANGELOG.md

@@ -13,6 +13,10 @@ Entries should include a reference to the pull request that introduced the chang
 
 [//]: # (<AUTOMATED_UPDATES_LOCATOR> : do not remove this line. Add your changelog bellow this line. This locator is used by CI pipeline to find the place where to put changelog entry.)
 
+## 4.5
+
+- [ENHANCEMENT] Single binary mode is now possible for more than 1 replica, with a gateway and object storage backend.
+
 ## 4.4.2
 
 - [CHANGE] Bump Loki version to 2.7.2 and GEL version to 1.6.1

production/helm/loki/Chart.yaml

@@ -3,8 +3,8 @@ apiVersion: v2
 name: loki
 description: Helm chart for Grafana Loki in simple, scalable mode
 type: application
-appVersion: 2.7.3
-version: 4.4.2
+appVersion: 2.7.2
+version: 4.5.0
 home: https://grafana.github.io/helm-charts
 sources:
   - https://github.com/grafana/loki

production/helm/loki/README.md

@@ -1,6 +1,6 @@
 # loki
 
-![Version: 4.4.2](https://img.shields.io/badge/Version-4.4.2-informational?style=flat-square) ![Type: application](https://img.shields.io/badge/Type-application-informational?style=flat-square) ![AppVersion: 2.7.3](https://img.shields.io/badge/AppVersion-2.7.3-informational?style=flat-square)
+![Version: 4.5.0](https://img.shields.io/badge/Version-4.5.0-informational?style=flat-square) ![Type: application](https://img.shields.io/badge/Type-application-informational?style=flat-square) ![AppVersion: 2.7.2](https://img.shields.io/badge/AppVersion-2.7.2-informational?style=flat-square)
 
 Helm chart for Grafana Loki in simple, scalable mode
 

production/helm/loki/templates/_helpers.tpl

@@ -50,14 +50,15 @@ Params:
 Return if deployment mode is simple scalable
 */}}
 {{- define "loki.deployment.isScalable" -}}
-  {{- eq (include "loki.isUsingObjectStorage" . ) "true" }}
+  {{- and (eq (include "loki.isUsingObjectStorage" . ) "true") (eq (int .Values.singleBinary.replicas) 0) }}
 {{- end -}}
 
 {{/*
 Return if deployment mode is single binary
 */}}
 {{- define "loki.deployment.isSingleBinary" -}}
-  {{- eq (include "loki.isUsingObjectStorage" . ) "false" }}
+  {{- $nonZeroReplicas := gt (int .Values.singleBinary.replicas) 0 }}
+  {{- or (eq (include "loki.isUsingObjectStorage" . ) "false") ($nonZeroReplicas) }}
 {{- end -}}
 
 {{/*
@@ -573,106 +574,87 @@ http {
       auth_basic off;
     }
 
+    {{- $backendHost := include "loki.backendFullname" .}}
+    {{- $readHost := include "loki.readFullname" .}}
+    {{- $writeHost := include "loki.writeFullname" .}}
+
+    {{- if .Values.read.legacyReadTarget }}
+    {{- $backendHost = include "loki.readFullname" . }}
+    {{- end }}
+
+    {{- if gt (int .Values.singleBinary.replicas) 0 }}
+    {{- $backendHost = include "loki.singleBinaryFullname" . }}
+    {{- $readHost = include "loki.singleBinaryFullname" .}}
+    {{- $writeHost = include "loki.singleBinaryFullname" .}}
+    {{- end }}
+
     location = /api/prom/push {
-      proxy_pass       http://{{ include "loki.writeFullname" . }}.{{ .Release.Namespace }}.svc.{{ .Values.global.clusterDomain }}:3100$request_uri;
+      proxy_pass       http://{{ $writeHost }}.{{ .Release.Namespace }}.svc.{{ .Values.global.clusterDomain }}:3100$request_uri;
     }
 
     location = /api/prom/tail {
-      proxy_pass       http://{{ include "loki.readFullname" . }}.{{ .Release.Namespace }}.svc.{{ .Values.global.clusterDomain }}:3100$request_uri;
+      proxy_pass       http://{{ $readHost }}.{{ .Release.Namespace }}.svc.{{ .Values.global.clusterDomain }}:3100$request_uri;
       proxy_set_header Upgrade $http_upgrade;
       proxy_set_header Connection "upgrade";
     }
 
     location ~ /api/prom/.* {
-      proxy_pass       http://{{ include "loki.readFullname" . }}.{{ .Release.Namespace }}.svc.{{ .Values.global.clusterDomain }}:3100$request_uri;
+      proxy_pass       http://{{ $readHost }}.{{ .Release.Namespace }}.svc.{{ .Values.global.clusterDomain }}:3100$request_uri;
     }
 
-    {{- if .Values.read.legacyReadTarget }}
-    location ~ /prometheus/api/v1/alerts.* {
-      proxy_pass       http://{{ include "loki.readFullname" . }}.{{ .Release.Namespace }}.svc.{{ .Values.global.clusterDomain }}:3100$request_uri;
-    }
-    location ~ /prometheus/api/v1/rules.* {
-      proxy_pass       http://{{ include "loki.readFullname" . }}.{{ .Release.Namespace }}.svc.{{ .Values.global.clusterDomain }}:3100$request_uri;
-    }
-    location ~ /ruler/.* {
-      proxy_pass       http://{{ include "loki.readFullname" . }}.{{ .Release.Namespace }}.svc.{{ .Values.global.clusterDomain }}:3100$request_uri;
-    }
-    {{- else }}
     location ~ /prometheus/api/v1/alerts.* {
-      proxy_pass       http://{{ include "loki.backendFullname" . }}.{{ .Release.Namespace }}.svc.{{ .Values.global.clusterDomain }}:3100$request_uri;
+      proxy_pass       http://{{ $backendHost }}.{{ .Release.Namespace }}.svc.{{ .Values.global.clusterDomain }}:3100$request_uri;
     }
     location ~ /prometheus/api/v1/rules.* {
-      proxy_pass       http://{{ include "loki.backendFullname" . }}.{{ .Release.Namespace }}.svc.{{ .Values.global.clusterDomain }}:3100$request_uri;
+      proxy_pass       http://{{ $backendHost }}.{{ .Release.Namespace }}.svc.{{ .Values.global.clusterDomain }}:3100$request_uri;
     }
     location ~ /ruler/.* {
-      proxy_pass       http://{{ include "loki.backendFullname" . }}.{{ .Release.Namespace }}.svc.{{ .Values.global.clusterDomain }}:3100$request_uri;
+      proxy_pass       http://{{ $backendHost }}.{{ .Release.Namespace }}.svc.{{ .Values.global.clusterDomain }}:3100$request_uri;
     }
-    {{- end }}
 
     location = /loki/api/v1/push {
-      proxy_pass       http://{{ include "loki.writeFullname" . }}.{{ .Release.Namespace }}.svc.{{ .Values.global.clusterDomain }}:3100$request_uri;
+      proxy_pass       http://{{ $writeHost }}.{{ .Release.Namespace }}.svc.{{ .Values.global.clusterDomain }}:3100$request_uri;
     }
 
     location = /loki/api/v1/tail {
-      proxy_pass       http://{{ include "loki.readFullname" . }}.{{ .Release.Namespace }}.svc.{{ .Values.global.clusterDomain }}:3100$request_uri;
+      proxy_pass       http://{{ $readHost }}.{{ .Release.Namespace }}.svc.{{ .Values.global.clusterDomain }}:3100$request_uri;
       proxy_set_header Upgrade $http_upgrade;
       proxy_set_header Connection "upgrade";
     }
 
-    {{- if .Values.read.legacyReadTarget }}
     location ~ /compactor/.* {
-      proxy_pass       http://{{ include "loki.readFullname" . }}.{{ .Release.Namespace }}.svc.{{ .Values.global.clusterDomain }}:3100$request_uri;
+      proxy_pass       http://{{ $backendHost }}.{{ .Release.Namespace }}.svc.{{ .Values.global.clusterDomain }}:3100$request_uri;
     }
-    {{- else }}
-    location ~ /compactor/.* {
-      proxy_pass       http://{{ include "loki.backendFullname" . }}.{{ .Release.Namespace }}.svc.{{ .Values.global.clusterDomain }}:3100$request_uri;
-    }
-    {{- end }}
 
     location ~ /distributor/.* {
-      proxy_pass       http://{{ include "loki.writeFullname" . }}.{{ .Release.Namespace }}.svc.{{ .Values.global.clusterDomain }}:3100$request_uri;
+      proxy_pass       http://{{ $writeHost }}.{{ .Release.Namespace }}.svc.{{ .Values.global.clusterDomain }}:3100$request_uri;
     }
 
     location ~ /ring {
-      proxy_pass       http://{{ include "loki.writeFullname" . }}.{{ .Release.Namespace }}.svc.{{ .Values.global.clusterDomain }}:3100$request_uri;
+      proxy_pass       http://{{ $writeHost }}.{{ .Release.Namespace }}.svc.{{ .Values.global.clusterDomain }}:3100$request_uri;
     }
 
     location ~ /ingester/.* {
-      proxy_pass       http://{{ include "loki.writeFullname" . }}.{{ .Release.Namespace }}.svc.{{ .Values.global.clusterDomain }}:3100$request_uri;
+      proxy_pass       http://{{ $writeHost }}.{{ .Release.Namespace }}.svc.{{ .Values.global.clusterDomain }}:3100$request_uri;
     }
 
-    {{- if .Values.read.legacyReadTarget }}
-    location ~ /store-gateway/.* {
-      proxy_pass       http://{{ include "loki.readFullname" . }}.{{ .Release.Namespace }}.svc.{{ .Values.global.clusterDomain }}:3100$request_uri;
-    }
-    {{- else }}
     location ~ /store-gateway/.* {
-      proxy_pass       http://{{ include "loki.backendFullname" . }}.{{ .Release.Namespace }}.svc.{{ .Values.global.clusterDomain }}:3100$request_uri;
+      proxy_pass       http://{{ $backendHost }}.{{ .Release.Namespace }}.svc.{{ .Values.global.clusterDomain }}:3100$request_uri;
     }
-    {{- end }}
 
-    {{- if .Values.read.legacyReadTarget }}
-    location ~ /query-scheduler/.* {
-      proxy_pass       http://{{ include "loki.readFullname" . }}.{{ .Release.Namespace }}.svc.{{ .Values.global.clusterDomain }}:3100$request_uri;
-    }
-    location ~ /scheduler/.* {
-      proxy_pass       http://{{ include "loki.readFullname" . }}.{{ .Release.Namespace }}.svc.{{ .Values.global.clusterDomain }}:3100$request_uri;
-    }
-    {{- else }}
     location ~ /query-scheduler/.* {
-      proxy_pass       http://{{ include "loki.backendFullname" . }}.{{ .Release.Namespace }}.svc.{{ .Values.global.clusterDomain }}:3100$request_uri;
+      proxy_pass       http://{{ $backendHost }}.{{ .Release.Namespace }}.svc.{{ .Values.global.clusterDomain }}:3100$request_uri;
     }
     location ~ /scheduler/.* {
-      proxy_pass       http://{{ include "loki.backendFullname" . }}.{{ .Release.Namespace }}.svc.{{ .Values.global.clusterDomain }}:3100$request_uri;
+      proxy_pass       http://{{ $backendHost }}.{{ .Release.Namespace }}.svc.{{ .Values.global.clusterDomain }}:3100$request_uri;
     }
-    {{- end }}
 
     location ~ /loki/api/.* {
-      proxy_pass       http://{{ include "loki.readFullname" . }}.{{ .Release.Namespace }}.svc.{{ .Values.global.clusterDomain }}:3100$request_uri;
+      proxy_pass       http://{{ $readHost }}.{{ .Release.Namespace }}.svc.{{ .Values.global.clusterDomain }}:3100$request_uri;
     }
 
     location ~ /admin/api/.* {
-      proxy_pass       http://{{ include "loki.writeFullname" . }}.{{ .Release.Namespace }}.svc.{{ .Values.global.clusterDomain }}:3100$request_uri;
+      proxy_pass       http://{{ $writeHost }}.{{ .Release.Namespace }}.svc.{{ .Values.global.clusterDomain }}:3100$request_uri;
     }
 
     {{- with .Values.gateway.nginxConfig.serverSnippet }}

production/helm/loki/templates/gateway/configmap-gateway.yaml

@@ -1,5 +1,4 @@
-{{- $isSimpleScalable := eq (include "loki.deployment.isScalable" .) "true" -}}
-{{- if and .Values.gateway.enabled $isSimpleScalable }}
+{{- if and .Values.gateway.enabled  }}
 apiVersion: v1
 kind: ConfigMap
 metadata:

production/helm/loki/templates/gateway/deployment-gateway.yaml

@@ -1,5 +1,4 @@
-{{- $isSimpleScalable := eq (include "loki.deployment.isScalable" .) "true" -}}
-{{- if and $isSimpleScalable .Values.gateway.enabled }}
+{{- if .Values.gateway.enabled }}
 apiVersion: apps/v1
 kind: Deployment
 metadata:

production/helm/loki/templates/gateway/hpa.yaml

@@ -1,6 +1,5 @@
-{{- $isSimpleScalable := eq (include "loki.deployment.isScalable" .) "true" -}}
 {{- $autoscalingv2 := .Capabilities.APIVersions.Has "autoscaling/v2" -}}
-{{- if and $isSimpleScalable .Values.gateway.autoscaling.enabled }}
+{{- if .Values.gateway.autoscaling.enabled }}
 {{- if $autoscalingv2 }}
 apiVersion: autoscaling/v2
 {{- else }}

production/helm/loki/templates/gateway/ingress-gateway.yaml

@@ -1,5 +1,4 @@
-{{- $isSimpleScalable := eq (include "loki.deployment.isScalable" .) "true" -}}
-{{- if and $isSimpleScalable .Values.gateway.enabled -}}
+{{- if and .Values.gateway.enabled -}}
 {{- if .Values.gateway.ingress.enabled -}}
 {{- $ingressApiIsStable := eq (include "loki.ingress.isStable" .) "true" -}}
 {{- $ingressSupportsIngressClassName := eq (include "loki.ingress.supportsIngressClassName" .) "true" -}}

production/helm/loki/templates/gateway/poddisruptionbudget-gateway.yaml

@@ -1,5 +1,4 @@
-{{- $isSimpleScalable := eq (include "loki.deployment.isScalable" .) "true" -}}
-{{- if and $isSimpleScalable .Values.gateway.enabled }}
+{{- if and .Values.gateway.enabled }}
 {{- if gt (int .Values.gateway.replicas) 1 }}
 apiVersion: {{ include "loki.podDisruptionBudget.apiVersion" . }}
 kind: PodDisruptionBudget

production/helm/loki/templates/gateway/secret-gateway.yaml

@@ -1,6 +1,5 @@
-{{- $isSimpleScalable := eq (include "loki.deployment.isScalable" .) "true" -}}
 {{- with .Values.gateway }}
-{{- if and $isSimpleScalable .enabled .basicAuth.enabled (not .basicAuth.existingSecret) }}
+{{- if and .enabled .basicAuth.enabled (not .basicAuth.existingSecret) }}
 apiVersion: v1
 kind: Secret
 metadata:

production/helm/loki/templates/gateway/service-gateway.yaml

@@ -1,5 +1,4 @@
-{{- $isSimpleScalable := eq (include "loki.deployment.isScalable" .) "true" -}}
-{{- if and $isSimpleScalable .Values.gateway.enabled }}
+{{- if .Values.gateway.enabled }}
 apiVersion: v1
 kind: Service
 metadata:

production/helm/loki/templates/single-binary/_helpers-single-binary.tpl

@@ -22,3 +22,13 @@ singleBinary priority class name
 priorityClassName: {{ $pcn }}
 {{- end }}
 {{- end }}
+
+{{/* singleBinary replicas calculation */}}
+{{- define "loki.singleBinaryReplicas" -}}
+{{- $replicas := 1 }}
+{{- $usingObjectStorage := eq (include "loki.isUsingObjectStorage" .) "true" }}
+{{- if and $usingObjectStorage (gt (int .Values.singleBinary.replicas) 1)}}
+{{- $replicas = int .Values.singleBinary.replicas -}}
+{{- end }}
+{{- printf "%d" $replicas }}
+{{- end }}

production/helm/loki/templates/single-binary/service.yaml

@@ -4,7 +4,7 @@
 apiVersion: v1
 kind: Service
 metadata:
-  name: {{ include "loki.name" . }}
+  name: {{ include "loki.singleBinaryFullname" . }}
   labels:
     {{- include "loki.labels" . | nindent 4 }}
 spec:
@@ -23,4 +23,4 @@ spec:
     {{- with .Values.singleBinary.selectorLabels }}
       {{- tpl (toYaml .) $ | nindent 4 }}
     {{- end }}
-{{- end }}
+{{- end }}

production/helm/loki/templates/single-binary/statefulset.yaml

@@ -9,7 +9,7 @@ metadata:
     {{- include "loki.singleBinaryLabels" . | nindent 4 }}
     app.kubernetes.io/part-of: memberlist
 spec:
-  replicas: {{ .Values.singleBinary.replicas }}
+  replicas: {{ include "loki.singleBinaryReplicas" . }}
   podManagementPolicy: Parallel
   updateStrategy:
     rollingUpdate:

production/helm/loki/templates/tokengen/clusterrole-tokengen.yaml

@@ -1,5 +1,5 @@
-{{- $isSimpleScalable := eq (include "loki.deployment.isScalable" .) "true" -}}
-{{ if and $isSimpleScalable .Values.enterprise.tokengen.enabled .Values.enterprise.enabled }}
+{{ if and .Values.enterprise.tokengen.enabled .Values.enterprise.enabled }}
+---
 apiVersion: rbac.authorization.k8s.io/v1
 kind: ClusterRole
 metadata:

production/helm/loki/templates/tokengen/clusterrolebinding-tokengen.yaml

@@ -1,5 +1,4 @@
-{{- $isSimpleScalable := eq (include "loki.deployment.isScalable" .) "true" -}}
-{{ if and $isSimpleScalable .Values.enterprise.tokengen.enabled .Values.enterprise.enabled }}
+{{ if and .Values.enterprise.tokengen.enabled .Values.enterprise.enabled }}
 ---
 apiVersion: rbac.authorization.k8s.io/v1
 kind: ClusterRoleBinding

production/helm/loki/templates/tokengen/job-tokengen.yaml

@@ -1,5 +1,4 @@
-{{- $isSimpleScalable := eq (include "loki.deployment.isScalable" .) "true" -}}
-{{ if and $isSimpleScalable .Values.enterprise.tokengen.enabled .Values.enterprise.enabled }}
+{{ if and .Values.enterprise.tokengen.enabled .Values.enterprise.enabled }}
 ---
 apiVersion: batch/v1
 kind: Job

production/helm/loki/templates/tokengen/serviceaccount-tokengen.yaml

@@ -1,5 +1,4 @@
-{{- $isSimpleScalable := eq (include "loki.deployment.isScalable" .) "true" -}}
-{{ if and $isSimpleScalable .Values.enterprise.tokengen.enabled .Values.enterprise.enabled }}
+{{ if and .Values.enterprise.tokengen.enabled .Values.enterprise.enabled }}
 ---
 apiVersion: v1
 kind: ServiceAccount

production/helm/loki/templates/validate.yaml

@@ -14,3 +14,15 @@
 {{- fail "Helm test requires a prometheusAddress for an instance scraping the Loki canary's metrics"}}
 {{- end }}
 
+{{- $singleBinaryReplicas := int .Values.singleBinary.replicas }}
+{{- $isUsingFilesystem := eq (include "loki.isUsingObjectStorage" .) "false" }}
+{{- $atLeastOneScalableReplica := or (gt (int .Values.backend.replicas) 0) (gt (int .Values.read.replicas) 0) (gt (int .Values.write.replicas) 0) }}
+
+{{- if and $isUsingFilesystem (gt $singleBinaryReplicas 1) }}
+{{- fail "Cannot run more than 1 Single Binary replica without an object storage backend."}}
+{{- end }}
+
+{{- if and $isUsingFilesystem (and (eq $singleBinaryReplicas 0) $atLeastOneScalableReplica) }}
+{{- fail "Cannot run Scalable targets (backend, read, write) without an object storage backend."}}
+{{- end }}
+

production/helm/loki/values.yaml

@@ -924,7 +924,7 @@ backend:
 # Configuration for the single binary node(s)
 singleBinary:
   # -- Number of replicas for the single binary
-  replicas: 1
+  replicas: 0
   autoscaling:
     # -- Enable autoscaling, this is only used if `queryIndex.enabled: true`
     enabled: false

tools/dev/k3d/Makefile

@@ -15,6 +15,15 @@ enterprise-logs: prepare-gel helm-cluster
 	$(MAKE) -C $(CURDIR) helm-install-enterprise-logs
 	echo "Helm installation finished. You can tear down this cluster with make down."
 
+enterprise-logs-ha-single-binary: prepare-gel helm-cluster
+	$(MAKE) -C $(CURDIR) apply-enterprise-helm-cluster
+	echo "Waiting 5s for cluster to be ready for helm installation."
+	echo "The helm install will take a while. It's useful to monitor progress using a tool like k9s."
+	# wait 5s for tk apply to finish and cluster is ready for helm install
+	sleep 5
+	$(MAKE) -C $(CURDIR) helm-install-enterprise-logs-ha-single-binary
+	echo "Helm installation finished. You can tear down this cluster with make down."
+
 loki: prepare helm-cluster
 	$(MAKE) -C $(CURDIR) apply-loki-helm-cluster
 	echo "Waiting 5s for cluster to be ready for helm installation."
@@ -23,6 +32,14 @@ loki: prepare helm-cluster
 	$(MAKE) -C $(CURDIR) helm-install-loki
 	echo "Helm installation finished. You can tear down this cluster with make down."
 
+loki-ha-single-binary: prepare helm-cluster
+	$(MAKE) -C $(CURDIR) apply-loki-helm-cluster
+	echo "Waiting 5s for cluster to be ready for helm installation."
+	# wait 5s for tk apply to finish and cluster is ready for helm install
+	sleep 5
+	$(MAKE) -C $(CURDIR) helm-install-loki-ha-single-binary
+	echo "Helm installation finished. You can tear down this cluster with make down."
+
 helm-cluster: prepare
 	$(CURDIR)/scripts/create_cluster.sh helm-cluster $(REGISTRY_PORT)
 	# wait 5s for the cluster to be ready
@@ -102,6 +119,15 @@ helm-upgrade-enterprise-logs:
 helm-uninstall-enterprise-logs:
 	helm uninstall enterprise-logs-test-fixture -n loki
 
+helm-install-enterprise-logs-ha-single-binary:
+	helm install enterprise-logs-test-fixture "$(HELM_DIR)" -n loki --create-namespace --values "$(CURDIR)/environments/helm-cluster/values/enterprise-logs-ha-single-binary.yaml"
+
+helm-upgrade-enterprise-logs-ha-single-binary:
+	helm upgrade enterprise-logs-test-fixture "$(HELM_DIR)" -n loki --values "$(CURDIR)/environments/helm-cluster/values/enterprise-logs-ha-single-binary.yaml"
+
+helm-uninstall-enterprise-logs-ha-single-binary:
+	helm uninstall enterprise-logs-test-fixture -n loki
+
 helm-install-loki:
 	helm install loki "$(HELM_DIR)" -n loki --create-namespace --values "$(CURDIR)/environments/helm-cluster/values/loki.yaml"
 
@@ -110,3 +136,12 @@ helm-upgrade-loki:
 
 helm-uninstall-loki:
 	helm uninstall loki -n loki
+
+helm-install-loki-ha-single-binary:
+	helm install loki-single-binary "$(HELM_DIR)" -n loki --create-namespace --values "$(CURDIR)/environments/helm-cluster/values/loki-ha-single-binary.yaml"
+
+helm-upgrade-loki-ha-single-binary:
+	helm upgrade loki-single-binary "$(HELM_DIR)" -n loki --values "$(CURDIR)/environments/helm-cluster/values/loki-ha-single-binary.yaml"
+
+helm-uninstall-loki-binary:
+	helm uninstall loki-single-binary -n loki

tools/dev/k3d/environments/helm-cluster/main.jsonnet

@@ -2,6 +2,8 @@ local k = import 'github.com/grafana/jsonnet-libs/ksonnet-util/kausal.libsonnet'
 local tanka = import 'github.com/grafana/jsonnet-libs/tanka-util/main.libsonnet';
 local container = k.core.v1.container;
 local configMap = k.core.v1.configMap;
+local deployment = k.apps.v1.deployment;
+local volume = k.core.v1.volume;
 
 local grafana = import 'grafana/grafana.libsonnet';
 local envVar = if std.objectHasAll(k.core.v1, 'envVar') then k.core.v1.envVar else k.core.v1.container.envType;
@@ -152,10 +154,20 @@ local tenant = 'loki';
             + grafana.addDatasource('prometheus', $.prometheus_datasource)
             + grafana.addDatasource('loki', $.loki_datasource) + {
     grafana_deployment+:
-      k.apps.v1.deployment.emptyVolumeMount('grafana-var', '/var/lib/grafana')
-      + k.apps.v1.deployment.emptyVolumeMount('grafana-plugins', '/etc/grafana/provisioning/plugins')
-      + k.apps.v1.deployment.configVolumeMount('%s-dashboards-1' % dashboardsPrefix, '/var/lib/grafana/dashboards/loki-1')
-      + k.apps.v1.deployment.configVolumeMount('%s-dashboards-2' % dashboardsPrefix, '/var/lib/grafana/dashboards/loki-2'),
+      deployment.emptyVolumeMount('grafana-var', '/var/lib/grafana')
+      + deployment.emptyVolumeMount('grafana-plugins', '/etc/grafana/provisioning/plugins')
+      + deployment.configVolumeMount(
+        '%s-dashboards-1' % dashboardsPrefix,
+        '/var/lib/grafana/dashboards/loki-1',
+        {},
+        volume.configMap.withOptional(true)  //no dashboards for single binary mode
+      )
+      + deployment.configVolumeMount(
+        '%s-dashboards-2' % dashboardsPrefix,
+        '/var/lib/grafana/dashboards/loki-2',
+        {},
+        volume.configMap.withOptional(true)  //no dashboards for single binary mode
+      ),
 
     dashboard_provisioning_config_map:
       configMap.new('grafana-dashboard-provisioning') +

tools/dev/k3d/environments/helm-cluster/values/enterprise-logs-ha-single-binary.yaml

@@ -0,0 +1,35 @@
+---
+loki:
+  querier:
+    multi_tenant_queries_enabled: true
+singleBinary:
+  replicas: 3
+enterprise:
+  enabled: true
+  adminToken:
+    secret: "gel-admin-token"
+    additionalNamespaces:
+      - k3d-helm-cluster
+  useExternalLicense: true
+  externalLicenseName: gel-license
+  provisioner:
+    provisionedSecretPrefix: "provisioned-secret"
+    additionalTenants:
+      - name: team-a
+        namespace: k3d-helm-cluster
+monitoring:
+  dashboards:
+    namespace: k3d-helm-cluster
+  selfMonitoring:
+    tenant:
+      name: loki
+      secretNamespace: k3d-helm-cluster
+  serviceMonitor:
+    labels:
+      release: "prometheus"
+  rules:
+    namespace: k3d-helm-cluster
+    labels:
+      release: "prometheus"
+minio:
+  enabled: true

tools/dev/k3d/environments/helm-cluster/values/loki-ha-single-binary.yaml

@@ -0,0 +1,19 @@
+---
+singleBinary:
+  replicas: 3
+monitoring:
+  dashboards:
+    namespace: k3d-helm-cluster
+  selfMonitoring:
+    tenant:
+      name: loki
+      secretNamespace: k3d-helm-cluster
+  serviceMonitor:
+    labels:
+      release: "prometheus"
+  rules:
+    namespace: k3d-helm-cluster
+    labels:
+      release: "prometheus"
+minio:
+  enabled: true

tools/dev/k3d/environments/helm-cluster/values/loki.yaml

@@ -9,5 +9,9 @@ monitoring:
   serviceMonitor:
     labels:
       release: "prometheus"
+  rules:
+    namespace: k3d-helm-cluster
+    labels:
+      release: "prometheus"
 minio:
   enabled: true