From cb9960d5987d5511b4d15f304e8844ab83c8beb2 Mon Sep 17 00:00:00 2001 From: Sashank Agarwal Date: Tue, 22 Mar 2022 13:42:07 +0530 Subject: [PATCH] operator: Document how to connect to LokiStack gateway component (#5615) --- operator/CHANGELOG.md | 1 + operator/docs/forwarding_logs_to_gateway.md | 117 +++++++++++++++++++- operator/docs/hack_loki_operator.md | 6 + operator/index.md | 1 + 4 files changed, 120 insertions(+), 5 deletions(-) diff --git a/operator/CHANGELOG.md b/operator/CHANGELOG.md index c7340657e6..d00bd3c5fc 100644 --- a/operator/CHANGELOG.md +++ b/operator/CHANGELOG.md @@ -1,5 +1,6 @@ ## Main +- [5615](https://github.com/grafana/loki/pull/5615) **sasagarw**: Document how to connect to LokiStack gateway component - [5655](https://github.com/grafana/loki/pull/5655) **xperimental**: Update Loki operand to 2.4.2 - [5579](https://github.com/grafana/loki/pull/5579) **Red-GV**: Add playbook for responding to operator alerts - [5640](https://github.com/grafana/loki/pull/5640) **sasagarw**: Update CSV to point to candidate channel and use openshift-operators-redhat ns (OpenShift) diff --git a/operator/docs/forwarding_logs_to_gateway.md b/operator/docs/forwarding_logs_to_gateway.md index 73bb6e36bf..188141d730 100644 --- a/operator/docs/forwarding_logs_to_gateway.md +++ b/operator/docs/forwarding_logs_to_gateway.md @@ -6,11 +6,118 @@ __Please read the [hacking guide](./hack_loki_operator.md) before proceeding wit _Note: While this document will only give instructions for two methods of log forwarding into the gateway, the examples given in the Promtail and Fluentd sections can be extrapolated to other log forwarders._ -## Openshift Logging - -Although there is a way to [forward logs to an external Loki instance](https://docs.openshift.com/container-platform/4.9/logging/cluster-logging-external.html#cluster-logging-collector-log-forward-loki_cluster-logging-external), [Openshift Logging](https://github.com/openshift/cluster-logging-operator) does not currently have support to send logs through the Lokistack Gateway. - -Support will be added in the near future. +## OpenShift Logging + +[OpenShift Logging](https://github.com/openshift/cluster-logging-operator) supports [forwarding logs to an external Loki instance](https://docs.openshift.com/container-platform/4.9/logging/cluster-logging-external.html#cluster-logging-collector-log-forward-loki_cluster-logging-external). This can also be used to forward logs to LokiStack gateway. + +* Deploy the Loki Operator and an `lokistack` instance with the [gateway flag enabled](./hack_loki_operator.md#hacking-on-loki-operator-on-openshift). + +* Deploy the [OpenShift Logging Operator](https://github.com/openshift/cluster-logging-operator/blob/master/docs/HACKING.md) from the Operator Hub or using the following command locally: + + ```console + make deploy-image deploy-catalog install + ``` + +* Create a Cluster Logging instance in the `openshift-logging` namespace with only `collection` defined. + + ```yaml + apiVersion: logging.openshift.io/v1 + kind: ClusterLogging + metadata: + name: instance + namespace: openshift-logging + spec: + collection: + logs: + type: fluentd + fluentd: {} + ``` + +* The LokiStack Gateway requires a bearer token for communication with fluentd. Therefore, create a secret with `token` key and the path to the file. + + ```console + kubectl -n openshift-logging create secret generic lokistack-gateway-bearer-token \ + --from-literal=token="/var/run/secrets/kubernetes.io/serviceaccount/token" + ``` + +* Create the following `ClusterRole` and `ClusterRoleBinding` which will allow the cluster to authenticate the user(s) submitting the logs: + + ```yaml + --- + apiVersion: rbac.authorization.k8s.io/v1 + kind: ClusterRole + metadata: + name: lokistack-dev-tenant-logs + rules: + - apiGroups: + - 'loki.grafana.com' + resources: + - application + - infrastructure + - audit + resourceNames: + - logs + verbs: + - 'create' + --- + apiVersion: rbac.authorization.k8s.io/v1 + kind: ClusterRoleBinding + metadata: + name: lokistack-dev-tenant-logs + roleRef: + apiGroup: rbac.authorization.k8s.io + kind: ClusterRole + name: lokistack-dev-tenant-logs + subjects: + - kind: ServiceAccount + name: logcollector + namespace: openshift-logging + ``` + +* Now create a ClusterLogForwarder CR to forward logs to LokiStack: + + ```yaml + apiVersion: logging.openshift.io/v1 + kind: ClusterLogForwarder + metadata: + name: instance + namespace: openshift-logging + spec: + outputs: + - name: loki-app + type: loki + url: http://lokistack-dev-gateway-http.openshift-logging.svc:8080/api/logs/v1/application + secret: + name: lokistack-gateway-bearer-token + - name: loki-infra + type: loki + url: http://lokistack-dev-gateway-http.openshift-logging.svc:8080/api/logs/v1/infrastructure + secret: + name: lokistack-gateway-bearer-token + - name: loki-audit + type: loki + url: http://lokistack-dev-gateway-http.openshift-logging.svc:8080/api/logs/v1/audit + secret: + name: lokistack-gateway-bearer-token + pipelines: + - name: send-app-logs + inputRefs: + - application + outputRefs: + - loki-app + - name: send-infra-logs + inputRefs: + - infrastructure + outputRefs: + - loki-infra + - name: send-audit-logs + inputRefs: + - audit + outputRefs: + - loki-audit + ``` + + _Note:_ You can add/remove any pipeline from the ClusterLogForwarder spec in case if you want to limit the logs being sent. ## Forwarding Clients diff --git a/operator/docs/hack_loki_operator.md b/operator/docs/hack_loki_operator.md index bb9d150eed..004f1f67f0 100644 --- a/operator/docs/hack_loki_operator.md +++ b/operator/docs/hack_loki_operator.md @@ -85,6 +85,12 @@ It will undeploy controller from the configured Kubernetes cluster in [~/.kube/c ### Installation of Loki Operator +* Create the `openshift-operators-redhat` namespace in the cluster: + + ```console + kubectl create ns openshift-operators-redhat + ``` + * Build and push the container image [2] and then deploy the operator with: ```console diff --git a/operator/index.md b/operator/index.md index 57dd92f19a..224e6f0348 100644 --- a/operator/index.md +++ b/operator/index.md @@ -14,6 +14,7 @@ This is the Kubernetes Operator for Loki provided by the Grafana Loki SIG operat #### Sending Logs Through the Gateway Component * The [forwarding logs to LokiStack guide](https://github.com/grafana/loki/tree/master/operator/docs/forwarding_logs_to_gateway.md) provides instructions for configuring forwarding clients to ship logs to Loki through the gateway component. +* This section details [how to connect to OpenShift Logging](https://github.com/grafana/loki/tree/master/operator/docs/forwarding_logs_to_gateway.md#openshift-logging) installation to the gateway. * This section details [how to connect a Promtail](https://github.com/grafana/loki/tree/master/operator/docs/forwarding_logs_to_gateway.md#promtail) installation to the gateway. * This section details [how to connect a Grafana Fluentd plugin](https://github.com/grafana/loki/tree/master/operator/docs/forwarding_logs_to_gateway.md#fluentd) installation to the gateway.