operator: Document how to connect to LokiStack gateway component (#5615)

pull/5700/head
Sashank Agarwal 3 years ago committed by GitHub
parent 4f2f0886d7
commit cb9960d598
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
  1. 1
      operator/CHANGELOG.md
  2. 117
      operator/docs/forwarding_logs_to_gateway.md
  3. 6
      operator/docs/hack_loki_operator.md
  4. 1
      operator/index.md

@ -1,5 +1,6 @@
## Main
- [5615](https://github.com/grafana/loki/pull/5615) **sasagarw**: Document how to connect to LokiStack gateway component
- [5655](https://github.com/grafana/loki/pull/5655) **xperimental**: Update Loki operand to 2.4.2
- [5579](https://github.com/grafana/loki/pull/5579) **Red-GV**: Add playbook for responding to operator alerts
- [5640](https://github.com/grafana/loki/pull/5640) **sasagarw**: Update CSV to point to candidate channel and use openshift-operators-redhat ns (OpenShift)

@ -6,11 +6,118 @@ __Please read the [hacking guide](./hack_loki_operator.md) before proceeding wit
_Note: While this document will only give instructions for two methods of log forwarding into the gateway, the examples given in the Promtail and Fluentd sections can be extrapolated to other log forwarders._
## Openshift Logging
Although there is a way to [forward logs to an external Loki instance](https://docs.openshift.com/container-platform/4.9/logging/cluster-logging-external.html#cluster-logging-collector-log-forward-loki_cluster-logging-external), [Openshift Logging](https://github.com/openshift/cluster-logging-operator) does not currently have support to send logs through the Lokistack Gateway.
Support will be added in the near future.
## OpenShift Logging
[OpenShift Logging](https://github.com/openshift/cluster-logging-operator) supports [forwarding logs to an external Loki instance](https://docs.openshift.com/container-platform/4.9/logging/cluster-logging-external.html#cluster-logging-collector-log-forward-loki_cluster-logging-external). This can also be used to forward logs to LokiStack gateway.
* Deploy the Loki Operator and an `lokistack` instance with the [gateway flag enabled](./hack_loki_operator.md#hacking-on-loki-operator-on-openshift).
* Deploy the [OpenShift Logging Operator](https://github.com/openshift/cluster-logging-operator/blob/master/docs/HACKING.md) from the Operator Hub or using the following command locally:
```console
make deploy-image deploy-catalog install
```
* Create a Cluster Logging instance in the `openshift-logging` namespace with only `collection` defined.
```yaml
apiVersion: logging.openshift.io/v1
kind: ClusterLogging
metadata:
name: instance
namespace: openshift-logging
spec:
collection:
logs:
type: fluentd
fluentd: {}
```
* The LokiStack Gateway requires a bearer token for communication with fluentd. Therefore, create a secret with `token` key and the path to the file.
```console
kubectl -n openshift-logging create secret generic lokistack-gateway-bearer-token \
--from-literal=token="/var/run/secrets/kubernetes.io/serviceaccount/token"
```
* Create the following `ClusterRole` and `ClusterRoleBinding` which will allow the cluster to authenticate the user(s) submitting the logs:
```yaml
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: lokistack-dev-tenant-logs
rules:
- apiGroups:
- 'loki.grafana.com'
resources:
- application
- infrastructure
- audit
resourceNames:
- logs
verbs:
- 'create'
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: lokistack-dev-tenant-logs
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: lokistack-dev-tenant-logs
subjects:
- kind: ServiceAccount
name: logcollector
namespace: openshift-logging
```
* Now create a ClusterLogForwarder CR to forward logs to LokiStack:
```yaml
apiVersion: logging.openshift.io/v1
kind: ClusterLogForwarder
metadata:
name: instance
namespace: openshift-logging
spec:
outputs:
- name: loki-app
type: loki
url: http://lokistack-dev-gateway-http.openshift-logging.svc:8080/api/logs/v1/application
secret:
name: lokistack-gateway-bearer-token
- name: loki-infra
type: loki
url: http://lokistack-dev-gateway-http.openshift-logging.svc:8080/api/logs/v1/infrastructure
secret:
name: lokistack-gateway-bearer-token
- name: loki-audit
type: loki
url: http://lokistack-dev-gateway-http.openshift-logging.svc:8080/api/logs/v1/audit
secret:
name: lokistack-gateway-bearer-token
pipelines:
- name: send-app-logs
inputRefs:
- application
outputRefs:
- loki-app
- name: send-infra-logs
inputRefs:
- infrastructure
outputRefs:
- loki-infra
- name: send-audit-logs
inputRefs:
- audit
outputRefs:
- loki-audit
```
_Note:_ You can add/remove any pipeline from the ClusterLogForwarder spec in case if you want to limit the logs being sent.
## Forwarding Clients

@ -85,6 +85,12 @@ It will undeploy controller from the configured Kubernetes cluster in [~/.kube/c
### Installation of Loki Operator
* Create the `openshift-operators-redhat` namespace in the cluster:
```console
kubectl create ns openshift-operators-redhat
```
* Build and push the container image [2] and then deploy the operator with:
```console

@ -14,6 +14,7 @@ This is the Kubernetes Operator for Loki provided by the Grafana Loki SIG operat
#### Sending Logs Through the Gateway Component
* The [forwarding logs to LokiStack guide](https://github.com/grafana/loki/tree/master/operator/docs/forwarding_logs_to_gateway.md) provides instructions for configuring forwarding clients to ship logs to Loki through the gateway component.
* This section details [how to connect to OpenShift Logging](https://github.com/grafana/loki/tree/master/operator/docs/forwarding_logs_to_gateway.md#openshift-logging) installation to the gateway.
* This section details [how to connect a Promtail](https://github.com/grafana/loki/tree/master/operator/docs/forwarding_logs_to_gateway.md#promtail) installation to the gateway.
* This section details [how to connect a Grafana Fluentd plugin](https://github.com/grafana/loki/tree/master/operator/docs/forwarding_logs_to_gateway.md#fluentd) installation to the gateway.

Loading…
Cancel
Save