Docs: Local install edits (#2220)

* Update README.md

* Update README.md

* Update README.md

* Update tanka.md

* content edits

* Create install-for-manual-build.md

* Update README.md

* Update helm.md

* Update local.md

* Create get-logs-into-loki.md

* Update local.md

* Update get-logs-into-loki.md

* Update get-logs-into-loki.md

* Update local.md

* Update get-logs-into-loki.md

* Update README.md

* Update README.md

* Update installation.md

* Update README.md

* Update docs/getting-started/get-logs-into-loki.md

Co-authored-by: Cyril Tovena <cyril.tovena@gmail.com>

* Update docs/installation/local.md

Co-authored-by: Owen Diehl <ow.diehl@gmail.com>

* Update docs/installation/local.md

Co-authored-by: Owen Diehl <ow.diehl@gmail.com>

* applied edits

* Update docs/getting-started/get-logs-into-loki.md

Co-authored-by: Ed Welch <edward.welch@grafana.com>

* Update get-logs-into-loki.md

* Update docs/getting-started/get-logs-into-loki.md

Co-authored-by: Ed Welch <edward.welch@grafana.com>

* Update docs/getting-started/get-logs-into-loki.md

Co-authored-by: Ed Welch <edward.welch@grafana.com>

* Update docs/installation/README.md

Co-authored-by: Ed Welch <edward.welch@grafana.com>

* applied edits

* Update local.md

Co-authored-by: Cyril Tovena <cyril.tovena@gmail.com>
Co-authored-by: Owen Diehl <ow.diehl@gmail.com>
Co-authored-by: Ed Welch <edward.welch@grafana.com>
pull/2250/head
Diana Payton 5 years ago committed by GitHub
parent 15378cf0e6
commit 687662ea08
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
  1. 16
      docs/clients/README.md
  2. 12
      docs/clients/promtail/README.md
  3. 10
      docs/clients/promtail/installation.md
  4. 74
      docs/getting-started/get-logs-into-loki.md
  5. 25
      docs/installation/README.md
  6. 15
      docs/installation/docker.md
  7. 34
      docs/installation/helm.md
  8. 26
      docs/installation/install-from-source.md
  9. 70
      docs/installation/local.md
  10. 13
      docs/installation/tanka.md

@ -1,13 +1,13 @@
# Loki Clients
# Loki clients
Loki supports the following official clients for sending logs:
1. [Promtail](./promtail/README.md)
2. [Docker Driver](./docker-driver/README.md)
3. [Fluentd](./fluentd/README.md)
4. [Fluent Bit](../../cmd/fluent-bit/README.md)
- [Promtail](./promtail/README.md)
- [Docker Driver](./docker-driver/README.md)
- [Fluentd](./fluentd/README.md)
- [Fluent Bit](../../cmd/fluent-bit/README.md)
## Picking a Client
## Picking a client
While all clients can be used simultaneously to cover multiple use cases, which
client is initially picked to send logs depends on your use case.
@ -41,9 +41,9 @@ and you already have configured `Parser` and `Filter` plugins.
Fluentd also works well for extracting metrics from logs when using its
Prometheus plugin.
# Unofficial Clients
# Unofficial clients
Please note that the Loki API is not stable yet and breaking changes may occur
Please note that the Loki API is not stable yet, so breaking changes might occur
when using or writing a third-party client.
- [promtail-client](https://github.com/afiskon/promtail-client) (Go)

@ -6,14 +6,14 @@ deployed to every machine that has applications needed to be monitored.
It primarily:
1. Discovers targets
2. Attaches labels to log streams
3. Pushes them to the Loki instance.
- Discovers targets
- Attaches labels to log streams
- Pushes them to the Loki instance.
Currently, Promtail can tail logs from two sources: local log files and the
systemd journal (on AMD64 machines only).
## Log File Discovery
## Log file discovery
Before Promtail can ship any data from log files to Loki, it needs to find out
information about its environment. Specifically, this means discovering
@ -32,12 +32,12 @@ Just like Prometheus, `promtail` is configured using a `scrape_configs` stanza.
drop, and the final metadata to attach to the log line. Refer to the docs for
[configuring Promtail](configuration.md) for more details.
## Receiving Logs From Syslog
## Receiving logs From Syslog
When the [Syslog Target](./scraping.md#syslog-target) is being used, logs
can be written with the syslog protocol to the configured port.
## Labeling and Parsing
## Labeling and parsing
During service discovery, metadata is determined (pod name, filename, etc.) that
may be attached to the log line as a label for easier identification when

@ -1,4 +1,4 @@
# Installing Promtail
# Install Promtail
Promtail is distributed as a [binary](#binary), [Docker container](#docker), and
[Helm chart](#helm).
@ -12,7 +12,7 @@ Every release includes binaries for Promtail which can be found on the
```bash
# modify tag to most recent version
$ docker pull grafana/promtail:1.5.0
docker pull grafana/promtail:1.5.0
```
## Helm
@ -23,13 +23,13 @@ Make sure that Helm is
Then you can add Loki's chart repository to Helm:
```bash
$ helm repo add loki https://grafana.github.io/loki/charts
helm repo add loki https://grafana.github.io/loki/charts
```
And the chart repository can be updated by running:
```bash
$ helm repo update
helm repo update
```
Finally, Promtail can be deployed with:
@ -40,7 +40,7 @@ $ helm upgrade --install promtail loki/promtail --set "loki.serviceName=loki"
## Kubernetes
### DaemonSet (Recommended)
### DaemonSet (recommended)
A `DaemonSet` will deploy `promtail` on every node within a Kubernetes cluster.

@ -0,0 +1,74 @@
# Get logs into Loki
After you [install and run Loki](./installation/local.md), you probably want to get logs from other applications into it.
To get application logs into Loki, you need to edit the [Promtail](./clients/promtail/README.md) config file.
Detailed information about configuring Promtail is available in [Promtail configuration](./clients/promtail/configuration.md).
The following instructions should help you get started.
1. If you haven't already, download a Promtail configuration file. Keep track of where it is, because you will need to cite it when you run the binary.
```
wget https://github.com/grafana/loki/blob/master/cmd/promtail/promtail-local-config.yaml
```
2. Open the config file in the text editor of your choice. It should look similar to this:
```
server:
http_listen_port: 9080
grpc_listen_port: 0
positions:
filename: /tmp/positions.yaml
clients:
- url: http://loki:3100/loki/api/v1/push
scrape_configs:
- job_name: system
static_configs:
- targets:
- localhost
labels:
job: varlogs
__path__: /var/log/*log
```
The seven lines under `scrape_configs` are what send the logs that Loki generates to Loki, which then outputs them in the command line and http://localhost:3100/metrics.
3. Copy the seven lines under `scrape_configs`, and then paste them under the original job (you can also just edit the original seven lines).
Below is an example that sends logs from a default Grafana installation to Loki. We updated the following fields:
- job_name - This differentiates the logs collected from other log groups.
- targets - Optional for static_configs, however is often defined because in older versions of Promtail it was not optional. This was an artifact from directly using the Prometheus service discovery code which required this entry.
- labels - Static label to apply to every log line scraped by this definition. Good examples would be environment name, job name, or app name.
- __path__ - The path to where the logs are stored that I want Loki to consume.
```
- job_name: grafana
static_configs:
- targets:
- grafana
labels:
job: grafana
__path__: "C:/Program Files/GrafanaLabs/grafana/data/log/grafana.log"
```
4. Enter the following command to run Promtail. Examples below assume you have put the config file in the same directory as the binary.
**Windows**
```
`.\promtail-windows-amd64.exe --config.file=promtail-local-config.yaml`
```
**Linux**
```
./promtail-linux-amd64 -config.file=promtail-local-config.yaml
```
You should now see your application logs. If you are using Grafana, you might need to refresh your instance in order to see the logs.

@ -1,6 +1,21 @@
# Installing Loki
# Install Loki
1. [Installing using Tanka (recommended)](./tanka.md)
2. [Installing through Helm](./helm.md)
3. [Installing through Docker or Docker Compose](./docker.md)
4. [Installing locally](./local.md)
## Installation methods
Instructions for different methods of installing Loki and Promtail.
- [Install using Tanka (recommended)](./tanka.md)
- [Install through Helm](./helm.md)
- [Install through Docker or Docker Compose](./docker.md)
- [Install and run locally](./local.md)
- [Install from source](./install-from-source.md)
## General process
In order to run Loki, you must:
1. Download and install both Loki and Promtail.
1. Download config files for both programs.
1. Start Loki.
1. Update the Promtail config file to get your logs into Loki.
1. Start Promtail.

@ -1,7 +1,7 @@
# Installing Loki with Docker or Docker Compose
# Install Loki with Docker or Docker Compose
You can install Loki with Docker or Docker Compose for evaluating, testing, or developing Loki.
For production, we recommend Tanka or Helm.
You can install Loki and Promtail with Docker or Docker Compose if you are evaluating, testing, or developing Loki.
For production, we recommend installing with Tanka or Helm.
## Prerequisites
@ -25,8 +25,7 @@ When finished, `loki-config.yaml` and `promtail-config.yaml` are downloaded in t
Navigate to http://localhost:3100/metrics to view the metrics and http://localhost:3100/ready for readiness.
As of v1.5.0, image is configured to run by default as user loki with UID `10001` and GID `10001`. You can use a different user, specially if you are using bind mounts, by specifying uid with docker run command
by specifying `--user=UID` with numeric UID suited to your needs.
As of v1.5.0, image is configured to run by default as user loki with UID `10001` and GID `10001`. You can use a different user, specially if you are using bind mounts, by specifying the UID with a `docker run` command and using `--user=UID` with numeric UID suited to your needs.
**Windows**
@ -46,7 +45,9 @@ Navigate to http://localhost:3100/metrics to view the output.
## Install with Docker Compose
Run the following commands in your command line. They work for Windows or Linux systems.
```bash
$ wget https://raw.githubusercontent.com/grafana/loki/v1.5.0/production/docker-compose.yaml -O docker-compose.yaml
$ docker-compose -f docker-compose.yaml up
wget https://raw.githubusercontent.com/grafana/loki/v1.5.0/production/docker-compose.yaml -O docker-compose.yaml
docker-compose -f docker-compose.yaml up
```

@ -1,19 +1,20 @@
# Installing Loki with Helm
# Install Loki with Helm
## Prerequisites
Make sure you have Helm [installed](https://helm.sh/docs/using_helm/#installing-helm) and
[deployed](https://helm.sh/docs/using_helm/#installing-tiller) to your cluster. Then add
[Loki's chart repository](https://github.com/grafana/loki/tree/master/production/helm/loki) to Helm:
[deployed](https://helm.sh/docs/using_helm/#installing-tiller) to your cluster.
Add [Loki's chart repository](https://github.com/grafana/loki/tree/master/production/helm/loki) to Helm:
```bash
$ helm repo add loki https://grafana.github.io/loki/charts
helm repo add loki https://grafana.github.io/loki/charts
```
You can update the chart repository by running:
To update the chart repository, run:
```bash
$ helm repo update
helm repo update
```
## Deploy Loki to your cluster
@ -21,32 +22,32 @@ $ helm repo update
### Deploy with default config
```bash
$ helm upgrade --install loki loki/loki-stack
helm upgrade --install loki loki/loki-stack
```
### Deploy in a custom namespace
```bash
$ helm upgrade --install loki --namespace=loki loki/loki
helm upgrade --install loki --namespace=loki loki/loki
```
### Deploy with custom config
```bash
$ helm upgrade --install loki loki/loki --set "key1=val1,key2=val2,..."
helm upgrade --install loki loki/loki --set "key1=val1,key2=val2,..."
```
### Deploy Loki Stack (Loki, Promtail, Grafana, Prometheus)
```bash
$ helm upgrade --install loki loki/loki-stack --set grafana.enabled=true,prometheus.enabled=true,prometheus.alertmanager.persistentVolume.enabled=false,prometheus.server.persistentVolume.enabled=false
helm upgrade --install loki loki/loki-stack --set grafana.enabled=true,prometheus.enabled=true,prometheus.alertmanager.persistentVolume.enabled=false,prometheus.server.persistentVolume.enabled=false
```
### Deploy Loki Stack (Loki, Fluent Bit, Grafana, Prometheus)
```bash
$ helm upgrade --install loki loki/loki-stack \
--set fluent-bit.enabled=true,promtail.enabled=false,grafana.enabled=true,prometheus.enabled=true,prometheus.alertmanager.persistentVolume.enabled=false,prometheus.server.persistentVolume.enabled=false
helm upgrade --install loki loki/loki-stack \
--set fluent-bit.enabled=true,promtail.enabled=false,grafana.enabled=true,prometheus.enabled=true,prometheus.alertmanager.persistentVolume.enabled=false,prometheus.server.persistentVolume.enabled=false
```
## Deploy Grafana to your cluster
@ -54,19 +55,19 @@ $ helm upgrade --install loki loki/loki-stack \
To install Grafana on your cluster with Helm, use the following command:
```bash
$ helm install stable/grafana -n loki-grafana
helm install stable/grafana -n loki-grafana
```
To get the admin password for the Grafana pod, run the following command:
```bash
$ kubectl get secret --namespace <YOUR-NAMESPACE> loki-grafana -o jsonpath="{.data.admin-password}" | base64 --decode ; echo
kubectl get secret --namespace <YOUR-NAMESPACE> loki-grafana -o jsonpath="{.data.admin-password}" | base64 --decode ; echo
```
To access the Grafana UI, run the following command:
```bash
$ kubectl port-forward --namespace <YOUR-NAMESPACE> service/loki-grafana 3000:80
kubectl port-forward --namespace <YOUR-NAMESPACE> service/loki-grafana 3000:80
```
Navigate to `http://localhost:3000` and login with `admin` and the password
@ -79,8 +80,7 @@ If Loki and Promtail are deployed on different clusters you can add an Ingress
in front of Loki. By adding a certificate you create an HTTPS endpoint. For
extra security you can also enable Basic Authentication on the Ingress.
In Promtail, set the following values to communicate using HTTPS and basic
authentication:
In Promtail, set the following values to communicate using HTTPS and basic authentication:
```yaml
loki:

@ -0,0 +1,26 @@
# Build from source
In order to build Loki manually, you need to clone the GitHub repo and then `make Loki`.
## Prerequisites
- Go 1.13 or later
- Make
- Docker (for updating protobuf files and yacc files)
## Build manually on your local system
Clone Loki to `$GOPATH/src/github.com/grafana/loki`:
```bash
git clone https://github.com/grafana/loki $GOPATH/src/github.com/grafana/loki
```
Then change into that directory and run `make loki`:
```bash
cd $GOPATH/src/github.com/grafana/loki
make loki
```
A file at ./cmd/loki/loki will be created and is the final built binary.

@ -1,11 +1,43 @@
# Installing Loki Locally
# Install and run Loki locally
## Release Binaries
In order to log events with Loki, you must download and install both Promtail and Loki.
- Loki is the logging engine.
- Promtail sends logs to Loki.
## Install and run
1. Navigate to the [release page](https://github.com/grafana/loki/releases/).
2. Scroll down to the Assets section under the version that you want to install.
3. Download the Loki and Promtail .zip files that correspond to your system.
**Note:** Do not download LogCLI or Loki Canary at this time. [LogCLI](./getting-started/logcli.md) allows you to run Loki queries in a command line interface. [Loki Canary](./operations/loki-canary.md) is a tool to audit Loki performance.
4. Unzip the package contents into the same directory. This is where the two programs will run.
5. In the command line, change directory (`cd` on most systems) to the directory with Loki and Promtail. Copy and paste the commands below into your command line to download generic configuration files:
```
wget https://raw.githubusercontent.com/grafana/loki/master/cmd/loki/loki-local-config.yaml
wget https://raw.githubusercontent.com/grafana/loki/master/cmd/promtail/promtail-local-config.yaml
```
6. Enter the following command to start Loki:
**Windows**
```
.\loki-windows-amd64.exe --config.file=loki-local-config.yaml
```
**Linux**
```
./promtail-linux-amd64 -config.file=promtail-local-config.yaml
```
Loki runs and displays Loki logs in your command line and on http://localhost:3100/metrics.
Congratulations, Loki is installed and running! Next, you might want edit the Promtail config file to [get logs into Loki](./getting-started/get-logs-into-loki.md).
## Release binaries - openSUSE Linux only
Every release includes binaries for Loki which can be found on the
[Releases page](https://github.com/grafana/loki/releases).
## Community openSUSE Linux packages
The community provides packages of Loki for openSUSE Linux. To install:
@ -13,35 +45,9 @@ The community provides packages of Loki for openSUSE Linux. To install:
1. Add the repository `https://download.opensuse.org/repositories/security:/logging/`
to your system. For example, if you are using Leap 15.1, run
`sudo zypper ar https://download.opensuse.org/repositories/security:/logging/openSUSE_Leap_15.1/security:logging.repo ; sudo zypper ref`
2. Install the Loki package with `zypper in loki`
3. Enable the Loki and Promtail services:
1. Install the Loki package with `zypper in loki`
1. Enable the Loki and Promtail services:
- `systemd start loki && systemd enable loki`
- `systemd start promtail && systemd enable promtail`
4. Modify the configuration files as needed: `/etc/loki/promtail.yaml` and
1. Modify the configuration files as needed: `/etc/loki/promtail.yaml` and
`/etc/loki/loki.yaml`.
## Manual Build
### Prerequisites
- Go 1.13 or later
- Make
- Docker (for updating protobuf files and yacc files)
### Building
Clone Loki to `$GOPATH/src/github.com/grafana/loki`:
```bash
$ git clone https://github.com/grafana/loki $GOPATH/src/github.com/grafana/loki
```
Then change into that directory and run `make loki`:
```bash
$ cd $GOPATH/src/github.com/grafana/loki
$ make loki
# A file at ./cmd/loki/loki will be created and is the
# final built binary.
```

@ -1,4 +1,4 @@
# Installing Loki with Tanka
# Install Loki with Tanka
[Tanka](https://tanka.dev) is a reimplementation of
[Ksonnet](https://ksonnet.io) that Grafana Labs created after Ksonnet was
@ -6,7 +6,7 @@ deprecated. Tanka is used by Grafana Labs to run Loki in production.
## Prerequisites
Grab the latest version of Tanka (at least version v0.5.0) for the `tk env`
Install the latest version of Tanka (at least version v0.5.0) for the `tk env`
commands. Prebuilt binaries for Tanka can be found at the [Tanka releases
URL](https://github.com/grafana/tanka/releases).
@ -23,7 +23,7 @@ tk env add environments/loki --namespace=loki --server=<Kubernetes API server>
## Deploying
Grab the Loki & Promtail module using `jb`:
Download and install the Loki and Promtail module using `jb`:
```bash
go get -u github.com/jsonnet-bundler/jsonnet-bundler/cmd/jb
@ -75,8 +75,7 @@ loki + promtail + gateway {
}
```
Notice that `container_root_path` is your own data root for the Docker Daemon,
run `docker info | grep "Root Dir"` to get it.
Notice that `container_root_path` is your own data root for the Docker Daemon.
Run `docker info | grep "Root Dir"` to get the root path.
Run `tk show environments/loki` to see the manifests that will be deployed to the cluster and
finally run `tk apply environments/loki` to deploy it.
Run `tk show environments/loki` to see the manifests that will be deployed to the cluster. Run `tk apply environments/loki` to deploy the manifests.

Loading…
Cancel
Save