description: How to create and use a simple local Loki cluster for testing and evaluation purposes.
description: How to create and use a local Loki cluster for testing and evaluation purposes.
killercoda:
comment: |
The killercoda front matter and the HTML comments that start '<!-- INTERACTIVE ' are used by a transformation tool that converts this Markdown source into a Killercoda tutorial.
You can find the tutorial in https://github.com/grafana/killercoda/tree/staging/loki/loki-quickstart.
Changes to this source file affect the Killercoda tutorial.
For more information about the transformation tool, refer to https://github.com/grafana/killercoda/blob/staging/docs/transformer.md.
preprocessing:
substitutions:
- regexp: evaluate-loki-([^-]+)-
replacement: evaluate-loki_${1}_
title: Loki Quickstart Demo
description: This sandbox provides an online enviroment for testing the Loki quickstart demo.
details:
finish:
text: finish.md
backend:
imageid: ubuntu
---
<!-- INTERACTIVE page intro.md START -->
# Quickstart to run Loki locally
If you want to experiment with Loki, you can run Loki locally using the Docker Compose file that ships with Loki. It runs Loki in a [monolithic deployment](https://grafana.com/docs/loki/<LOKI_VERSION>/get-started/deployment-modes/#monolithic-mode) mode and includes a sample application to generate logs.
The Docker Compose configuration instantiates the following components, each in its own container:
The Docker Compose configuration runs the following components, each in its own container:
- **flog**: which generates log lines.
[flog](https://github.com/mingrammer/flog) is a log generator for common log formats.
- **Grafana Alloy**: which scrapes the log lines from flog, and pushes them to Loki through the gateway.
- **Gateway** (nginx) which receives requests and redirects them to the appropriate container based on the request's URL.
- **Loki read component**: which runs a Query Frontend and a Querier.
- **Loki write component**: which runs a Distributor and an Ingester.
- **Loki backend component**: which runs an Index Gateway, Compactor, Ruler, Bloom Compactor (experimental), and Bloom Gateway (experimental).
- **Minio**: which Loki uses to store its index and chunks.
- **Grafana**: which provides visualization of the log lines captured within Loki.
{{<figuremax-width="75%"src="/media/docs/loki/get-started-flog-v3.png"caption="Getting started sample application"alt="Getting started sample application">}}
<!-- INTERACTIVE page intro.md END -->
## Before you begin
- **flog** a sample application which generates log lines. [flog](https://github.com/mingrammer/flog) is a log generator for common log formats.
- **Grafana Alloy** which scrapes the log lines from flog, and pushes them to Loki through the gateway.
- **Gateway** (NGINX) which receives requests and redirects them to the appropriate container based on the request's URL.
- One Loki **read** component (Query Frontend, Querier).
- One Loki **write** component (Distributor, Ingester).
- **Minio** an S3-compatible object store which Loki uses to store its index and chunks.
- **Grafana** which provides visualization of the log lines captured within Loki.
{{<admonitiontype="tip">}}
Alternatively, you can try out this example in our interactive learning environment: [Loki Quickstart Sandbox](https://killercoda.com/grafana-labs/course/loki/loki-quickstart).
{{<figuremax-width="75%"src="/media/docs/loki/get-started-flog-v3.png"caption="Getting started sample application"alt="Getting started sample application">}}
It's a fully configured environment with all the dependencies already installed.
The Interactive Learning Environment is currently in trial. Please provide feedback, report bugs, and raise issues in the [Grafana Killercoda Repository](https://github.com/grafana/killercoda).
The Interactive Learning Environment is in trial.
Provide feedback, report bugs, and raise issues in the [Grafana Killercoda repository](https://github.com/grafana/killercoda).
{{</admonition>}}
Try out this demo within our interactive learning environment: [Loki Quickstart Sandbox](https://killercoda.com/grafana-labs/course/loki/loki-quickstart)
Try out this demo within our interactive learning environment: [Loki Quickstart Sandbox](https://killercoda.com/grafana-labs/course/loki/loki-quickstart)
- A free Killercoda account is required to verify you are not a bot.
- Tutorial instructions are located on the left-hand side of the screen. Click to move on to the next section.
- All commands run inside the interactive terminal. Grafana can also be accessed via the URL links provided within the sandbox.
- You must have a free Killercoda account to verify you aren't a bot.
- Tutorial instructions are located on the left-side of the screen.
Click to move on to the next section.
- All commands run inside the interactive terminal.
- You can access Grafana with the URL links provided within the sandbox.
With `evaluate-loki` as the current working directory, start the demo environment using `docker compose`:
With `evaluate-loki` as the current working directory, start the demo environment using `docker compose`:
<!-- INTERACTIVE ignore START -->
```bash
docker compose up -d
```
<!-- INTERACTIVE ignore END -->
{{<docs/ignore>}}
<!-- INTERACTIVE exec START -->
```bash
docker-compose up -d
```
```bash
docker compose up -d
```
<!-- INTERACTIVE exec END -->
You should see something similar to the following:
{{< /docs/ignore >}}
```bash
✔ Network evaluate-loki_loki Created 0.1s
✔ Container evaluate-loki-minio-1 Started 0.6s
✔ Container evaluate-loki-flog-1 Started 0.6s
✔ Container evaluate-loki-backend-1 Started 0.8s
✔ Container evaluate-loki-write-1 Started 0.8s
✔ Container evaluate-loki-read-1 Started 0.8s
✔ Container evaluate-loki-gateway-1 Started 1.1s
✔ Container evaluate-loki-grafana-1 Started 1.4s
✔ Container evaluate-loki-alloy-1 Started 1.4s
```
At the end of the command, you should see something similar to the following:
<!-- INTERACTIVE ignore START -->
```console
✔ Network evaluate-loki_loki Created 0.1s
✔ Container evaluate-loki-minio-1 Started 0.6s
✔ Container evaluate-loki-flog-1 Started 0.6s
✔ Container evaluate-loki-backend-1 Started 0.8s
✔ Container evaluate-loki-write-1 Started 0.8s
✔ Container evaluate-loki-read-1 Started 0.8s
✔ Container evaluate-loki-gateway-1 Started 1.1s
✔ Container evaluate-loki-grafana-1 Started 1.4s
✔ Container evaluate-loki-alloy-1 Started 1.4s
```
<!-- INTERACTIVE ignore END -->
{{<docs/ignore>}}
```console
Creating evaluate-loki_flog_1 ... done
Creating evaluate-loki_minio_1 ... done
Creating evaluate-loki_read_1 ... done
Creating evaluate-loki_write_1 ... done
Creating evaluate-loki_gateway_1 ... done
Creating evaluate-loki_alloy_1 ... done
Creating evaluate-loki_grafana_1 ... done
Creating evaluate-loki_backend_1 ... done
```
{{< /docs/ignore >}}
1. (Optional) Verify that the Loki cluster is up and running.
- The read component returns `ready` when you point a web browser at [http://localhost:3101/ready](http://localhost:3101/ready). The message `Query Frontend not ready: not ready: number of schedulers this worker is connected to is 0` will show prior to the read component being ready.
- The write component returns `ready` when you point a web browser at [http://localhost:3102/ready](http://localhost:3102/ready). The message `Ingester not ready: waiting for 15s after being ready` will show prior to the write component being ready.
- The read component returns `ready` when you browse to [http://localhost:3101/ready](http://localhost:3101/ready).
The message `Query Frontend not ready: not ready: number of schedulers this worker is connected to is 0` shows until the read component is ready.
- The write component returns `ready` when you browse to [http://localhost:3102/ready](http://localhost:3102/ready).
The message `Ingester not ready: waiting for 15s after being ready` shows until the write component is ready.
1. (Optional) Verify that Grafana Alloy is running.
- Grafana Alloy's UI can be accessed at [http://localhost:12345](http://localhost:12345).
- You can access the Grafana Alloy UI at [http://localhost:12345](http://localhost:12345).
<!-- INTERACTIVE page step1.md END -->
<!-- INTERACTIVE page step2.md START -->
## View your logs in Grafana
## Viewing your logs in Grafana
After you have collected logs, you will want to view them.
You can view your logs using the command line interface, [LogCLI](/docs/loki/<LOKI_VERSION>/query/logcli/), but the easiest way to view your logs is with Grafana.
Once you have collected logs, you will want to view them. You can view your logs using the command line interface, [LogCLI](/docs/loki/<LOKI_VERSION>/query/logcli/), but the easiest way to view your logs is with Grafana.
1. Use Grafana to query the Loki data source.
1. Use Grafana to query the Loki data source.
The test environment includes [Grafana](https://grafana.com/docs/grafana/latest/), which you can use to query and observe the sample logs generated by the flog application.
The test environment includes [Grafana](https://grafana.com/docs/grafana/latest/), which you can use to query and observe the sample logs generated by the flog application. You can access the Grafana cluster by navigating to [http://localhost:3000](http://localhost:3000). The Grafana instance provided with this demo has a Loki [data source](https://grafana.com/docs/grafana/latest/datasources/loki/) already configured.
You can access the Grafana cluster by browsing to [http://localhost:3000](http://localhost:3000).
The Grafana instance in this demonstration has a Loki [data source](https://grafana.com/docs/grafana/latest/datasources/loki/) already configured.
1. From the Grafana main menu, click the **Explore** icon (1) to launch the Explore tab. To learn more about Explore, refer the [Explore](https://grafana.com/docs/grafana/latest/explore/) documentation.
1. From the menu in the dashboard header, select the Loki data source (2). This displays the Loki query editor. In the query editor you use the Loki query language, [LogQL](https://grafana.com/docs/loki/<LOKI_VERSION>/query/), to query your logs.
To learn more about the query editor, refer to the [query editor documentation](https://grafana.com/docs/grafana/latest/datasources/loki/query-editor/).
1. From the Grafana main menu, click the **Explore** icon (1) to open the Explore tab.
To learn more about Explore, refer to the [Explore](https://grafana.com/docs/grafana/latest/explore/) documentation.
1. From the menu in the dashboard header, select the Loki data source (2).
This displays the Loki query editor.
In the query editor you use the Loki query language, [LogQL](https://grafana.com/docs/loki/<LOKI_VERSION>/query/), to query your logs.
To learn more about the query editor, refer to the [query editor documentation](https://grafana.com/docs/grafana/latest/datasources/loki/query-editor/).
1. The Loki query editor has two modes (3):
@ -115,47 +220,61 @@ Once you have collected logs, you will want to view them. You can view your log
1. Click **Code** (3) to work in Code mode in the query editor.
Here are some basic sample queries to get you started using LogQL. Note that these queries assume that you followed the instructions to create a directory called `evaluate-loki`. If you installed in a different directory, you’ll need to modify these queries to match your installation directory. After copying any of these queries into the query editor, click **Run Query** (4) to execute the query.
Here are some sample queries to get you started using LogQL.
These queries assume that you followed the instructions to create a directory called `evaluate-loki`.
If you installed in a different directory, you’ll need to modify these queries to match your installation directory.
1. View all the log lines which have the container label "flog":
After copying any of these queries into the query editor, click **Run Query** (4) to execute the query.
```bash
{container="evaluate-loki-flog-1"}
```
1. View all the log lines which have the container label `evaluate-loki-flog-1`:
In Loki, this is called a log stream. Loki uses [labels](https://grafana.com/docs/loki/<LOKI_VERSION>/get-started/labels/) as metadata to describe log streams. Loki queries always start with a label selector. In the query above, the label selector is `container`.
```bash
{container="evaluate-loki-flog-1"}
```
1. To view all the log lines which have the container label "grafana":
In Loki, this is a log stream.
```bash
{container="evaluate-loki-grafana-1"}
```
Loki uses [labels](https://grafana.com/docs/loki/<LOKI_VERSION>/get-started/labels/) as metadata to describe log streams.
1. Find all the log lines in the container=flog stream that contain the string "status":
Loki queries always start with a label selector.
In the previous query, the label selector is `{container="evaluate-loki-flog-1"}`.
```bash
{container="evaluate-loki-flog-1"} |= `status`
```
1. To view all the log lines which have the container label `evaluate-loki-grafana-1`:
1. Find all the log lines in the container=flog stream where the JSON field "status" is "404":
1. Find all the log lines in the `{container="evaluate-loki-flog-1"}` stream that contain the string `status`:
1. Calculate the number of logs per second where the JSON field "status" is "404":
```bash
{container="evaluate-loki-flog-1"} |= `status`
```
```bash
sum by(container) (rate({container="evaluate-loki-flog-1"} | json | status=`404` [$__auto]))
```
1. Find all the log lines in the `{container="evaluate-loki-flog-1"}` stream where the JSON field `status` has the value `404`:
The final query above is a metric query which returns a time series. This will trigger Grafana to draw a graph of the results. You can change the type of graph for a different view of the data. Click **Bars** to view a bar graph of the data.
To see every log line that does not contain the value 401:
To see every log line that doesn't contain the text `401`:
```bash
{container="evaluate-loki-flog-1"} != "401"
@ -197,8 +316,12 @@ To see every log line that does not contain the value 401:
For more examples, refer to the [query documentation](https://grafana.com/docs/loki/<LOKI_VERSION>/query/query_examples/).
<!-- INTERACTIVE page step2.md END -->
## Complete metrics, logs, traces, and profiling example
If you would like to use a demo that includes Mimir, Loki, Tempo, and Grafana, you can use [Introduction to Metrics, Logs, Traces, and Profiling in Grafana](https://github.com/grafana/intro-to-mlt). `Intro-to-mltp` provides a self-contained environment for learning about Mimir, Loki, Tempo, and Grafana.
If you would like to run a demonstration environment that includes Mimir, Loki, Tempo, and Grafana, you can use [Introduction to Metrics, Logs, Traces, and Profiling in Grafana](https://github.com/grafana/intro-to-mlt).
It's a self-contained environment for learning about Mimir, Loki, Tempo, and Grafana.
The project includes detailed explanations of each component and annotated configurations for a single-instance deployment. Data from `intro-to-mltp` can also be pushed to Grafana Cloud.
The project includes detailed explanations of each component and annotated configurations for a single-instance deployment.
You can also push the data from the environment to [Grafana Cloud](https://grafana.com/cloud/).
Jack Baldry
1 year ago
committed by