apitech
/
loki
mirror of https://github.com/grafana/loki
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
Like Prometheus, but for logs.
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
1387 Commits
1055 Branches
354 Tags
708 MiB
Tag: Branch: Tree: 8b924a52cb
Branches Tags
${ item.name }
Create tag ${ searchTerm }
Create branch ${ searchTerm }
from '8b924a52cb'
${ noResults }
loki/docs/overview/comparisons.md

52 lines
2.8 KiB
Raw Normal View History Unescape

Documentation Rewrite (#982) * docs: create structure of docs overhaul This commit removes all old docs and lays out the table of contents and framework for how the new documentation will be intended to be read. * docs: add design docs back in * docs: add community documentation * docs: add LogQL docs * docs: port existing operations documentation * docs: add new placeholder file for promtail configuration docs * docs: add TOC for operations/storage * docs: add Loki API documentation * docs: port troubleshooting document * docs: add docker-driver documentation * docs: link to configuration from main docker-driver document * docs: update API for new paths * docs: fix broken links in api.md and remove json marker from examples * docs: incorporate api changes from #1009 * docs: port promtail documentation * docs: add TOC to promtail configuration reference * docs: fix promtail spelling errors * docs: add loki configuration reference * docs: add TOC to configuration * docs: add loki configuration example * docs: add Loki overview with brief explanation about each component * docs: add comparisons document * docs: add info on table manager and update storage/README.md * docs: add getting started * docs: incorporate config yaml changes from #755 * docs: fix typo in releases url for promtail * docs: add installation instructions * docs: add more configuration examples * docs: add information on fluentd client fluent-bit has been temporarily removed until the PR for it is merged. * docs: PR review feedback * docs: add architecture document * docs: add missing information from old docs * `localy` typo Co-Authored-By: Ed Welch <ed@oqqer.com> * docs: s/ran/run/g * Typo * Typo * Tyop * Typo * docs: fixed typo * docs: PR feedback * docs: @cyriltovena PR feedback * docs: add more details to promtail url config option * docs: expand promtail's pipelines document with extra detail * docs: remove reference to Stage interface in pipelines.md * docs: fixed some spelling * docs: clarify promtail configuration and scraping * docs: attempt #2 at explaining promtail's usage of machine hostname * docs: spelling fixes * docs: add reference to promtail custom metrics and fix silly typo * docs: cognizant -> aware * docs: typo * docs: typos * docs: add which components expose which API endpoints in microservices mode * docs: change ksonnet installation to tanka * docs: address most @pracucci feedback * docs: fix all spelling errors so reviewers don't have to keep finding them :) * docs: incorporate changes to API endpoints made in #1022 * docs: add missing loki metrics * docs: add missing promtail metrics * docs: @pstribrany feedback * docs: more @pracucci feedback * docs: move metrics into a table * docs: update push path references to /loki/api/v1/push * docs: add detail to further explain limitations of monolithic mode * docs: add alternative names to modes_of_operation diagram * docs: add log ordering requirement * docs: add procedure for updating docs with latest version * docs: separate out stages documentation into one document per stage * docs: list supported stores in storage documentation * docs: add info on duplicate log lines in pipelines * docs: add line_format as key feature to fluentd * docs: hopefully final commit :)
6 years ago
# Loki compared to other log systems
## Loki / Promtail / Grafana vs EFK
The EFK (Elasticsearch, Fluentd, Kibana) stack is used to ingest, visualize, and
query for logs from various sources.
Data in Elasticsearch is stored on-disk as unstructured JSON objects. Both the
keys for each object and the contents of each key are indexed. Data can then be
queried using a JSON object to define a query (called the Query DSL) or through
the Lucene query language.
In comparison, Loki in single-binary mode can store data on-disk, but in
horizontally-scalable mode data is stored in a cloud storage system such as S3,
GCS, or Cassandra. Logs are stored in plaintext form tagged with a set of label
names and values, where only the label pairs are indexed. This tradeoff makes it
cheaper to operate than a full index and allows developers to aggressively log
from their applications. Logs in Loki are queried using [LogQL](../logql.md).
However, because of this design tradeoff, LogQL queries that filter based on
content (i.e., text within the log lines) require loading all chunks within the
search window that match the labels defined in the query.
Fluentd is usually used to collect and forward logs to Elasticsearch. Fluentd is
called a data collector which can ingest logs from many sources, process it, and
forward it to one or more targets.
In comparison, Promtail's use case is specifically tailored to Loki. Its main mode
of operation is to discover log files stored on disk and forward them associated
with a set of labels to Loki. Promtail can do service discovery for Kubernetes
pods running on the same node as Promtail, act as a container sidecar or a
Docker logging driver, read logs from specified folders, and tail the systemd
journal.
The way Loki represents logs by a set of label pairs is similar to how
[Prometheus](https://prometheus.io) represents metrics. When deployed in an
environment alongside Prometheus, logs from Promtail usually have the same
labels as your applications metrics thanks to using the same service
discovery mechanisms. Having logs and metrics with the same levels enables users
to seamlessly context switch between metrics and logs, helping with root cause
analysis.
Kibana is used to visualize and search Elasticsearch data and is very powerful
for doing analytics on that data. Kibana provides many visualization tools to do
data analysis, such as location maps, machine learning for anomaly detection,
and graphs to discover relationships in data. Alerts can be configured to notify
users when an unexpected condition occurs.
In comparison, Grafana is tailored specifically towards time series data from
sources like Prometheus and Loki. Dashboards can be set up to visualize metrics
(log support coming soon) and an explore view can be used to make ad-hoc queries
against your data. Like Kibana, Grafana supports alerting based on your metrics.