apitech
/
loki
mirror of https://github.com/grafana/loki
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
Like Prometheus, but for logs.
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
5135 Commits
1062 Branches
354 Tags
713 MiB
Tag: Branch: Tree: c4d2966fdf
Branches Tags
${ item.name }
Create tag ${ searchTerm }
Create branch ${ searchTerm }
from 'c4d2966fdf'
${ noResults }
loki/docs/sources/installation/tanka.md

115 lines
4.5 KiB
Raw Normal View History Unescape

Sync docs to website (#2378) * update docs location * add workflow * update docs content * update content * cleanup
5 years ago
---
title: Tanka
Fix documentation linter errors (#8229)
3 years ago
description: Install Grafana Loki with Tanka
Docs: clarify distinction between single binary and microservices. (#4374) * Docs: clarify distinction between single binary and microservices. * Update clarification of mode per review suggestions
4 years ago
weight: 10
Sync docs to website (#2378) * update docs location * add workflow * update docs content * update content * cleanup
5 years ago
---
Fix documentation linter errors (#8229)
3 years ago
# Tanka
Documentation Rewrite (#982) * docs: create structure of docs overhaul This commit removes all old docs and lays out the table of contents and framework for how the new documentation will be intended to be read. * docs: add design docs back in * docs: add community documentation * docs: add LogQL docs * docs: port existing operations documentation * docs: add new placeholder file for promtail configuration docs * docs: add TOC for operations/storage * docs: add Loki API documentation * docs: port troubleshooting document * docs: add docker-driver documentation * docs: link to configuration from main docker-driver document * docs: update API for new paths * docs: fix broken links in api.md and remove json marker from examples * docs: incorporate api changes from #1009 * docs: port promtail documentation * docs: add TOC to promtail configuration reference * docs: fix promtail spelling errors * docs: add loki configuration reference * docs: add TOC to configuration * docs: add loki configuration example * docs: add Loki overview with brief explanation about each component * docs: add comparisons document * docs: add info on table manager and update storage/README.md * docs: add getting started * docs: incorporate config yaml changes from #755 * docs: fix typo in releases url for promtail * docs: add installation instructions * docs: add more configuration examples * docs: add information on fluentd client fluent-bit has been temporarily removed until the PR for it is merged. * docs: PR review feedback * docs: add architecture document * docs: add missing information from old docs * `localy` typo Co-Authored-By: Ed Welch <ed@oqqer.com> * docs: s/ran/run/g * Typo * Typo * Tyop * Typo * docs: fixed typo * docs: PR feedback * docs: @cyriltovena PR feedback * docs: add more details to promtail url config option * docs: expand promtail's pipelines document with extra detail * docs: remove reference to Stage interface in pipelines.md * docs: fixed some spelling * docs: clarify promtail configuration and scraping * docs: attempt #2 at explaining promtail's usage of machine hostname * docs: spelling fixes * docs: add reference to promtail custom metrics and fix silly typo * docs: cognizant -> aware * docs: typo * docs: typos * docs: add which components expose which API endpoints in microservices mode * docs: change ksonnet installation to tanka * docs: address most @pracucci feedback * docs: fix all spelling errors so reviewers don't have to keep finding them :) * docs: incorporate changes to API endpoints made in #1022 * docs: add missing loki metrics * docs: add missing promtail metrics * docs: @pstribrany feedback * docs: more @pracucci feedback * docs: move metrics into a table * docs: update push path references to /loki/api/v1/push * docs: add detail to further explain limitations of monolithic mode * docs: add alternative names to modes_of_operation diagram * docs: add log ordering requirement * docs: add procedure for updating docs with latest version * docs: separate out stages documentation into one document per stage * docs: list supported stores in storage documentation * docs: add info on duplicate log lines in pipelines * docs: add line_format as key feature to fluentd * docs: hopefully final commit :)
6 years ago
[Tanka](https://tanka.dev) is a reimplementation of
[Ksonnet](https://ksonnet.io) that Grafana Labs created after Ksonnet was
Docs: correctly represent product name (#4416) The product name is Grafana Loki. The first use of the product name on each web page should be the full product name. After that, within each page, it is fine to use just Loki. This PR also set up some frontmatter with weights where it was missing on a few files, and correct where "Loki" was not capitalized.
4 years ago
deprecated. Tanka is used by Grafana Labs to run Grafana Loki in production.
Documentation Rewrite (#982) * docs: create structure of docs overhaul This commit removes all old docs and lays out the table of contents and framework for how the new documentation will be intended to be read. * docs: add design docs back in * docs: add community documentation * docs: add LogQL docs * docs: port existing operations documentation * docs: add new placeholder file for promtail configuration docs * docs: add TOC for operations/storage * docs: add Loki API documentation * docs: port troubleshooting document * docs: add docker-driver documentation * docs: link to configuration from main docker-driver document * docs: update API for new paths * docs: fix broken links in api.md and remove json marker from examples * docs: incorporate api changes from #1009 * docs: port promtail documentation * docs: add TOC to promtail configuration reference * docs: fix promtail spelling errors * docs: add loki configuration reference * docs: add TOC to configuration * docs: add loki configuration example * docs: add Loki overview with brief explanation about each component * docs: add comparisons document * docs: add info on table manager and update storage/README.md * docs: add getting started * docs: incorporate config yaml changes from #755 * docs: fix typo in releases url for promtail * docs: add installation instructions * docs: add more configuration examples * docs: add information on fluentd client fluent-bit has been temporarily removed until the PR for it is merged. * docs: PR review feedback * docs: add architecture document * docs: add missing information from old docs * `localy` typo Co-Authored-By: Ed Welch <ed@oqqer.com> * docs: s/ran/run/g * Typo * Typo * Tyop * Typo * docs: fixed typo * docs: PR feedback * docs: @cyriltovena PR feedback * docs: add more details to promtail url config option * docs: expand promtail's pipelines document with extra detail * docs: remove reference to Stage interface in pipelines.md * docs: fixed some spelling * docs: clarify promtail configuration and scraping * docs: attempt #2 at explaining promtail's usage of machine hostname * docs: spelling fixes * docs: add reference to promtail custom metrics and fix silly typo * docs: cognizant -> aware * docs: typo * docs: typos * docs: add which components expose which API endpoints in microservices mode * docs: change ksonnet installation to tanka * docs: address most @pracucci feedback * docs: fix all spelling errors so reviewers don't have to keep finding them :) * docs: incorporate changes to API endpoints made in #1022 * docs: add missing loki metrics * docs: add missing promtail metrics * docs: @pstribrany feedback * docs: more @pracucci feedback * docs: move metrics into a table * docs: update push path references to /loki/api/v1/push * docs: add detail to further explain limitations of monolithic mode * docs: add alternative names to modes_of_operation diagram * docs: add log ordering requirement * docs: add procedure for updating docs with latest version * docs: separate out stages documentation into one document per stage * docs: list supported stores in storage documentation * docs: add info on duplicate log lines in pipelines * docs: add line_format as key feature to fluentd * docs: hopefully final commit :)
6 years ago
Docs: clarify distinction between single binary and microservices. (#4374) * Docs: clarify distinction between single binary and microservices. * Update clarification of mode per review suggestions
4 years ago
The Tanka installation runs the Loki cluster in microservices mode.
Documentation Rewrite (#982) * docs: create structure of docs overhaul This commit removes all old docs and lays out the table of contents and framework for how the new documentation will be intended to be read. * docs: add design docs back in * docs: add community documentation * docs: add LogQL docs * docs: port existing operations documentation * docs: add new placeholder file for promtail configuration docs * docs: add TOC for operations/storage * docs: add Loki API documentation * docs: port troubleshooting document * docs: add docker-driver documentation * docs: link to configuration from main docker-driver document * docs: update API for new paths * docs: fix broken links in api.md and remove json marker from examples * docs: incorporate api changes from #1009 * docs: port promtail documentation * docs: add TOC to promtail configuration reference * docs: fix promtail spelling errors * docs: add loki configuration reference * docs: add TOC to configuration * docs: add loki configuration example * docs: add Loki overview with brief explanation about each component * docs: add comparisons document * docs: add info on table manager and update storage/README.md * docs: add getting started * docs: incorporate config yaml changes from #755 * docs: fix typo in releases url for promtail * docs: add installation instructions * docs: add more configuration examples * docs: add information on fluentd client fluent-bit has been temporarily removed until the PR for it is merged. * docs: PR review feedback * docs: add architecture document * docs: add missing information from old docs * `localy` typo Co-Authored-By: Ed Welch <ed@oqqer.com> * docs: s/ran/run/g * Typo * Typo * Tyop * Typo * docs: fixed typo * docs: PR feedback * docs: @cyriltovena PR feedback * docs: add more details to promtail url config option * docs: expand promtail's pipelines document with extra detail * docs: remove reference to Stage interface in pipelines.md * docs: fixed some spelling * docs: clarify promtail configuration and scraping * docs: attempt #2 at explaining promtail's usage of machine hostname * docs: spelling fixes * docs: add reference to promtail custom metrics and fix silly typo * docs: cognizant -> aware * docs: typo * docs: typos * docs: add which components expose which API endpoints in microservices mode * docs: change ksonnet installation to tanka * docs: address most @pracucci feedback * docs: fix all spelling errors so reviewers don't have to keep finding them :) * docs: incorporate changes to API endpoints made in #1022 * docs: add missing loki metrics * docs: add missing promtail metrics * docs: @pstribrany feedback * docs: more @pracucci feedback * docs: move metrics into a table * docs: update push path references to /loki/api/v1/push * docs: add detail to further explain limitations of monolithic mode * docs: add alternative names to modes_of_operation diagram * docs: add log ordering requirement * docs: add procedure for updating docs with latest version * docs: separate out stages documentation into one document per stage * docs: list supported stores in storage documentation * docs: add info on duplicate log lines in pipelines * docs: add line_format as key feature to fluentd * docs: hopefully final commit :)
6 years ago
## Prerequisites
Update the docs to install Loki with Tanka (#4083) * Update to the Loki tanka install docs to reflect changes made in tanka. Also pulled up the docs piece about the bolt-db-shipper-shared-store so users know they must modify it, rather than seeing it as a note further down. I think we're probably still missing one more update to the yaml to add the `schema_config` override, but that can come as a separate pr. * Adding the schema config block. Think we should update here because we otherwise default to gcs, which will throw an error if the user wants to use aws s3. * Updated the value we recommend for 'from' * Update docs/sources/installation/tanka.md Co-authored-by: Sandeep Sukhani <sandeep.d.sukhani@gmail.com> * Apply suggestions from code review Thank you Karen! Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> * Tried to get more clarify around how I"m describing the object storage type. * Update docs/sources/installation/tanka.md Typos begone! Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> Co-authored-by: Sandeep Sukhani <sandeep.d.sukhani@gmail.com> Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com>
4 years ago
Install the latest version of Tanka (version v0.17.1 or a more recent version) for the `tk env`
Documentation Rewrite (#982) * docs: create structure of docs overhaul This commit removes all old docs and lays out the table of contents and framework for how the new documentation will be intended to be read. * docs: add design docs back in * docs: add community documentation * docs: add LogQL docs * docs: port existing operations documentation * docs: add new placeholder file for promtail configuration docs * docs: add TOC for operations/storage * docs: add Loki API documentation * docs: port troubleshooting document * docs: add docker-driver documentation * docs: link to configuration from main docker-driver document * docs: update API for new paths * docs: fix broken links in api.md and remove json marker from examples * docs: incorporate api changes from #1009 * docs: port promtail documentation * docs: add TOC to promtail configuration reference * docs: fix promtail spelling errors * docs: add loki configuration reference * docs: add TOC to configuration * docs: add loki configuration example * docs: add Loki overview with brief explanation about each component * docs: add comparisons document * docs: add info on table manager and update storage/README.md * docs: add getting started * docs: incorporate config yaml changes from #755 * docs: fix typo in releases url for promtail * docs: add installation instructions * docs: add more configuration examples * docs: add information on fluentd client fluent-bit has been temporarily removed until the PR for it is merged. * docs: PR review feedback * docs: add architecture document * docs: add missing information from old docs * `localy` typo Co-Authored-By: Ed Welch <ed@oqqer.com> * docs: s/ran/run/g * Typo * Typo * Tyop * Typo * docs: fixed typo * docs: PR feedback * docs: @cyriltovena PR feedback * docs: add more details to promtail url config option * docs: expand promtail's pipelines document with extra detail * docs: remove reference to Stage interface in pipelines.md * docs: fixed some spelling * docs: clarify promtail configuration and scraping * docs: attempt #2 at explaining promtail's usage of machine hostname * docs: spelling fixes * docs: add reference to promtail custom metrics and fix silly typo * docs: cognizant -> aware * docs: typo * docs: typos * docs: add which components expose which API endpoints in microservices mode * docs: change ksonnet installation to tanka * docs: address most @pracucci feedback * docs: fix all spelling errors so reviewers don't have to keep finding them :) * docs: incorporate changes to API endpoints made in #1022 * docs: add missing loki metrics * docs: add missing promtail metrics * docs: @pstribrany feedback * docs: more @pracucci feedback * docs: move metrics into a table * docs: update push path references to /loki/api/v1/push * docs: add detail to further explain limitations of monolithic mode * docs: add alternative names to modes_of_operation diagram * docs: add log ordering requirement * docs: add procedure for updating docs with latest version * docs: separate out stages documentation into one document per stage * docs: list supported stores in storage documentation * docs: add info on duplicate log lines in pipelines * docs: add line_format as key feature to fluentd * docs: hopefully final commit :)
6 years ago
commands. Prebuilt binaries for Tanka can be found at the [Tanka releases
URL](https://github.com/grafana/tanka/releases).
In your config repo, if you don't have a Tanka application, create a folder and
call `tk init` inside of it. Then create an environment for Loki and provide the
URL for the Kubernetes API server to deploy to (e.g., `https://localhost:6443`):
```
Update tanka.md doc. (#1598) * Update tanka.md Fixed and tested see https://github.com/grafana/loki/issues/1591 * Update tanka.md * Update docs/installation/tanka.md Co-Authored-By: Robert Fratto <robert.fratto@grafana.com> Co-authored-by: Robert Fratto <robertfratto@gmail.com>
6 years ago
mkdir <application name>
cd <application name>
tk init
tk env add environments/loki --namespace=loki --server=<Kubernetes API server>
Documentation Rewrite (#982) * docs: create structure of docs overhaul This commit removes all old docs and lays out the table of contents and framework for how the new documentation will be intended to be read. * docs: add design docs back in * docs: add community documentation * docs: add LogQL docs * docs: port existing operations documentation * docs: add new placeholder file for promtail configuration docs * docs: add TOC for operations/storage * docs: add Loki API documentation * docs: port troubleshooting document * docs: add docker-driver documentation * docs: link to configuration from main docker-driver document * docs: update API for new paths * docs: fix broken links in api.md and remove json marker from examples * docs: incorporate api changes from #1009 * docs: port promtail documentation * docs: add TOC to promtail configuration reference * docs: fix promtail spelling errors * docs: add loki configuration reference * docs: add TOC to configuration * docs: add loki configuration example * docs: add Loki overview with brief explanation about each component * docs: add comparisons document * docs: add info on table manager and update storage/README.md * docs: add getting started * docs: incorporate config yaml changes from #755 * docs: fix typo in releases url for promtail * docs: add installation instructions * docs: add more configuration examples * docs: add information on fluentd client fluent-bit has been temporarily removed until the PR for it is merged. * docs: PR review feedback * docs: add architecture document * docs: add missing information from old docs * `localy` typo Co-Authored-By: Ed Welch <ed@oqqer.com> * docs: s/ran/run/g * Typo * Typo * Tyop * Typo * docs: fixed typo * docs: PR feedback * docs: @cyriltovena PR feedback * docs: add more details to promtail url config option * docs: expand promtail's pipelines document with extra detail * docs: remove reference to Stage interface in pipelines.md * docs: fixed some spelling * docs: clarify promtail configuration and scraping * docs: attempt #2 at explaining promtail's usage of machine hostname * docs: spelling fixes * docs: add reference to promtail custom metrics and fix silly typo * docs: cognizant -> aware * docs: typo * docs: typos * docs: add which components expose which API endpoints in microservices mode * docs: change ksonnet installation to tanka * docs: address most @pracucci feedback * docs: fix all spelling errors so reviewers don't have to keep finding them :) * docs: incorporate changes to API endpoints made in #1022 * docs: add missing loki metrics * docs: add missing promtail metrics * docs: @pstribrany feedback * docs: more @pracucci feedback * docs: move metrics into a table * docs: update push path references to /loki/api/v1/push * docs: add detail to further explain limitations of monolithic mode * docs: add alternative names to modes_of_operation diagram * docs: add log ordering requirement * docs: add procedure for updating docs with latest version * docs: separate out stages documentation into one document per stage * docs: list supported stores in storage documentation * docs: add info on duplicate log lines in pipelines * docs: add line_format as key feature to fluentd * docs: hopefully final commit :)
6 years ago
```
Update tanka installation docs to refer to tanka section about `jb` (#4208) installation instead of providing a cli command that may not work. Signed-off-by: Callum Styan <callumstyan@gmail.com>
4 years ago
Install `jsonnet-bundler` (`jb`), find instructions for your platform in Tanka's [installation docs](https://tanka.dev/install#jsonnet-bundler).
Documentation Rewrite (#982) * docs: create structure of docs overhaul This commit removes all old docs and lays out the table of contents and framework for how the new documentation will be intended to be read. * docs: add design docs back in * docs: add community documentation * docs: add LogQL docs * docs: port existing operations documentation * docs: add new placeholder file for promtail configuration docs * docs: add TOC for operations/storage * docs: add Loki API documentation * docs: port troubleshooting document * docs: add docker-driver documentation * docs: link to configuration from main docker-driver document * docs: update API for new paths * docs: fix broken links in api.md and remove json marker from examples * docs: incorporate api changes from #1009 * docs: port promtail documentation * docs: add TOC to promtail configuration reference * docs: fix promtail spelling errors * docs: add loki configuration reference * docs: add TOC to configuration * docs: add loki configuration example * docs: add Loki overview with brief explanation about each component * docs: add comparisons document * docs: add info on table manager and update storage/README.md * docs: add getting started * docs: incorporate config yaml changes from #755 * docs: fix typo in releases url for promtail * docs: add installation instructions * docs: add more configuration examples * docs: add information on fluentd client fluent-bit has been temporarily removed until the PR for it is merged. * docs: PR review feedback * docs: add architecture document * docs: add missing information from old docs * `localy` typo Co-Authored-By: Ed Welch <ed@oqqer.com> * docs: s/ran/run/g * Typo * Typo * Tyop * Typo * docs: fixed typo * docs: PR feedback * docs: @cyriltovena PR feedback * docs: add more details to promtail url config option * docs: expand promtail's pipelines document with extra detail * docs: remove reference to Stage interface in pipelines.md * docs: fixed some spelling * docs: clarify promtail configuration and scraping * docs: attempt #2 at explaining promtail's usage of machine hostname * docs: spelling fixes * docs: add reference to promtail custom metrics and fix silly typo * docs: cognizant -> aware * docs: typo * docs: typos * docs: add which components expose which API endpoints in microservices mode * docs: change ksonnet installation to tanka * docs: address most @pracucci feedback * docs: fix all spelling errors so reviewers don't have to keep finding them :) * docs: incorporate changes to API endpoints made in #1022 * docs: add missing loki metrics * docs: add missing promtail metrics * docs: @pstribrany feedback * docs: more @pracucci feedback * docs: move metrics into a table * docs: update push path references to /loki/api/v1/push * docs: add detail to further explain limitations of monolithic mode * docs: add alternative names to modes_of_operation diagram * docs: add log ordering requirement * docs: add procedure for updating docs with latest version * docs: separate out stages documentation into one document per stage * docs: list supported stores in storage documentation * docs: add info on duplicate log lines in pipelines * docs: add line_format as key feature to fluentd * docs: hopefully final commit :)
6 years ago
## Deploying
Update the docs to install Loki with Tanka (#4083) * Update to the Loki tanka install docs to reflect changes made in tanka. Also pulled up the docs piece about the bolt-db-shipper-shared-store so users know they must modify it, rather than seeing it as a note further down. I think we're probably still missing one more update to the yaml to add the `schema_config` override, but that can come as a separate pr. * Adding the schema config block. Think we should update here because we otherwise default to gcs, which will throw an error if the user wants to use aws s3. * Updated the value we recommend for 'from' * Update docs/sources/installation/tanka.md Co-authored-by: Sandeep Sukhani <sandeep.d.sukhani@gmail.com> * Apply suggestions from code review Thank you Karen! Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> * Tried to get more clarify around how I"m describing the object storage type. * Update docs/sources/installation/tanka.md Typos begone! Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> Co-authored-by: Sandeep Sukhani <sandeep.d.sukhani@gmail.com> Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com>
4 years ago
Download and install the Loki and Promtail module using `jb` (version v0.4.0 or a more recent version):
Documentation Rewrite (#982) * docs: create structure of docs overhaul This commit removes all old docs and lays out the table of contents and framework for how the new documentation will be intended to be read. * docs: add design docs back in * docs: add community documentation * docs: add LogQL docs * docs: port existing operations documentation * docs: add new placeholder file for promtail configuration docs * docs: add TOC for operations/storage * docs: add Loki API documentation * docs: port troubleshooting document * docs: add docker-driver documentation * docs: link to configuration from main docker-driver document * docs: update API for new paths * docs: fix broken links in api.md and remove json marker from examples * docs: incorporate api changes from #1009 * docs: port promtail documentation * docs: add TOC to promtail configuration reference * docs: fix promtail spelling errors * docs: add loki configuration reference * docs: add TOC to configuration * docs: add loki configuration example * docs: add Loki overview with brief explanation about each component * docs: add comparisons document * docs: add info on table manager and update storage/README.md * docs: add getting started * docs: incorporate config yaml changes from #755 * docs: fix typo in releases url for promtail * docs: add installation instructions * docs: add more configuration examples * docs: add information on fluentd client fluent-bit has been temporarily removed until the PR for it is merged. * docs: PR review feedback * docs: add architecture document * docs: add missing information from old docs * `localy` typo Co-Authored-By: Ed Welch <ed@oqqer.com> * docs: s/ran/run/g * Typo * Typo * Tyop * Typo * docs: fixed typo * docs: PR feedback * docs: @cyriltovena PR feedback * docs: add more details to promtail url config option * docs: expand promtail's pipelines document with extra detail * docs: remove reference to Stage interface in pipelines.md * docs: fixed some spelling * docs: clarify promtail configuration and scraping * docs: attempt #2 at explaining promtail's usage of machine hostname * docs: spelling fixes * docs: add reference to promtail custom metrics and fix silly typo * docs: cognizant -> aware * docs: typo * docs: typos * docs: add which components expose which API endpoints in microservices mode * docs: change ksonnet installation to tanka * docs: address most @pracucci feedback * docs: fix all spelling errors so reviewers don't have to keep finding them :) * docs: incorporate changes to API endpoints made in #1022 * docs: add missing loki metrics * docs: add missing promtail metrics * docs: @pstribrany feedback * docs: more @pracucci feedback * docs: move metrics into a table * docs: update push path references to /loki/api/v1/push * docs: add detail to further explain limitations of monolithic mode * docs: add alternative names to modes_of_operation diagram * docs: add log ordering requirement * docs: add procedure for updating docs with latest version * docs: separate out stages documentation into one document per stage * docs: list supported stores in storage documentation * docs: add info on duplicate log lines in pipelines * docs: add line_format as key feature to fluentd * docs: hopefully final commit :)
6 years ago
```bash
fix(ksonnet): don't depend on specific k8s version (#2525) * fix(ksonnet): don't depend on specific k8s version * docs(tanka): remove k8s-alpha refs * fix(ksonnet): backwards compatibility with ksonnet
5 years ago
jb init # not required if you already ran `tk init`
Add explicit `main` to pull loki and promtail to install it via Tanka (#3848) Rationale is, by default `jb` command, assumes `master` branch, but Loki repo renamed the `master` to `main` branch.
5 years ago
jb install github.com/grafana/loki/production/ksonnet/loki@main
jb install github.com/grafana/loki/production/ksonnet/promtail@main
Documentation Rewrite (#982) * docs: create structure of docs overhaul This commit removes all old docs and lays out the table of contents and framework for how the new documentation will be intended to be read. * docs: add design docs back in * docs: add community documentation * docs: add LogQL docs * docs: port existing operations documentation * docs: add new placeholder file for promtail configuration docs * docs: add TOC for operations/storage * docs: add Loki API documentation * docs: port troubleshooting document * docs: add docker-driver documentation * docs: link to configuration from main docker-driver document * docs: update API for new paths * docs: fix broken links in api.md and remove json marker from examples * docs: incorporate api changes from #1009 * docs: port promtail documentation * docs: add TOC to promtail configuration reference * docs: fix promtail spelling errors * docs: add loki configuration reference * docs: add TOC to configuration * docs: add loki configuration example * docs: add Loki overview with brief explanation about each component * docs: add comparisons document * docs: add info on table manager and update storage/README.md * docs: add getting started * docs: incorporate config yaml changes from #755 * docs: fix typo in releases url for promtail * docs: add installation instructions * docs: add more configuration examples * docs: add information on fluentd client fluent-bit has been temporarily removed until the PR for it is merged. * docs: PR review feedback * docs: add architecture document * docs: add missing information from old docs * `localy` typo Co-Authored-By: Ed Welch <ed@oqqer.com> * docs: s/ran/run/g * Typo * Typo * Tyop * Typo * docs: fixed typo * docs: PR feedback * docs: @cyriltovena PR feedback * docs: add more details to promtail url config option * docs: expand promtail's pipelines document with extra detail * docs: remove reference to Stage interface in pipelines.md * docs: fixed some spelling * docs: clarify promtail configuration and scraping * docs: attempt #2 at explaining promtail's usage of machine hostname * docs: spelling fixes * docs: add reference to promtail custom metrics and fix silly typo * docs: cognizant -> aware * docs: typo * docs: typos * docs: add which components expose which API endpoints in microservices mode * docs: change ksonnet installation to tanka * docs: address most @pracucci feedback * docs: fix all spelling errors so reviewers don't have to keep finding them :) * docs: incorporate changes to API endpoints made in #1022 * docs: add missing loki metrics * docs: add missing promtail metrics * docs: @pstribrany feedback * docs: more @pracucci feedback * docs: move metrics into a table * docs: update push path references to /loki/api/v1/push * docs: add detail to further explain limitations of monolithic mode * docs: add alternative names to modes_of_operation diagram * docs: add log ordering requirement * docs: add procedure for updating docs with latest version * docs: separate out stages documentation into one document per stage * docs: list supported stores in storage documentation * docs: add info on duplicate log lines in pipelines * docs: add line_format as key feature to fluentd * docs: hopefully final commit :)
6 years ago
```
Update the docs to install Loki with Tanka (#4083) * Update to the Loki tanka install docs to reflect changes made in tanka. Also pulled up the docs piece about the bolt-db-shipper-shared-store so users know they must modify it, rather than seeing it as a note further down. I think we're probably still missing one more update to the yaml to add the `schema_config` override, but that can come as a separate pr. * Adding the schema config block. Think we should update here because we otherwise default to gcs, which will throw an error if the user wants to use aws s3. * Updated the value we recommend for 'from' * Update docs/sources/installation/tanka.md Co-authored-by: Sandeep Sukhani <sandeep.d.sukhani@gmail.com> * Apply suggestions from code review Thank you Karen! Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> * Tried to get more clarify around how I"m describing the object storage type. * Update docs/sources/installation/tanka.md Typos begone! Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> Co-authored-by: Sandeep Sukhani <sandeep.d.sukhani@gmail.com> Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com>
4 years ago
Revise the YAML contents of `environments/loki/main.jsonnet`, updating these variables:
Update tanka.md with steps for using k8s-alpha lib (#2528) * Update tanka.md * Update docs/sources/installation/tanka.md Co-authored-by: Diana Payton <52059945+oddlittlebird@users.noreply.github.com> * Update docs/sources/installation/tanka.md Co-authored-by: Diana Payton <52059945+oddlittlebird@users.noreply.github.com> Co-authored-by: Diana Payton <52059945+oddlittlebird@users.noreply.github.com>
5 years ago
Update the docs to install Loki with Tanka (#4083) * Update to the Loki tanka install docs to reflect changes made in tanka. Also pulled up the docs piece about the bolt-db-shipper-shared-store so users know they must modify it, rather than seeing it as a note further down. I think we're probably still missing one more update to the yaml to add the `schema_config` override, but that can come as a separate pr. * Adding the schema config block. Think we should update here because we otherwise default to gcs, which will throw an error if the user wants to use aws s3. * Updated the value we recommend for 'from' * Update docs/sources/installation/tanka.md Co-authored-by: Sandeep Sukhani <sandeep.d.sukhani@gmail.com> * Apply suggestions from code review Thank you Karen! Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> * Tried to get more clarify around how I"m describing the object storage type. * Update docs/sources/installation/tanka.md Typos begone! Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> Co-authored-by: Sandeep Sukhani <sandeep.d.sukhani@gmail.com> Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com>
4 years ago
- Update the `username`, `password`, and the relevant `htpasswd` variable values.
Fix documentation linter errors (#8229)
3 years ago
- Update the S3 or GCS variable values, depending on your object storage type. See [storage_config](/docs/loki/latest/configuration/#storage_config) for more configuration details.
Update the docs to install Loki with Tanka (#4083) * Update to the Loki tanka install docs to reflect changes made in tanka. Also pulled up the docs piece about the bolt-db-shipper-shared-store so users know they must modify it, rather than seeing it as a note further down. I think we're probably still missing one more update to the yaml to add the `schema_config` override, but that can come as a separate pr. * Adding the schema config block. Think we should update here because we otherwise default to gcs, which will throw an error if the user wants to use aws s3. * Updated the value we recommend for 'from' * Update docs/sources/installation/tanka.md Co-authored-by: Sandeep Sukhani <sandeep.d.sukhani@gmail.com> * Apply suggestions from code review Thank you Karen! Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> * Tried to get more clarify around how I"m describing the object storage type. * Update docs/sources/installation/tanka.md Typos begone! Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> Co-authored-by: Sandeep Sukhani <sandeep.d.sukhani@gmail.com> Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com>
4 years ago
- Remove from the configuration the S3 or GCS object storage variables that are not part of your setup.
feat(storage): add support for IBM cloud object storage as storage client (#8826) **What this PR does / why we need it**: Add support for IBM cloud object storage as storage client **Which issue(s) this PR fixes**: Fixes NA **Checklist** - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [x] Documentation added - [x] Tests updated - [ ] `CHANGELOG.md` updated --------- Signed-off-by: Shahul <shahulsonhal@gmail.com> Co-authored-by: Aman Kumar Singh <amankrsingh2110@gmail.com> Co-authored-by: Suruthi-G-K <shruthi.suruthi@gmail.com> Co-authored-by: tareqmamari <tariq.mamari@de.ibm.com> Co-authored-by: shahulsonhal <shahulsonhal@gmail.com> Co-authored-by: Aditya C S <aditya.gnu@gmail.com> Co-authored-by: Tareq Al-Maamari <tariq.mamari@gmail.com>
3 years ago
- Update the value of `boltdb_shipper_shared_store` to the type of object storage you are using. Options are `gcs`, `s3`, `azure`, `cos` or `filesystem`. Update the `object_store` variable under the `schema_config` section to the same value.
Update the docs to install Loki with Tanka (#4083) * Update to the Loki tanka install docs to reflect changes made in tanka. Also pulled up the docs piece about the bolt-db-shipper-shared-store so users know they must modify it, rather than seeing it as a note further down. I think we're probably still missing one more update to the yaml to add the `schema_config` override, but that can come as a separate pr. * Adding the schema config block. Think we should update here because we otherwise default to gcs, which will throw an error if the user wants to use aws s3. * Updated the value we recommend for 'from' * Update docs/sources/installation/tanka.md Co-authored-by: Sandeep Sukhani <sandeep.d.sukhani@gmail.com> * Apply suggestions from code review Thank you Karen! Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> * Tried to get more clarify around how I"m describing the object storage type. * Update docs/sources/installation/tanka.md Typos begone! Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> Co-authored-by: Sandeep Sukhani <sandeep.d.sukhani@gmail.com> Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com>
4 years ago
- Update the Promtail configuration `container_root_path` variable's value to reflect your root path for the Docker daemon. Run `docker info | grep "Root Dir"` to acquire your root path.
- Update the `from` value in the Loki `schema_config` section to no more than 14 days prior to the current date. The `from` date represents the first day for which the `schema_config` section is valid. For example, if today is `2021-01-15`, set `from` to `2021-01-01`. This recommendation is based on Loki's default acceptance of log lines up to 14 days in the past. The `reject_old_samples_max_age` configuration variable controls the acceptance range.
Update tanka.md with steps for using k8s-alpha lib (#2528) * Update tanka.md * Update docs/sources/installation/tanka.md Co-authored-by: Diana Payton <52059945+oddlittlebird@users.noreply.github.com> * Update docs/sources/installation/tanka.md Co-authored-by: Diana Payton <52059945+oddlittlebird@users.noreply.github.com> Co-authored-by: Diana Payton <52059945+oddlittlebird@users.noreply.github.com>
5 years ago
Documentation Rewrite (#982) * docs: create structure of docs overhaul This commit removes all old docs and lays out the table of contents and framework for how the new documentation will be intended to be read. * docs: add design docs back in * docs: add community documentation * docs: add LogQL docs * docs: port existing operations documentation * docs: add new placeholder file for promtail configuration docs * docs: add TOC for operations/storage * docs: add Loki API documentation * docs: port troubleshooting document * docs: add docker-driver documentation * docs: link to configuration from main docker-driver document * docs: update API for new paths * docs: fix broken links in api.md and remove json marker from examples * docs: incorporate api changes from #1009 * docs: port promtail documentation * docs: add TOC to promtail configuration reference * docs: fix promtail spelling errors * docs: add loki configuration reference * docs: add TOC to configuration * docs: add loki configuration example * docs: add Loki overview with brief explanation about each component * docs: add comparisons document * docs: add info on table manager and update storage/README.md * docs: add getting started * docs: incorporate config yaml changes from #755 * docs: fix typo in releases url for promtail * docs: add installation instructions * docs: add more configuration examples * docs: add information on fluentd client fluent-bit has been temporarily removed until the PR for it is merged. * docs: PR review feedback * docs: add architecture document * docs: add missing information from old docs * `localy` typo Co-Authored-By: Ed Welch <ed@oqqer.com> * docs: s/ran/run/g * Typo * Typo * Tyop * Typo * docs: fixed typo * docs: PR feedback * docs: @cyriltovena PR feedback * docs: add more details to promtail url config option * docs: expand promtail's pipelines document with extra detail * docs: remove reference to Stage interface in pipelines.md * docs: fixed some spelling * docs: clarify promtail configuration and scraping * docs: attempt #2 at explaining promtail's usage of machine hostname * docs: spelling fixes * docs: add reference to promtail custom metrics and fix silly typo * docs: cognizant -> aware * docs: typo * docs: typos * docs: add which components expose which API endpoints in microservices mode * docs: change ksonnet installation to tanka * docs: address most @pracucci feedback * docs: fix all spelling errors so reviewers don't have to keep finding them :) * docs: incorporate changes to API endpoints made in #1022 * docs: add missing loki metrics * docs: add missing promtail metrics * docs: @pstribrany feedback * docs: more @pracucci feedback * docs: move metrics into a table * docs: update push path references to /loki/api/v1/push * docs: add detail to further explain limitations of monolithic mode * docs: add alternative names to modes_of_operation diagram * docs: add log ordering requirement * docs: add procedure for updating docs with latest version * docs: separate out stages documentation into one document per stage * docs: list supported stores in storage documentation * docs: add info on duplicate log lines in pipelines * docs: add line_format as key feature to fluentd * docs: hopefully final commit :)
6 years ago
```jsonnet
local gateway = import 'loki/gateway.libsonnet';
local loki = import 'loki/loki.libsonnet';
local promtail = import 'promtail/promtail.libsonnet';
loki + promtail + gateway {
_config+:: {
namespace: 'loki',
htpasswd_contents: 'loki:$apr1$H4yGiGNg$ssl5/NymaGFRUvxIV1Nyr.',
Sync docs to website (#2378) * update docs location * add workflow * update docs content * update content * cleanup
5 years ago
Update the docs to install Loki with Tanka (#4083) * Update to the Loki tanka install docs to reflect changes made in tanka. Also pulled up the docs piece about the bolt-db-shipper-shared-store so users know they must modify it, rather than seeing it as a note further down. I think we're probably still missing one more update to the yaml to add the `schema_config` override, but that can come as a separate pr. * Adding the schema config block. Think we should update here because we otherwise default to gcs, which will throw an error if the user wants to use aws s3. * Updated the value we recommend for 'from' * Update docs/sources/installation/tanka.md Co-authored-by: Sandeep Sukhani <sandeep.d.sukhani@gmail.com> * Apply suggestions from code review Thank you Karen! Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> * Tried to get more clarify around how I"m describing the object storage type. * Update docs/sources/installation/tanka.md Typos begone! Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> Co-authored-by: Sandeep Sukhani <sandeep.d.sukhani@gmail.com> Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com>
4 years ago
// S3 variables -- Remove if not using s3
Update tanka.md doc. (#1598) * Update tanka.md Fixed and tested see https://github.com/grafana/loki/issues/1591 * Update tanka.md * Update docs/installation/tanka.md Co-Authored-By: Robert Fratto <robert.fratto@grafana.com> Co-authored-by: Robert Fratto <robertfratto@gmail.com>
6 years ago
storage_backend: 's3,dynamodb',
s3_access_key: 'key',
s3_secret_access_key: 'secret access key',
s3_address: 'url',
s3_bucket_name: 'loki-test',
dynamodb_region: 'region',
Sync docs to website (#2378) * update docs location * add workflow * update docs content * update content * cleanup
5 years ago
Update the docs to install Loki with Tanka (#4083) * Update to the Loki tanka install docs to reflect changes made in tanka. Also pulled up the docs piece about the bolt-db-shipper-shared-store so users know they must modify it, rather than seeing it as a note further down. I think we're probably still missing one more update to the yaml to add the `schema_config` override, but that can come as a separate pr. * Adding the schema config block. Think we should update here because we otherwise default to gcs, which will throw an error if the user wants to use aws s3. * Updated the value we recommend for 'from' * Update docs/sources/installation/tanka.md Co-authored-by: Sandeep Sukhani <sandeep.d.sukhani@gmail.com> * Apply suggestions from code review Thank you Karen! Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> * Tried to get more clarify around how I"m describing the object storage type. * Update docs/sources/installation/tanka.md Typos begone! Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> Co-authored-by: Sandeep Sukhani <sandeep.d.sukhani@gmail.com> Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com>
4 years ago
// GCS variables -- Remove if not using gcs
Update tanka.md doc. (#1598) * Update tanka.md Fixed and tested see https://github.com/grafana/loki/issues/1591 * Update tanka.md * Update docs/installation/tanka.md Co-Authored-By: Robert Fratto <robert.fratto@grafana.com> Co-authored-by: Robert Fratto <robertfratto@gmail.com>
6 years ago
storage_backend: 'bigtable,gcs',
bigtable_instance: 'instance',
bigtable_project: 'project',
gcs_bucket_name: 'bucket',
Sync docs to website (#2378) * update docs location * add workflow * update docs content * update content * cleanup
5 years ago
Update the docs to install Loki with Tanka (#4083) * Update to the Loki tanka install docs to reflect changes made in tanka. Also pulled up the docs piece about the bolt-db-shipper-shared-store so users know they must modify it, rather than seeing it as a note further down. I think we're probably still missing one more update to the yaml to add the `schema_config` override, but that can come as a separate pr. * Adding the schema config block. Think we should update here because we otherwise default to gcs, which will throw an error if the user wants to use aws s3. * Updated the value we recommend for 'from' * Update docs/sources/installation/tanka.md Co-authored-by: Sandeep Sukhani <sandeep.d.sukhani@gmail.com> * Apply suggestions from code review Thank you Karen! Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> * Tried to get more clarify around how I"m describing the object storage type. * Update docs/sources/installation/tanka.md Typos begone! Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> Co-authored-by: Sandeep Sukhani <sandeep.d.sukhani@gmail.com> Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com>
4 years ago
//Set this variable based on the type of object storage you're using.
boltdb_shipper_shared_store: 'my-object-storage-backend-type',
//Update the object_store and from fields
loki+: {
schema_config: {
configs: [{
from: 'YYYY-MM-DD',
store: 'boltdb-shipper',
object_store: 'my-object-storage-backend-type',
schema: 'v11',
index: {
prefix: '%s_index_' % $._config.table_prefix,
period: '%dh' % $._config.index_period_hours,
},
}],
},
},
//Update the container_root_path if necessary
Update tanka.md doc. (#1598) * Update tanka.md Fixed and tested see https://github.com/grafana/loki/issues/1591 * Update tanka.md * Update docs/installation/tanka.md Co-Authored-By: Robert Fratto <robert.fratto@grafana.com> Co-authored-by: Robert Fratto <robertfratto@gmail.com>
6 years ago
promtail_config+: {
clients: [{
Update tanka.md so that promtail.yml is in the correct format (#1671)
6 years ago
scheme:: 'http',
hostname:: 'gateway.%(namespace)s.svc' % $._config,
username:: 'loki',
password:: 'password',
container_root_path:: '/var/lib/docker',
Update tanka.md doc. (#1598) * Update tanka.md Fixed and tested see https://github.com/grafana/loki/issues/1591 * Update tanka.md * Update docs/installation/tanka.md Co-Authored-By: Robert Fratto <robert.fratto@grafana.com> Co-authored-by: Robert Fratto <robertfratto@gmail.com>
6 years ago
}],
Documentation Rewrite (#982) * docs: create structure of docs overhaul This commit removes all old docs and lays out the table of contents and framework for how the new documentation will be intended to be read. * docs: add design docs back in * docs: add community documentation * docs: add LogQL docs * docs: port existing operations documentation * docs: add new placeholder file for promtail configuration docs * docs: add TOC for operations/storage * docs: add Loki API documentation * docs: port troubleshooting document * docs: add docker-driver documentation * docs: link to configuration from main docker-driver document * docs: update API for new paths * docs: fix broken links in api.md and remove json marker from examples * docs: incorporate api changes from #1009 * docs: port promtail documentation * docs: add TOC to promtail configuration reference * docs: fix promtail spelling errors * docs: add loki configuration reference * docs: add TOC to configuration * docs: add loki configuration example * docs: add Loki overview with brief explanation about each component * docs: add comparisons document * docs: add info on table manager and update storage/README.md * docs: add getting started * docs: incorporate config yaml changes from #755 * docs: fix typo in releases url for promtail * docs: add installation instructions * docs: add more configuration examples * docs: add information on fluentd client fluent-bit has been temporarily removed until the PR for it is merged. * docs: PR review feedback * docs: add architecture document * docs: add missing information from old docs * `localy` typo Co-Authored-By: Ed Welch <ed@oqqer.com> * docs: s/ran/run/g * Typo * Typo * Tyop * Typo * docs: fixed typo * docs: PR feedback * docs: @cyriltovena PR feedback * docs: add more details to promtail url config option * docs: expand promtail's pipelines document with extra detail * docs: remove reference to Stage interface in pipelines.md * docs: fixed some spelling * docs: clarify promtail configuration and scraping * docs: attempt #2 at explaining promtail's usage of machine hostname * docs: spelling fixes * docs: add reference to promtail custom metrics and fix silly typo * docs: cognizant -> aware * docs: typo * docs: typos * docs: add which components expose which API endpoints in microservices mode * docs: change ksonnet installation to tanka * docs: address most @pracucci feedback * docs: fix all spelling errors so reviewers don't have to keep finding them :) * docs: incorporate changes to API endpoints made in #1022 * docs: add missing loki metrics * docs: add missing promtail metrics * docs: @pstribrany feedback * docs: more @pracucci feedback * docs: move metrics into a table * docs: update push path references to /loki/api/v1/push * docs: add detail to further explain limitations of monolithic mode * docs: add alternative names to modes_of_operation diagram * docs: add log ordering requirement * docs: add procedure for updating docs with latest version * docs: separate out stages documentation into one document per stage * docs: list supported stores in storage documentation * docs: add info on duplicate log lines in pipelines * docs: add line_format as key feature to fluentd * docs: hopefully final commit :)
6 years ago
},
replication_factor: 3,
consul_replicas: 1,
},
}
```
fix(ksonnet): don't depend on specific k8s version (#2525) * fix(ksonnet): don't depend on specific k8s version * docs(tanka): remove k8s-alpha refs * fix(ksonnet): backwards compatibility with ksonnet
5 years ago
Run `tk show environments/loki` to see the manifests that will be deployed to
the cluster. Run `tk apply environments/loki` to deploy the manifests.