[Docs] Create top level Manage section (#10461)

**What this PR does / why we need it**:

Part of the Epic to create a consistent documentation information
architecture across the three DB projects.

**Which issue(s) this PR fixes**:
Fixes #8718 

**Special notes for your reviewer**:
The original Issue called for moving files under Manage, but rather than
rename the folder and break links, I chose to make the "move" by
updating the topic metadata.
This PR also includes updates to topic metadata and removes or updates
page weights.
pull/10490/head
J Stickler 2 years ago committed by GitHub
parent 965f19a3b9
commit 1f408305c4
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
  1. 13
      docs/sources/operations/_index.md
  2. 16
      docs/sources/operations/authentication.md
  3. 2
      docs/sources/operations/automatic-stream-sharding.md
  4. 2
      docs/sources/operations/autoscaling_queriers.md
  5. 3
      docs/sources/operations/blocking-queries.md
  6. 2
      docs/sources/operations/caching.md
  7. 15
      docs/sources/operations/loki-canary/_index.md
  8. 3
      docs/sources/operations/multi-tenancy.md
  9. 3
      docs/sources/operations/observability.md
  10. 7
      docs/sources/operations/overrides-exporter.md
  11. 2
      docs/sources/operations/query-fairness/_index.md
  12. 2
      docs/sources/operations/recording-rules.md
  13. 9
      docs/sources/operations/request-validation-rate-limits.md
  14. 9
      docs/sources/operations/scalability.md
  15. 2
      docs/sources/operations/shuffle-sharding/_index.md
  16. 19
      docs/sources/operations/storage/_index.md
  17. 18
      docs/sources/operations/storage/boltdb-shipper.md
  18. 8
      docs/sources/operations/storage/filesystem.md
  19. 6
      docs/sources/operations/storage/logs-deletion.md
  20. 8
      docs/sources/operations/storage/retention.md
  21. 4
      docs/sources/operations/storage/schema/_index.md
  22. 8
      docs/sources/operations/storage/table-manager/_index.md
  23. 10
      docs/sources/operations/storage/tsdb.md
  24. 4
      docs/sources/operations/storage/wal.md
  25. 7
      docs/sources/operations/troubleshooting.md
  26. 3
      docs/sources/operations/zone-ingesters.md
  27. 6
      docs/sources/setup/install/_index.md
  28. 2
      docs/sources/storage/_index.md

@ -1,11 +1,14 @@
---
title: Operations
description: Operations
weight: 900
title: Manage Loki
menuTitle: Manage
description: How to manage your Loki implementation.
weight: 800
---
# Operations
# Manage Loki
This section covers operational topics in Loki:
This section describes the decisions Loki operators and users make and the actions they perform to deploy, configure, and maintain Loki.
This section includes the following topics for managing and tuning Loki:
{{< section >}}

@ -1,23 +1,27 @@
---
title: Authentication
description: Authentication
weight: 10
menuTitle:
description: Describes Loki's authentication.
weight:
---
# Authentication
Grafana Loki does not come with any included authentication layer. Operators are
expected to run an authenticating reverse proxy in front of your services. A helpful list of open-source reverse proxies to use:
expected to run an authenticating reverse proxy in front of your services.
The scalable monolithic [deployment mode]({{< relref "../get-started/deployment-modes" >}}) requires a reverse proxy to be deployed in front of Loki, to direct client API requests to either the read or write nodes. The Loki Helm chart includes a default reverse proxy configuration, using Nginx.
A list of open-source reverse proxies you can use:
- [Pomerium](https://www.pomerium.com/docs), which has a [guide for securing Grafana](https://www.pomerium.com/docs/guides/grafana)
- [NGINX](https://docs.nginx.com/nginx/) using their [guide on restricting access with HTTP basic authentication](https://docs.nginx.com/nginx/admin-guide/security-controls/configuring-http-basic-authentication/)
- [OAuth2 proxy](https://github.com/oauth2-proxy/oauth2-proxy)
- [HAProxy](https://www.haproxy.org/)
Note that when using Loki in multi-tenant mode, Loki requires the HTTP header
`X-Scope-OrgID` to be set to a string identifying the tenant; the responsibility
of populating this value should be handled by the authenticating reverse proxy.
Read the [multi-tenancy]({{< relref "./multi-tenancy" >}}) documentation for more information.
For more inforamtion, read the [multi-tenancy]({{< relref "./multi-tenancy" >}}) documentation.
For information on authenticating Promtail, please see the docs for [how to
For information on authenticating Promtail, see the documentation for [how to
configure Promtail]({{< relref "../send-data/promtail/configuration" >}}).

@ -2,7 +2,7 @@
title: Automatic stream sharding
menuTitle: Automatic stream sharding
description: Automatic stream sharding can control issues around the per-stream rate limit
weight: 110
weight:
---
# Automatic stream sharding

@ -2,7 +2,7 @@
title: Autoscaling Loki queriers
menuTitle: Autoscaling queriers
description: Kubernetes deployments of a microservices mode Loki cluster can use KEDA to autoscale the quantity of queriers.
weight: 30
weight:
---
# Autoscaling Loki queriers

@ -1,7 +1,8 @@
---
title: Blocking Queries
menuTitle:
description: Blocking Queries
weight: 60
weight:
---
# Blocking Queries

@ -2,7 +2,7 @@
title: Caching
menuTitle: Caching
description: Enable and configure memcached for caching.
weight: 100
weight:
keywords:
- memcached
- caching

@ -1,22 +1,23 @@
---
title: Loki Canary
description: Loki Canary is a standalone app that audits the log-capturing performance of a Grafana Loki cluster.
weight: 60
menuTitle:
description: Loki Canary audits the log-capturing performance of a Grafana Loki cluster to ensure Loki is ingesting logs without data loss.
weight:
---
# Loki Canary
Loki Canary is a standalone app that audits the log-capturing performance of
a Grafana Loki cluster.
Loki Canary is a standalone app that audits the log-capturing performance of a Grafana Loki cluster.
This component emits and periodically queries for logs, making sure that Loki is ingesting logs without any data loss.
When something is wrong with Loki, the Canary often provides the first indication.
Loki Canary generates artificial log lines.
These log lines are sent to the Loki cluster.
Loki Canary communicates with the Loki cluster to capture metrics about the
artificial log lines,
such that Loki Canary forms information about the performance of the
Loki cluster.
such that Loki Canary forms information about the performance of the Loki cluster.
The information is available as Prometheus time series metrics.
![block_diagram](./loki-canary-block.png)
{{< figure max-width="75%" src="./loki-canary-block.png">}}
Loki Canary writes a log to a file and stores the timestamp in an internal
array. The contents look something like this:

@ -1,7 +1,8 @@
---
title: Multi-tenancy
menuTitle:
description: Multi-tenancy
weight: 50
weight:
---
# Multi-tenancy

@ -1,7 +1,8 @@
---
title: Observability
menuTitle:
description: Observing Grafana Loki
weight: 20
weight:
---
# Observability

@ -1,7 +1,8 @@
---
title: "Overrides exporter"
description: "Overrides Exporter"
weight: 20
title: Overrides exporter
menuTitle:
description: Overrides Exporter
weight:
---
# Overrides exporter

@ -2,7 +2,7 @@
title: Query fairness within tenants
menuTitle: Query fairness
description: The scheduler can guarantee query fairness across multiple actors within a single tenant.
weight: 101
weight:
---
# Query fairness within tenants

@ -1,6 +1,8 @@
---
title: Recording Rules
menuTitle:
description: Working with recording rules.
weight:
---
# Recording Rules

@ -1,10 +1,11 @@
---
title: Request Validation & Rate-Limit Errors
description: Request Validation & Rate-Limit Errors
weight: 30
title: Request Validation and Rate-Limit Errors
menuTitle:
description: Request Validation and Rate-Limit Errors
weight:
---
# Request Validation & Rate-Limit Errors
# Request Validation and Rate-Limit Errors
Loki will reject requests if they exceed a usage threshold (rate-limit error) or if they are invalid (validation error).

@ -1,9 +1,10 @@
---
title: Scalability
description: Scaling with Grafana Loki
weight: 30
title: Scale Loki
menuTitle: Scale
description: Describes how to scale Grafana Loki
weight:
---
# Scalability
# Scale Loki
When scaling Loki, operators should consider running several Loki processes
partitioned by role (ingester, distributor, querier) rather than a single Loki

@ -2,7 +2,7 @@
title: Shuffle sharding
menuTitle: Shuffle sharding
description: Shuffle sharding can isolate a tenant workload from other tenant workloads, providing a better sharing of resources.
weight: 100
weight:
---
# Shuffle sharding

@ -1,11 +1,12 @@
---
title: Grafana Loki Storage
description: Grafana Loki Storage
weight: 40
title: Manage storage
menuTitle: Storage
description: Describes Loki's storage needs and supported stores.
weight:
---
# Grafana Loki Storage
# Manage storage
[High level storage overview here]({{< relref "../../storage/_index.md" >}})
You can read a high level overview of Loki storage [here]({{< relref "../../storage/_index.md" >}})
Grafana Loki needs to store two different types of data: **chunks** and **indexes**.
@ -22,9 +23,9 @@ how to configure the storage and the index.
For more information:
1. [Table Manager]({{< relref "./table-manager" >}})
1. [Retention]({{< relref "./retention" >}})
1. [Logs Deletion]({{< relref "./logs-deletion" >}})
- [Table Manager]({{< relref "./table-manager" >}})
- [Retention]({{< relref "./retention" >}})
- [Logs Deletion]({{< relref "./logs-deletion" >}})
## Supported Stores
@ -133,7 +134,7 @@ Resources: `arn:aws:iam::<aws_account_id>:role/<role_name>`
When using IBM Cloud Object Storage (COS) as object storage, IAM `Writer` role is needed.
See the [IBM Cloud Object Storage section]({{< relref "../../storage#ibm-cloud-object-storage" >}}) on the storage page for a detailed setup guide.
See the [IBM Cloud Object Storage section]({{< relref "../../storage#ibm-deployment-cos-single-store" >}}) on the storage page for a detailed setup guide.
## Chunk Format

@ -1,18 +1,24 @@
---
title: Single Store (boltdb-shipper)
description: Single Store (boltdb-shipper)
title: Single Store BoltDB (boltdb-shipper)
menuTitle: BoltDB Shipper
description: Describes the deprecated boltdb-shipper single store usage.
weight: 200
---
# Single Store (boltdb-shipper)
# Single Store BoltDB (boltdb-shipper)
Note that single store (boltdb-shipper) is a legacy storage option and is not recommended for new deployments. It is recommended to use the [TSDB]({{< relref "./tsdb" >}}) index instead.
{{% admonition type="note" %}}
Note that single store BoltDB Shipper is a legacy storage option and is not recommended for new deployments. The [TSDB]({{< relref "./tsdb" >}}) index is the recommended index.
{{% /admonition %}}
BoltDB Shipper lets you run Grafana Loki without any dependency on NoSQL stores for storing index.
It locally stores the index in BoltDB files instead and keeps shipping those files to a shared object store i.e the same object store which is being used for storing chunks.
It also keeps syncing BoltDB files from shared object store to a configured local directory for getting index entries created by other services of same Loki cluster.
This helps run Loki with one less dependency and also saves costs in storage since object stores are likely to be much cheaper compared to cost of a hosted NoSQL store or running a self hosted instance of Cassandra.
**Note:** BoltDB shipper works best with 24h periodic index files. It is a requirement to have index period set to 24h for either active or upcoming usage of boltdb-shipper.
If boltdb-shipper already has created index files with 7 days period, and you want to retain previous data then just add a new schema config using boltdb-shipper with a future date and index files period set to 24h.
{{% admonition type="note" %}}
BoltDB shipper works best with 24h periodic index files. It is a requirement to have the index period set to 24h for either active or upcoming usage of boltdb-shipper.
If boltdb-shipper already has created index files with 7 days period, and you want to retain previous data, add a new schema config using boltdb-shipper with a future date and index files period set to 24h.
{{% /admonition %}}
## Example Configuration

@ -1,8 +1,10 @@
---
title: Filesystem
description: Filesystem Object Store
title: Filesystem object store
menuTitle: Filesystem object store
description: Describes the filesystem object store
weight: 300
---
# Filesystem
# Filesystem object store
The filesystem object store is the easiest to get started with Grafana Loki but there are some pros/cons to this approach.

@ -1,8 +1,8 @@
---
title: Log entry deletion
menuTitle: "Log entry deletion"
description: "Log entries from a specified stream may be deleted."
weight: 60
menuTitle: Log entry deletion
description: Log entries from a specified stream may be deleted.
weight: 700
---
# Log entry deletion

@ -1,8 +1,10 @@
---
title: Retention
description: Grafana Loki Storage Retention
title: Log retention
menuTitle: Log retention
description: Describes Grafana Loki Storage Retention
weight: 600
---
# Retention
# Log retention
Retention in Grafana Loki is achieved either through the [Table Manager](#table-manager) or the [Compactor](#compactor).

@ -1,6 +1,8 @@
---
title: Storage schema
description: Storage schema
menuTitle:
description: Describes the Loki storage schema
weight: 400
---
# Storage schema

@ -1,9 +1,15 @@
---
title: Table manager
description: Table manager
menuTitle:
description: Describes the Loki table manager component.
weight:
---
# Table manager
{{% admonition type="note" %}}
Table manager is only needed if you are using a multi-store [backend]({{< relref "../../storage" >}}). If you are using either TSDB,(recommended) or BoltDB you do not need the Table Manager.
{{% /admonition %}}
Grafana Loki supports storing indexes and chunks in table-based data storages. When
such a storage type is used, multiple tables are created over the time: each
table - also called periodic table - contains the data for a specific time

@ -1,10 +1,10 @@
---
title: TSDB
description: TSDB index
weight: 1000
title: Single Store TSDB (tsdb)
menuTitle: TSDB
description: Describes the tsdb single store usage.
weight: 100
---
# TSDB
# Single Store TSDB (tsdb)
Starting with Loki v2.8, TSDB is the recommended Loki index. It is heavily inspired by the Prometheus's TSDB [sub-project](https://github.com/prometheus/prometheus/tree/main/tsdb). For a deeper explanation you can read Loki maintainer Owen's [blog post](https://lokidex.com/posts/tsdb/). The short version is that this new index is more efficient, faster, and more scalable. It also resides in object storage like the [boltdb-shipper]({{< relref "./boltdb-shipper" >}}) index which preceded it.

@ -1,6 +1,8 @@
---
title: Write Ahead Log
description: Write Ahead Log
menuTitle:
description: Describes the Loki Write Ahead Log (WAL).
weight: 500
---
# Write Ahead Log

@ -1,11 +1,12 @@
---
title: Troubleshooting
title: Troubleshooting Loki
menuTitle: Troubleshooting
description: Troubleshooting Grafana Loki
weight: 80
weight:
aliases:
- /docs/loki/latest/getting-started/troubleshooting/
---
# Troubleshooting
# Troubleshooting Loki
## "Loki: Bad Gateway. 502"

@ -1,7 +1,8 @@
---
title: Zone aware ingesters
menuTitle:
description: How to migrate from a single ingester statefulset to three zone aware ingester StatefulSets
weight: 10
weight:
---
# Zone aware ingesters

@ -1,13 +1,13 @@
---
title: Installation
title: Install Loki
menuTitle: Install
description: Describes how to install Loki.
description: Overview of methods for installing Loki.
aliases:
- ../installation/
weight: 200
---
# Installation
# Install Loki
There are several methods of installing Loki and Promtail:

@ -20,7 +20,7 @@ More detailed information can be found on the [operations page]({{< relref "../o
Loki 2.8 adds TSDB as a new mode for the Single Store and is now the recommended way to persist data in Loki.
Some more storage details can also be found in the [operations section]({{< relref "../operations/storage/_index.md" >}}).
You can find more storage information in the [manage section]({{< relref "../operations/storage/_index.md" >}}).
## Single Store

Loading…
Cancel
Save