**What this PR does / why we need it**:
Went through and looked for places where Loki should be capitalized
because we are talking about the product or project (vs. in
configuration examples, etc. where lower case is acceptable or
required).
- Logs are often unstructured data, it can be very difficult to extract reliable data from some unstructured formats, often requiring the use of complicated regular expressions.
- Easy to abuse. Easy to create a Label with high cardinality, even possibly by accident with a rogue regular expression.
- Where do we extract metrics and labels at the client (Promtail or other?) or Loki? Extraction at the server (Loki) side has some pros/cons. Can we do both? At least with labels we could define a set of expected labels and if loki doesn’t receive them they could be extracted.
- Where do we extract metrics and labels at the client (Promtail or other?) or Loki? Extraction at the server (Loki) side has some pros/cons. Can we do both? At least with labels we could define a set of expected labels and if Loki doesn’t receive them they could be extracted.
- Server side extraction would improve interoperability at the expense of increase server workload and cost.
- Are there discoverability questions/concerns with metrics exposed via loki vs the agent? Maybe this is better/easier to manage?
- Are there discoverability questions/concerns with metrics exposed via Loki vs the agent? Maybe this is better/easier to manage?
- Potentially more difficult to manage configuration with the server side having to match configs to incoming log streams
@ -219,7 +219,7 @@ The query frontend splits larger queries into multiple smaller queries, executin
#### Metric Queries
The query frontend supports caching metric query results and reuses them on subsequent queries. If the cached results are incomplete, the query frontend calculates the required subqueries and executes them in parallel on downstream queriers. The query frontend can optionally align queries with their step parameter to improve the cacheability of the query results. The result cache is compatible with any loki caching backend (currently memcached, redis, and an in-memory cache).
The query frontend supports caching metric query results and reuses them on subsequent queries. If the cached results are incomplete, the query frontend calculates the required subqueries and executes them in parallel on downstream queriers. The query frontend can optionally align queries with their step parameter to improve the cacheability of the query results. The result cache is compatible with any Loki caching backend (currently memcached, redis, and an in-memory cache).
| `loki_log_messages_total` | Counter | DEPRECATED. Use internal_log_messages_total for the same functionality. Total number of log messages created by loki itself. |
| `loki_internal_log_messages_total` | Counter | Total number of log messages created by loki itself. |
| `loki_log_messages_total` | Counter | DEPRECATED. Use internal_log_messages_total for the same functionality. Total number of log messages created by Loki itself. |
| `loki_internal_log_messages_total` | Counter | Total number of log messages created by Loki itself. |
| `loki_request_duration_seconds` | Histogram | Number of received HTTP requests. |
The Loki Distributors expose the following metrics:
| Url | Url of loki server API endpoint. | http://localhost:3100/loki/api/v1/push |
| Url | Url of Loki server API endpoint. | http://localhost:3100/loki/api/v1/push |
| TenantID | The tenant ID used by default to push logs to Loki. If omitted or empty it assumes Loki is running in single-tenant mode and no `X-Scope-OrgID` header is sent. | "" |
| BatchWait | Time to wait before send a log batch to Loki, full or not. | 1s |
| BatchSize | Log batch size to send a log batch to Loki (unit: Bytes). | 10 KiB (10 * 1024 Bytes) |
| Timeout | Maximum time to wait for loki server to respond to a request. | 10s |
| Timeout | Maximum time to wait for Loki server to respond to a request. | 10s |
| MinBackoff | Initial backoff time between retries. | 500ms |
| MaxBackoff | Maximum backoff time between retries. | 5m |
| MaxRetries | Maximum number of retries when sending batches. Setting it to `0` will retry indefinitely. | 10 |
@ -1141,7 +1141,7 @@ The `group_id` defined the unique consumer group id to use for consuming logs. E
- If all promtail instances have the same consumer group, then the records will effectively be load balanced over the promtail instances.
- If all promtail instances have different consumer groups, then each record will be broadcast to all promtail instances.
The `group_id` is useful if you want to effectively send the data to multiple loki instances and/or other sinks.
The `group_id` is useful if you want to effectively send the data to multiple Loki instances and/or other sinks.
The `assignor` configuration allow you to select the rebalancing strategy to use for the consumer group.
Rebalancing is the process where a group of consumer instances (belonging to the same group) co-ordinate to own a mutually exclusive set of partitions of topics that the group is subscribed to.
@ -22,9 +22,9 @@ We recommend having a Grafana instance available to monitor both the existing an
**To Migrate from a "read & write" to a "backend, read & write" deployment**
1. Make sure your deployment is using a new enough version of loki
1. Make sure your deployment is using a new enough version of Loki
This feature landed as an option in the helm chart while still in the `main` branch of Loki. As a result, depending on when you run this migration, you may neeed to manually override the Loki or GEL image being used to one that has the third, `backend` target available. For loki, add the following to your `values.yaml`.
This feature landed as an option in the helm chart while still in the `main` branch of Loki. As a result, depending on when you run this migration, you may neeed to manually override the Loki or GEL image being used to one that has the third, `backend` target available. For Loki, add the following to your `values.yaml`.
J Stickler
2 years ago
committed by