@ -861,9 +863,9 @@ The `local_storage_config` configures a (local) filesystem as a general storage
[directory: <filename> | default = ""]
```
## frontend_worker_config
## frontend_worker
The `frontend_worker_config` configures the worker - running within the Loki querier - picking up and executing queries enqueued by the query-frontend.
The `frontend_worker` configures the worker - running within the Loki querier - picking up and executing queries enqueued by the query-frontend.
```yaml
# Address of query frontend service, in host:port format.
@ -887,9 +889,9 @@ The `frontend_worker_config` configures the worker - running within the Loki que
[scheduler_address: <string> | default = ""]
```
## ingester_client_config
## ingester_client
The `ingester_client_config` block configures how connections to ingesters
The `ingester_client` block configures how connections to ingesters
operate.
```yaml
@ -918,9 +920,9 @@ pool_config:
[grpc_client_config: <grpc_client_config>]
```
## ingester_config
## ingester
The `ingester_config` block configures the Loki Ingesters.
The `ingester` block configures the Loki Ingesters.
```yaml
# Configures how the lifecycle of the ingester will operate
@ -1842,9 +1844,9 @@ chunks:
[row_shards: <int> | default = 16]
```
## compactor_config
## compactor
The `compactor_config` block configures the compactor component. This component periodically
The `compactor` block configures the compactor component. This component periodically
compacts index shards to more performant forms.
```yaml
@ -2173,9 +2175,9 @@ backoff_config:
[max_retries: <int> | default = 10]
```
## table_manager_config
## table_manager
The `table_manager_config` block configures the Loki table-manager.
The `table_manager` block configures the Loki table-manager.
```yaml
# Master 'off-switch' for table capacity updates, e.g. when troubleshooting.
@ -2299,9 +2301,9 @@ The `auto_scaling_config` block configures autoscaling for DynamoDB.
[target: <float> | default = 80]
```
## tracing_config
## tracing
The `tracing_config` block configures tracing for Jaeger. Currently limited to disable auto-configuration per [environment variables](https://www.jaegertracing.io/docs/1.16/client-features/) only.
The `tracing` block configures tracing for Jaeger. Currently limited to disable auto-configuration per [environment variables](https://www.jaegertracing.io/docs/1.16/client-features/) only.
```yaml
# Whether or not tracing should be enabled.
@ -2309,14 +2311,14 @@ The `tracing_config` block configures tracing for Jaeger. Currently limited to d
[enabled: <boolean>: default = true]
```
## common_config
## common
The `common_config` block sets common definitions to be shared by different components.
This way, one doesn't have to replicate configs in multiple places.
The `common` block sets common definitions to be shared by different components.
This way, one doesn't have to replicate configuration in multiple places.
```yaml
# A common storage config to be used by the different Loki components.
[storage: <common_storage_config>]
# A common storage configuration to be used by the different Loki components.
[storage: <storage>]
# When defined, the given prefix will be present in front of the endpoint paths.
[path_prefix: <string>]
@ -2337,10 +2339,11 @@ This way, one doesn't have to replicate configs in multiple places.
[ring: <ring_config>]
```
### common_storage_config
### storage
The `common_storage_config` block defines a common storage to be reused by different components as a way to facilitate storage configuration.
If any specific configs for an object storage client have been provided elsewhere in the configuration file, the specific config will supersede the common storage config.
The common `storage` block defines a common storage to be reused by different
components as a way to facilitate storage configuration.
If any specific configuration for an object storage client have been provided elsewhere in the configuration file, the specific configuration will supersede the common storage configuration.
@ -13,7 +13,7 @@ It's a common occurrence to start running Grafana Loki as a single binary while
### Parallelization
One of the most important functions of the query frontend is the ability to split larger queries into smaller ones, execute them in parallel, and stitch the results back together. How often it splits them is determined by the `querier.split-queries-by-interval` flag or the yaml config `queryrange.split_queriers_by_interval`. With this set to `1h`, the frontend will dissect a day long query into 24 one hour queries, distribute them to the queriers, and collect the results. This is immensely helpful in production environments as it not only allows us to perform larger queries via aggregation, but also evens the work distribution across queriers so that one or two are not stuck with impossibly large queries while others are left idle.
One of the most important functions of the query frontend is the ability to split larger queries into smaller ones, execute them in parallel, and stitch the results back together. How often it splits them is determined by the `querier.split-queries-by-interval` flag or the yaml config `query_range.split_queriers_by_interval`. With this set to `1h`, the frontend will dissect a day long query into 24 one hour queries, distribute them to the queriers, and collect the results. This is immensely helpful in production environments as it not only allows us to perform larger queries via aggregation, but also evens the work distribution across queriers so that one or two are not stuck with impossibly large queries while others are left idle.
* [**Loki no longer requires logs to be sent in perfect chronological order.**](../../configuration/#accept-out-of-order-writes) Support for out of order logs is one of the most highly requested features for Loki. The strict ordering constraint has been removed.
* Scaling Loki is now easier with a hybrid deployment mode that falls between our single binary and our microservices. The [Simple scalable deployment](../../fundamentals/architecture/#modes-of-operation) scales Loki with new `read` and `write` targets. Where previously you would have needed Kubernetes and the microservices approach to start tapping into Loki’s potential, it’s now possible to do this in a simpler way.
* The new [`common_config` section](../../configuration/#common_config) results in a 70% smaller Loki configuration. Pair that with updated defaults and Loki comes out of the box with more appropriate defaults and limits. Check out the [example local configuration](https://github.com/grafana/loki/blob/main/cmd/loki/loki-local-config.yaml) as the new reference for running Loki.
* The new [`common` section](../../configuration/#common) results in a 70% smaller Loki configuration. Pair that with updated defaults and Loki comes out of the box with more appropriate defaults and limits. Check out the [example local configuration](https://github.com/grafana/loki/blob/main/cmd/loki/loki-local-config.yaml) as the new reference for running Loki.
* [**Recording rules**](../../rules/#recording-rules) are no longer an experimental feature. We've given them a more resilient implementation which leverages the existing write ahead log code in Prometheus.
* The new [**Promtail Kafka Consumer**](../../clients/promtail/scraping/#kafka) can easily get your logs out of Kafka and into Loki.
* There are **nice LogQL enhancements**, thanks to the amazing Loki community. LogQL now has [group_left and group_right](../../logql/#many-to-one-and-one-to-many-vector-matches). And, the `label_format` and `line_format` functions now support [working with dates and times](../../logql/template_functions/#now).
Karen Miller
5 years ago
committed by