docs: Updates to support public preview of Blooms in Cloud (#15108)

pull/13867/merge
J Stickler 1 year ago committed by GitHub
parent 0c388c30ad
commit 3da33cd4ef
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
  1. 13
      docs/sources/get-started/labels/structured-metadata.md
  2. 6
      docs/sources/operations/bloom-filters.md
  3. 7
      docs/sources/query/query_accceleration.md

@ -26,6 +26,19 @@ You should only use structured metadata in the following situations:
- If you are using [Explore Logs](https://grafana.com/docs/grafana-cloud/visualizations/simplified-exploration/logs/) to visualize and explore your Loki logs. You must set `discover_log_levels` and `allow_structured_metadata` to `true` in your Loki configuration.
- If you are a large-scale customer, who is ingesting more than 75TB of logs a month and are using [Bloom filters](https://grafana.com/docs/loki/<LOKI_VERSION>/operations/bloom-filters/) (Experimental), starting in [Loki 3.3](https://grafana.com/docs/loki/<LOKI_VERSION>/release-notes/v3-3/) Bloom filters now utilize structured metadata.
## Enable or disable structured metadata
You enable structured metadata in the Loki config.yaml file.
```yaml
limits_config:
allow_structured_metadata: true
volume_enabled: true
retention_period: 672h # 28 days retention
```
You can disable Structured Metadata by setting `allow_structured_metadata: false` in the `limits_config` section or set the command line argument `-validation.allow-structured-metadata=false`. Note structured metadata is required to support ingesting OTLP data.
## Attaching structured metadata to log lines
You have the option to attach structured metadata to log lines in the push payload along with each log line and the timestamp.

@ -13,9 +13,9 @@ aliases:
# Bloom filters (Experimental)
{{< admonition type="warning" >}}
This feature is an [experimental feature](/docs/release-life-cycle/). Engineering and on-call support is not available. No SLA is provided.
Note that this feature is intended for users who are ingesting more than 75TB of logs a month, as it is designed to accelerate queries against large volumes of logs.
Query acceleration via Bloom filters is enabled for select Grafana Cloud customers ingesting more that 75TB of logs a month.
In Loki and Grafana Enterprise Logs (GEL), Query acceleration using blooms is an [experimental feature](/docs/release-life-cycle/). Engineering and on-call support is not available. No SLA is provided. Note that this feature is intended for users who are ingesting more than 75TB of logs a month, as it is designed to accelerate queries against large volumes of logs.
In Grafana Cloud, Query acceleration using Bloom filters is enabled as a [public preview](/docs/release-life-cycle/) for select large-scale customers that are ingesting more that 75TB of logs a month. Limited support and no SLA are provided.
{{< /admonition >}}
Loki leverages [bloom filters](https://en.wikipedia.org/wiki/Bloom_filter) to speed up queries by reducing the amount of data Loki needs to load from the store and iterate through.

@ -12,15 +12,16 @@ keywords:
{{< admonition type="warning" >}}
In Loki and Grafana Enterprise Logs (GEL), Query acceleration using blooms is an [experimental feature](/docs/release-life-cycle/). Engineering and on-call support is not available. No SLA is provided.
In Grafana Cloud, Query acceleration using blooms is enabled for large-scale customers that send more than 75TB of logs a month as a [public preview](/docs/release-life-cycle/) with limited support and no SLA.
In Grafana Cloud, Query acceleration using Bloom filters is enabled as a [public preview](/docs/release-life-cycle/) for select large-scale customers that are ingesting more that 75TB of logs a month. Limited support and no SLA are provided.
{{< /admonition >}}
If [bloom filters][] are enabled, you can write LogQL queries using [structured metadata][] to benefit from query acceleration.
## Prerequisites
* [Bloom filters][bloom filters] must be enabled.
* Logs must be sending [structured metadata][].
* [Bloom filters][bloom filters] must be enabled. OpenSource and Enterprise customers must [enable bloom filters](https://grafana.com/docs/loki/<LOKI_VERSION>/operations/bloom-filters/#enable-bloom-filters). Grafana Support enables Bloom Filters for Cloud Customers who are part of the public preview.
* Logs must be sending [structured metadata][]. OpenSource and Enterprise customers must [enable structured metadata](https://grafana.com/docs/loki/<LOKI_VERSION>/get-started/labels/structured-metadata/#enable-or-disable-structured-metadata). Structured metadata is enabled by default in Grafana Cloud.
### Query blooms

Loading…
Cancel
Save