Docs: Update query and resource caching documentation to improve clarity and add additional context (#70556)

* update caching docs

* fix validation issue

* fix missing anchors

* fix up compatible data sources section

* Update docs/sources/administration/data-source-management/index.md

Co-authored-by: Stephanie Hingtgen <stephanie.hingtgen@grafana.com>

* change one word

* change one more word

* add a little more meat to the benefits section

* remove unnecessary alias

* Update docs/sources/administration/data-source-management/index.md

Co-authored-by: Christopher Moyer <35463610+chri2547@users.noreply.github.com>

* Update docs/sources/administration/data-source-management/index.md

Co-authored-by: Christopher Moyer <35463610+chri2547@users.noreply.github.com>

* Update docs/sources/administration/data-source-management/index.md

Co-authored-by: Christopher Moyer <35463610+chri2547@users.noreply.github.com>

* Update docs/sources/administration/data-source-management/index.md

Co-authored-by: Christopher Moyer <35463610+chri2547@users.noreply.github.com>

* Update docs/sources/administration/data-source-management/index.md

Co-authored-by: Christopher Moyer <35463610+chri2547@users.noreply.github.com>

* add link to section

* change word

* add note on benefits of resource caching

---------

Co-authored-by: Stephanie Hingtgen <stephanie.hingtgen@grafana.com>
Co-authored-by: Christopher Moyer <35463610+chri2547@users.noreply.github.com>
pull/69656/head
Michael Mandrus 2 years ago committed by GitHub
parent d64b6264ff
commit 6c25342ecb
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
  1. 38
      docs/sources/administration/data-source-management/index.md
  2. 2
      docs/sources/dashboards/dashboard-public/index.md
  3. 2
      docs/sources/introduction/grafana-enterprise.md
  4. 2
      docs/sources/setup-grafana/configure-grafana/_index.md
  5. 2
      docs/sources/whatsnew/whats-new-in-v7-5.md
  6. 2
      docs/sources/whatsnew/whats-new-in-v8-0.md
  7. 2
      docs/sources/whatsnew/whats-new-in-v8-1.md

@ -81,29 +81,33 @@ You can assign data source permissions to users, teams, and roles which will all
<div class="clearfix"></div> <div class="clearfix"></div>
## Query caching ## Query and resource caching
When query caching is enabled, Grafana temporarily stores the results of data source queries. When you or another user submit the exact same query again, the results will come back from the cache instead of from the data source (like Splunk or ServiceNow) itself. When you enable query and resource caching, Grafana temporarily stores the results of data source queries and resource requests. When you or another user submit the same query or resource request again, the results will come back from the cache instead of from the data source.
Query caching works for all backend data sources. You can enable the cache globally and configure the cache duration (also called Time to Live, or TTL). When using Grafana, a query pertains to a request for data frames to be modified or displayed. A resource relates to any HTTP requests made by a plugin, such as the Amazon Timestream plugin requesting a list of available databases from AWS. For more information on data source queries and resources, please see the developers page on [backend plugins]({{< relref "../../developers/plugins/backend/" >}}).
The caching feature works for **all** backend data sources. You can enable the cache globally in Grafana's [configuration]({{< relref "../../setup-grafana/configure-grafana/enterprise-configuration/#caching" >}}), and configure a cache duration (also called Time to Live, or TTL) for each data source individually.
{{% admonition type="note" %}} {{% admonition type="note" %}}
Available in [Grafana Enterprise]({{< relref "../../introduction/grafana-enterprise/" >}}) and [Grafana Cloud Pro and Advanced](/docs/grafana-cloud/). Available in [Grafana Enterprise]({{< relref "../../introduction/grafana-enterprise/" >}}) and [Grafana Cloud Pro and Advanced](/docs/grafana-cloud/).
{{% /admonition %}} {{% /admonition %}}
The following cache backends are available: in-memory, Redis, and Memcached. The following cache backend options are available: in-memory, Redis, and Memcached.
{{% admonition type="note" %}} {{% admonition type="note" %}}
Storing cached queries in-memory can increase Grafana's memory footprint. In production environments, a Redis or Memcached backend is highly recommended. Storing cached queries in-memory can increase Grafana's memory footprint. In production environments, a Redis or Memcached backend is highly recommended.
{{% /admonition %}} {{% /admonition %}}
When a panel queries a cached data source, the time until this query fetches fresh data is determined by the panel's **interval.** This means that wider panels and dashboards with shorter time ranges fetch new data more frequently than narrower panels and dashboards with longer time ranges. When a panel queries a data source with cached data, it will either fetch fresh data or use cached data depending on the panel's **interval.** The interval is used to round the query time range to a nearby cached time range, increasing the likelihood of cache hits. Therefore, wider panels and dashboards with shorter time ranges fetch new data more often than narrower panels and dashboards with longer time ranges.
A panel's interval is visible in the [query options]({{< relref "../../panels-visualizations/query-transform-data/" >}}). It is calculated as follows: `time range / max data points`. Max data points are calculated based on the width of the panel. For example, a wide panel with `1000 data points` on a dashboard with a time range of `last 7 days` will retrieve fresh data every 10 minutes: `7d / 1000 = 10m`. In this example, cached data for this panel will be served for up to 10 minutes before Grafana needs to query the data source again for new data.
Interval is visible in a panel's [query options]({{< relref "../../panels-visualizations/query-transform-data/" >}}). It is calculated like this: `(max data points) / time range`. Max data points are calculated based on the width of the panel. For example, a full-width panel on a dashboard with a time range of `last 7 days` will retrieve fresh data every 10 minutes. In this example, cached data for this panel will be served for up to 10 minutes before Grafana queries the data source again and returns new data. You can configure a panel to retrieve data more often by increasing the **Max data points** setting in the panel's [query options]({{< relref "../../panels-visualizations/query-transform-data/" >}}).
You can make a panel retrieve fresh data more frequently by increasing the **Max data points** setting in the panel's [query options]({{< relref "../../panels-visualizations/query-transform-data/" >}}). ### Caching benefits
### Query caching benefits By reducing the number of queries and requests sent to data sources, caching can provide the following benefits:
- Faster dashboard load times, especially for popular dashboards. - Faster dashboard load times, especially for popular dashboards.
- Reduced API costs. - Reduced API costs.
@ -111,21 +115,13 @@ You can make a panel retrieve fresh data more frequently by increasing the **Max
### Data sources that work with query caching ### Data sources that work with query caching
Query caching works for all [Enterprise data sources](/grafana/plugins/?type=datasource&enterprise=1) as well as the following [built-in data sources]({{< relref "../../datasources/" >}}): Query caching works for Grafana's [built-in data sources]({{< relref "../../datasources/#built-in-core-data-sources" >}}), and [backend data source plugins](https://grafana.com/grafana/plugins/?type=datasource) that extend the `DataSourceWithBackend` class in the plugins SDK.
- CloudWatch Metrics
- Google Cloud Monitoring
- InfluxDB
- Microsoft SQL Server
- MySQL
- Postgres
- Tempo
Some data sources, such as Elasticsearch, Prometheus, and Loki, cache queries themselves, so Grafana query caching does not improve performance. To verify that a data source works with query caching, follow the [instructions below](#enable-and-configure-query-caching) to **Enable and Configure query caching**. If caching is enabled in Grafana but the Caching tab is not visible for the given data source, then query caching is not available for that data source.
Query caching also works for all data sources that include a backend. More specifically, caching works with data sources that extend the `DataSourceWithBackend` class in the plugins SDK. {{% admonition type="note" %}}
Some data sources, such as Elasticsearch, Prometheus, and Loki, cache queries themselves, so Grafana _query_ caching does not significantly improve performance. However, _resource_ caching may help. See the developers page on [plugin resources]({{< relref "../../developers/plugins/backend/#resources" >}}) for details.
To tell if a data source works with query caching, follow the instructions below to **Enable and Configure query caching**. If caching is enabled in Grafana but the Caching tab is not visible for the given data source, then query caching is not available for that data source. {{% /admonition %}}
### Enable and configure query caching ### Enable and configure query caching

@ -17,7 +17,7 @@ This feature is in [public preview](/docs/release-life-cycle/).
{{% admonition type="caution" %}} {{% admonition type="caution" %}}
Making your dashboard public could result in a large number of queries to the data sources used by your dashboard. Making your dashboard public could result in a large number of queries to the data sources used by your dashboard.
This can be mitigated by utilizing the enterprise [caching]({{< relref "../../administration/data-source-management/#query-caching" >}}) and/or rate limiting features. This can be mitigated by utilizing the enterprise [caching]({{< relref "../../administration/data-source-management/#query-and-resource-caching" >}}) and/or rate limiting features.
{{% /admonition %}} {{% /admonition %}}

@ -54,7 +54,7 @@ Grafana Enterprise adds the following features:
- [Role-based access control]({{< relref "../administration/roles-and-permissions/access-control" >}}) to control access with role-based permissions. - [Role-based access control]({{< relref "../administration/roles-and-permissions/access-control" >}}) to control access with role-based permissions.
- [Data source permissions]({{< relref "../administration/data-source-management#data-source-permissions" >}}) to restrict query access to specific teams and users. - [Data source permissions]({{< relref "../administration/data-source-management#data-source-permissions" >}}) to restrict query access to specific teams and users.
- [Data source query caching]({{< relref "../administration/data-source-management#query-caching" >}}) to temporarily store query results in Grafana to reduce data source load and rate limiting. - [Data source query and resource caching]({{< relref "../administration/data-source-management#query-and-resource-caching" >}}) to temporarily store query results in Grafana to reduce data source load and rate limiting.
- [Reporting]({{< relref "../dashboards/create-reports" >}}) to generate a PDF report from any dashboard and set up a schedule to have it emailed to whomever you choose. - [Reporting]({{< relref "../dashboards/create-reports" >}}) to generate a PDF report from any dashboard and set up a schedule to have it emailed to whomever you choose.
- [Export dashboard as PDF]({{< relref "../dashboards/share-dashboards-panels#export-dashboard-as-pdf" >}}) - [Export dashboard as PDF]({{< relref "../dashboards/share-dashboards-panels#export-dashboard-as-pdf" >}})
- [Custom branding]({{< relref "../setup-grafana/configure-grafana/configure-custom-branding" >}}) to customize Grafana from the brand and logo to the footer links. - [Custom branding]({{< relref "../setup-grafana/configure-grafana/configure-custom-branding" >}}) to customize Grafana from the brand and logo to the footer links.

@ -424,7 +424,7 @@ Set to `true` to add metrics and tracing for database queries. The default value
## [remote_cache] ## [remote_cache]
Caches authentication details and session information in the configured database, Redis or Memcached. This setting does not configure [Query Caching in Grafana Enterprise]({{< relref "../../administration/data-source-management#query-caching" >}}). Caches authentication details and session information in the configured database, Redis or Memcached. This setting does not configure [Query Caching in Grafana Enterprise]({{< relref "../../administration/data-source-management#query-and-resource-caching" >}}).
### type ### type

@ -122,7 +122,7 @@ Query caching advantages:
Caching currently works for all backend data sources. You can enable the cache globally or per data source, and you can configure the cache duration per data source. The cache is currently in-memory. Caching currently works for all backend data sources. You can enable the cache globally or per data source, and you can configure the cache duration per data source. The cache is currently in-memory.
For more information, refer to [Query caching]({{< relref "../administration/data-source-management#query-caching" >}}). For more information, refer to [Query caching]({{< relref "../administration/data-source-management#query-and-resource-caching" >}}).
### Use template variable in reports ### Use template variable in reports

@ -303,7 +303,7 @@ This results in faster average load times for dashboards and fewer duplicate que
You can enable caching per data source, and time-to-live (TTL) can be configured globally and per data source. Query caching can be set up with Redis, Memcached, or a simple in-memory cache. You can enable caching per data source, and time-to-live (TTL) can be configured globally and per data source. Query caching can be set up with Redis, Memcached, or a simple in-memory cache.
For more information, refer to the [Data source query caching docs]({{< relref "../administration/data-source-management#query-caching" >}}). For more information, refer to the [Data source query caching docs]({{< relref "../administration/data-source-management#query-and-resource-caching" >}}).
### Reporting updates ### Reporting updates

@ -153,7 +153,7 @@ We’ve enhanced the scheduler for Reports to be more flexible, so you can send
### Encrypt data in the query cache ### Encrypt data in the query cache
Query caching was released in Grafana 8.0 and allows you to temporarily store the results of data source queries in a cache, so that Grafana reads repeated queries from there instead of from the data source itself. This reduces load on data sources, improves dashboard load times, and can save money for data sources that charge per query. To learn more about query caching see its [overview]({{< relref "../administration/data-source-management#query-caching" >}}) page. To find out how to turn on encryption, refer to the [caching configuration]({{< relref "../setup-grafana/configure-grafana/enterprise-configuration#caching" >}}) documentation. Query caching was released in Grafana 8.0 and allows you to temporarily store the results of data source queries in a cache, so that Grafana reads repeated queries from there instead of from the data source itself. This reduces load on data sources, improves dashboard load times, and can save money for data sources that charge per query. To learn more about query caching see its [overview]({{< relref "../administration/data-source-management#query-and-resource-caching" >}}) page. To find out how to turn on encryption, refer to the [caching configuration]({{< relref "../setup-grafana/configure-grafana/enterprise-configuration#caching" >}}) documentation.
You can now encrypt the query data cached by Grafana. This improves the security of query data, especially when your cache (like Redis) is shared with other services. You can now encrypt the query data cached by Grafana. This improves the security of query data, especially when your cache (like Redis) is shared with other services.

Loading…
Cancel
Save