Docs: Add a query variable topic (#25852)

* Update global-variables.md

* Create _index.md

* Create add-query-variable.md

* added variable task files

* Update templates-and-variables.md

* added content

* content edits

* Update formatting-multi-value-variables.md

* added content and updated menu

* Update add-query-variable.md

* applied edits

* Update docs/sources/variables/formatting-multi-value-variables.md

Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>

* Update docs/sources/variables/formatting-multi-value-variables.md

Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>

* Update docs/sources/variables/formatting-multi-value-variables.md

Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>

* applied edits

* Update docs/sources/variables/filter-variables-with-regex.md

Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>

* Update docs/sources/variables/formatting-multi-value-variables.md

Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>

* Update docs/sources/variables/formatting-multi-value-variables.md

Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>

* Update formatting-multi-value-variables.md

Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>

docs/sources/menu.yaml

@@ -249,12 +249,18 @@
   children:
     - link: /variables/templates-and-variables/
       name: Overview
+    - link: /variables/add-query-variable/
+      name: Add a query variable
     - link: /variables/global-variables/
       name: Global variables
     - link: /variables/url-variables/
       name: URL variables
     - link: /variables/advanced-variable-format-options/
       name: Advanced variable formats
+    - link: /variables/formatting-multi-value-variables/
+      name: Formatting multi-value variables
+    - link: /variables/filter-variables-with-regex/
+      name: Filter variables with regex
     - link: /variables/repeat-panels-or-rows/
       name: Repeat panels or rows
 - name: What's new in Grafana

docs/sources/variables/add-interval-variable.md

@@ -0,0 +1,22 @@
++++
+title = "Add an interval variable"
+type = "docs"
+[menu.docs]
+weight = 500
+draft = "true"
++++
+
+# Add an interval variable
+
+> Note: This is a draft topic, not included in the live build.
+
+
+Use the `Interval` type to create a variable that represents a time span (eg. `1m`,`1h`, `1d`). There is also a special `auto` option that will change depending on the current time range. You can specify how many times the current time range should be divided to calculate the current `auto` timespan.
+
+This variable type is useful as a parameter to group by time (for InfluxDB), Date histogram interval (for Elasticsearch) or as a *summarize* function parameter (for Graphite).
+
+Example using the template variable `myinterval` of type `Interval` in a graphite function:
+
+```
+summarize($myinterval, sum, false)
+```

docs/sources/variables/add-query-variable.md

@@ -0,0 +1,94 @@
++++
+title = "Add a query variable"
+type = "docs"
+[menu.docs]
+weight = 500
++++
+
+# Add a query variable
+
+Query variables allow you to write a data source query that can return a list of metric names, tag values, or keys. For example, a query variable might return a list of server names, sensor IDs, or data centers. The variable values change as they dynamically fetch options with a data source query.
+
+Query expressions can contain references to other variables and in effect create linked variables. Grafana detects this and automatically refreshes a variable when one of its linked variables change.
+
+## Query expressions
+
+Query expressions are different for each data source. For more information, refer to the documentation for your [data source]({{< relref "../features/datasources/_index.md" >}}).
+
+## Enter General options
+
+1. Navigate to the dashboard you want to make a variable for and then click the **Dashboard settings** (gear) icon at the top of the page.
+1. On the Variables tab, click **New**.
+1. Enter a **Name** for your variable.
+1. In the **Type** list, select **Query**.
+1. (optional) In **Label**, enter the display name of the variable dropdown. If you don't enter a display name, then the dropdown label will be the variable name.
+1. Choose a **Hide** option:
+   - **No selection (blank) -** The variable dropdown displays the variable **Name** or **Label** value. This is the default.
+   - **Label -** The variable dropdown only displays the selected variable value and a down arrow.
+   - **Variable -** No variable dropdown is displayed on the dashboard.
+
+## Enter Query Options
+
+1. In the **Data source** list, select the target data source for the query. For more information about data sources, refer to [Add a data source]({{< relref "../features/datasources/add-a-data-source.md" >}}).
+1. In the **Refresh** list, select when the variable should update options.
+   - **Never -** Variables queries are cached and values are not updated. This is fine if the values never change, but problematic if they are dynamic and change a lot.
+   - **On Dashboard Load -** Queries the data source every time the dashboard loads. This slows down dashboard loading, because the variable query needs to be completed before dashboard can be initialized. 
+   - **On Time Range Change -** Queries the data source when the dashboard time range changes. Only use this option if your variable options query contains a time range filter or is dependent on the dashboard time range.
+1. In the **Query** field, enter a query. 
+   - The query field varies according to your data source. Some data sources have custom query editors.
+   - If you need more room in a single input field query editor, then hover your cursor over the lines in the lower right corner of the field and drag downward to expand.
+2. (optional) In the **Regex** field, type a regex expression to filter or capture specific parts of the names return by your data source query. To see examples, refer to [Filter variables with regex]({{< relref "filter-variables-with-regex.md" >}}).
+3. In the **Sort** list, select the sort order for values to be displayed in the dropdown list. The default option, **Disabled**, means that the order of options returned by your data source query will be used.
+
+## Enter Selection Options
+
+All selection options are optional, and they are off by default.
+
+### Multi-value
+
+If you turn this on, then the variable dropdown list allows users to select multiple options at the same time. For more information, refer to [Formatting multi-value variables]({{< relref "formatting-multi-value-variables.md" >}}).
+
+### Include All option
+
+Grafana adds an `All` option to the variable dropdown list. If a user selects this option, then all variable options are selected.
+
+### Custom all value
+
+This option is only visible if the **Include All option** is selected.
+
+Enter regex, globs, or lucene syntax in the **Custom all value** field to define the value of the `All` option.
+
+By default the `All` value includes all options in combined expression. This can become very long and can have performance problems. Sometimes it can be better to specify a custom all value, like a wildcard regex. 
+
+In order to have custom regex, globs, or lucene syntax in the **Custom all value** option, it is never escaped so you will have to think about what is a valid value for your data source.
+
+## Enter Value groups/tags (experimental feature)
+
+If you have many options in the dropdown for a multi-value variable, then you can use this feature to group the values into selectable tags.
+
+{{< docs-imagebox img="/img/docs/v50/variable_dropdown_tags.png" max-width="300px" >}}
+
+This feature is off by default. Click **Enabled** to turn on the feature.
+
+### Tags query
+
+Enter a data source query that should return a list of tags. The tags query returns a list of tags that each represent a group, and the tag values query returns a list of group members. 
+
+For example, the tags query could be a list of regions (Europe, Asia, Americas), and then if the user selects the Europe tag, then the tag values query would return a list of countries -- Sweden, Germany, France, and so on.
+
+If you have a variable with a lot of values (say all the countries in the world), then this allows you to easily select a group of them. If the user selects the tag Europe, all the countries in Europe would be selected.
+
+In this [example dashboard](https://play.grafana.org/d/ZUPhFVGGk/graphite-with-experimental-tags?orgId=1), the server variable has tags enabled. 
+
+### Tag values query
+
+Enter a data source query that should return a list of values for a specified tag key. Use `$tag` in the query to refer the currently selected tag.
+
+The `$tag` variable will have the value of the tag that the user chooses. 
+
+For example, if you have a Graphite query for tags, `regions.*`, that returns a list of regions. The the values query could be `regions.$tag.*`, which if the user chooses Europe would be interpolated to `regions.Europe.*`.
+
+## Final steps
+
+1. In **Preview of values**, Grafana displays a list of the current variable values. Review them to ensure they match what you expect.
+1. Click **Add** to add the variable to the dashboard.

docs/sources/variables/filter-variables-with-regex.md

@@ -0,0 +1,71 @@
+# Filter variables with regex
+
+Using the Regex Query option, you filter the list of options returned by the variable query or modify the options returned.
+
+This page shows how to use regex to filter/modify values in the variable dropdown
+
+Using the Regex Query Option, you filter the list of options returned by the Variable query or modify the options returned. For more information, refer to the Mozilla guide on [Regular expressions](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide/Regular_Expressions).
+
+Examples of filtering on the following list of options:
+
+```text
+backend_01
+backend_02
+backend_03
+backend_04
+```
+
+## Filter so that only the options that end with `01` or `02` are returned:
+
+Regex:
+
+```regex
+/.*[01|02]/
+```
+
+Result:
+
+```text
+backend_01
+backend_02
+```
+
+## Filter and modify the options using a regex capture group to return part of the text:
+
+Regex:
+
+```regex
+/.*(01|02)/
+```
+
+Result:
+
+```text
+01
+02
+```
+
+## Filter and modify - Prometheus Example
+
+List of options:
+
+```text
+up{instance="demo.robustperception.io:9090",job="prometheus"} 1 1521630638000
+up{instance="demo.robustperception.io:9093",job="alertmanager"} 1 1521630638000
+up{instance="demo.robustperception.io:9100",job="node"} 1 1521630638000
+```
+
+Regex:
+
+```regex
+/.*instance="([^"]*).*/
+```
+
+Result:
+
+```text
+demo.robustperception.io:9090
+demo.robustperception.io:9093
+demo.robustperception.io:9100
+```
+```

docs/sources/variables/formatting-multi-value-variables.md

@@ -0,0 +1,32 @@
++++
+title = "Formatting multi-value variables"
+type = "docs"
+[menu.docs]
+weight = 500
++++
+
+# Formatting multi-value variables
+
+Interpolating a variable with multiple values selected is tricky as it is not straight forward how to format the multiple values into a string that is valid in the given context where the variable is used. Grafana tries to solve this by allowing each data source plugin to inform the templating interpolation engine what format to use for multiple values.
+
+> **Note:** The **Custom all value** option on the variable must be blank for Grafana to format all values into a single string. If leave it blank, then the Grafana concatenates (adds together) all the values in the query. Something like `value1,value2,value3`. If a custom `all` value is used, then instead the value will be something like `*` or `all`.
+
+## Multi-value variables with a Graphite data source
+
+Graphite uses glob expressions. A variable with multiple values would, in this case, be interpolated as `{host1,host2,host3}` if the current variable value was *host1*, *host2*, and *host3*.
+
+## Multi-value variables with a Prometheus or InfluxDB data source
+
+InfluxDB and Prometheus use regex expressions, so the same variable would be interpolated as `(host1|host2|host3)`. Every value would also be regex escaped. If not, a value with a regex control character would break the regex expression.
+
+## Multi-value variables with an Elastic data source
+
+Elasticsearch uses lucene query syntax, so the same variable would be formatted as `("host1" OR "host2" OR "host3")`. In this case, every value must be escaped so that the value only contains lucene control words and quotation marks.
+
+## Formatting troubles
+
+Automatic escaping and formatting can cause problems and it can be tricky to grasp the logic behind it. Especially for InfluxDB and Prometheus where the use of regex syntax requires that the variable is used in regex operator context.
+
+If you do not want Grafana to do this automatic regex escaping and formatting, then you must do one of the following:
+- Turn off the **Multi-value** or **Include All option** options.
+- Use the [raw variable format]({{< relref "advanced-variable-format-options.md#raw" >}}).

docs/sources/variables/global-variables.md

@@ -47,12 +47,14 @@ This variable is the `$__interval` variable in milliseconds, not a time interval
 This variable is only available in the Singlestat panel and can be used in the prefix or suffix fields on the Options tab. The variable will be replaced with the series name or alias.
 
 ## $__org
+
 > Only available in Grafana v6.7+
 
 This variable is the ID of the current organization.
 `${__org.name}` is the name of the current organization.
 
 ## $__user
+
 > Only available in Grafana v7.1+
 
 `${__user.id}` is the ID of the current user.

docs/sources/variables/templates-and-variables.md

@@ -36,6 +36,20 @@ wmi_system_threads{instance=~"$server"}
 
 Variable values are always synced to the URL using the syntax `var-<varname>=value`.
 
+## Examples of templates and variables
+
+To see variable and template examples, go to any of the dashboards listed below.
+
+- [Elasticsearch Templated dashboard](https://play.grafana.org/dashboard/db/elasticsearch-templated)
+- [Graphite Templated Nested dashboard](https://play.grafana.org/dashboard/db/graphite-templated-nested)
+- [InfluxDB Templated dashboard](https://play.grafana.org/dashboard/db/influxdb-templated)
+
+Variables are listed in dropdown lists across the top of the screen. Select different variables to see how the visualizations change. 
+
+To see variable settings, navigate to **Dashboard Settings > Variables**. Click a variable in the list to see its settings.
+
+Variables can be used in titles, descriptions, text panels, and queries. Queries with text that starts with `$` are templates. Not all panels will have template queries.
+
 ## Variable syntax
 
 Panel titles and metric queries can refer to variables using two different syntaxes:
@@ -54,196 +68,48 @@ documentation topic for details on value escaping during interpolation.
 
 For advanced syntax to override data source default formatting, refer to [Advanced variable format options]({{< relref "advanced-variable-format-options.md" >}}).
 
-## Examples of templates and variables
-
-To see variable and template examples, go to any of the dashboards listed below.
-
-- [Elasticsearch Templated dashboard](https://play.grafana.org/dashboard/db/elasticsearch-templated)
-- [Graphite Templated Nested dashboard](https://play.grafana.org/dashboard/db/graphite-templated-nested)
-- [InfluxDB Templated dashboard](https://play.grafana.org/dashboard/db/influxdb-templated)
-
-Variables are listed in dropdown lists across the top of the screen. Select different variables to see how the visualizations change. 
-
-To see variable settings, navigate to **Dashboard Settings > Variables**. Click a variable in the list to see its settings.
-
-Variables can be used in titles, descriptions, text panels, and queries. Queries with text that starts with `$` are templates. Not all panels will have template queries.
-
 ## Variable types
 
-Grafana also has global built-in variables that can be used in expressions in the query editor. Refer to [Global variables]({{< relref "global-variables" >}}) for more information.
+Grafana has global built-in variables that can be used in expressions in the query editor. Refer to [Global variables]({{< relref "global-variables" >}}) for more information.
 
 You can also define the following types of variables in Grafana. 
 
-Type | Description
-------- | --------
-*Query* | This variable type allows you to write a data source query that usually returns a list of metric names, tag values or keys. For example, a query that returns a list of server names, sensor ids or data centers.
-*Interval* | This variable can represent time spans. Instead of hard-coding a group by time or date histogram interval, use a variable of this type.
-*Data source* | This type allows you to quickly change the data source for an entire Dashboard. Useful if you have multiple instances of a data source in for example different environments.
-*Custom* | Define the variable options manually using a comma separated list.
-*Constant* | Define a hidden constant. Useful for metric path prefixes for dashboards you want to share. During dashboard export, constant variables will be made into an import option.
-*Ad hoc filters* | Very special kind of variable that only works with some data sources, InfluxDB, Prometheus, and Elasticsearch currently. It allows you to add key/value filters that will automatically be added to all metric queries that use the specified data source.
-*Text box* | This variable type will display as a free text input field with an optional default value.
-
-## Adding a variable
-
-{{< docs-imagebox img="/img/docs/v50/variables_var_list.png" max-width="800px" >}}
-
-You add variables via Dashboard cogs menu > Templating. This opens up a list of variables and a `New` button to create a new variable.
-
-### Basic variable options
-
-Option | Description
-------- | --------
-*Name* | The name of the variable, this is the name you use when you refer to your variable in your metric queries. Must be unique and contain no white-spaces.
-*Label* | The name of the dropdown for this variable.
-*Hide* | Options to hide the dropdown select box.
-*Type* | Defines the variable type.
-
-### Query options
-
-This variable type is the most powerful and complex as it can dynamically fetch its options using a data source query.
-
-Option | Description
-------- | --------
-*Data source* | The data source target for the query.
-*Refresh* | Controls when to update the variable option list (values in the dropdown). **On Dashboard Load** will slow down dashboard load as the variable query needs to be completed before dashboard can be initialized. Set this only to **On Time Range Change** if your variable options query contains a time range filter or is dependent on dashboard time range.
-*Query* | The data source specific query expression.
-*Regex* | Regex to filter or capture specific parts of the names return by your data source query. Optional.
-*Sort* | Define sort order for options in dropdown. **Disabled** means that the order of options returned by your data source query will be used.
-
-#### Using regex to filter/modify values in the Variable dropdown
+### Query
 
-Using the Regex Query Option, you filter the list of options returned by the Variable query or modify the options returned.
+Query variables allow you to write a data source query that might return a list of metric names, tag values, keys, server names, sensor IDs, data centers, etc.
 
-Examples of filtering on the following list of options:
+For instructions, refer to [Add a query variable]({{< relref "add-query-variable.md" >}}).
 
-```text
-backend_01
-backend_02
-backend_03
-backend_04
-```
-
-##### Filter so that only the options that end with `01` or `02` are returned:
-
-Regex:
-
-```regex
-/.*[01|02]/
-```
+### Custom
 
-Result:
+Define the variable options manually using a comma-separated list.
 
-```text
-backend_01
-backend_02
-```
-
-##### Filter and modify the options using a regex capture group to return part of the text:
+### Text box
 
-Regex:
+Text box variables display as a free text input field with an optional default value.
 
-```regex
-/.*(01|02)/
-```
+### Constant
 
-Result:
-
-```text
-01
-02
-```
+Define a hidden constant. Useful for metric path prefixes for dashboards you want to share. During dashboard export, constant variables are made into an import option.
 
-#### Filter and modify - Prometheus Example
-
-List of options:
-
-```text
-up{instance="demo.robustperception.io:9090",job="prometheus"} 1 1521630638000
-up{instance="demo.robustperception.io:9093",job="alertmanager"} 1 1521630638000
-up{instance="demo.robustperception.io:9100",job="node"} 1 1521630638000
-```
-
-Regex:
-
-```regex
-/.*instance="([^"]*).*/
-```
-
-Result:
-
-```text
-demo.robustperception.io:9090
-demo.robustperception.io:9093
-demo.robustperception.io:9100
-```
+### Data source
 
-### Query expressions
+Data source variables allow you to quickly change the data source for an entire dashboard. Useful if you have multiple instances of a data source in for example different environments.
 
-The query expressions are different for each data source.
+### Interval
 
-- [Graphite templating queries]({{< relref "../features/datasources/graphite.md#templating" >}})
-- [Elasticsearch templating queries]({{< relref "../features/datasources/elasticsearch.md#templating" >}})
-- [InfluxDB templating queries]({{< relref "../features/datasources/influxdb.md#templating" >}})
-- [Prometheus templating queries]({{< relref "../features/datasources/prometheus.md#templating" >}})
-- [OpenTSDB templating queries]({{< relref "../features/datasources/opentsdb.md#templating" >}})
-- [AzureMonitor templating queries]({{< relref "../features/datasources/azuremonitor.md#templating-with-variables-for-the-azure-monitor-service" >}})
+Interval variables represent time spans. Instead of hard-coding a group by time or date histogram interval, use an interval variable.
 
-One thing to note is that query expressions can contain references to other variables and in effect create linked variables.
-Grafana will detect this and automatically refresh a variable when one of it's containing variables change.
-
-## Selection options
-
-Option | Description
-------- | --------
-*Multi-value* | If enabled, the variable will support the selection of multiple options at the same time.
-*Include All option* | Add a special `All` option whose value includes all options.
-*Custom all value* | By default the `All` value will include all options in combined expression. This can become very long and can have performance problems. Many times it can be better to specify a custom all value, like a wildcard regex. To make it possible to have custom regex, globs or lucene syntax in the **Custom all value** option it is never escaped so you will have to think about what is a valid value for your data source.
-
-### Formatting multiple values
-
-Interpolating a variable with multiple values selected is tricky, as it is not straightforward how to format the multiple values into a string that
-is valid in the given context where the variable is used. Grafana tries to solve this by allowing each data source plugin to
-inform the templating interpolation engine what format to use for multiple values.
-
-Note that the *Custom all value* option on the variable will have to be left blank for Grafana to format all values into a single string.
-
-**Graphite**, for example, uses glob expressions. A variable with multiple values would, in this case, be interpolated as `{host1,host2,host3}` if
-the current variable value was *host1*, *host2* and *host3*.
-
-**InfluxDB and Prometheus** uses regex expressions, so the same variable
-would be interpolated as `(host1|host2|host3)`. Every value would also be regex escaped if not, a value with a regex control character would
-break the regex expression.
-
-**Elasticsearch** uses lucene query syntax, so the same variable would, in this case, be formatted as `("host1" OR "host2" OR "host3")`. In this case every value
-needs to be escaped so that the value can contain lucene control words and quotation marks.
-
-#### Formatting troubles
-
-Automatic escaping and formatting can cause problems and it can be tricky to grasp the logic behind it.
-Especially for InfluxDB and Prometheus where the use of regex syntax requires that the variable is used in regex operator context.
-If you do not want Grafana to do this automatic regex escaping and formatting your only option is to disable the *Multi-value* or *Include All option*
-options.
-
-### Value groups/tags
-
-If you have a lot of options in the dropdown for a multi-value variable. You can use this feature to group the values into selectable tags.
-
-Option | Description
-------- | --------
-*Tags query* | Data source query that should return a list of tags
-*Tag values query* | Data source query that should return a list of values for a specified tag key. Use `$tag` in the query to refer the currently selected tag.
-
-{{< docs-imagebox img="/img/docs/v50/variable_dropdown_tags.png" max-width="300px" >}}
-
-### Interval variables
-
-Use the `Interval` type to create a variable that represents a time span (eg. `1m`,`1h`, `1d`). There is also a special `auto` option that will change depending on the current time range. You can specify how many times the current time range should be divided to calculate the current `auto` timespan.
+You can use the `Interval` type to create a variable that represents a time span (eg. `1m`,`1h`, `1d`). There is also a special `auto` option that will change depending on the current time range. You can specify how many times the current time range should be divided to calculate the current `auto` timespan.
 
 This variable type is useful as a parameter to group by time (for InfluxDB), Date histogram interval (for Elasticsearch) or as a *summarize* function parameter (for Graphite).
 
-Example using the template variable `myinterval` of type `Interval` in a graphite function:
+Example using the template variable `myinterval` of type `Interval` in a Graphite function:
 
 ```
 summarize($myinterval, sum, false)
 ```
+
+### Ad hoc filters
+
+Ad hoc filter variables only work with  InfluxDB, Prometheus, and Elasticsearch data sources. This variable type allows you to add key/value filters that are automatically added to all metric queries that use the specified data source.