mirror of https://github.com/grafana/grafana
Docs: refactors notifications in alerting (#57734)
* Refactors noticifacations section * adds hover over error icon detailpull/57743/head
brendamuir
3 years ago
committed by
GitHub
parent
50fb47dba0
commit
279ac50d59
@ -1,27 +0,0 @@ |
||||
--- |
||||
aliases: |
||||
- /docs/grafana/latest/alerting/alert-groups/filter-alerts/ |
||||
- /docs/grafana/latest/alerting/unified-alerting/alert-groups/ |
||||
description: Alert groups |
||||
keywords: |
||||
- grafana |
||||
- alerting |
||||
- alerts |
||||
- groups |
||||
title: Filter alerts |
||||
weight: 445 |
||||
--- |
||||
|
||||
# Filter alerts by group |
||||
|
||||
You can use the following filters to view alerts that match specific criteria. |
||||
|
||||
## Search by label |
||||
|
||||
To use this filter option: |
||||
|
||||
In **Search**, enter an existing label to view alerts matching the label. For example, `environment=production,region=~US|EU,severity!=warning`. |
||||
|
||||
## Filter alerts by state |
||||
|
||||
In **States**, select from Active, Suppressed, or Unprocessed states to view alerts matching your selected state. All other alerts are hidden. |
||||
@ -1,32 +0,0 @@ |
||||
--- |
||||
aliases: |
||||
- /docs/grafana/latest/alerting/alert-groups/ |
||||
- /docs/grafana/latest/alerting/alert-groups/view-alert-grouping/ |
||||
- /docs/grafana/latest/alerting/unified-alerting/alert-groups/ |
||||
description: Alert groups |
||||
keywords: |
||||
- grafana |
||||
- alerting |
||||
- alerts |
||||
- groups |
||||
title: View alert groups |
||||
weight: 445 |
||||
--- |
||||
|
||||
# View alert groups |
||||
|
||||
To view alert groups: |
||||
|
||||
1. In the Grafana menu, click the **Alerting** (bell) icon to open the Alerting page listing existing alerts. |
||||
1. Click **Alert groups** to open the page listing existing groups. |
||||
1. From the **Alertmanager** drop-down, select an external Alertmanager as your data source. By default, the `Grafana` Alertmanager is selected. |
||||
1. From **custom group by** drop-down, select a combination of labels to view a grouping other than the default. This is useful for debugging and verifying your grouping of notification policies. |
||||
|
||||
If an alert does not contain labels specified either in the grouping of the root policy or the custom grouping, then the alert is added to a catch all group with a header of `No grouping`. |
||||
|
||||
## Filter alerts |
||||
|
||||
You can use the following filters to view alerts that match specific criteria: |
||||
|
||||
- **Search by label:** In **Search**, enter an existing label to view alerts matching the label. For example, `environment=production,region=~US|EU,severity!=warning` |
||||
- **Filter alerts by state:** In **States**, select from Active, Suppressed, or Unprocessed states to view alerts matching your selected state. All other alerts are hidden. |
||||
@ -1,21 +0,0 @@ |
||||
--- |
||||
aliases: |
||||
- /docs/grafana/latest/alerting/contact-points/ |
||||
- /docs/grafana/latest/alerting/unified-alerting/contact-points/ |
||||
description: Create or edit contact point |
||||
keywords: |
||||
- grafana |
||||
- alerting |
||||
- guide |
||||
- contact point |
||||
- notification channel |
||||
- create |
||||
title: Contact points |
||||
weight: 410 |
||||
--- |
||||
|
||||
# Contact points |
||||
|
||||
Use contact points to define how your contacts are notified when an alert fires. A contact point can have one or more contact point types, for example, email, slack, webhook, and so on. When an alert fires, a notification is sent to all contact point types listed for a contact point. Optionally, use message templating to customize notification messages for the contact point types. |
||||
|
||||
You can configure Grafana managed contact points as well as contact points for an external Alertmanager data source. |
||||
@ -1,69 +0,0 @@ |
||||
--- |
||||
aliases: |
||||
- /docs/grafana/latest/alerting/contact-points/message-templating/ |
||||
- /docs/grafana/latest/alerting/contact-points/message-templating/example-template/ |
||||
- /docs/grafana/latest/alerting/message-templating/ |
||||
- /docs/grafana/latest/alerting/unified-alerting/message-templating/ |
||||
- /docs/grafana/latest/alerting/fundamentals/contact-points/example-template/ |
||||
keywords: |
||||
- grafana |
||||
- alerting |
||||
- guide |
||||
- contact point |
||||
- templating |
||||
title: Example message template |
||||
weight: 115 |
||||
--- |
||||
|
||||
# Example of a custom template |
||||
|
||||
Here's an example of how to use a custom template. You can also use the default template included in the setup. |
||||
|
||||
Step 1: Configure a template to render a single alert. |
||||
|
||||
``` |
||||
{{ define "myalert" }} |
||||
[{{.Status}}] {{ .Labels.alertname }} |
||||
|
||||
Labels: |
||||
{{ range .Labels.SortedPairs }} |
||||
{{ .Name }}: {{ .Value }} |
||||
{{ end }} |
||||
|
||||
{{ if gt (len .Annotations) 0 }} |
||||
Annotations: |
||||
{{ range .Annotations.SortedPairs }} |
||||
{{ .Name }}: {{ .Value }} |
||||
{{ end }} |
||||
{{ end }} |
||||
|
||||
{{ if gt (len .SilenceURL ) 0 }} |
||||
Silence alert: {{ .SilenceURL }} |
||||
{{ end }} |
||||
{{ if gt (len .DashboardURL ) 0 }} |
||||
Go to dashboard: {{ .DashboardURL }} |
||||
{{ end }} |
||||
{{ end }} |
||||
``` |
||||
|
||||
Step 2: Configure a template to render entire notification message. |
||||
|
||||
``` |
||||
{{ define "mymessage" }} |
||||
{{ if gt (len .Alerts.Firing) 0 }} |
||||
{{ len .Alerts.Firing }} firing: |
||||
{{ range .Alerts.Firing }} {{ template "myalert" .}} {{ end }} |
||||
{{ end }} |
||||
{{ if gt (len .Alerts.Resolved) 0 }} |
||||
{{ len .Alerts.Resolved }} resolved: |
||||
{{ range .Alerts.Resolved }} {{ template "myalert" .}} {{ end }} |
||||
{{ end }} |
||||
{{ end }} |
||||
``` |
||||
|
||||
Step 3: Add `mymessage` in the notification message field. |
||||
|
||||
``` |
||||
Alert summary: |
||||
{{ template "mymessage" . }} |
||||
``` |
||||
@ -1,72 +0,0 @@ |
||||
--- |
||||
aliases: |
||||
- /docs/grafana/latest/alerting/contact-points/message-templating/template-data/ |
||||
- /docs/grafana/latest/alerting/message-templating/template-data/ |
||||
- /docs/grafana/latest/alerting/unified-alerting/message-templating/template-data/ |
||||
- /docs/grafana/latest/alerting/fundamentals/contact-points/template-data/ |
||||
keywords: |
||||
- grafana |
||||
- alerting |
||||
- guide |
||||
- contact point |
||||
- templating |
||||
title: Template data |
||||
weight: 120 |
||||
--- |
||||
|
||||
# Template data |
||||
|
||||
Template data is passed on to message templates as well as sent as payload to webhook pushes. |
||||
|
||||
| Name | Type | Notes | |
||||
| ----------------- | -------- | -------------------------------------------------------------------------------------------------------------------- | |
||||
| Receiver | string | Name of the contact point that the notification is being sent to. | |
||||
| Status | string | `firing` if at least one alert is firing, otherwise `resolved`. | |
||||
| Alerts | Alert | List of alert objects that are included in this notification (see below). | |
||||
| GroupLabels | KeyValue | Labels these alerts were grouped by. | |
||||
| CommonLabels | KeyValue | Labels common to all the alerts included in this notification. | |
||||
| CommonAnnotations | KeyValue | Annotations common to all the alerts included in this notification. | |
||||
| ExternalURL | string | Back link to the Grafana that sent the notification. If using external Alertmanager, back link to this Alertmanager. | |
||||
|
||||
The `Alerts` type exposes functions for filtering alerts: |
||||
|
||||
- `Alerts.Firing` returns a list of firing alerts. |
||||
- `Alerts.Resolved` returns a list of resolved alerts. |
||||
|
||||
## Alert |
||||
|
||||
| Name | Type | Notes | |
||||
| ------------ | --------- | ---------------------------------------------------------------------------------------------------------------------------------------------- | |
||||
| Status | string | `firing` or `resolved`. | |
||||
| Labels | KeyValue | A set of labels attached to the alert. | |
||||
| Annotations | KeyValue | A set of annotations attached to the alert. | |
||||
| StartsAt | time.Time | Time the alert started firing. | |
||||
| EndsAt | time.Time | Only set if the end time of an alert is known. Otherwise set to a configurable timeout period from the time since the last alert was received. | |
||||
| GeneratorURL | string | A back link to Grafana or external Alertmanager. | |
||||
| SilenceURL | string | Link to grafana silence for with labels for this alert pre-filled. Only for Grafana managed alerts. | |
||||
| DashboardURL | string | Link to grafana dashboard, if alert rule belongs to one. Only for Grafana managed alerts. | |
||||
| PanelURL | string | Link to grafana dashboard panel, if alert rule belongs to one. Only for Grafana managed alerts. | |
||||
| Fingerprint | string | Fingerprint that can be used to identify the alert. | |
||||
| ValueString | string | A string that contains the labels and value of each reduced expression in the alert. | |
||||
|
||||
## KeyValue |
||||
|
||||
`KeyValue` is a set of key/value string pairs that represent labels and annotations. |
||||
|
||||
Here is an example containing two annotations: |
||||
|
||||
```json |
||||
{ |
||||
"summary": "alert summary", |
||||
"description": "alert description" |
||||
} |
||||
``` |
||||
|
||||
In addition to direct access of data (labels and annotations) stored as KeyValue, there are also methods for sorting, removing and transforming. |
||||
|
||||
| Name | Arguments | Returns | Notes | |
||||
| ----------- | --------- | --------------------------------------- | ----------------------------------------------------------- | |
||||
| SortedPairs | | Sorted list of key & value string pairs | |
||||
| Remove | []string | KeyValue | Returns a copy of the Key/Value map without the given keys. | |
||||
| Names | | []string | List of label names | |
||||
| Values | | []string | List of label values | |
||||
@ -0,0 +1,24 @@ |
||||
--- |
||||
aliases: |
||||
- /docs/grafana/latest/alerting/silences/ |
||||
- /docs/grafana/latest/alerting/unified-alerting/silences/ |
||||
- /docs/grafana/latest/alerting/fundamentals/silences |
||||
description: Silences |
||||
keywords: |
||||
- grafana |
||||
- alerting |
||||
- silence |
||||
- mute |
||||
title: Silences |
||||
weight: 420 |
||||
--- |
||||
|
||||
# Silences |
||||
|
||||
Use silences to stop notifications from one or more alerting rules. Silences do not prevent alert rules from being evaluated. They also do not stop alert instances from being shown in the user interface. Silences only stop notifications from getting created. A silence lasts for only a specified window of time. |
||||
|
||||
You can configure Grafana managed silences as well as silences for an external Alertmanager data source. |
||||
|
||||
## Useful links |
||||
|
||||
[Create silences](https://grafana.com/docs/grafana/latest/alerting/manage-notifications/create-silence/) |
||||
@ -0,0 +1,65 @@ |
||||
--- |
||||
aliases: |
||||
- /docs/grafana/latest/alerting/silences/create-silence/ |
||||
- /docs/grafana/latest/alerting/unified-alerting/silences/ |
||||
- /docs/grafana/latest/alerting/silences/edit-silence/ |
||||
- /docs/grafana/latest/alerting/unified-alerting/silences/ |
||||
- /docs/grafana/latest/alerting/silences/linking-to-silence-form/ |
||||
- /docs/grafana/latest/alerting/unified-alerting/silences/ |
||||
- /docs/grafana/latest/alerting/silences/remove-silence/ |
||||
- /docs/grafana/latest/alerting/unified-alerting/silences/ |
||||
description: Add silence alert notification |
||||
keywords: |
||||
- grafana |
||||
- alerting |
||||
- silence |
||||
- mute |
||||
title: Manage silences |
||||
weight: 450 |
||||
--- |
||||
|
||||
# Manage silences |
||||
|
||||
Silences stop notifications from getting created and last for only a specified window of time. |
||||
|
||||
## Add silences |
||||
|
||||
To add a silence, complete the following steps. |
||||
|
||||
1. In the Grafana menu, click the **Alerting** (bell) icon to open the Alerting page listing existing alerts. |
||||
2. On the Alerting page, click **Silences** to open the page listing existing silences. |
||||
3. From Alertmanager drop-down, select an external Alertmanager to create and manage silences for the external data source. Otherwise, keep the default option of Grafana. |
||||
4. Click **New Silence** to open the Create silence page. |
||||
5. In **Silence start and end**, select the start and end date to indicate when the silence should go into effect and expire. |
||||
6. Optionally, in **Duration**, specify how long the silence is enforced. This automatically updates the end time in the **Silence start and end** field. |
||||
7. In the **Name** and **Value** fields, enter one or more _Matching Labels_. Matchers determine which rules the silence will apply to. |
||||
8. In **Comment**, add details about the silence. |
||||
9. In **Creator**, enter the name of the silence owner or keep the default owner. |
||||
10. Click **Create**. |
||||
|
||||
## Edit silences |
||||
|
||||
To edit a silence, complete the following steps. |
||||
|
||||
1. In the Grafana menu, click the **Alerting** (bell) icon to open the Alerting page listing existing alerts. |
||||
2. Click **Silences** to view the list of existing silences. |
||||
3. Find the silence you want to edit, then click **Edit** (pen icon). |
||||
4. Make the desired changes, then click **Submit** to save your changes. |
||||
|
||||
## Create a URL to link to a silence form |
||||
|
||||
When linking to a silence form, provide the default matching labels and comment via `matcher` and `comment` query parameters. The `matcher` parameter should be in the following format `[label][operator][value]` where the `operator` parameter can be one of the following: `=` (equals, not regex), `!=` (not equals, not regex), `=~` (equals, regex), `!~` (not equals, regex). |
||||
The URL can contain many query parameters with the key `matcher`. |
||||
For example, to link to silence form with matching labels `severity=critical` & `cluster!~europe-.*` and comment `Silence critical EU alerts`, create a URL `https://mygrafana/alerting/silence/new?matcher=severity%3Dcritical&matcher=cluster!~europe-*&comment=Silence%20critical%20EU%20alert`. |
||||
|
||||
To link to a new silence page for an external Alertmanager, add a `alertmanager` query parameter with the Alertmanager data source name. |
||||
|
||||
## Remove silences |
||||
|
||||
To remove a silence, complete the following steps. |
||||
|
||||
1. In the Grafana menu, click the **Alerting** (bell) icon to open the Alerting page listing existing alerts. |
||||
1. Click **Silences** to view the list of existing silences. |
||||
1. Select the silence you want to end, then click **Unsilence**. |
||||
|
||||
> **Note:** You cannot remove a silence manually. Silences that have ended are retained and listed for five days. |
||||
@ -0,0 +1,38 @@ |
||||
--- |
||||
aliases: |
||||
- /docs/grafana/latest/alerting/manage-notifications/view-notification-errors/ |
||||
keywords: |
||||
- grafana |
||||
- alerting |
||||
- notification |
||||
- errors |
||||
- contact points |
||||
title: View notification errors |
||||
weight: 150 |
||||
--- |
||||
|
||||
# View notification errors |
||||
|
||||
View notification errors and understand why they failed to be sent or were not received. |
||||
|
||||
**Note:** |
||||
This feature only works if you are using Grafana Alertmanager. |
||||
|
||||
To view notification errors, complete the following steps. |
||||
|
||||
1. Navigate to Alerting -> Contact points. |
||||
|
||||
If any contact points are failing, a message at the right-hand corner of the screen alerts the user to the fact that there are errors and how many. |
||||
|
||||
2. Click on the contact point to view the details of errors for each contact point. |
||||
|
||||
Error details are displayed if you hover over the Error icon. |
||||
|
||||
If a contact point has more than one integration, you see all errors for each of the integrations listed. |
||||
|
||||
3. In the Health column, check the status of the notification. |
||||
|
||||
This can be either OK, No attempts, or Error. |
||||
|
||||
## Useful links |
||||
[Receivers API](https://editor.swagger.io/?url=https://raw.githubusercontent.com/grafana/grafana/main/pkg/services/ngalert/api/tooling/post.json) |
||||
@ -1,27 +0,0 @@ |
||||
--- |
||||
aliases: |
||||
- /docs/grafana/latest/alerting/silences/ |
||||
- /docs/grafana/latest/alerting/unified-alerting/silences/ |
||||
description: Silences |
||||
keywords: |
||||
- grafana |
||||
- alerting |
||||
- silence |
||||
- mute |
||||
title: Silences |
||||
weight: 450 |
||||
--- |
||||
|
||||
# About alerting silences |
||||
|
||||
Use silences to stop notifications from one or more alerting rules. Silences do not prevent alert rules from being evaluated. Nor do they not stop alerting instances from being shown in the user interface. Silences only stop notifications from getting created. A silence lasts for only a specified window of time. |
||||
|
||||
You can configure Grafana managed silences as well as silences for an [external Alertmanager data source]({{< relref "../../datasources/alertmanager/" >}}). For more information, see [Alertmanager]({{< relref "../fundamentals/alertmanager/" >}}). |
||||
|
||||
See also: |
||||
|
||||
- [How label matching works]({{< relref "../fundamentals/annotation-label/labels-and-label-matchers/" >}}) |
||||
- [Create a silence]({{< relref "create-silence/" >}}) |
||||
- [Create a URL to link to a silence form]({{< relref "linking-to-silence-form/" >}}) |
||||
- [Edit silences]({{< relref "edit-silence/" >}}) |
||||
- [Remove silences]({{< relref "remove-silence/" >}}) |
||||
@ -1,30 +0,0 @@ |
||||
--- |
||||
aliases: |
||||
- /docs/grafana/latest/alerting/silences/create-silence/ |
||||
- /docs/grafana/latest/alerting/unified-alerting/silences/ |
||||
description: Add silence alert notification |
||||
keywords: |
||||
- grafana |
||||
- alerting |
||||
- silence |
||||
- mute |
||||
title: Create a silence |
||||
weight: 450 |
||||
--- |
||||
|
||||
# Create a silence |
||||
|
||||
Silences stop notifications from getting created and last for only a specified window of time. |
||||
|
||||
To add a silence: |
||||
|
||||
1. In the Grafana menu, click the **Alerting** (bell) icon to open the Alerting page listing existing alerts. |
||||
2. On the Alerting page, click **Silences** to open the page listing existing silences. |
||||
3. From Alertmanager drop-down, select an external Alertmanager to create and manage silences for the external data source. Otherwise, keep the default option of Grafana. |
||||
4. Click **New Silence** to open the Create silence page. |
||||
5. In **Silence start and end**, select the start and end date to indicate when the silence should go into effect and expire. |
||||
6. Optionally, in **Duration**, specify how long the silence is enforced. This automatically updates the end time in the **Silence start and end** field. |
||||
7. In the **Name** and **Value** fields, enter one or more _Matching Labels_. Matchers determine which rules the silence will apply to. For more information, see [how label matching works]({{< relref "../fundamentals/annotation-label/labels-and-label-matchers/" >}}). |
||||
8. In **Comment**, add details about the silence. |
||||
9. In **Creator**, enter the name of the silence owner or keep the default owner. |
||||
10. Click **Create**. |
||||
@ -1,24 +0,0 @@ |
||||
--- |
||||
aliases: |
||||
- /docs/grafana/latest/alerting/silences/edit-silence/ |
||||
- /docs/grafana/latest/alerting/unified-alerting/silences/ |
||||
description: Edit alerting silences |
||||
keywords: |
||||
- grafana |
||||
- alerting |
||||
- silence |
||||
- mute |
||||
title: Edit silences |
||||
weight: 450 |
||||
--- |
||||
|
||||
# Edit silences |
||||
|
||||
Silences stop notifications from getting created. |
||||
|
||||
To edit a silence: |
||||
|
||||
1. In the Grafana menu, click the **Alerting** (bell) icon to open the Alerting page listing existing alerts. |
||||
2. Click **Silences** to view the list of existing silences. |
||||
3. Find the silence you want to edit, then click **Edit** (pen icon). |
||||
4. Make the desired changes, then click **Submit** to save your changes. |
||||
@ -1,21 +0,0 @@ |
||||
--- |
||||
aliases: |
||||
- /docs/grafana/latest/alerting/silences/linking-to-silence-form/ |
||||
- /docs/grafana/latest/alerting/unified-alerting/silences/ |
||||
description: Linking to a silence form |
||||
keywords: |
||||
- grafana |
||||
- alerting |
||||
- silence |
||||
- mute |
||||
title: Create URL to link to silence form |
||||
weight: 451 |
||||
--- |
||||
|
||||
# Create a URL to link to a silence form |
||||
|
||||
When linking to a silence form, provide the default matching labels and comment via `matcher` and `comment` query parameters. The `matcher` parameter should be in the following format `[label][operator][value]` where the `operator` parameter can be one of the following: `=` (equals, not regex), `!=` (not equals, not regex), `=~` (equals, regex), `!~` (not equals, regex). |
||||
The URL can contain many query parameters with the key `matcher`. |
||||
For example, to link to silence form with matching labels `severity=critical` & `cluster!~europe-.*` and comment `Silence critical EU alerts`, create a URL `https://mygrafana/alerting/silence/new?matcher=severity%3Dcritical&matcher=cluster!~europe-*&comment=Silence%20critical%20EU%20alert`. |
||||
|
||||
To link to a new silence page for an [external Alertmanager]({{< relref "../../datasources/alertmanager/" >}}), add a `alertmanager` query parameter with the Alertmanager data source name. |
||||
@ -1,23 +0,0 @@ |
||||
--- |
||||
aliases: |
||||
- /docs/grafana/latest/alerting/silences/remove-silence/ |
||||
- /docs/grafana/latest/alerting/unified-alerting/silences/ |
||||
description: Remove alerting silences |
||||
keywords: |
||||
- grafana |
||||
- alerting |
||||
- silence |
||||
- mute |
||||
title: Remove silences |
||||
weight: 451 |
||||
--- |
||||
|
||||
# Remove silences |
||||
|
||||
Silences stop notifications from getting created. |
||||
|
||||
1. In the Grafana menu, click the **Alerting** (bell) icon to open the Alerting page listing existing alerts. |
||||
1. Click **Silences** to view the list of existing silences. |
||||
1. Select the silence you want to end, then click **Unsilence**. |
||||
|
||||
> **Note:** You cannot remove a silence manually. Silences that have ended are retained and listed for five days. |
||||
Loading…
Reference in new issue