@ -134,6 +134,4 @@ Grafana Alerting is based on the Prometheus model for designing alerting systems
{{<figuresrc="/media/docs/alerting/alerting-alertmanager-architecture.png"max-width="750px"alt="A diagram with the alert generator and alert manager architecture">}}
Grafana uses a custom Alertmanager to manage and deliver alert notifications, as detailed in this guide. This custom Alertmanager extends the capabilities of the [Prometheus Alertmanager](https://prometheus.io/docs/alerting/latest/alertmanager/).
If you already run a Prometheus Alertmanager instance, you can configure Grafana Alerting to forward alerts to your [external Alertmanager for handling notifications](ref:configure-alertmanager).
Grafana provides a custom Alertmanager, extending the Prometheus Alertmanager, to manage and deliver alert notifications. If you run a Prometheus or Mimir Alertmanager, you can configure Grafana Alerting to manage them and handle notifications for Grafana-managed alerts. For details, refer to [configure Alertmanagers](ref:configure-alertmanager).
Grafana Alerting is based on the architecture of the Prometheus alerting system. Grafana sends firing and resolved alerts to an Alertmanager, which is responsible for [handling notifications](ref:notifications).
Grafana Alerting is based on the architecture of the Prometheus alerting system. Grafana sends firing and resolved alerts to an Alertmanager, which is responsible for [handling notifications](ref:notifications). This architecture decouples alert rule evaluation from notification handling, improving scalability.
{{<figuresrc="/media/docs/alerting/alerting-alertmanager-architecture.png"max-width="750px"alt="A diagram with the alert generator and alert manager architecture">}}
This architecture decouples alert rule evaluation from notification handling, allowing alerts to be forwarded to other Alertmanagers.
Grafana can use different Alertmanagers. It’s important to note that each Alertmanager manages its own independent alerting resources, such as:
Grafana can enable one or more Alertmanagers to receive Grafana-managed alerts for notification handling. It’s important to note that each Alertmanager manages its own independent alerting resources, such as:
- Contact points and notification templates
- Notification policies and mute timings
- Silences
- Active notifications
Use the `Choose Alertmanager` on these pages to switch between Alertmanagers.
Use the `Choose Alertmanager`dropdown on these pages to switch between Alertmanagers and view or manage their resources.
{{<figuresrc="/media/docs/alerting/alerting-choose-alertmanager.png"max-width="750px"alt="A screenshot choosing an Alertmanager in the notification policies UI">}}
## Types of Alertmanagers in Grafana
Grafana can be configured to manage both Grafana-managed and data source-managed alerts using various Alertmanagers, depending on your infrastructure and alerting requirements.
Grafana can be configured to handle alert notifications using various Alertmanagers.
- **Grafana Alertmanager**: Grafana has its own built-in Alertmanager, referred to as "Grafana" in the user interface. It is the default Alertmanager and can only handle Grafana-managed alerts.
- **Grafana Alertmanager**: Grafana includes a built-in Alertmanager that extends the [Prometheus Alertmanager](https://prometheus.io/docs/alerting/latest/alertmanager/). This is the default Alertmanager and is referred to as "Grafana" in the user interface. It can only handle Grafana-managed alerts.
- **Cloud Alertmanager**: Each Grafana Cloud instance comes preconfigured with an additional Alertmanager (`grafanacloud-STACK_NAME-ngalertmanager`) from the Mimir (Prometheus) instance running in the Grafana Cloud Stack.
@ -59,44 +62,47 @@ Grafana can be configured to manage both Grafana-managed and data source-managed
Some Grafana Cloud services, such as **Kubernetes Monitoring** and **Synthetic Monitoring** use the Cloud Alertmanager to create and manage alerts.
- **Other Alertmanagers**: Grafana Alerting also supports sending alerts to other Alertmanagers, such as the [Prometheus Alertmanager](https://prometheus.io/docs/alerting/latest/alertmanager/), which can handle Grafana-managed alerts and data sources-managed alerts such as alerts from Loki, Mimir, and Prometheus.
- **Other Alertmanagers**: Grafana Alerting also supports sending alerts to other Alertmanagers, such as the [Prometheus Alertmanager](https://prometheus.io/docs/alerting/latest/alertmanager/), which can handle both Grafana-managed and data source-managed alerts.
You can use a combination of Alertmanagers. The decision often depends on your alerting setup and where your alerts are being generated. Here are two examples of when you may want to add an Alertmanager and send your alerts there instead of using the built-in Grafana Alertmanager.
Grafana Alerting supports using a combination of Alertmanagers and can [enable other Alertmanagers to receive Grafana-managed alerts](#enable-an-alertmanager-to-receive-grafana-managed-alerts). The decision often depends on your alerting setup and where your alerts are generated.
1. You may already have Alertmanagers on-premises in your own Cloud infrastructure that you still want to use because you have other alert generators, such as Prometheus.
2. You want to use both Prometheus on-premises and hosted Grafana to send alerts to the same Alertmanager that runs in your Cloud infrastructure.
For example, if you already have an Alertmanager running in your on-premises or cloud infrastructure to handle Prometheus alerts, you can forward Grafana-managed alerts to the same Alertmanager for unified notification handling.
## Add an Alertmanager
From Grafana, you can configure and administer your own Alertmanager to receive Grafana alerts.
Alertmanagers should be configured as data sources using Grafana Configuration from the main Grafana navigation menu. To add an Alertmanager, complete the following steps.
After adding an Alertmanager, you can use the Grafana Alerting UI to manage notification policies, contact points, and other alerting resources from within Grafana, with support for HTTP basic authentication credentials.
For provisioning instructions, refer to the [Alertmanager data source documentation](ref:alertmanager-data-source).
Alertmanagers should be configured as data sources using Grafana Configuration from the main Grafana navigation menu. To add an Alertmanager, complete the following steps.
After adding an Alertmanager, you can use the Grafana Alerting UI to manage notification policies, contact points, silences, and other alerting resources from within Grafana.
{{% admonition type="note" %}}
When using Prometheus, you can manage silences in the Grafana Alerting UI. However, other Alertmanager resources such as contact points, notification policies, and templates are read-only because the Prometheus Alertmanager HTTP API does not support updates for these resources.
{{% /admonition %}}
1. Click **Connections** in the left-side menu.
1. On the Connections page, search for `Alertmanager`.
1. Click the **Create a new data source** button.
When using multiple Alertmanagers, use the `Choose Alertmanager` dropdown to switch between Alertmanagers.
If you don't see this button, you may need to install the plugin, relaunch your Cloud instance, and then repeat steps 1 and 2.
## Enable an Alertmanager to receive Grafana-managed alerts
1. Fill out the fields on the page, as required.
After enabling **Receive Grafana Alerts** in the Data Source Settings, you must also configure the Alertmanager in the Alerting Settings page. Grafana supports enabling one or multiple Alertmanagers to receive all generated Grafana-managed alerts.
If you are provisioning your data source, set the flag `handleGrafanaManagedAlerts` in the `jsonData` field to `true` to send Grafana-managed alerts to this Alertmanager.
1. In the left-side menu, click **Alerts & IRM** and then **Alerting**.
1. Click **Settings** to view the list of configured Alertmanagers.
1. For the selected Alertmanager, click the **Enable/Disable** button to toggle receiving Grafana-managed alerts. When activated, the Alertmanager displays `Receiving Grafana-managed alerts`.
**Note:** Prometheus, Grafana Mimir, and Cortex implementations of Alertmanager are supported. For Prometheus, contact points and notification policies are read-only in the Grafana Alerting UI.
All Grafana-managed alerts are forwarded to Alertmanagers marked as `Receiving Grafana-managed alerts`.
{{% admonition type="note" %}}
Grafana Alerting does not support sending alerts to the AWS Managed Service for Prometheus due to the lack of sigv4 support in Prometheus.
Grafana Alerting does not support forwarding Grafana-managed alerts to the AlertManager in Amazon Managed Service for Prometheus. For more details, refer to [this GitHub issue](https://github.com/grafana/grafana/issues/64064).
{{% /admonition %}}
## Manage Alertmanager configurations
On the Settings page, you can manage your Alertmanager configurations and configure where Grafana-managed alert instances are forwarded.
On the Settings page, you can also manage your Alertmanager configurations.
- Manage which Alertmanagers receive alert instances from Grafana-managed rules without navigating and editing data sources.
- Manage version snapshots for the built-in Alertmanager, which allows administrators to roll back unintentional changes or mistakes in the Alertmanager configuration.
- Compare the historical snapshot with the latest configuration to see which changes were made.
Grafana includes built-in support for Alertmanager implementations in Prometheus and Mimir.
Once you add it as a data source, you can use the [Grafana Alerting UI](ref:alerting) to manage silences, contact points, and notification policies.
To switch between Grafana and any configured Alertmanager data sources, you can select your preference from a drop-down option in those databases' data source settings pages.
## Alertmanager implementations
Once you add an Alertmanager as a data source, you can use the `Choose Alertmanager` drop-down on [Grafana Alerting](ref:alerting) to view and manage Alertmanager resources, such as silences, contact points, and notification policies. Additionally, you can enable the Alertmanager to receive Grafana-managed alerts.
The data source supports [Prometheus](https://prometheus.io/) and [Grafana Mimir](/docs/mimir/latest/) (default) implementations of Alertmanager.
You can specify the implementation in the data source's Settings page.
When using Prometheus, contact points and notification policies are read-only in the Grafana Alerting UI, because it doesn't support updates to the configuration using HTTP API.
For more details about using other Alertmanagers, refer to [Alertmanagers in the Grafana Alerting documentation](ref:configure-alertmanager).
## Configure the data source
## Alertmanager implementations
To configure basic settings for the data source, complete the following steps:
The data source supports [Prometheus](https://prometheus.io/) and [Grafana Mimir](/docs/mimir/latest/) (default) implementations of Alertmanager. You can specify the implementation in the data source's Settings page.
1. Click **Connections** in the left-side menu.
1. Under Your connections, click **Data sources**.
1. Enter `Alertmanager` in the search bar.
1. Click **Alertmanager**.
When using Prometheus, you can manage silences in the Grafana Alerting UI. However, other Alertmanager resources such as contact points, notification policies, and templates are read-only because the Prometheus Alertmanager HTTP API does not support updates for these resources.
The **Settings** tab of the data source is displayed.
## Configure the data source
1. Set the data source's basic configuration options:
To configure basic settings for the data source, complete the following steps:
| **Name** | Sets the name you use to refer to the data source |
| **Default** | Sets whether the data source is pre-selected for new panels and queries |
| **Alertmanager Implementation** | Alertmanager implementation. **Mimir**, **Cortex,** and **Prometheus** are supported |
| **Receive Grafana Alerts** | When enabled the Alertmanager receives alert instances from Grafana-managed alert rules. **Important:** It works only if Grafana alerting is configured to send its alert instances to external Alertmanagers |
| **HTTP URL** | Sets the HTTP protocol, IP, and port of your Alertmanager instance, such as `https://alertmanager.example.org:9093` |
| **Access** | Only **Server** access mode is functional |
| **Name** | Sets the name you use to refer to the data source |
| **Default** | Sets whether the data source is pre-selected for new panels and queries |
| **Alertmanager Implementation** | Alertmanager implementation. **Mimir**, **Cortex,** and **Prometheus** are supported |
| **Receive Grafana Alerts** | When enabled, the Alertmanager can receive Grafana-managed alerts. **Important:** This works only if receiving alerts is enabled for the Alertmanager in the Grafana Alerting Settings page |
| **HTTP URL** | Sets the HTTP protocol, IP, and port of your Alertmanager instance, such as `https://alertmanager.example.org:9093` |
| **Access** | Only **Server** access mode is functional |
grafana-delivery-bot[bot]
6 months ago
committed by