The open and composable observability and data visualization platform. Visualize metrics, logs, and traces from multiple sources like Prometheus, Loki, Elasticsearch, InfluxDB, Postgres and many more.
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
 
 
 
 
 
 
grafana/docs/sources/alerting/alerting-rules/create-data-source-managed-...

9.4 KiB

aliases canonical description keywords labels title weight refs
[../unified-alerting/alerting-rules/create-mimir-loki-managed-rule/ ../unified-alerting/alerting-rules/edit-cortex-loki-namespace-group/ ../unified-alerting/alerting-rules/edit-mimir-loki-namespace-group/ ../alerting-rules/create-mimir-loki-managed-rule/] https://grafana.com/docs/grafana/latest/alerting/alerting-rules/create-data-source-managed-rule/ Configure data source-managed alert rules alert for an external Grafana Mimir or Loki instance [grafana alerting guide rules create] [{products [cloud enterprise oss]}] Configure data source-managed alert rules 200 [{shared-configure-prometheus-data-source-alerting {pattern /docs/grafana/} {destination /docs/grafana/<GRAFANA_VERSION>/datasources/prometheus/configure/}] [{pattern /docs/grafana-cloud/} {destination /docs/grafana-cloud/connect-externally-hosted/data-sources/prometheus/configure/}} {configure-grafana-managed-rules {pattern /docs/grafana/} {destination /docs/grafana/<GRAFANA_VERSION>/alerting/alerting-rules/create-grafana-managed-rule/}] [{pattern /docs/grafana-cloud/} {destination /docs/grafana-cloud/alerting-and-irm/alerting/alerting-rules/create-grafana-managed-rule/notification-policies/}} {notification-policies {pattern /docs/grafana/} {destination /docs/grafana/<GRAFANA_VERSION>/alerting/fundamentals/notifications/notification-policies/}] [{pattern /docs/grafana-cloud/} {destination /docs/grafana-cloud/alerting-and-irm/alerting/fundamentals/notifications/notification-policies/}} {pending-period {pattern /docs/grafana/} {destination /docs/grafana/<GRAFANA_VERSION>/alerting/fundamentals/alert-rule-evaluation/#pending-period}] [{pattern /docs/grafana-cloud/} {destination /docs/grafana-cloud/alerting-and-irm/alerting/fundamentals/alert-rule-evaluation/#pending-period}} {alert-rules {pattern /docs/grafana/} {destination /docs/grafana/<GRAFANA_VERSION>/alerting/fundamentals/alert-rules/}] [{pattern /docs/grafana-cloud/} {destination /docs/grafana-cloud/alerting-and-irm/alerting/fundamentals/alert-rules/}} {alert-rule-labels {pattern /docs/grafana/} {destination /docs/grafana/<GRAFANA_VERSION>/alerting/fundamentals/alert-rules/annotation-label/#labels}] [{pattern /docs/grafana-cloud/} {destination /docs/grafana-cloud/alerting-and-irm/alerting/fundamentals/alert-rules/annotation-label/#labels}} {alert-rule-evaluation {pattern /docs/grafana/} {destination /docs/grafana/<GRAFANA_VERSION>/alerting/fundamentals/alert-rule-evaluation/}] [{pattern /docs/grafana-cloud/} {destination /docs/grafana-cloud/alerting-and-irm/alerting/fundamentals/alert-rule-evaluation/}} {shared-provision-alerting-resources {pattern /docs/grafana/} {destination /docs/grafana/<GRAFANA_VERSION>/alerting/set-up/provision-alerting-resources/}] [{pattern /docs/grafana-cloud/} {destination /docs/grafana-cloud/alerting-and-irm/alerting/set-up/provision-alerting-resources/}} {shared-alert-rule-template {pattern /docs/grafana/} {destination /docs/grafana/<GRAFANA_VERSION>/alerting/alerting-rules/templates/}] [{pattern /docs/grafana-cloud/} {destination /docs/grafana-cloud/alerting-and-irm/alerting/alerting-rules/templates/}} {shared-annotations {pattern /docs/grafana/} {destination /docs/grafana/<GRAFANA_VERSION>/alerting/fundamentals/alert-rules/annotation-label/#annotations}] [{pattern /docs/grafana-cloud/} {destination /docs/grafana-cloud/alerting-and-irm/alerting/fundamentals/alert-rules/annotation-label/#annotations}} {shared-link-alert-rules-to-panels {pattern /docs/grafana/} {destination /docs/grafana/<GRAFANA_VERSION>/alerting/alerting-rules/link-alert-rules-to-panels/}] [{pattern /docs/grafana-cloud/} {destination /docs/grafana-cloud/alerting-and-irm/alerting/alerting-rules/link-alert-rules-to-panels/}}]

Configure data source-managed alert rules

Data source-managed alert rules can only be created using Grafana Mimir or Grafana Loki data sources.

The rules are stored within the data source. In a distributed architecture, they can scale horizontally to provide high-availability. For more details, refer to alert rule types.

We recommend using Grafana-managed alert rules whenever possible and opting for data source-managed alert rules when scaling your alerting setup is necessary.

Rules from a Prometheus data source appear in the Data source-managed section of the Alert rules page when Manage alerts via Alerting UI is enabled.

However, Grafana can only create and edit data source-managed rules for Mimir and Loki, not for a Prometheus instance.

To create or edit data source-managed alert rules, follow these instructions.

Before you begin

Verify that you have write permission to the Mimir or Loki data source. Otherwise, you cannot create or update data source-managed alert rules.

Enable the Ruler API

For more information, refer to the Mimir Ruler API or Loki Ruler API.

  • Mimir - use the /prometheus prefix. The Prometheus data source supports both Grafana Mimir and Prometheus, and Grafana expects that both the Query API and Ruler API are under the same URL. You cannot provide a separate URL for the Ruler API.

  • Loki - The local rule storage type, default for the Loki data source, supports only viewing of rules. To edit rules, configure one of the other rule storage types.

Permissions

Alert rules for Mimir or Loki instances can be edited or deleted by users with Editor or Admin roles.

If you do not want to manage alert rules for a particular data source, go to its settings and clear the Manage alerts via Alerting UI checkbox.

Provisioning

Note that if you delete an alert resource created in the UI, you can no longer retrieve it.

To backup and manage alert rules, you can provision alerting resources using options such as configuration files, Terraform, or the Alerting API.

{{< docs/shared lookup="alerts/configure-alert-rule-name.md" source="grafana" version="<GRAFANA_VERSION>" >}}

Define query and condition

Define a query to get the data you want to measure and a condition that needs to be met before an alert rule fires.

{{% admonition type="note" %}} By default, new alert rules are Grafana-managed. To switch to Data source-managed, follow these instructions. {{% /admonition %}}

  1. Select a Prometheus-based data source from the drop-down list.

    You can also click Open advanced data source picker to find more options.

  2. Enter a PromQL or LogQL query, including the alert condition.

  3. In the Rule type option, select Data source-managed.

  4. Click Preview alerts.

Set alert evaluation behavior

Use alert rule evaluation to determine how frequently an alert rule should be evaluated and how quickly it should change its state.

  1. Select a namespace or click + New namespace.

  2. Select an evaluation group or click + New evaluation group.

    If you are creating a new evaluation group, specify the interval for the group.

    All rules within the same group are evaluated sequentially over the same time interval. You can reorder them from the Alert rules page.

  3. Enter a pending period.

    The pending period is the period in which an alert rule can be in breach of the condition until it fires.

    Once a condition is met, the alert goes into the Pending state. If the condition remains active for the duration specified, the alert transitions to the Firing state, else it reverts to the Normal state.

Configure labels and notifications

Add labels to your alert rules to set which notification policy should handle your firing alert instances.

All alert rules and instances, irrespective of their labels, match the default notification policy. If there are no nested policies, or no nested policies match the labels in the alert rule or alert instance, then the default notification policy is the matching policy.

  1. Add labels if you want to change the way your notifications are routed.

    Add custom labels by selecting existing key-value pairs from the drop down, or add new labels by entering the new key or value.

{{< docs/shared lookup="alerts/configure-notification-message.md" source="grafana" version="<GRAFANA_VERSION>" >}}