The open and composable observability and data visualization platform. Visualize metrics, logs, and traces from multiple sources like Prometheus, Loki, Elasticsearch, InfluxDB, Postgres and many more.
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
grafana/pkg/services/ngalert/api/lotex_ruler.go

223 lines
5.1 KiB

package api
import (
"bytes"
"fmt"
"net/http"
"net/url"
Inhouse alerting api (#33129) * init * autogens AM route * POST dashboards/db spec * POST alert-notifications spec * fix description * re inits vendor, updates grafana to master * go mod updates * alerting routes * renames to receivers * prometheus endpoints * align config endpoint with cortex, include templates * Change grafana receiver type * Update receivers.go * rename struct to stop swagger thrashing * add rules API * index html * standalone swagger ui html page * Update README.md * Expose GrafanaManagedAlert properties * Some fixes - /api/v1/rules/{Namespace} should return a map - update ExtendedUpsertAlertDefinitionCommand properties * am alerts routes * rename prom swagger section for clarity, remove example endpoints * Add missing json and yaml tags * folder perms * make folders POST again * fix grafana receiver type * rename fodler->namespace for perms * make ruler json again * PR fixes * silences * fix Ok -> Ack * Add id to POST /api/v1/silences (#9) Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Add POST /api/v1/alerts (#10) Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * fix silences * Add testing endpoints * removes grpc replace directives * [wip] starts validation * pkg cleanup * go mod tidy * ignores vendor dir * Change response type for Cortex/Loki alerts * receiver unmarshaling tests * ability to split routes between AM & Grafana * api marshaling & validation * begins work on routing lib * [hack] ignores embedded field in generation * path specific datasource for alerting * align endpoint names with cloud * single route per Alerting config * removes unused routing pkg * regens spec * adds datasource param to ruler/prom route paths * Modifications for supporting migration * Apply suggestions from code review * hack for cleaning circular refs in swagger definition * generates files * minor fixes for prom endpoints * decorate prom apis with required: true where applicable * Revert "generates files" This reverts commit ef7e97558477d79bcad416e043b04dbd04a2c8f7. * removes server autogen * Update imported structs from ngalert * Fix listing rules response * Update github.com/prometheus/common dependency * Update get silence response * Update get silences response * adds ruler validation & backend switching * Fix GET /alertmanager/{DatasourceId}/config/api/v1/alerts response * Distinct gettable and postable grafana receivers * Remove permissions routes * Latest JSON specs * Fix testing routes * inline yaml annotation on apirulenode * yaml test & yamlv3 + comments * Fix yaml annotations for embedded type * Rename DatasourceId path parameter * Implement Backend.String() * backend zero value is a real backend * exports DiscoveryBase * Fix GO initialisms * Silences: Use PostableSilence as the base struct for creating silences * Use type alias instead of struct embedding * More fixes to alertmanager silencing routes * post and spec JSONs * Split rule config to postable/gettable * Fix empty POST /silences payload Recreating the generated JSON specs fixes the issue without further modifications * better yaml unmarshaling for nested yaml docs in cortex-am configs * regens spec * re-adds config.receivers * omitempty to align with prometheus API behavior * Prefix routes with /api * Update Alertmanager models * Make adjustments to follow the Alertmanager API * ruler: add for and annotations to grafana alert (#45) * Modify testing API routes * Fix grafana rule for field type * Move PostableUserConfig validation to this library * Fix PostableUserConfig YAML encoding/decoding * Use common fields for grafana and lotex rules * Add namespace id in GettableGrafanaRule * Apply suggestions from code review * fixup * more changes * Apply suggestions from code review * aligns structure pre merge * fix new imports & tests * updates tooling readme * goimports * lint * more linting!! * revive lint Co-authored-by: Sofia Papagiannaki <papagian@gmail.com> Co-authored-by: Domas <domasx2@gmail.com> Co-authored-by: Sofia Papagiannaki <papagian@users.noreply.github.com> Co-authored-by: Ganesh Vernekar <15064823+codesome@users.noreply.github.com> Co-authored-by: gotjosh <josue@grafana.com> Co-authored-by: David Parrott <stomp.box.yo@gmail.com> Co-authored-by: Kyle Brandt <kyle@grafana.com>
4 years ago
apimodels "github.com/grafana/grafana/pkg/services/ngalert/api/tooling/definitions"
"github.com/grafana/grafana/pkg/web"
"gopkg.in/yaml.v3"
"github.com/grafana/grafana/pkg/api/response"
"github.com/grafana/grafana/pkg/infra/log"
"github.com/grafana/grafana/pkg/models"
)
Alerting: Add support to distinguish Prometheus datasource subtypes (Mimir, Cortex and Vanilla Prometheus) (#46771) * Add basic UI for custom ruler URL * Add build info fetching for alerting data sources * Add keeping data sources build info in the store * Use data source build info to construct data source urls * Remove unused code * Add custom ruler support in prometheus api calls * Migrate actions * Use thunk condition to prevent multiple data source buildinfo fetches * Unify prom and ruler rules loading * Upgrade RuleEditor tests * Upgrade RuleList tests * Upgrade PanelAlertTab tests * Upgrade actions tests * Build info refactoring * Get rid of lotex ruler support action * Add prom ruler availability checking when the buildinfo is not available * Add rulerUrlBuilder tests * Improve prometheus data source validation, small build info refactoring * Change prefix based on Prometheus subtype * Use the correct path * Revert config routing * Add deprecation notice for /api/prom prefix * Add tests to the datasource subtype * Remove custom ruler support * Remove deprecation notice * Prevent fetching ruler rules when ruler api is not available * Add build info tests * Unify naming of ruler methods * Fix test * Change buildinfo data source validation * Use strings for subtype params and unveil mimir * organise imports * frontend changes and wordsmithing * fix test suite * add a nicer verbose message for prometheus datasources * detect Mimir datasource * fix test * fix buildinfo test for Mimir * shrink vectors * add some code documentation * DRY prepareRulesFilterQueryParams * clarify that Prometheus does not support managing rules * Improve buildinfo error handling Co-authored-by: gotjosh <josue.abreu@gmail.com> Co-authored-by: gillesdemey <gilles.de.mey@gmail.com>
3 years ago
const (
Prometheus = "prometheus"
Cortex = "cortex"
Mimir = "mimir"
)
const (
PrometheusDatasourceType = "prometheus"
LokiDatasourceType = "loki"
mimirPrefix = "/config/v1/rules"
prometheusPrefix = "/rules"
lokiPrefix = "/api/prom/rules"
subtypeQuery = "subtype"
)
var dsTypeToRulerPrefix = map[string]string{
Alerting: Add support to distinguish Prometheus datasource subtypes (Mimir, Cortex and Vanilla Prometheus) (#46771) * Add basic UI for custom ruler URL * Add build info fetching for alerting data sources * Add keeping data sources build info in the store * Use data source build info to construct data source urls * Remove unused code * Add custom ruler support in prometheus api calls * Migrate actions * Use thunk condition to prevent multiple data source buildinfo fetches * Unify prom and ruler rules loading * Upgrade RuleEditor tests * Upgrade RuleList tests * Upgrade PanelAlertTab tests * Upgrade actions tests * Build info refactoring * Get rid of lotex ruler support action * Add prom ruler availability checking when the buildinfo is not available * Add rulerUrlBuilder tests * Improve prometheus data source validation, small build info refactoring * Change prefix based on Prometheus subtype * Use the correct path * Revert config routing * Add deprecation notice for /api/prom prefix * Add tests to the datasource subtype * Remove custom ruler support * Remove deprecation notice * Prevent fetching ruler rules when ruler api is not available * Add build info tests * Unify naming of ruler methods * Fix test * Change buildinfo data source validation * Use strings for subtype params and unveil mimir * organise imports * frontend changes and wordsmithing * fix test suite * add a nicer verbose message for prometheus datasources * detect Mimir datasource * fix test * fix buildinfo test for Mimir * shrink vectors * add some code documentation * DRY prepareRulesFilterQueryParams * clarify that Prometheus does not support managing rules * Improve buildinfo error handling Co-authored-by: gotjosh <josue.abreu@gmail.com> Co-authored-by: gillesdemey <gilles.de.mey@gmail.com>
3 years ago
PrometheusDatasourceType: prometheusPrefix,
LokiDatasourceType: lokiPrefix,
}
var subtypeToPrefix = map[string]string{
Prometheus: prometheusPrefix,
Cortex: prometheusPrefix,
Mimir: mimirPrefix,
}
type LotexRuler struct {
log log.Logger
*AlertingProxy
}
func NewLotexRuler(proxy *AlertingProxy, log log.Logger) *LotexRuler {
return &LotexRuler{
log: log,
AlertingProxy: proxy,
}
}
func (r *LotexRuler) RouteDeleteNamespaceRulesConfig(ctx *models.ReqContext, namespace string) response.Response {
legacyRulerPrefix, err := r.validateAndGetPrefix(ctx)
if err != nil {
return ErrResp(500, err, "")
}
return r.withReq(
ctx,
http.MethodDelete,
withPath(
*ctx.Req.URL,
fmt.Sprintf("%s/%s", legacyRulerPrefix, namespace),
),
nil,
messageExtractor,
nil,
)
}
func (r *LotexRuler) RouteDeleteRuleGroupConfig(ctx *models.ReqContext, namespace string, group string) response.Response {
legacyRulerPrefix, err := r.validateAndGetPrefix(ctx)
if err != nil {
return ErrResp(500, err, "")
}
return r.withReq(
ctx,
http.MethodDelete,
withPath(
*ctx.Req.URL,
fmt.Sprintf(
"%s/%s/%s",
legacyRulerPrefix,
namespace,
group,
),
),
nil,
messageExtractor,
nil,
)
}
func (r *LotexRuler) RouteGetNamespaceRulesConfig(ctx *models.ReqContext, namespace string) response.Response {
legacyRulerPrefix, err := r.validateAndGetPrefix(ctx)
if err != nil {
return ErrResp(500, err, "")
}
return r.withReq(
ctx,
http.MethodGet,
withPath(
*ctx.Req.URL,
fmt.Sprintf(
"%s/%s",
legacyRulerPrefix,
namespace,
),
),
nil,
yamlExtractor(apimodels.NamespaceConfigResponse{}),
nil,
)
}
func (r *LotexRuler) RouteGetRulegGroupConfig(ctx *models.ReqContext, namespace string, group string) response.Response {
legacyRulerPrefix, err := r.validateAndGetPrefix(ctx)
if err != nil {
return ErrResp(500, err, "")
}
return r.withReq(
ctx,
http.MethodGet,
withPath(
*ctx.Req.URL,
fmt.Sprintf(
"%s/%s/%s",
legacyRulerPrefix,
namespace,
group,
),
),
nil,
yamlExtractor(&apimodels.GettableRuleGroupConfig{}),
nil,
)
}
func (r *LotexRuler) RouteGetRulesConfig(ctx *models.ReqContext) response.Response {
legacyRulerPrefix, err := r.validateAndGetPrefix(ctx)
if err != nil {
return ErrResp(500, err, "")
}
return r.withReq(
ctx,
http.MethodGet,
withPath(
*ctx.Req.URL,
legacyRulerPrefix,
),
nil,
yamlExtractor(apimodels.NamespaceConfigResponse{}),
nil,
)
}
func (r *LotexRuler) RoutePostNameRulesConfig(ctx *models.ReqContext, conf apimodels.PostableRuleGroupConfig, ns string) response.Response {
legacyRulerPrefix, err := r.validateAndGetPrefix(ctx)
if err != nil {
return ErrResp(500, err, "")
}
yml, err := yaml.Marshal(conf)
if err != nil {
return ErrResp(500, err, "Failed marshal rule group")
}
u := withPath(*ctx.Req.URL, fmt.Sprintf("%s/%s", legacyRulerPrefix, ns))
return r.withReq(ctx, http.MethodPost, u, bytes.NewBuffer(yml), jsonExtractor(nil), nil)
}
func (r *LotexRuler) validateAndGetPrefix(ctx *models.ReqContext) (string, error) {
datasourceUID := web.Params(ctx.Req)[":DatasourceUID"]
if datasourceUID == "" {
return "", fmt.Errorf("datasource UID is invalid")
}
ds, err := r.DataProxy.DataSourceCache.GetDatasourceByUID(ctx.Req.Context(), datasourceUID, ctx.SignedInUser, ctx.SkipCache)
if err != nil {
return "", err
}
// Validate URL
if ds.Url == "" {
return "", fmt.Errorf("URL for this data source is empty")
}
prefix, ok := dsTypeToRulerPrefix[ds.Type]
if !ok {
return "", fmt.Errorf("unexpected datasource type. expecting loki or prometheus")
}
Alerting: Add support to distinguish Prometheus datasource subtypes (Mimir, Cortex and Vanilla Prometheus) (#46771) * Add basic UI for custom ruler URL * Add build info fetching for alerting data sources * Add keeping data sources build info in the store * Use data source build info to construct data source urls * Remove unused code * Add custom ruler support in prometheus api calls * Migrate actions * Use thunk condition to prevent multiple data source buildinfo fetches * Unify prom and ruler rules loading * Upgrade RuleEditor tests * Upgrade RuleList tests * Upgrade PanelAlertTab tests * Upgrade actions tests * Build info refactoring * Get rid of lotex ruler support action * Add prom ruler availability checking when the buildinfo is not available * Add rulerUrlBuilder tests * Improve prometheus data source validation, small build info refactoring * Change prefix based on Prometheus subtype * Use the correct path * Revert config routing * Add deprecation notice for /api/prom prefix * Add tests to the datasource subtype * Remove custom ruler support * Remove deprecation notice * Prevent fetching ruler rules when ruler api is not available * Add build info tests * Unify naming of ruler methods * Fix test * Change buildinfo data source validation * Use strings for subtype params and unveil mimir * organise imports * frontend changes and wordsmithing * fix test suite * add a nicer verbose message for prometheus datasources * detect Mimir datasource * fix test * fix buildinfo test for Mimir * shrink vectors * add some code documentation * DRY prepareRulesFilterQueryParams * clarify that Prometheus does not support managing rules * Improve buildinfo error handling Co-authored-by: gotjosh <josue.abreu@gmail.com> Co-authored-by: gillesdemey <gilles.de.mey@gmail.com>
3 years ago
// If the datasource is Loki, there's nothing else for us to do - it doesn't have subtypes.
if ds.Type == LokiDatasourceType {
return prefix, nil
}
// A Prometheus datasource, can have many subtypes: Cortex, Mimir and vanilla Prometheus.
// Based on these subtypes, we want to use a different proxying path.
subtype := ctx.Query(subtypeQuery)
subTypePrefix, ok := subtypeToPrefix[subtype]
if !ok {
r.log.Debug(
"Unable to determine prometheus datasource subtype, using default prefix",
"datasource", ds.Uid, "datasourceType", ds.Type, "subtype", subtype, "prefix", prefix)
Alerting: Add support to distinguish Prometheus datasource subtypes (Mimir, Cortex and Vanilla Prometheus) (#46771) * Add basic UI for custom ruler URL * Add build info fetching for alerting data sources * Add keeping data sources build info in the store * Use data source build info to construct data source urls * Remove unused code * Add custom ruler support in prometheus api calls * Migrate actions * Use thunk condition to prevent multiple data source buildinfo fetches * Unify prom and ruler rules loading * Upgrade RuleEditor tests * Upgrade RuleList tests * Upgrade PanelAlertTab tests * Upgrade actions tests * Build info refactoring * Get rid of lotex ruler support action * Add prom ruler availability checking when the buildinfo is not available * Add rulerUrlBuilder tests * Improve prometheus data source validation, small build info refactoring * Change prefix based on Prometheus subtype * Use the correct path * Revert config routing * Add deprecation notice for /api/prom prefix * Add tests to the datasource subtype * Remove custom ruler support * Remove deprecation notice * Prevent fetching ruler rules when ruler api is not available * Add build info tests * Unify naming of ruler methods * Fix test * Change buildinfo data source validation * Use strings for subtype params and unveil mimir * organise imports * frontend changes and wordsmithing * fix test suite * add a nicer verbose message for prometheus datasources * detect Mimir datasource * fix test * fix buildinfo test for Mimir * shrink vectors * add some code documentation * DRY prepareRulesFilterQueryParams * clarify that Prometheus does not support managing rules * Improve buildinfo error handling Co-authored-by: gotjosh <josue.abreu@gmail.com> Co-authored-by: gillesdemey <gilles.de.mey@gmail.com>
3 years ago
return prefix, nil
}
r.log.Debug("Determined prometheus datasource subtype",
"datasource", ds.Uid, "datasourceType", ds.Type, "subtype", subtype)
Alerting: Add support to distinguish Prometheus datasource subtypes (Mimir, Cortex and Vanilla Prometheus) (#46771) * Add basic UI for custom ruler URL * Add build info fetching for alerting data sources * Add keeping data sources build info in the store * Use data source build info to construct data source urls * Remove unused code * Add custom ruler support in prometheus api calls * Migrate actions * Use thunk condition to prevent multiple data source buildinfo fetches * Unify prom and ruler rules loading * Upgrade RuleEditor tests * Upgrade RuleList tests * Upgrade PanelAlertTab tests * Upgrade actions tests * Build info refactoring * Get rid of lotex ruler support action * Add prom ruler availability checking when the buildinfo is not available * Add rulerUrlBuilder tests * Improve prometheus data source validation, small build info refactoring * Change prefix based on Prometheus subtype * Use the correct path * Revert config routing * Add deprecation notice for /api/prom prefix * Add tests to the datasource subtype * Remove custom ruler support * Remove deprecation notice * Prevent fetching ruler rules when ruler api is not available * Add build info tests * Unify naming of ruler methods * Fix test * Change buildinfo data source validation * Use strings for subtype params and unveil mimir * organise imports * frontend changes and wordsmithing * fix test suite * add a nicer verbose message for prometheus datasources * detect Mimir datasource * fix test * fix buildinfo test for Mimir * shrink vectors * add some code documentation * DRY prepareRulesFilterQueryParams * clarify that Prometheus does not support managing rules * Improve buildinfo error handling Co-authored-by: gotjosh <josue.abreu@gmail.com> Co-authored-by: gillesdemey <gilles.de.mey@gmail.com>
3 years ago
return subTypePrefix, nil
}
func withPath(u url.URL, newPath string) *url.URL {
// TODO: handle path escaping
u.Path = newPath
return &u
}