61 Commits (a40fa8389388a72e0775e98c2299f870e93adad0)

Author	SHA1	Message	Date
Yuriy Tseretyan	984c95de63	Do not store EvaluationString in Evaluation. (#44606) ... * do not store evaluation string in Evaluation. * reduce number of buckets to store for a single state	3 years ago
George Robinson	5e2280ceee	Add metrics to ngalert scheduler (#44602) ... This pull request adds metrics to the ngalert scheduler so we can see how long it takes to evaluate a tick.	3 years ago
idafurjes	56c3875bb9	Chore: Remove context.TODO (#43458) ... * Remove context.TODO() from services * Fix live test	3 years ago
George Robinson	c932dc959c	Alerting: Add Ref ID to DatasourceNoData and DatasourceError alerts (#42630)	4 years ago
gotjosh	357e9ed1ea	Alerting: Fix Annotation Creation when the alerting state changes (#42479) ... * Fix Annotation creation - Remove validation of panelID, now annotations are created irrespective on whether they're attached to a panel or not. - Alwasy attach the annotation to an AlertID * Fix annotation creation * fix tests	4 years ago
Santiago	a21d1e50f1	avoid template execution errors on missing values (#41617)	4 years ago
gotjosh	dd5a2e5128	Alerting: Clear alerting rule evaluation errors after intermittent failures (#42386) ... * Alerting: Clear alerting rule evaluation errors after intermittent failures When an alert transitioned in a way that `alerting -> error -> (alerting|nodata)`, the error provided by the `error` state would never be cleared thus the API and UI would show the health as an error.	4 years ago
George Robinson	1b26d4d88e	Alerting: Create DatasourceError alert if evaluation returns error (#41869) ... * Alerting: Create DatasourceError alert if evaluation returns error * Alerting: Add docs for DatasourceError alert * Alerting: Fix DatasourceError alert does not have dashboard_uid label * Alerting: Add break when datasource_uid found * Alerting: Update TestProcessEvalResults	4 years ago
Santiago	a45e4ff73f	graphLink and tableLink template functions (#41369) ... * graphLink and tableLink functions, docs updated * Code review changes * extract query struct outside of graphLink and tableLink functions * Fix docs * Update docs/sources/alerting/unified-alerting/alerting-rules/alert-annotation-label.md Co-authored-by: Jean-Philippe Quéméner <JohnnyQQQQ@users.noreply.github.com> * Update docs/sources/alerting/unified-alerting/alerting-rules/alert-annotation-label.md Co-authored-by: Jean-Philippe Quéméner <JohnnyQQQQ@users.noreply.github.com> * Update docs/sources/alerting/unified-alerting/alerting-rules/alert-annotation-label.md Co-authored-by: achatterjee-grafana <70489351+achatterjee-grafana@users.noreply.github.com> * Update docs/sources/alerting/unified-alerting/alerting-rules/alert-annotation-label.md Co-authored-by: achatterjee-grafana <70489351+achatterjee-grafana@users.noreply.github.com> * Fix linting errors Co-authored-by: Jean-Philippe Quéméner <JohnnyQQQQ@users.noreply.github.com> Co-authored-by: achatterjee-grafana <70489351+achatterjee-grafana@users.noreply.github.com>	4 years ago
Yuriy Tseretyan	610643a668	Alerting: Special alert instance if rule is in state NoData (#40540) ... * do not suppress NoData state * extract conversion of state to postable alert + tests * create a special alert instance if nodata * use NoData when converting from Keep Last State instead of Alerting * add silence during migration if NoData is mapped to KeepLastState.	4 years ago
Yuriy Tseretyan	1b5b747885	Alerting: Additional Tests for State Manager (#41291) ... * rename fakeInstanceStore to FakeInstanceStore * update test for state manager to initialize instance store with FakeInstanceStore	4 years ago
Yuriy Tseretyan	5836def6c2	Alerting: declare constants for __dashboardUid__ and __panelId__ literals (#39976)	4 years ago
idafurjes	2759b16ef5	Chore: Add context for dashboards (#39844) ... * Add context for dashboards * Remove GetDashboardCtx * Remove ctx.TODO	4 years ago
Santiago	562cd9e44e	Alerting template functions (#39261) ... * Alerting: (wip) add template funcs * Alerting: (wip) numeric template functions * Alerting: (wip) template functions * Test for the "args" function * Alerting: (wip) Documentation for template functions * Alerting: template functions - refactor * code review changes * disable linter error * Use Prometheus implementation of TemplateExpander * Update docs/sources/alerting/unified-alerting/alerting-rules/create-grafana-managed-rule.md Co-authored-by: achatterjee-grafana <70489351+achatterjee-grafana@users.noreply.github.com> * change templateCaptureValue to support using template functions * Update pkg/services/ngalert/state/template.go Co-authored-by: gotjosh <josue.abreu@gmail.com> * Test and documentation added for reReplaceAll template function * complete missing functions, documentation and tests * Use the alert instance's evaluation time for expanding the template * strvalue graphlink and tablelink functions * delete duplicate test * make strvalue return an empty string Co-authored-by: achatterjee-grafana <70489351+achatterjee-grafana@users.noreply.github.com> Co-authored-by: gotjosh <josue.abreu@gmail.com>	4 years ago
Sofia Papagiannaki	012d4f0905	Alerting: Remove `ngalert` feature toggle and introduce two new settings for enabling Grafana 8 alerts and disabling them for specific organisations (#38746) ... * Remove `ngalert` feature toggle * Update frontend Remove all references of ngalert feature toggle * Update docs * Disable unified alerting for specific orgs * Add backend tests * Apply suggestions from code review Co-authored-by: achatterjee-grafana <70489351+achatterjee-grafana@users.noreply.github.com> * Disabled unified alerting by default * Ensure backward compatibility with old ngalert feature toggle * Apply suggestions from code review Co-authored-by: gotjosh <josue@grafana.com>	4 years ago
George Robinson	27609dc2c5	Fix alerts with evaluation interval more than 30 seconds resolving in Alertmanager (#39513)	4 years ago
gotjosh	fcbcfd232b	Alerting: Move spammy log line to debug in the state manager (#39410)	4 years ago
Santiago	c3cf95f383	Revert "Alerting: add template funcs (#38404)" (#39258) ... This reverts commit d6fb0181fb.	4 years ago
Santiago	d6fb0181fb	Alerting: add template funcs (#38404) ... * Alerting: (wip) add template funcs * Alerting: (wip) numeric template functions * Alerting: (wip) template functions * Test for the "args" function * Alerting: (wip) Documentation for template functions * Alerting: template functions - refactor * code review changes * disable linter error * Use Prometheus implementation of TemplateExpander * Update docs/sources/alerting/unified-alerting/alerting-rules/create-grafana-managed-rule.md Co-authored-by: achatterjee-grafana <70489351+achatterjee-grafana@users.noreply.github.com> Co-authored-by: achatterjee-grafana <70489351+achatterjee-grafana@users.noreply.github.com>	4 years ago
Marcus Efraimsson	fa9857499b	Chore: GetDashboardQuery should be dispatched using DispatchCtx (#36877) ... * Chore: GetDashboardQuery should be dispatched using DispatchCtx * Fix after merge * Changes after review * Various fixes * Use GetDashboardCtx function instead of GetDashboard	4 years ago
gotjosh	a2f4344bf2	Alerting: Refactor & fix unified alerting metrics structure (#39151) ... * Alerting: Refactor & fix unified alerting metrics structure Fixes and refactors the metrics structure we have for the ngalert service. Now, each component has its own metric struct that includes the JUST the metrics it uses. Additionally, I have fixed the configuration metrics and added new metrics to determine if we have discovered and started all the necessary configurations of an instance. This allows us to alert on `grafana_alerting_discovered_configurations - grafana_alerting_active_configurations != 0` to know whether an alertmanager instance did not start successfully.	4 years ago
George Robinson	5caf6cb369	Change templateCaptureValue to support using template functions (#38766) ... * Change templateCaptureValue to support using template functions This commit changes templateCaptureValue to use float64 for the value instead of *float64. This change means that annotations and labels can use the float64 value with functions such as printf and avoid having to check for nil. It also means that absent values are now printed as 0. * Use math.NaN() instead of 0 for absent value	4 years ago
gotjosh	dd502f22eb	Alerting: Fix alert flapping in the internal alertmanager (#38648) ... * Alerting: Fix alert flapping in the alertmanager fixes a bug that caused Alerts that are evaluated at low intervals (sub 1 minute), to flap in the Alertmanager. Mostly due to a combination of `EndsAt` and resend delay. The Alertmanager uses `EndsAt` as a heuristic to know whenever it should resolve a firing alert, in the case that it hasn't heard back from the alert generation system. Because grafana sent the alert with an `EndsAt` which is equal to the `For` of the alert itself, and we had a hard-coded 1 minute re-send delay (only applicable to firing alerts) this meant that a firing alert would resolve in the Alertmanager before we re-notify that it still firing. This commit, increases the `EndsAt` by 3x the the resend delay or alert interval (depending on which one is higher). The resendDelay has been decreased to 30 seconds.	4 years ago
Arve Knudsen	78596a6756	Migrate to Wire for dependency injection (#32289) ... Fixes #30144 Co-authored-by: dsotirakis <sotirakis.dim@gmail.com> Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> Co-authored-by: Ida Furjesova <ida.furjesova@grafana.com> Co-authored-by: Jack Westbrook <jack.westbrook@gmail.com> Co-authored-by: Will Browne <wbrowne@users.noreply.github.com> Co-authored-by: Leon Sorokin <leeoniya@gmail.com> Co-authored-by: Andrej Ocenas <mr.ocenas@gmail.com> Co-authored-by: spinillos <selenepinillos@gmail.com> Co-authored-by: Karl Persson <kalle.persson@grafana.com> Co-authored-by: Leonard Gram <leo@xlson.com>	4 years ago
Kyle Brandt	aef67994a1	Annotations: Fix alerting annotation coloring (#37412) ... Co-authored-by: Ryan McKinley <ryantxu@gmail.com>	4 years ago
Kyle Brandt	aa904a5a04	NGAlert: Send resolve signal to alertmanager on alerting -> Normal (#37363)	4 years ago
David Parrott	b5f464412d	Alerting: automatically remove stale alerting states (#36767) ... * initial attempt at automatic removal of stale states * test case, need espected states * finish unit test * PR feedback * still multiply by time.second * pr feedback	4 years ago
George Robinson	2f4c893cf3	Expand the value string in annotations and labels of alerts (#37051) ... This commit makes it possible to use the value string in annotations and labels for alerts with "{{ $value }}"	4 years ago
George Robinson	456dac1303	Expand the value of math and reduce expressions in annotations and labels (#36611) ... * Expand the value of math and reduce expressions in annotations and labels This commit makes it possible to use the values of reduce and math expressions in annotations and labels via their RefIDs. It uses the Stringer interface to ensure that "{{ $values.A }}" still prints the value in decimal format while also making the labels for each RefID available with "{{ $values.A.Labels }}" and the float64 value with "{{ $values.A.Value }}"	4 years ago
David Parrott	19f18bcecc	Alerting: annotation on state change (#36535) ... * WIP * Add annotation on alert state change * move annotation creation to manager * praise the linter! * add debug msg when creating annotation	4 years ago
David Parrott	310d3ebe3d	change template expansion missing value handling (#36679)	4 years ago
gotjosh	a86ad1190c	Alerting: Refactor state manager as a dependency (#36513) ... * Alerting: Refactor state manager as a dependency Within the scheduler, the state manager was being passed around a certain number of functions. I've introduced it as a dependency to keep the "service" interfaces as clean and homogeneous as possible. This is relevant, because I'm going to introduce live reload of these components as part of my next PR and it is better if dependencies are self-contained. * remove unused functions * Fix a few more tests * Make sure the `stateManager` is declared before the schedule	4 years ago
David Parrott	4732f832f7	Alerting: recalculate EndsAt (#35830) ... * setEndsAt * one more test case * add should clause to tests	4 years ago
Ganesh Vernekar	dcd4bf1615	Alerting: Fill the empty GeneratorURL (#35740) ... Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>	4 years ago
Ganesh Vernekar	8417088969	Alerting: Expand `{{$labels.xyz}}` template in labels and annotations (#35159) ... * Alerting: Expand `{{$labels.xyz}}` template in labels and annotations Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com> * Fix annotation not updating for same alert Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>	4 years ago
David Parrott	20d356947c	set state correctly and test (#34680)	4 years ago
David Parrott	7a83d1f9ff	Alerting resend delay for sending to notifiers (#34312) ... * adds resend delay to avoid saturating notifier * correct method signatures * pr feedback	4 years ago
David Parrott	25485100b0	Alerting: Trim results when at processing instead of on ticker (#34248) ... * Trim results when at processing instead of on ticker * User RWMutex correctly * remove comment	4 years ago
David Parrott	bbb7bbf891	Alerting: Remove back end logic for supporting KeepLastState (#34242) ... * Removed back end logic for supporting KeepLastState * Map keep_state correctly in migrations	4 years ago
Kyle Brandt	63b2dd06a5	Alerting: Set "value" with evalmatches in G Managed (#34075) ... When, and currently only when using a classic condition, evaluation information is added (which is like the EvalMatches from dashboard alerting). This is returned via the API and can be included in notifications by reading the `__value__` label attached `.Alerts` in the template. It is a string.	4 years ago
Owen Diehl	1367f7171e	Alerting/ruler metrics (#34144) ... * adds active configurations metric * rule evaluation metrics * ruler metrics * pr feedback	4 years ago
Kyle Brandt	babb17afd6	Alerting/Chore: Move tests from tests package (#34059) ... Instead put in package folder but with package name suffixed with _test This enables code coverage within the pkg while still allow the tests to operate from external to package perspective (only exported things).	4 years ago
Kyle Brandt	fae093bbe2	Alerting: Fix state cache getOrCreate panic (#33777)	4 years ago
David Parrott	b1a8c67689	Alerting return evaluation errors to /rules (#33663) ... * Set and return errors produced by evaluation results * test fixup	4 years ago
David Parrott	39099bf3c0	Alerting nested state cache (#33666) ... * nest cache by orgID, ruleUID, stateID * update accessors to use new cache structure * test and linter fixup * fix panic Co-authored-by: Kyle Brandt <kyle@grafana.com> * add comment to identify what's going on with nested maps in cache Co-authored-by: Kyle Brandt <kyle@grafana.com>	4 years ago
Kyle Brandt	48358efc13	Alerting: remove State cache entries on Ruler Delete (#33638) ... for https://github.com/grafana/alerting-squad/issues/133	4 years ago
Owen Diehl	070627d11e	better handle metrics for state transitions (#33648)	4 years ago
Kyle Brandt	7823842c5d	Alerting: Load annotations from rule into State cache (#33542) ... for https://github.com/grafana/alerting-squad/issues/127	4 years ago
Owen Diehl	5e48b54549	Alerting/metrics (#33547) ... * moves alerting metrics to their own pkg * adds grafana_alerting_alerts (by state) metric * alerts_received_{total,invalid} * embed alertmanager alerting struct in ng metrics & remove duplicated notification metrics (already embed alertmanager notifier metrics) * use silence metrics from alertmanager lib * fix - manager has metrics * updates ngalert tests * comment lint Signed-off-by: Owen Diehl <ow.diehl@gmail.com> * cleaner prom registry code * removes ngalert global metrics * new registry use in all tests * ngalert metrics impl service, hack testinfra code to prevent duplicate metric registrations * nilmetrics unexported	4 years ago
Sofia Papagiannaki	1e380e869e	[Alerting]: some fixes (#33538) ... * Fix fialure when adding state annotations * Fix get org rules API Do not fail response if user has no access to view a namespace. Do not include the namespace in the response instead. * lint	4 years ago