104 Commits (2e2e89a81663dfdab284e945bc508c33a6eadddb)

Author	SHA1	Message	Date
Ryan McKinley	429da7fd68	SQL/Blob: Add support for blob storage to SQL backend (#98192)	1 year ago
Dave Henderson	ced7a84f8b	chore(metrics): Add metrics & traces to DB migration (#97181) ... Signed-off-by: Dave Henderson <dave.henderson@grafana.com>	1 year ago
Jeff Levin	a21a232a8e	Revert read replica POC (#93551) ... * Revert "chore: add replDB to team service (#91799)" This reverts commit c6ae2d7999. * Revert "experiment: use read replica for Get and Find Dashboards (#91706)" This reverts commit 54177ca619. * Revert "QuotaService: refactor to use ReplDB for Get queries (#91333)" This reverts commit 299c142f6a. * Revert "refactor replCfg to look more like plugins/plugin config (#91142)" This reverts commit ac0b4bb34d. * Revert "chore (replstore): fix registration with multiple sql drivers, again (#90990)" This reverts commit daedb358dd. * Revert "Chore (sqlstore): add validation and testing for repl config (#90683)" This reverts commit af19f039b6. * Revert "ReplStore: Add support for round robin load balancing between multiple read replicas (#90530)" This reverts commit 27b52b1507. * Revert "DashboardStore: Use ReplDB and get dashboard quotas from the ReadReplica (#90235)" This reverts commit 8a6107cd35. * Revert "accesscontrol service read replica (#89963)" This reverts commit 77a4869fca. * Revert "Fix: add mapping for the new mysqlRepl driver (#89551)" This reverts commit ab5a079bcc. * Revert "fix: sql instrumentation dual registration error (#89508)" This reverts commit d988f5c3b0. * Revert "Experimental Feature Toggle: databaseReadReplica (#89232)" This reverts commit 50244ed4a1.	1 year ago
Kristin Laemmert	daedb358dd	chore (replstore): fix registration with multiple sql drivers, again (#90990) ... * replstore: fix registration with multiple sql drivers, again * only compile regex once	1 year ago
Kristin Laemmert	ab5a079bcc	Fix: add mapping for the new mysqlRepl driver (#89551)	2 years ago
Dan Cech	5dfe4cf407	Storage: Watch tests (#85496) ... * basic watch tests working * refactor to read previous event within poller * add watch test files * cleanup * watch tests passing * debug cleanup * special handling for canceled context * cleanup * fix wire * fix sqlite_sequence cleanup issue * move watch tests to integration, wait for provisioned dashboards * handle context deadline exceeded and eof errors * add comment about sleep	2 years ago
Dan Cech	1f1461734c	Storage: Add support for sortBy selector (#80680) ... * add support for sortBy field selector * use label selectors instead of field selectors * set entity_labels on create & update * make entity server integration tests work * test fixes * be more consistent with handling of empty body, meta or status * workaround for database is locked errors during migration * fix double import of sqlite3 * rename functions and tidy up * refactor update * disable integration tests until we can fix the database locking issue	2 years ago
Sofia Papagiannaki	5e88d29814	Folders: Introduce folder service function for fetching folders by org and UIDs that contain optionally the folder full path (#80716) ... * Folders: Expose function for getting all org folders with specific UIDs * Return all org folders if UIDs is empty * Filter out not accessible folders by the user * Modify query to optionally returning a string that contains the UIDs of all parent folders separated by slash.	2 years ago
Arati R	ca9d147a44	Give dialects control over how insert and update queries are performed (#79946) ... * Refactor insert, update * Add separate insert, update methods * Refactor Insert, Update signatures	2 years ago
Charandas	48612063dd	Grafana app platform: an aggregator cmd and package (#79948)	2 years ago
Dan Cech	67b2972052	Chore: add/update sqlstore-related helper functions (#77408) ... * add/update sqlstore-related helper functions * add documentation & tests for InsertQuery and UpdateQuery, make generated SQL deterministic by sorting columns * remove old log line	2 years ago
Marcus Efraimsson	edc34fc836	Instrumentation: Improve instrumentation for database migrations (#74659)	2 years ago
Ryan McKinley	025b2f3011	Chore: use any rather than interface{} (#74066)	2 years ago
Matthew Jacobson	44972d0cd5	SQLStore: Fix Postgres dialect treating "false" migrator default as true (#69353) ... * SQLStore: Fix Postgres dialect treating "false" migrator default as true Previously, when creating a migration you could choose a default value for a new boolean column that looked correct but would be interpreted incorrectly by the Postgres dialect. For example, values such as "false" or "FALSE" would be treated as true by the Postgres dialect. This refactors how migration dialects determine the Default column value for boolean type columns. Each dialect now uses the same base code to parse the Default literal and panics if an unknown value is encountered. So, now AddColumnMigration and AddTableMigration will ensure that across dialects: - The exact same Default literals will be allowed. - The literals are converted to equivalent defaults in their DDL. - An error will be thrown if an invalid literal is provided.	3 years ago
Dan Cech	a6279b2d62	Database: Make dialects independent of xorm Engine (#69955) ... * make dialects independent of xorm Engine * goimports	3 years ago
Sofia Papagiannaki	74e87ccbbd	SQLStore: Align SQLite IsUniqueConstraintViolation() with other backend implementations (#69224) ... * Add integration test for primary key and unique constrain violation * Align SQLite IsUniqueConstraintViolation implementation with other backends	3 years ago
Ryan McKinley	26658d172f	Migrations: Support independent migration log for entity api (#68511)	3 years ago
Ieva	035bf29146	RBAC: Remove the option to disable RBAC and add automated permission migrations for instances that had RBAC disabled (#66652) ... * RBAC: Stop reading enabeld from ini file and always set to true * Migrations: Add a migration for rbac to reset data migrations if rbac was disabled * If rbac was disabled we reset the data and data migrations that rbac has to perform to get it to a correct state * Migrator: Store migration logs on migrator and add function to clear it from the in-memory stored logs * update tests --------- Co-authored-by: Karl Persson <kalle.persson@grafana.com>	3 years ago
Serge Zaitsev	bc2813ef06	Chore: Fix goimports grouping in pkg/services (#62420) ... * fix goimports * fix goimports order	3 years ago
Kyle Brandt	55d2d872ec	Chore: Missed deprecations due to overly broad lint exclusion (#59732)	3 years ago
Ryan McKinley	eb1cc80941	Storage: include SQL implementation (#58018)	3 years ago
Alexander Weaver	ea8c5ce929	SQLStore: Create utility for bulk inserts (#57632) ... * Add batchsize to dialects * Some intermediary progress * Reflection-based implementation * Validate batch size * Implement a few simple tests * Fix linter error * Add debug message when falling back to default batch size	3 years ago
Joe Blubaugh	b476ae62fb	Alerting: Write and Delete multiple alert instances. (#55350) ... Prior to this change, all alert instance writes and deletes happened individually, in their own database transaction. This change batches up writes or deletes for a given rule's evaluation loop into a single transaction before applying it. These new transactions are off by default, guarded by the feature toggle "alertingBigTransactions" Before: ``` goos: darwin goarch: arm64 pkg: github.com/grafana/grafana/pkg/services/ngalert/store BenchmarkAlertInstanceOperations-8 398 2991381 ns/op 1133537 B/op 27703 allocs/op --- BENCH: BenchmarkAlertInstanceOperations-8 util.go:127: alert definition: {orgID: 1, UID: FovKXiRVzm} with title: "an alert definition FTvFXmRVkz" interval: 60 created util.go:127: alert definition: {orgID: 1, UID: foDFXmRVkm} with title: "an alert definition fovFXmRVkz" interval: 60 created util.go:127: alert definition: {orgID: 1, UID: VQvFuigVkm} with title: "an alert definition VwDKXmR4kz" interval: 60 created PASS ok github.com/grafana/grafana/pkg/services/ngalert/store 1.619s ``` After: ``` goos: darwin goarch: arm64 pkg: github.com/grafana/grafana/pkg/services/ngalert/store BenchmarkAlertInstanceOperations-8 1440 816484 ns/op 352297 B/op 6529 allocs/op --- BENCH: BenchmarkAlertInstanceOperations-8 util.go:127: alert definition: {orgID: 1, UID: 302r_igVzm} with title: "an alert definition q0h9lmR4zz" interval: 60 created util.go:127: alert definition: {orgID: 1, UID: 71hrlmR4km} with title: "an alert definition nJ29_mR4zz" interval: 60 created util.go:127: alert definition: {orgID: 1, UID: Cahr_mR4zm} with title: "an alert definition ja2rlmg4zz" interval: 60 created PASS ok github.com/grafana/grafana/pkg/services/ngalert/store 1.383s ``` So we cut time by about 75% and memory allocations by about 60% when storing and deleting 100 instances.	3 years ago
Emil Tullstedt	b287047052	Chore: Upgrade Go to 1.19.1 (#54902) ... * WIP * Set public_suffix to a pre Ruby 2.6 version * we don't need to install python * Stretch->Buster * Bump versions in lib.star * Manually update linter Sort of messy, but the .mod-file need to contain all dependencies that use 1.16+ features, otherwise they're assumed to be compiled with -lang=go1.16 and cannot access generics et al. Bingo doesn't seem to understand that, but it's possible to manually update things to get Bingo happy. * undo reformatting * Various lint improvements * More from the linter * goimports -w ./pkg/ * Disable gocritic * Add/modify linter exceptions * lint + flatten nested list Go 1.19 doesn't support nested lists, and there wasn't an obvious workaround. https://go.dev/doc/comment#lists	3 years ago
Joe Blubaugh	22c937340e	Revert "Alerting: Write and Delete multiple alert instances. (#54072)" (#54885) ... This reverts commit 5e4fd94413.	3 years ago
Joe Blubaugh	5e4fd94413	Alerting: Write and Delete multiple alert instances. (#54072) ... Prior to this change, all alert instance writes and deletes happened individually, in their own database transaction. This change batches up writes or deletes for a given rule's evaluation loop into a single transaction before applying it. Before: ``` goos: darwin goarch: arm64 pkg: github.com/grafana/grafana/pkg/services/ngalert/store BenchmarkAlertInstanceOperations-8 398 2991381 ns/op 1133537 B/op 27703 allocs/op --- BENCH: BenchmarkAlertInstanceOperations-8 util.go:127: alert definition: {orgID: 1, UID: FovKXiRVzm} with title: "an alert definition FTvFXmRVkz" interval: 60 created util.go:127: alert definition: {orgID: 1, UID: foDFXmRVkm} with title: "an alert definition fovFXmRVkz" interval: 60 created util.go:127: alert definition: {orgID: 1, UID: VQvFuigVkm} with title: "an alert definition VwDKXmR4kz" interval: 60 created PASS ok github.com/grafana/grafana/pkg/services/ngalert/store 1.619s ``` After: ``` goos: darwin goarch: arm64 pkg: github.com/grafana/grafana/pkg/services/ngalert/store BenchmarkAlertInstanceOperations-8 1440 816484 ns/op 352297 B/op 6529 allocs/op --- BENCH: BenchmarkAlertInstanceOperations-8 util.go:127: alert definition: {orgID: 1, UID: 302r_igVzm} with title: "an alert definition q0h9lmR4zz" interval: 60 created util.go:127: alert definition: {orgID: 1, UID: 71hrlmR4km} with title: "an alert definition nJ29_mR4zz" interval: 60 created util.go:127: alert definition: {orgID: 1, UID: Cahr_mR4zm} with title: "an alert definition ja2rlmg4zz" interval: 60 created PASS ok github.com/grafana/grafana/pkg/services/ngalert/store 1.383s ``` So we cut time by about 75% and memory allocations by about 60% when storing and deleting 100 instances. This change also updates some of our tests so that they run successfully against postgreSQL - we were using random Int64s, but postgres integers, which our tables use, max out at 2^31-1	3 years ago
Joe Blubaugh	a0f96ed4e1	SQLStore: Support Upserting multiple rows. (#52228) ... This will be used to reduce write load when the alerting system writes a large number of events.	4 years ago
Kat Yang	31630edf0c	Chore: Remove Wrapf (#50128) ... * Chore: Remove Wrapf * Remove all Wrapf refs * Remove last Wrapf ref * Fix lint errors * Remove Wrap and Wrapf definitions * Remove unnecessary colon	4 years ago
Joan López de la Franca Beltran	219e848e73	SQLStore: Prevent migration_id duplicates (#47774) ... * SQLStore: Prevent migration_id duplicates * Migrations: Remove non-executed migration (duplicated id) Co-authored-by: Leonard Gram <leo@xlson.com>	4 years ago
Joan López de la Franca Beltran	5f1305d280	SQLStore: Add deprecation comments for breaking migrations (#49740) ... * Migrator: Extend support to rename columns * SQLStore: Add deprecation comments for breaking migrations	4 years ago
Kat Yang	3c3039f5b3	Chore: Remove Wrap (#50048) ... * Chore: Remove Wrap and Wrapf * Fix: Add error check	4 years ago
Joan López de la Franca Beltran	e43879e55d	Encryption: Add support for multiple data keys per day (#47765) ... * Add database migrations * Use short uids as data key ids * Add support for manual data key rotation * Fix duplicated mutex unlocks * Fix migration * Manage current data keys per name * Adjust key re-encryption and test * Modify rename column migration for MySQL compatibility * Refactor secrets manager and data keys cache * Multiple o11y adjustments * Fix stats query * Apply suggestions from code review Co-authored-by: Tania <yalyna.ts@gmail.com> * Fix linter * Docs: Rotate data encryption keys API endpoint Co-authored-by: Tania <yalyna.ts@gmail.com>	4 years ago
Leonard Gram	f46038ed3a	Search: sort results correctly when using postgres (#46466) ... * Search: sort results correctly when using postgres postgresql puts nulls first while both mysql and sqlite puts them last * linting	4 years ago
Sofia Papagiannaki	d718ee1918	SQLStore: Prevent concurrent migrations (#44101) ... * SQLStore: Prevent concurrent migrations * Hide behind a feature toggle * Configurable locking attempt timeout * Update docs/sources/administration/configuration.md Co-authored-by: Igor Suleymanov <radiohead@users.noreply.github.com> Co-authored-by: achatterjee-grafana <70489351+achatterjee-grafana@users.noreply.github.com>	4 years ago
Armand Grillet	6523486122	Alerting: Make Unified Alerting enabled by default for those who do not use legacy alerting (#42200) ... * update AlertingEnabled and UnifiedAlertingSettings.Enabled to be pointers * add a pseudo migration to fix the AlertingEnabled and UnifiedAlertingSettings.Enabled if the latter is not defined * update the default configuration file to make default value for both 'enabled' flags be undefined Misc * update Migrator to expose DB engine. This is needed for a ualert migration to access the database while the list of migrations is created. * add more verbose failure when migrations do not match Co-authored-by: gotjosh <josue@grafana.com> Co-authored-by: Yuriy Tseretyan <yuriy.tseretyan@grafana.com> Co-authored-by: gillesdemey <gilles.de.mey@gmail.com>	4 years ago
Arve Knudsen	78596a6756	Migrate to Wire for dependency injection (#32289) ... Fixes #30144 Co-authored-by: dsotirakis <sotirakis.dim@gmail.com> Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> Co-authored-by: Ida Furjesova <ida.furjesova@grafana.com> Co-authored-by: Jack Westbrook <jack.westbrook@gmail.com> Co-authored-by: Will Browne <wbrowne@users.noreply.github.com> Co-authored-by: Leon Sorokin <leeoniya@gmail.com> Co-authored-by: Andrej Ocenas <mr.ocenas@gmail.com> Co-authored-by: spinillos <selenepinillos@gmail.com> Co-authored-by: Karl Persson <kalle.persson@grafana.com> Co-authored-by: Leonard Gram <leo@xlson.com>	4 years ago
Sofia Papagiannaki	04d5dcb7c8	Alerting: modify DB table, accessors and migration to restrict org access (#37414) ... * Alerting: modify table and accessors to limit org access appropriately * Update migration to create multiple Alertmanager configs * Apply suggestions from code review Co-authored-by: gotjosh <josue@grafana.com> * replace mg.ClearMigrationEntry() mg.ClearMigrationEntry() would create a new session. This commit introduces a new migration for clearing an entry from migration log for replacing mg.ClearMigrationEntry() so that all dashboard alert migration operations will run inside the same transaction. It adds also `SkipMigrationLog()` in Migrator interface for skipping adding an entry in the migration_log. Co-authored-by: gotjosh <josue@grafana.com>	4 years ago
Sofia Papagiannaki	423474325a	Alerting: fix postgres migration on conflict (#35556)	5 years ago
Kyle Brandt	dc3e17ba5b	Alerting: Change rule migration to be based on feature flag (#33792) ... makes it so the feature flag can be turned on off, and the migration will be cleared and rerun. All existing NG alert rules, configuration settings, etc are removed when disabling the feature flag. for https://github.com/grafana/alerting-squad/issues/142 Co-authored-by: Sofia Papagiannaki <sofia@grafana.com>	5 years ago
Sofia Papagiannaki	49e9f7e3b6	Chore: Fix mysql integration tests (#33606)	5 years ago
Alexander Emelin	0c2bcbf2bc	Live: persisting last message in cache for broadcast scope (#32938)	5 years ago
Carl Bergquist	c899bf3592	log skipped, performed and duration for migrations (#31722) ... Signed-off-by: bergquist <carl.bergquist@gmail.com>	5 years ago
Sofia Papagiannaki	8c31e25926	AlertingNG: Save alert instances (#30223) ... * AlertingNG: Save alert instances Co-authored-by: Kyle Brandt <kyle@grafana.com> * Rename alert instance fields/columns * Include definition title in listing alert instances * Delete instances when deleting defintion Co-authored-by: Kyle Brandt <kyle@grafana.com>	5 years ago
Arve Knudsen	25048ebdf8	Chore: Add CloudWatch HTTP API tests (#29691) ... * CloudWatch: Add HTTP API tests Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	5 years ago
Sofia Papagiannaki	3cac10e598	AlertingNG: Create a scheduler to evaluate alert definitions (#29305) ... * Always use cache: stop passing skipCache among ngalert functions * Add updated column * Scheduler initial draft * Add retry on failure * Allow settting/updating alert definition interval Set default interval if no interval is provided during alert definition creation. Keep existing alert definition interval if no interval is provided during alert definition update. * Parameterise alerting.Ticker to run on custom interval * Allow updating alert definition interval without having to provide the queries and expressions * Add schedule tests * Use xorm tags for having initialisms with consistent case in Go * Add ability to pause/unpause the scheduler * Add alert definition versioning * Optimise scheduler to fetch alert definition only when it's necessary * Change MySQL data column to mediumtext * Delete alert definition versions * Increase default scheduler interval to 10 seconds * Fix setting OrgID on updates * Add validation for alert definition name length * Recreate tables	5 years ago
Arve Knudsen	9593d57914	Chore: Enable errorlint linter (#29227) ... * Enable errorlint linter * Handle wrapped errors Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> Co-authored-by: Emil Tullstedt <emil.tullstedt@grafana.com>	5 years ago
Arve Knudsen	4dd7b7a82d	Chore: Remove unused Go code (#28852) ... * Chore: Remove more unused Go code Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	5 years ago
Arve Knudsen	b5379c5335	Chore: Fix SQL related Go variable naming (#28887) ... * Chore: Fix variable naming Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	5 years ago
Arve Knudsen	676d393ec9	Chore: Fix issues reported by staticcheck; enable stylecheck linter (#28866) ... * Chore: Fix issues reported by staticcheck Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Apply suggestions from code review Co-authored-by: Emil Tullstedt <emil.tullstedt@grafana.com>	5 years ago
Carl Bergquist	74d1d3c6a8	Instrumentation: Add counters and histograms for database queries (#28236) ... Signed-off-by: bergquist <carl.bergquist@gmail.com> Co-authored-by: Sofia Papagiannaki <papagian@users.noreply.github.com> Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com>	5 years ago