126 Commits (c4d3eb1cd03f30ea8e6aab71b2597bf3c6a6f46f)

Author	SHA1	Message	Date
Peter Štibraný	c4d3eb1cd0	Remove support for Google Spanner database. (#105846) ... * Remove support for Google Spanner database.	2 months ago
Serge Zaitsev	694b9dfe50	Chore: Replace xorm.io/xorm imports (#104458) ... * replace xorm.io/xorm imports * replace xorm from other go.mod files * clean up workspace * nolint does not make sense anymore as it is not a module * try if nolint directive helps * use nolint:all for xorm * add more nolints * try to skip xorm in linter config * exclude xorm differently * retrigger ci	3 months ago
Peter Štibraný	e85acf047a	Remove use of dialect.LikeStr (#104460) ... * Fix TestIntegrationUpdatingProvisionionedDashboards* tests when running on Spanner by fixing case-insensitive search for dashboard title. * Fix use of case-insensitive LIKE when running on Spanner. * Fix use of LikeStr in anonstore. * Fix use of LikeStr in ngalert/store and org/orgimpl. * Fix use of LikeStr in queryhistory search. * Fix use of LikeStr in serviceaccounts. * Fix use of LikeStr in serviceaccounts. * Fix use of LikeStr in services/team. * Remove LikeStr method.	3 months ago
Nathan Vērzemnieks	af59c4eb3e	Chore: bump grafana-plugin-sdk-go from v0.275.0 to v0.277.0 (#103971) ... Co-authored-by: Ryan McKinley <ryantxu@gmail.com>	3 months ago
Serge Zaitsev	047f5edae9	Chore: Add xorm.io/core into xorm (#103700) ... * add xorm.io/core into xorm * update workspace * format imports * update workspace	3 months ago
Peter Štibraný	54b8dde1e1	services/user: Fix userimpl tests when running on Spanner. (#103715) ... Fix userimpl tests when running on Spanner.	3 months ago
Mariell Hoversholm	757be6365a	CI: Bump golangci-lint to 2.0.2 (#103572)	3 months ago
Peter Štibraný	4efdef9b9c	Switch to using in-mem sequence generator only when explicitly configured in connection string. Move spanner-related functions to pkg/util/spanner. (#103363) ... * Switch to using in-mem sequence generator only when explicitly configured in connection string. Move spanner-related functions to pkg/util/spanner.	3 months ago
Peter Štibraný	f7f28757f6	Skip noop migrations, instead of executing SELECT 0. (#103086) ... * Skip noop migrations, instead of executing SELECT 0.	4 months ago
Peter Štibraný	00db0cf6e6	Remove dependency from OSS to enterprise packages by moving Spanner functions to xorm. (#102692) ... * Remove dependency from OSS to enterprise packages by moving Spanner functions to xorm.	4 months ago
Peter Štibraný	8db050af6b	Use in-memory sequence number generator when running integration tests against Spanner emulator. (#102522)	4 months ago
Peter Štibraný	0566ddbf4d	Speed up truncation in mysql, postgres and sqlite in integration tests. (#102439) ... Speed up truncation in sqlite, mysql and postgres integration tests.	4 months ago
Peter Štibraný	0c58d39e76	Spanner-related fixes (#102376) ... * Fix UNION syntax in resourcepermissions package. * Fix migrations in usermig package to work with Spanner. * Fix health query. * Use more connections for integration tests. * Add test-go-integration-spanner target to run integration tests against Spanner emulator. * Add test for enterprise. * Don't delete sequence number for migration_log.id column. * Only bump max open connections to 20 for Spanner. Lower integration test timeout.	4 months ago
Georges Chaudy	0bafd4e99d	unistore: update spanner ddl to include unistore tables (#102198) ... * provision unistore tables * update the tests to use generated namespace * update ddl * update ddl * clean streams * missing space * use engine.db.query	4 months ago
Peter Štibraný	11552e5c57	Speed up truncation of Spanner tables (used in integration tests) (#102278) ... Speed up truncation of Spanner tables.	4 months ago
Peter Štibraný	8625a6c131	Export SpannerConnectorConfigToClientOptions function (used by grafana-enterprise). (#102106) ... * Export SpannerConnectorConfigToClientOptions function (used by grafana-enterprise).	4 months ago
Peter Štibraný	f3df64b7f4	Pass BOOL values as real types (int/bool) instead of strings to SQL parameters. (#101961) ... * Pass BOOL values as real types (int/bool) instead of strings to SQL parameters. Fixes following integration tests when running with Spanner: * TestIntegrationDataAccess * GetDataSourcesByType/Get_prunable_data_sources * TestIntegrationUserAuthToken: * expires_correctly * can_properly_rotate_tokens * keeps_prev_token_valid_for_1_minute_after_it_is_confirmed * Fix more places where "true" or "false" string was passed as query parameter instead of bool value. * Removed unit test because it brought unwanted dependencies on xorm into multiple modules.	4 months ago
Peter Štibraný	0ce67dad74	Spanner: support for lastInsertId (#101770) ... * Support for returning inserted ID in Spanner. * Convert bytes to string when storing them into text column.	4 months ago
Peter Štibraný	32f651e9aa	Spanner: update DB snapshot (#101879) ... * Update spanner snapshot to include migrations from grafana-enterprise. * Move sql dump tool (test) to grafana-enterprise.	4 months ago
Peter Štibraný	fd6a4908f1	Support Spanner's UNION syntax, which needs to be UNION DISTINCT or UNION ALL. (#101768) ... * Support Spanner's UNION syntax, which needs to be UNION DISTINCT or UNION ALL.	4 months ago
Peter Štibraný	9e690bd620	Spanner support enhancements (#101634) ... * Adds ability to run integration tests against spanner (by using GRAFANA_TEST_DB=spanner env variable. SPANNER_DB variable then specifies database to use: spannertest, emulator or string like /projects/<project>/instances/<instance>/databases/<db>) * Adds feature to migration dialects to create database from a snapshot, instead of running individual migrations. * Adds first version of Spanner snapshot, prepared from "OSS" migrations. * Uses generated bit-reversed-positive values instead of auto_increment. (As an experiment)	5 months ago
Serge Zaitsev	165bca6417	Chore: Add initial/experimental xorm spanner driver (#101398) ... * make it build and start * run some migrations * add build tags, remove log * remove unused code * revert go.mod changes * move initialisation into dialect file * update workspace * update workspace once again * clean up dependencies * further cleanup * Address some review feedback. * Fix go.sum. --------- Co-authored-by: Peter Štibraný <pstibrany@gmail.com>	5 months ago
Ryan McKinley	429da7fd68	SQL/Blob: Add support for blob storage to SQL backend (#98192)	6 months ago
Dave Henderson	ced7a84f8b	chore(metrics): Add metrics & traces to DB migration (#97181) ... Signed-off-by: Dave Henderson <dave.henderson@grafana.com>	8 months ago
Jeff Levin	a21a232a8e	Revert read replica POC (#93551) ... * Revert "chore: add replDB to team service (#91799)" This reverts commit c6ae2d7999. * Revert "experiment: use read replica for Get and Find Dashboards (#91706)" This reverts commit 54177ca619. * Revert "QuotaService: refactor to use ReplDB for Get queries (#91333)" This reverts commit 299c142f6a. * Revert "refactor replCfg to look more like plugins/plugin config (#91142)" This reverts commit ac0b4bb34d. * Revert "chore (replstore): fix registration with multiple sql drivers, again (#90990)" This reverts commit daedb358dd. * Revert "Chore (sqlstore): add validation and testing for repl config (#90683)" This reverts commit af19f039b6. * Revert "ReplStore: Add support for round robin load balancing between multiple read replicas (#90530)" This reverts commit 27b52b1507. * Revert "DashboardStore: Use ReplDB and get dashboard quotas from the ReadReplica (#90235)" This reverts commit 8a6107cd35. * Revert "accesscontrol service read replica (#89963)" This reverts commit 77a4869fca. * Revert "Fix: add mapping for the new mysqlRepl driver (#89551)" This reverts commit ab5a079bcc. * Revert "fix: sql instrumentation dual registration error (#89508)" This reverts commit d988f5c3b0. * Revert "Experimental Feature Toggle: databaseReadReplica (#89232)" This reverts commit 50244ed4a1.	10 months ago
Kristin Laemmert	daedb358dd	chore (replstore): fix registration with multiple sql drivers, again (#90990) ... * replstore: fix registration with multiple sql drivers, again * only compile regex once	1 year ago
Kristin Laemmert	ab5a079bcc	Fix: add mapping for the new mysqlRepl driver (#89551)	1 year ago
Dan Cech	5dfe4cf407	Storage: Watch tests (#85496) ... * basic watch tests working * refactor to read previous event within poller * add watch test files * cleanup * watch tests passing * debug cleanup * special handling for canceled context * cleanup * fix wire * fix sqlite_sequence cleanup issue * move watch tests to integration, wait for provisioned dashboards * handle context deadline exceeded and eof errors * add comment about sleep	1 year ago
Dan Cech	1f1461734c	Storage: Add support for sortBy selector (#80680) ... * add support for sortBy field selector * use label selectors instead of field selectors * set entity_labels on create & update * make entity server integration tests work * test fixes * be more consistent with handling of empty body, meta or status * workaround for database is locked errors during migration * fix double import of sqlite3 * rename functions and tidy up * refactor update * disable integration tests until we can fix the database locking issue	1 year ago
Sofia Papagiannaki	5e88d29814	Folders: Introduce folder service function for fetching folders by org and UIDs that contain optionally the folder full path (#80716) ... * Folders: Expose function for getting all org folders with specific UIDs * Return all org folders if UIDs is empty * Filter out not accessible folders by the user * Modify query to optionally returning a string that contains the UIDs of all parent folders separated by slash.	2 years ago
Arati R	ca9d147a44	Give dialects control over how insert and update queries are performed (#79946) ... * Refactor insert, update * Add separate insert, update methods * Refactor Insert, Update signatures	2 years ago
Charandas	48612063dd	Grafana app platform: an aggregator cmd and package (#79948)	2 years ago
Dan Cech	67b2972052	Chore: add/update sqlstore-related helper functions (#77408) ... * add/update sqlstore-related helper functions * add documentation & tests for InsertQuery and UpdateQuery, make generated SQL deterministic by sorting columns * remove old log line	2 years ago
Marcus Efraimsson	edc34fc836	Instrumentation: Improve instrumentation for database migrations (#74659)	2 years ago
Ryan McKinley	025b2f3011	Chore: use any rather than interface{} (#74066)	2 years ago
Matthew Jacobson	44972d0cd5	SQLStore: Fix Postgres dialect treating "false" migrator default as true (#69353) ... * SQLStore: Fix Postgres dialect treating "false" migrator default as true Previously, when creating a migration you could choose a default value for a new boolean column that looked correct but would be interpreted incorrectly by the Postgres dialect. For example, values such as "false" or "FALSE" would be treated as true by the Postgres dialect. This refactors how migration dialects determine the Default column value for boolean type columns. Each dialect now uses the same base code to parse the Default literal and panics if an unknown value is encountered. So, now AddColumnMigration and AddTableMigration will ensure that across dialects: - The exact same Default literals will be allowed. - The literals are converted to equivalent defaults in their DDL. - An error will be thrown if an invalid literal is provided.	2 years ago
Dan Cech	a6279b2d62	Database: Make dialects independent of xorm Engine (#69955) ... * make dialects independent of xorm Engine * goimports	2 years ago
Sofia Papagiannaki	74e87ccbbd	SQLStore: Align SQLite IsUniqueConstraintViolation() with other backend implementations (#69224) ... * Add integration test for primary key and unique constrain violation * Align SQLite IsUniqueConstraintViolation implementation with other backends	2 years ago
Ryan McKinley	26658d172f	Migrations: Support independent migration log for entity api (#68511)	2 years ago
Ieva	035bf29146	RBAC: Remove the option to disable RBAC and add automated permission migrations for instances that had RBAC disabled (#66652) ... * RBAC: Stop reading enabeld from ini file and always set to true * Migrations: Add a migration for rbac to reset data migrations if rbac was disabled * If rbac was disabled we reset the data and data migrations that rbac has to perform to get it to a correct state * Migrator: Store migration logs on migrator and add function to clear it from the in-memory stored logs * update tests --------- Co-authored-by: Karl Persson <kalle.persson@grafana.com>	2 years ago
Serge Zaitsev	bc2813ef06	Chore: Fix goimports grouping in pkg/services (#62420) ... * fix goimports * fix goimports order	3 years ago
Kyle Brandt	55d2d872ec	Chore: Missed deprecations due to overly broad lint exclusion (#59732)	3 years ago
Ryan McKinley	eb1cc80941	Storage: include SQL implementation (#58018)	3 years ago
Alexander Weaver	ea8c5ce929	SQLStore: Create utility for bulk inserts (#57632) ... * Add batchsize to dialects * Some intermediary progress * Reflection-based implementation * Validate batch size * Implement a few simple tests * Fix linter error * Add debug message when falling back to default batch size	3 years ago
Joe Blubaugh	b476ae62fb	Alerting: Write and Delete multiple alert instances. (#55350) ... Prior to this change, all alert instance writes and deletes happened individually, in their own database transaction. This change batches up writes or deletes for a given rule's evaluation loop into a single transaction before applying it. These new transactions are off by default, guarded by the feature toggle "alertingBigTransactions" Before: ``` goos: darwin goarch: arm64 pkg: github.com/grafana/grafana/pkg/services/ngalert/store BenchmarkAlertInstanceOperations-8 398 2991381 ns/op 1133537 B/op 27703 allocs/op --- BENCH: BenchmarkAlertInstanceOperations-8 util.go:127: alert definition: {orgID: 1, UID: FovKXiRVzm} with title: "an alert definition FTvFXmRVkz" interval: 60 created util.go:127: alert definition: {orgID: 1, UID: foDFXmRVkm} with title: "an alert definition fovFXmRVkz" interval: 60 created util.go:127: alert definition: {orgID: 1, UID: VQvFuigVkm} with title: "an alert definition VwDKXmR4kz" interval: 60 created PASS ok github.com/grafana/grafana/pkg/services/ngalert/store 1.619s ``` After: ``` goos: darwin goarch: arm64 pkg: github.com/grafana/grafana/pkg/services/ngalert/store BenchmarkAlertInstanceOperations-8 1440 816484 ns/op 352297 B/op 6529 allocs/op --- BENCH: BenchmarkAlertInstanceOperations-8 util.go:127: alert definition: {orgID: 1, UID: 302r_igVzm} with title: "an alert definition q0h9lmR4zz" interval: 60 created util.go:127: alert definition: {orgID: 1, UID: 71hrlmR4km} with title: "an alert definition nJ29_mR4zz" interval: 60 created util.go:127: alert definition: {orgID: 1, UID: Cahr_mR4zm} with title: "an alert definition ja2rlmg4zz" interval: 60 created PASS ok github.com/grafana/grafana/pkg/services/ngalert/store 1.383s ``` So we cut time by about 75% and memory allocations by about 60% when storing and deleting 100 instances.	3 years ago
Emil Tullstedt	b287047052	Chore: Upgrade Go to 1.19.1 (#54902) ... * WIP * Set public_suffix to a pre Ruby 2.6 version * we don't need to install python * Stretch->Buster * Bump versions in lib.star * Manually update linter Sort of messy, but the .mod-file need to contain all dependencies that use 1.16+ features, otherwise they're assumed to be compiled with -lang=go1.16 and cannot access generics et al. Bingo doesn't seem to understand that, but it's possible to manually update things to get Bingo happy. * undo reformatting * Various lint improvements * More from the linter * goimports -w ./pkg/ * Disable gocritic * Add/modify linter exceptions * lint + flatten nested list Go 1.19 doesn't support nested lists, and there wasn't an obvious workaround. https://go.dev/doc/comment#lists	3 years ago
Joe Blubaugh	22c937340e	Revert "Alerting: Write and Delete multiple alert instances. (#54072)" (#54885) ... This reverts commit 5e4fd94413.	3 years ago
Joe Blubaugh	5e4fd94413	Alerting: Write and Delete multiple alert instances. (#54072) ... Prior to this change, all alert instance writes and deletes happened individually, in their own database transaction. This change batches up writes or deletes for a given rule's evaluation loop into a single transaction before applying it. Before: ``` goos: darwin goarch: arm64 pkg: github.com/grafana/grafana/pkg/services/ngalert/store BenchmarkAlertInstanceOperations-8 398 2991381 ns/op 1133537 B/op 27703 allocs/op --- BENCH: BenchmarkAlertInstanceOperations-8 util.go:127: alert definition: {orgID: 1, UID: FovKXiRVzm} with title: "an alert definition FTvFXmRVkz" interval: 60 created util.go:127: alert definition: {orgID: 1, UID: foDFXmRVkm} with title: "an alert definition fovFXmRVkz" interval: 60 created util.go:127: alert definition: {orgID: 1, UID: VQvFuigVkm} with title: "an alert definition VwDKXmR4kz" interval: 60 created PASS ok github.com/grafana/grafana/pkg/services/ngalert/store 1.619s ``` After: ``` goos: darwin goarch: arm64 pkg: github.com/grafana/grafana/pkg/services/ngalert/store BenchmarkAlertInstanceOperations-8 1440 816484 ns/op 352297 B/op 6529 allocs/op --- BENCH: BenchmarkAlertInstanceOperations-8 util.go:127: alert definition: {orgID: 1, UID: 302r_igVzm} with title: "an alert definition q0h9lmR4zz" interval: 60 created util.go:127: alert definition: {orgID: 1, UID: 71hrlmR4km} with title: "an alert definition nJ29_mR4zz" interval: 60 created util.go:127: alert definition: {orgID: 1, UID: Cahr_mR4zm} with title: "an alert definition ja2rlmg4zz" interval: 60 created PASS ok github.com/grafana/grafana/pkg/services/ngalert/store 1.383s ``` So we cut time by about 75% and memory allocations by about 60% when storing and deleting 100 instances. This change also updates some of our tests so that they run successfully against postgreSQL - we were using random Int64s, but postgres integers, which our tables use, max out at 2^31-1	3 years ago
Joe Blubaugh	a0f96ed4e1	SQLStore: Support Upserting multiple rows. (#52228) ... This will be used to reduce write load when the alerting system writes a large number of events.	3 years ago
Kat Yang	31630edf0c	Chore: Remove Wrapf (#50128) ... * Chore: Remove Wrapf * Remove all Wrapf refs * Remove last Wrapf ref * Fix lint errors * Remove Wrap and Wrapf definitions * Remove unnecessary colon	3 years ago