apitech
/
grafana
mirror of https://github.com/grafana/grafana
1
0
Fork 0
Code Issues Projects Releases Wiki Activity

Plugins: Enable feature toggle `pluginsDynamicAngularDetectionPatterns` by default (#84723)

Browse Source 
* Enable feature toggle pluginsDynamicAngularDetectionPatterns by default

* Change backgroundJobInterval to 24h

* re-generate feature toggles

* Use different intervals for cloud vs on-prem

* temporarily switch interval log to info level

* debug level again

* Simplify provideDynamic for tests

* re-generated feature toggles files

* PR review feedback

* PR review feedback: removed dependency from plugin management config
pull/84933/head
Giuseppe Guerra 1 year ago committed by GitHub
parent 6c1de260a2
commit eb31f71f60
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
7 changed files with 106 additions and 66 deletions
Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL
Show Stats Download Patch File Download Diff File
  1. 72
      docs/sources/setup-grafana/configure-grafana/feature-toggles/index.md
  2. 3
      pkg/services/featuremgmt/registry.go
  3. 2
      pkg/services/featuremgmt/toggles_gen.csv
  4. 9
      pkg/services/featuremgmt/toggles_gen.json
  5. 49
      pkg/services/pluginsintegration/angulardetectorsprovider/dynamic.go
  6. 31
      pkg/services/pluginsintegration/angulardetectorsprovider/dynamic_test.go
  7. 6
      pkg/services/pluginsintegration/angularinspector/angularinspector_test.go

72
docs/sources/setup-grafana/configure-grafana/feature-toggles/index.md
Unescape View File

@ -19,41 +19,42 @@ This page contains a list of available feature toggles. To learn how to turn on
Some features are enabled by default. You can disable these feature by setting the feature flag to "false" in the configuration.
| Feature toggle name | Description | Enabled by default |
| ------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------ |
| `disableEnvelopeEncryption` | Disable envelope encryption (emergency only) | |
| `publicDashboards` | [Deprecated] Public dashboards are now enabled by default; to disable them, use the configuration setting. This feature toggle will be removed in the next major version. | Yes |
| `featureHighlights` | Highlight Grafana Enterprise features | |
| `correlations` | Correlations page | |
| `exploreContentOutline` | Content outline sidebar | Yes |
| `cloudWatchCrossAccountQuerying` | Enables cross-account querying in CloudWatch datasources | Yes |
| `nestedFolderPicker` | Enables the new folder picker to work with nested folders. Requires the nestedFolders feature toggle | Yes |
| `logsContextDatasourceUi` | Allow datasource to provide custom UI for context view | Yes |
| `lokiQuerySplitting` | Split large interval queries into subqueries with smaller time intervals | Yes |
| `prometheusMetricEncyclopedia` | Adds the metrics explorer component to the Prometheus query builder as an option in metric select | Yes |
| `influxdbBackendMigration` | Query InfluxDB InfluxQL without the proxy | Yes |
| `prometheusDataplane` | Changes responses to from Prometheus to be compliant with the dataplane specification. In particular, when this feature toggle is active, the numeric `Field.Name` is set from 'Value' to the value of the `__name__` label. | Yes |
| `lokiMetricDataplane` | Changes metric responses from Loki to be compliant with the dataplane specification. | Yes |
| `dataplaneFrontendFallback` | Support dataplane contract field name change for transformations and field name matchers where the name is different | Yes |
| `enableElasticsearchBackendQuerying` | Enable the processing of queries and responses in the Elasticsearch data source through backend | Yes |
| `recordedQueriesMulti` | Enables writing multiple items from a single query within Recorded Queries | Yes |
| `logsExploreTableVisualisation` | A table visualisation for logs in Explore | Yes |
| `transformationsRedesign` | Enables the transformations redesign | Yes |
| `traceQLStreaming` | Enables response streaming of TraceQL queries of the Tempo data source | |
| `awsAsyncQueryCaching` | Enable caching for async queries for Redshift and Athena. Requires that the datasource has caching and async query support enabled | Yes |
| `prometheusConfigOverhaulAuth` | Update the Prometheus configuration page with the new auth component | Yes |
| `influxdbSqlSupport` | Enable InfluxDB SQL query language support with new querying UI | Yes |
| `alertingNoDataErrorExecution` | Changes how Alerting state manager handles execution of NoData/Error | Yes |
| `angularDeprecationUI` | Display Angular warnings in dashboards and panels | Yes |
| `alertingInsights` | Show the new alerting insights landing page | Yes |
| `panelMonitoring` | Enables panel monitoring through logs and measurements | Yes |
| `recoveryThreshold` | Enables feature recovery threshold (aka hysteresis) for threshold server-side expression | Yes |
| `lokiStructuredMetadata` | Enables the loki data source to request structured metadata from the Loki server | Yes |
| `managedPluginsInstall` | Install managed plugins directly from plugins catalog | Yes |
| `logRowsPopoverMenu` | Enable filtering menu displayed when text of a log line is selected | Yes |
| `lokiQueryHints` | Enables query hints for Loki | Yes |
| `alertingQueryOptimization` | Optimizes eligible queries in order to reduce load on datasources | |
| `betterPageScrolling` | Removes CustomScrollbar from the UI, relying on native browser scrollbars | Yes |
| Feature toggle name | Description | Enabled by default |
| ---------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------ |
| `disableEnvelopeEncryption` | Disable envelope encryption (emergency only) | |
| `publicDashboards` | [Deprecated] Public dashboards are now enabled by default; to disable them, use the configuration setting. This feature toggle will be removed in the next major version. | Yes |
| `featureHighlights` | Highlight Grafana Enterprise features | |
| `correlations` | Correlations page | |
| `exploreContentOutline` | Content outline sidebar | Yes |
| `cloudWatchCrossAccountQuerying` | Enables cross-account querying in CloudWatch datasources | Yes |
| `nestedFolderPicker` | Enables the new folder picker to work with nested folders. Requires the nestedFolders feature toggle | Yes |
| `logsContextDatasourceUi` | Allow datasource to provide custom UI for context view | Yes |
| `lokiQuerySplitting` | Split large interval queries into subqueries with smaller time intervals | Yes |
| `prometheusMetricEncyclopedia` | Adds the metrics explorer component to the Prometheus query builder as an option in metric select | Yes |
| `influxdbBackendMigration` | Query InfluxDB InfluxQL without the proxy | Yes |
| `prometheusDataplane` | Changes responses to from Prometheus to be compliant with the dataplane specification. In particular, when this feature toggle is active, the numeric `Field.Name` is set from 'Value' to the value of the `__name__` label. | Yes |
| `lokiMetricDataplane` | Changes metric responses from Loki to be compliant with the dataplane specification. | Yes |
| `dataplaneFrontendFallback` | Support dataplane contract field name change for transformations and field name matchers where the name is different | Yes |
| `enableElasticsearchBackendQuerying` | Enable the processing of queries and responses in the Elasticsearch data source through backend | Yes |
| `recordedQueriesMulti` | Enables writing multiple items from a single query within Recorded Queries | Yes |
| `pluginsDynamicAngularDetectionPatterns` | Enables fetching Angular detection patterns for plugins from GCOM and fallback to hardcoded ones | Yes |
| `logsExploreTableVisualisation` | A table visualisation for logs in Explore | Yes |
| `transformationsRedesign` | Enables the transformations redesign | Yes |
| `traceQLStreaming` | Enables response streaming of TraceQL queries of the Tempo data source | |
| `awsAsyncQueryCaching` | Enable caching for async queries for Redshift and Athena. Requires that the datasource has caching and async query support enabled | Yes |
| `prometheusConfigOverhaulAuth` | Update the Prometheus configuration page with the new auth component | Yes |
| `influxdbSqlSupport` | Enable InfluxDB SQL query language support with new querying UI | Yes |
| `alertingNoDataErrorExecution` | Changes how Alerting state manager handles execution of NoData/Error | Yes |
| `angularDeprecationUI` | Display Angular warnings in dashboards and panels | Yes |
| `alertingInsights` | Show the new alerting insights landing page | Yes |
| `panelMonitoring` | Enables panel monitoring through logs and measurements | Yes |
| `recoveryThreshold` | Enables feature recovery threshold (aka hysteresis) for threshold server-side expression | Yes |
| `lokiStructuredMetadata` | Enables the loki data source to request structured metadata from the Loki server | Yes |
| `managedPluginsInstall` | Install managed plugins directly from plugins catalog | Yes |
| `logRowsPopoverMenu` | Enable filtering menu displayed when text of a log line is selected | Yes |
| `lokiQueryHints` | Enables query hints for Loki | Yes |
| `alertingQueryOptimization` | Optimizes eligible queries in order to reduce load on datasources | |
| `betterPageScrolling` | Removes CustomScrollbar from the UI, relying on native browser scrollbars | Yes |
## Preview feature toggles
@ -130,7 +131,6 @@ Experimental features might be changed or removed without prior notice.
| `dashboardEmbed` | Allow embedding dashboard for external use in Code editors |
| `frontendSandboxMonitorOnly` | Enables monitor only in the plugin frontend sandbox (if enabled) |
| `lokiFormatQuery` | Enables the ability to format Loki queries |
| `pluginsDynamicAngularDetectionPatterns` | Enables fetching Angular detection patterns for plugins from GCOM and fallback to hardcoded ones |
| `vizAndWidgetSplit` | Split panels between visualizations and widgets |
| `prometheusIncrementalQueryInstrumentation` | Adds RudderStack events to incremental queries |
| `awsDatasourcesTempCredentials` | Support temporary security credentials in AWS plugins for Grafana Cloud customers |

3
pkg/services/featuremgmt/registry.go
Unescape View File

@ -510,9 +510,10 @@ var (
{
Name: "pluginsDynamicAngularDetectionPatterns",
Description: "Enables fetching Angular detection patterns for plugins from GCOM and fallback to hardcoded ones",
Stage: FeatureStageExperimental,
Stage: FeatureStageGeneralAvailability,
FrontendOnly: false,
Owner: grafanaPluginsPlatformSquad,
Expression: "true", // enabled by default
},
{
Name: "vizAndWidgetSplit",

2
pkg/services/featuremgmt/toggles_gen.csv
Unescape View File

@ -67,7 +67,7 @@ frontendSandboxMonitorOnly,experimental,@grafana/plugins-platform-backend,false,
sqlDatasourceDatabaseSelection,preview,@grafana/dataviz-squad,false,false,true
lokiFormatQuery,experimental,@grafana/observability-logs,false,false,true
recordedQueriesMulti,GA,@grafana/observability-metrics,false,false,false
pluginsDynamicAngularDetectionPatterns,experimental,@grafana/plugins-platform-backend,false,false,false
pluginsDynamicAngularDetectionPatterns,GA,@grafana/plugins-platform-backend,false,false,false
vizAndWidgetSplit,experimental,@grafana/dashboards-squad,false,false,true
prometheusIncrementalQueryInstrumentation,experimental,@grafana/observability-metrics,false,false,true
logsExploreTableVisualisation,GA,@grafana/observability-logs,false,false,true

1 Name Stage Owner requiresDevMode RequiresRestart FrontendOnly
67 sqlDatasourceDatabaseSelection preview @grafana/dataviz-squad false false true
68 lokiFormatQuery experimental @grafana/observability-logs false false true
69 recordedQueriesMulti GA @grafana/observability-metrics false false false
70 pluginsDynamicAngularDetectionPatterns experimental GA @grafana/plugins-platform-backend false false false
71 vizAndWidgetSplit experimental @grafana/dashboards-squad false false true
72 prometheusIncrementalQueryInstrumentation experimental @grafana/observability-metrics false false true
73 logsExploreTableVisualisation GA @grafana/observability-logs false false true

9
pkg/services/featuremgmt/toggles_gen.json
Unescape View File

@ -441,12 +441,15 @@
{
"metadata": {
"name": "pluginsDynamicAngularDetectionPatterns",
"resourceVersion": "1709648236447",
"creationTimestamp": "2024-03-05T14:17:16Z"
"resourceVersion": "1710847676897",
"creationTimestamp": "2024-03-05T14:17:16Z",
"annotations": {
"grafana.app/updatedTimestamp": "2024-03-19 11:27:56.897962695 +0000 UTC"
}
},
"spec": {
"description": "Enables fetching Angular detection patterns for plugins from GCOM and fallback to hardcoded ones",
"stage": "experimental",
"stage": "GA",
"codeowner": "@grafana/plugins-platform-backend"
}
},

49
pkg/services/pluginsintegration/angulardetectorsprovider/dynamic.go
Unescape View File

@ -12,18 +12,21 @@ import (
"sync"
"time"
"github.com/grafana/grafana/pkg/plugins/config"
"github.com/grafana/grafana/pkg/plugins/log"
"github.com/grafana/grafana/pkg/plugins/manager/loader/angular/angulardetector"
"github.com/grafana/grafana/pkg/services/featuremgmt"
"github.com/grafana/grafana/pkg/services/pluginsintegration/angularpatternsstore"
"github.com/grafana/grafana/pkg/setting"
)
var errNotModified = errors.New("not modified")
// backgroundJobInterval is the interval that passes between background job runs.
// It can be overwritten in tests.
var backgroundJobInterval = time.Hour * 1
// Intervals that passes between background job runs.
// Can be overwritten in tests.
var (
backgroundJobIntervalOnPrem = time.Hour * 24
backgroundJobIntervalCloud = time.Hour * 1
)
// Dynamic is an angulardetector.DetectorsProvider that calls GCOM to get Angular detection patterns,
// converts them to detectors and caches them for all future calls.
@ -45,20 +48,32 @@ type Dynamic struct {
// mux is the mutex used to read/write the cached detectors in a concurrency-safe way.
mux sync.RWMutex
// backgroundJobInterval is the interval that passes between background job runs.
backgroundJobInterval time.Duration
}
func ProvideDynamic(cfg *config.PluginManagementCfg, store angularpatternsstore.Service, features featuremgmt.FeatureToggles) (*Dynamic, error) {
func ProvideDynamic(cfg *setting.Cfg, store angularpatternsstore.Service, features featuremgmt.FeatureToggles) (*Dynamic, error) {
backgroundJobInterval := backgroundJobIntervalOnPrem
if cfg.StackID != "" {
// Use a shorter interval for cloud.
// (in cloud, cfg.StackID is always set).
backgroundJobInterval = backgroundJobIntervalCloud
}
d := &Dynamic{
log: log.New("plugin.angulardetectorsprovider.dynamic"),
features: features,
store: store,
httpClient: makeHttpClient(),
baseURL: cfg.GrafanaComURL,
log: log.New("plugin.angulardetectorsprovider.dynamic"),
features: features,
store: store,
httpClient: makeHttpClient(),
baseURL: cfg.GrafanaComURL,
backgroundJobInterval: backgroundJobInterval,
}
if d.IsDisabled() {
// Do not attempt to restore if the background service is disabled (no feature flag)
return d, nil
}
d.log.Debug("Providing dynamic angular detection patterns", "baseURL", d.baseURL, "interval", d.backgroundJobInterval)
// Perform the initial restore from db
st := time.Now()
@ -224,7 +239,7 @@ func (d *Dynamic) IsDisabled() bool {
}
// randomSkew returns a random time.Duration between 0 and maxSkew.
// This can be added to backgroundJobInterval to skew it by a random amount.
// This can be added to d.backgroundJobInterval to skew it by a random amount.
func (d *Dynamic) randomSkew(maxSkew time.Duration) time.Duration {
return time.Duration(rand.Float64() * float64(maxSkew))
}
@ -242,15 +257,15 @@ func (d *Dynamic) Run(ctx context.Context) error {
// Offset the background job interval a bit to skew GCOM calls from all instances,
// so GCOM is not overwhelmed with lots of requests all at the same time.
// Important when lots of HG instances restart at the same time.
skew := d.randomSkew(backgroundJobInterval / 4)
backgroundJobInterval += skew
skew := d.randomSkew(d.backgroundJobInterval / 4)
d.backgroundJobInterval += skew
d.log.Debug(
"Applied background job skew",
"skew", backgroundJobInterval, "interval", backgroundJobInterval,
"skew", d.backgroundJobInterval, "interval", d.backgroundJobInterval,
)
nextRunUntil := time.Until(lastUpdate.Add(backgroundJobInterval))
ticker := time.NewTicker(backgroundJobInterval)
nextRunUntil := time.Until(lastUpdate.Add(d.backgroundJobInterval))
ticker := time.NewTicker(d.backgroundJobInterval)
defer ticker.Stop()
var tick <-chan time.Time
@ -286,7 +301,7 @@ func (d *Dynamic) Run(ctx context.Context) error {
d.log.Info("Patterns update finished", "duration", time.Since(st))
// Restore default ticker if we run with a shorter interval the first time
ticker.Reset(backgroundJobInterval)
ticker.Reset(d.backgroundJobInterval)
tick = ticker.C
case <-ctx.Done():
return ctx.Err()

31
pkg/services/pluginsintegration/angulardetectorsprovider/dynamic_test.go
Unescape View File

@ -14,10 +14,10 @@ import (
"github.com/stretchr/testify/require"
"github.com/grafana/grafana/pkg/infra/kvstore"
"github.com/grafana/grafana/pkg/plugins/config"
"github.com/grafana/grafana/pkg/plugins/manager/loader/angular/angulardetector"
"github.com/grafana/grafana/pkg/services/featuremgmt"
"github.com/grafana/grafana/pkg/services/pluginsintegration/angularpatternsstore"
"github.com/grafana/grafana/pkg/setting"
)
func TestDynamicAngularDetectorsProvider(t *testing.T) {
@ -314,6 +314,22 @@ func TestDynamicAngularDetectorsProvider(t *testing.T) {
})
}
func TestDynamicAngularDetectorsProviderCloudVsOnPrem(t *testing.T) {
gcom := newDefaultGCOMScenario()
srv := gcom.newHTTPTestServer()
t.Cleanup(srv.Close)
t.Run("should use cloud interval if stack_id is set", func(t *testing.T) {
svc := provideDynamic(t, srv.URL, provideDynamicOpts{cfg: &setting.Cfg{StackID: "1234"}})
require.Equal(t, backgroundJobIntervalCloud, svc.backgroundJobInterval)
})
t.Run("should use on-prem interval if stack_id is not set", func(t *testing.T) {
svc := provideDynamic(t, srv.URL, provideDynamicOpts{cfg: &setting.Cfg{StackID: ""}})
require.Equal(t, backgroundJobIntervalOnPrem, svc.backgroundJobInterval)
})
}
func TestDynamicAngularDetectorsProviderBackgroundService(t *testing.T) {
mockGCOMPatterns := newMockGCOMPatterns()
gcom := newDefaultGCOMScenario()
@ -321,10 +337,10 @@ func TestDynamicAngularDetectorsProviderBackgroundService(t *testing.T) {
t.Cleanup(srv.Close)
t.Run("background service", func(t *testing.T) {
oldBackgroundJobInterval := backgroundJobInterval
backgroundJobInterval = time.Millisecond * 500
oldBackgroundJobInterval := backgroundJobIntervalOnPrem
backgroundJobIntervalOnPrem = time.Millisecond * 500
t.Cleanup(func() {
backgroundJobInterval = oldBackgroundJobInterval
backgroundJobIntervalOnPrem = oldBackgroundJobInterval
})
t.Run("is disabled if feature flag is not present", func(t *testing.T) {
@ -563,6 +579,7 @@ func newError500GCOMScenario() *gcomScenario {
type provideDynamicOpts struct {
store angularpatternsstore.Service
cfg *setting.Cfg
}
func provideDynamic(t *testing.T, gcomURL string, opts ...provideDynamicOpts) *Dynamic {
@ -573,8 +590,12 @@ func provideDynamic(t *testing.T, gcomURL string, opts ...provideDynamicOpts) *D
if opt.store == nil {
opt.store = angularpatternsstore.ProvideService(kvstore.NewFakeKVStore())
}
if opt.cfg == nil {
opt.cfg = setting.NewCfg()
}
opt.cfg.GrafanaComURL = gcomURL
d, err := ProvideDynamic(
&config.PluginManagementCfg{GrafanaComURL: gcomURL},
opt.cfg,
opt.store,
featuremgmt.WithFeatures(featuremgmt.FlagPluginsDynamicAngularDetectionPatterns),
)

6
pkg/services/pluginsintegration/angularinspector/angularinspector_test.go
Unescape View File

@ -7,19 +7,19 @@ import (
"github.com/stretchr/testify/require"
"github.com/grafana/grafana/pkg/infra/kvstore"
"github.com/grafana/grafana/pkg/plugins/config"
"github.com/grafana/grafana/pkg/plugins/manager/loader/angular/angulardetector"
"github.com/grafana/grafana/pkg/plugins/manager/loader/angular/angularinspector"
"github.com/grafana/grafana/pkg/services/featuremgmt"
"github.com/grafana/grafana/pkg/services/pluginsintegration/angulardetectorsprovider"
"github.com/grafana/grafana/pkg/services/pluginsintegration/angularpatternsstore"
"github.com/grafana/grafana/pkg/setting"
)
func TestProvideService(t *testing.T) {
t.Run("uses hardcoded inspector if feature flag is not present", func(t *testing.T) {
features := featuremgmt.WithFeatures()
dynamic, err := angulardetectorsprovider.ProvideDynamic(
&config.PluginManagementCfg{},
setting.NewCfg(),
angularpatternsstore.ProvideService(kvstore.NewFakeKVStore()),
features,
)
@ -37,7 +37,7 @@ func TestProvideService(t *testing.T) {
featuremgmt.FlagPluginsDynamicAngularDetectionPatterns,
)
dynamic, err := angulardetectorsprovider.ProvideDynamic(
&config.PluginManagementCfg{},
setting.NewCfg(),
angularpatternsstore.ProvideService(kvstore.NewFakeKVStore()),
features,
)

Write Preview
Loading…
Cancel
Save