The open and composable observability and data visualization platform. Visualize metrics, logs, and traces from multiple sources like Prometheus, Loki, Elasticsearch, InfluxDB, Postgres and many more.
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
grafana/pkg/api/plugin_resource_test.go

214 lines
8.6 KiB

package api
import (
"context"
"encoding/json"
"errors"
"io"
"net/url"
"path/filepath"
"strings"
"testing"
"github.com/grafana/grafana-azure-sdk-go/v2/azsettings"
"github.com/grafana/grafana-plugin-sdk-go/backend"
"github.com/prometheus/client_golang/prometheus"
"github.com/stretchr/testify/require"
"github.com/grafana/grafana/pkg/infra/db"
"github.com/grafana/grafana/pkg/infra/localcache"
"github.com/grafana/grafana/pkg/infra/log"
Plugins: Improve instrumentation by adding metrics and tracing (#61035) * WIP: Plugins tracing * Trace ID middleware * Add prometheus metrics and tracing to plugins updater * Add TODOs * Add instrumented http client * Add tracing to grafana update checker * Goimports * Moved plugins tracing to middleware * goimports, fix tests * Removed X-Trace-Id header * Fix comment in NewTracingHeaderMiddleware * Add metrics to instrumented http client * Add instrumented http client options * Removed unused function * Switch to contextual logger * Refactoring, fix tests * Moved InstrumentedHTTPClient and PrometheusMetrics to their own package * Tracing middleware: handle errors * Report span status codes when recording errors * Add tests for tracing middleware * Moved fakeSpan and fakeTracer to pkg/infra/tracing * Add TestHTTPClientTracing * Lint * Changes after PR review * Tests: Made "ended" in FakeSpan private, allow calling End only once * Testing: panic in FakeSpan if span already ended * Refactoring: Simplify Grafana updater checks * Refactoring: Simplify plugins updater error checks and logs * Fix wrong call to checkForUpdates -> instrumentedCheckForUpdates * Tests: Fix wrong call to checkForUpdates -> instrumentedCheckForUpdates * Log update checks duration, use Info log level for check succeeded logs * Add plugin context span attributes in tracing_middleware * Refactor prometheus metrics as httpclient middleware * Fix call to ProvidePluginsService in plugins_test.go * Propagate context to update checker outgoing http requests * Plugin client tracing middleware: Removed operation name in status * Fix tests * Goimports tracing_middleware.go * Goimports * Fix imports * Changed span name to plugins client middleware * Add span name assertion in TestTracingMiddleware * Removed Prometheus metrics middleware from grafana and plugins updatechecker * Add span attributes for ds name, type, uid, panel and dashboard ids * Fix http header reading in tracing middlewares * Use contexthandler.FromContext, add X-Query-Group-Id * Add test for RunStream * Fix imports * Changes from PR review * TestTracingMiddleware: Changed assert to require for didPanic assertion * Lint * Fix imports
2 years ago
"github.com/grafana/grafana/pkg/infra/tracing"
"github.com/grafana/grafana/pkg/plugins"
"github.com/grafana/grafana/pkg/plugins/backendplugin/coreplugin"
"github.com/grafana/grafana/pkg/plugins/manager/fakes"
"github.com/grafana/grafana/pkg/services/accesscontrol"
Caching: Refactor enterprise query caching middleware to a wire service (#65616) * define initial service and add to wire * update caching service interface * add skipQueryCache header handler and update metrics query function to use it * add caching service as a dependency to query service * working caching impl * propagate cache status to frontend in response * beginning of improvements suggested by Lean - separate caching logic from query logic. * more changes to simplify query function * Decided to revert renaming of function * Remove error status from cache request * add extra documentation * Move query caching duration metric to query package * add a little bit of documentation * wip: convert resource caching * Change return type of query service QueryData to a QueryDataResponse with Headers * update codeowners * change X-Cache value to const * use resource caching in endpoint handlers * write resource headers to response even if it's not a cache hit * fix panic caused by lack of nil check * update unit test * remove NONE header - shouldn't show up in OSS * Convert everything to use the plugin middleware * revert a few more things * clean up unused vars * start reverting resource caching, start to implement in plugin middleware * revert more, fix typo * Update caching interfaces - resource caching now has a separate cache method * continue wiring up new resource caching conventions - still in progress * add more safety to implementation * remove some unused objects * remove some code that I left in by accident * add some comments, fix codeowners, fix duplicate registration * fix source of panic in resource middleware * Update client decorator test to provide an empty response object * create tests for caching middleware * fix unit test * Update pkg/services/caching/service.go Co-authored-by: Arati R. <33031346+suntala@users.noreply.github.com> * improve error message in error log * quick docs update * Remove use of mockery. Update return signature to return an explicit hit/miss bool * create unit test for empty request context * rename caching metrics to make it clear they pertain to caching * Update pkg/services/pluginsintegration/clientmiddleware/caching_middleware.go Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * Add clarifying comments to cache skip middleware func * Add comment pointing to the resource cache update call * fix unit tests (missing dependency) * try to fix mystery syntax error * fix a panic * Caching: Introduce feature toggle to caching service refactor (#66323) * introduce new feature toggle * hide calls to new service behind a feature flag * remove licensing flag from toggle (misunderstood what it was for) * fix unit tests * rerun toggle gen --------- Co-authored-by: Arati R. <33031346+suntala@users.noreply.github.com> Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
2 years ago
"github.com/grafana/grafana/pkg/services/caching"
datasources "github.com/grafana/grafana/pkg/services/datasources/fakes"
"github.com/grafana/grafana/pkg/services/featuremgmt"
"github.com/grafana/grafana/pkg/services/oauthtoken/oauthtokentest"
"github.com/grafana/grafana/pkg/services/pluginsintegration"
"github.com/grafana/grafana/pkg/services/pluginsintegration/pluginaccesscontrol"
"github.com/grafana/grafana/pkg/services/pluginsintegration/pluginconfig"
"github.com/grafana/grafana/pkg/services/pluginsintegration/plugincontext"
pluginSettings "github.com/grafana/grafana/pkg/services/pluginsintegration/pluginsettings/service"
"github.com/grafana/grafana/pkg/services/quota/quotatest"
fakeSecrets "github.com/grafana/grafana/pkg/services/secrets/fakes"
"github.com/grafana/grafana/pkg/services/user"
"github.com/grafana/grafana/pkg/setting"
"github.com/grafana/grafana/pkg/tsdb/cloudwatch"
testdatasource "github.com/grafana/grafana/pkg/tsdb/grafana-testdata-datasource"
"github.com/grafana/grafana/pkg/web/webtest"
)
func TestCallResource(t *testing.T) {
staticRootPath, err := filepath.Abs("../../public/")
require.NoError(t, err)
cfg := setting.NewCfg()
cfg.StaticRootPath = staticRootPath
cfg.Azure = &azsettings.AzureSettings{}
coreRegistry := coreplugin.ProvideCoreRegistry(tracing.InitializeTracerForTest(), nil, &cloudwatch.CloudWatchService{}, nil, nil, nil, nil,
nil, nil, nil, nil, testdatasource.ProvideService(), nil, nil, nil, nil, nil, nil, nil, nil)
testCtx := pluginsintegration.CreateIntegrationTestCtx(t, cfg, coreRegistry)
pcp := plugincontext.ProvideService(cfg, localcache.ProvideService(), testCtx.PluginStore, &datasources.FakeCacheService{},
&datasources.FakeDataSourceService{}, pluginSettings.ProvideService(db.InitTestDB(t), fakeSecrets.NewFakeSecretsService()), pluginconfig.NewFakePluginRequestConfigProvider())
srv := SetupAPITestServer(t, func(hs *HTTPServer) {
hs.Cfg = cfg
hs.pluginContextProvider = pcp
hs.QuotaService = quotatest.New(false, nil)
hs.pluginStore = testCtx.PluginStore
hs.pluginClient = testCtx.PluginClient
hs.log = log.New("test")
})
t.Run("Test successful response is received for valid request", func(t *testing.T) {
req := srv.NewPostRequest("/api/plugins/grafana-testdata-datasource/resources/test", strings.NewReader(`{"test": "true"}`))
webtest.RequestWithSignedInUser(req, &user.SignedInUser{UserID: 1, OrgID: 1, Permissions: map[int64]map[string][]string{
1: accesscontrol.GroupScopesByActionContext(context.Background(), []accesscontrol.Permission{
{Action: pluginaccesscontrol.ActionAppAccess, Scope: pluginaccesscontrol.ScopeProvider.GetResourceAllScope()},
}),
}})
resp, err := srv.SendJSON(req)
require.NoError(t, err)
b, err := io.ReadAll(resp.Body)
require.NoError(t, err)
var body = make(map[string]any)
err = json.Unmarshal(b, &body)
require.NoError(t, err)
require.Equal(t, "Hello world from test datasource!", body["message"])
require.NoError(t, resp.Body.Close())
require.Equal(t, 200, resp.StatusCode)
})
t.Run("Test successful response is received for valid request with the colon character", func(t *testing.T) {
req := srv.NewPostRequest("/api/plugins/grafana-testdata-datasource/resources/test-*,*:test-*/_mapping", strings.NewReader(`{"test": "true"}`))
webtest.RequestWithSignedInUser(req, &user.SignedInUser{UserID: 1, OrgID: 1, Permissions: map[int64]map[string][]string{
1: accesscontrol.GroupScopesByActionContext(context.Background(), []accesscontrol.Permission{
{Action: pluginaccesscontrol.ActionAppAccess, Scope: pluginaccesscontrol.ScopeProvider.GetResourceAllScope()},
}),
}})
resp, err := srv.SendJSON(req)
require.NoError(t, err)
require.NoError(t, resp.Body.Close())
require.Equal(t, 200, resp.StatusCode)
})
t.Run("CallResource plugin resource request is created correctly", func(t *testing.T) {
type testdataCallResourceTestResponse struct {
Message string `json:"message"`
Request struct {
URL url.URL
Body map[string]any `json:"body"`
} `json:"request"`
}
for _, tc := range []struct {
name string
url string
exp func(t *testing.T, resp testdataCallResourceTestResponse)
}{
{
name: "Simple URL",
url: "/api/plugins/grafana-testdata-datasource/resources/test",
exp: func(t *testing.T, resp testdataCallResourceTestResponse) {
require.Equal(t, "Hello world from test datasource!", resp.Message)
require.Equal(t, "/test", resp.Request.URL.Path)
require.Equal(t, "true", resp.Request.Body["test"])
require.Len(t, resp.Request.Body, 1)
require.Empty(t, resp.Request.URL.RawQuery)
require.Empty(t, resp.Request.URL.Query())
},
},
{
name: "URL with query params",
url: "/api/plugins/grafana-testdata-datasource/resources/test?test=true&a=b",
exp: func(t *testing.T, resp testdataCallResourceTestResponse) {
require.Equal(t, "Hello world from test datasource!", resp.Message)
require.Equal(t, "/test", resp.Request.URL.Path)
require.Equal(t, "test=true&a=b", resp.Request.URL.RawQuery)
query := resp.Request.URL.Query()
require.Equal(t, "true", query.Get("test"))
require.Equal(t, "b", query.Get("a"))
require.Len(t, query, 2)
},
},
} {
t.Run(tc.name, func(t *testing.T) {
req := srv.NewPostRequest(tc.url, strings.NewReader(`{"test": "true"}`))
webtest.RequestWithSignedInUser(req, &user.SignedInUser{UserID: 1, OrgID: 1, Permissions: map[int64]map[string][]string{
1: accesscontrol.GroupScopesByActionContext(context.Background(), []accesscontrol.Permission{
{Action: pluginaccesscontrol.ActionAppAccess, Scope: pluginaccesscontrol.ScopeProvider.GetResourceAllScope()},
}),
}})
resp, err := srv.SendJSON(req)
require.NoError(t, err)
var body testdataCallResourceTestResponse
require.NoError(t, json.NewDecoder(resp.Body).Decode(&body))
tc.exp(t, body)
require.NoError(t, resp.Body.Close())
require.Equal(t, 200, resp.StatusCode)
})
}
})
pluginRegistry := fakes.NewFakePluginRegistry()
require.NoError(t, pluginRegistry.Add(context.Background(), &plugins.Plugin{
JSONData: plugins.JSONData{
ID: "grafana-testdata-datasource",
Backend: true,
},
}))
middlewares := pluginsintegration.CreateMiddlewares(cfg, &oauthtokentest.Service{}, tracing.InitializeTracerForTest(), &caching.OSSCachingService{}, featuremgmt.WithFeatures(), prometheus.DefaultRegisterer, pluginRegistry)
pc, err := backend.HandlerFromMiddlewares(&fakes.FakePluginClient{
CallResourceHandlerFunc: backend.CallResourceHandlerFunc(func(ctx context.Context,
req *backend.CallResourceRequest, sender backend.CallResourceResponseSender) error {
return errors.New("something went wrong")
}),
}, middlewares...)
require.NoError(t, err)
srv = SetupAPITestServer(t, func(hs *HTTPServer) {
hs.Cfg = cfg
hs.pluginContextProvider = pcp
hs.QuotaService = quotatest.New(false, nil)
hs.pluginStore = testCtx.PluginStore
hs.pluginClient = pc
hs.log = log.New("test")
})
t.Run("Test error is properly propagated to API response", func(t *testing.T) {
req := srv.NewGetRequest("/api/plugins/grafana-testdata-datasource/resources/scenarios")
webtest.RequestWithSignedInUser(req, &user.SignedInUser{UserID: 1, OrgID: 1, Permissions: map[int64]map[string][]string{
1: accesscontrol.GroupScopesByActionContext(context.Background(), []accesscontrol.Permission{
{Action: pluginaccesscontrol.ActionAppAccess, Scope: pluginaccesscontrol.ScopeProvider.GetResourceAllScope()},
}),
}})
resp, err := srv.SendJSON(req)
require.NoError(t, err)
bodyBytes, err := io.ReadAll(resp.Body)
require.NoError(t, err)
var responseBody struct {
Message string `json:"message"`
}
err = json.Unmarshal(bodyBytes, &responseBody)
require.NoError(t, err)
require.Equal(t, responseBody.Message, "Failed to call resource")
require.NoError(t, resp.Body.Close())
require.Equal(t, 500, resp.StatusCode)
})
}