The open and composable observability and data visualization platform. Visualize metrics, logs, and traces from multiple sources like Prometheus, Loki, Elasticsearch, InfluxDB, Postgres and many more.
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
grafana/public/app/features/dashboard/state/PanelQueryRunner.test.ts

295 lines
8.0 KiB

QueryProcessing: Observable query interface and RxJS for query & stream processing (#18899) * I needed to learn some rxjs and understand this more, so just playing around * Updated * Removed all the complete calls * Refactoring * StreamHandler -> observable start * progress * simple singal works * Handle update time range * added error handling * wrap old function * minor changes * handle data format in the subscribe function * Use replay subject to return last value to subscribers * Set loading state after no response in 50ms * added missing file * updated comment * Added cancelation of network requests * runRequest: Added unit test scenario framework * Progress on tests * minor refactor of unit tests * updated test * removed some old code * Shared queries work again, and also became so much simplier * unified query and observe methods * implict any fix * Fixed closed subject issue * removed comment * Use last returned data for loading state * WIP: Explore to runRequest makover step1 * Minor progress * Minor progress on explore and runRequest * minor progress * Things are starting to work in explore * Updated prometheus to use new observable query response, greatly simplified code * Revert refId change * Found better solution for key/refId/requestId problem * use observable with loki * tests compile * fix loki query prep * Explore: correct first response handling * Refactorings * Refactoring * Explore: Fixes LoadingState and GraphResults between runs (#18986) * Refactor: Adds state to DataQueryResponse * Fix: Fixes so we do not empty results before new data arrives Fixes: #17409 * Transformations work * observable test data * remove single() from loki promise * Fixed comment * Explore: Fixes failing Loki and Prometheus unit tests (#18995) * Tests: Makes datasource tests work again * Fix: Fixes loki datasource so highligthing works * Chore: Runs Prettier * Fixed query runner tests * Delay loading state indication to 200ms * Fixed test * fixed unit tests * Clear cached calcs * Fixed bug getProcesedDataFrames * Fix the correct test is a better idea * Fix: Fixes so queries in Explore are only run if Graph/Table is shown (#19000) * Fix: Fixes so queries in Explore are only run if Graph/Table is shown Fixes: #18618 * Refactor: Removes unnecessary condition * PanelData: provide legacy data only when needed (#19018) * no legacy * invert logic... now compiles * merge getQueryResponseData and getDataRaw * update comment about query editor * use single getData() function * only send legacy when it is used in explore * pre process rather than post process * pre process rather than post process * Minor refactoring * Add missing tags to test datasource response * MixedDatasource: Adds query observable pattern to MixedDatasource (#19037) * start mixed datasource * Refactor: Refactors into observable parttern * Tests: Fixes tests * Tests: Removes console.log * Refactor: Adds unique requestId
6 years ago
import { PanelQueryRunner } from './PanelQueryRunner';
// Importing this way to be able to spy on grafana/data
import * as grafanaData from '@grafana/data';
import { DataConfigSource, DataQueryRequest, GrafanaTheme, PanelData, ScopedVars } from '@grafana/data';
import { DashboardModel } from './index';
import { setEchoSrv } from '@grafana/runtime';
import { Echo } from '../../../core/services/echo/Echo';
jest.mock('app/core/services/backend_srv');
jest.mock('app/core/config', () => ({
config: { featureToggles: { transformations: true } },
getConfig: () => ({
featureToggles: {},
}),
}));
const dashboardModel = new DashboardModel({
panels: [{ id: 1, type: 'graph' }],
});
QueryProcessing: Observable query interface and RxJS for query & stream processing (#18899) * I needed to learn some rxjs and understand this more, so just playing around * Updated * Removed all the complete calls * Refactoring * StreamHandler -> observable start * progress * simple singal works * Handle update time range * added error handling * wrap old function * minor changes * handle data format in the subscribe function * Use replay subject to return last value to subscribers * Set loading state after no response in 50ms * added missing file * updated comment * Added cancelation of network requests * runRequest: Added unit test scenario framework * Progress on tests * minor refactor of unit tests * updated test * removed some old code * Shared queries work again, and also became so much simplier * unified query and observe methods * implict any fix * Fixed closed subject issue * removed comment * Use last returned data for loading state * WIP: Explore to runRequest makover step1 * Minor progress * Minor progress on explore and runRequest * minor progress * Things are starting to work in explore * Updated prometheus to use new observable query response, greatly simplified code * Revert refId change * Found better solution for key/refId/requestId problem * use observable with loki * tests compile * fix loki query prep * Explore: correct first response handling * Refactorings * Refactoring * Explore: Fixes LoadingState and GraphResults between runs (#18986) * Refactor: Adds state to DataQueryResponse * Fix: Fixes so we do not empty results before new data arrives Fixes: #17409 * Transformations work * observable test data * remove single() from loki promise * Fixed comment * Explore: Fixes failing Loki and Prometheus unit tests (#18995) * Tests: Makes datasource tests work again * Fix: Fixes loki datasource so highligthing works * Chore: Runs Prettier * Fixed query runner tests * Delay loading state indication to 200ms * Fixed test * fixed unit tests * Clear cached calcs * Fixed bug getProcesedDataFrames * Fix the correct test is a better idea * Fix: Fixes so queries in Explore are only run if Graph/Table is shown (#19000) * Fix: Fixes so queries in Explore are only run if Graph/Table is shown Fixes: #18618 * Refactor: Removes unnecessary condition * PanelData: provide legacy data only when needed (#19018) * no legacy * invert logic... now compiles * merge getQueryResponseData and getDataRaw * update comment about query editor * use single getData() function * only send legacy when it is used in explore * pre process rather than post process * pre process rather than post process * Minor refactoring * Add missing tags to test datasource response * MixedDatasource: Adds query observable pattern to MixedDatasource (#19037) * start mixed datasource * Refactor: Refactors into observable parttern * Tests: Fixes tests * Tests: Removes console.log * Refactor: Adds unique requestId
6 years ago
jest.mock('app/features/dashboard/services/DashboardSrv', () => ({
getDashboardSrv: () => {
return {
getCurrent: () => dashboardModel,
};
},
}));
interface ScenarioContext {
setup: (fn: () => void) => void;
// Options used in setup
maxDataPoints?: number | null;
dsInterval?: string;
minInterval?: string;
scopedVars: ScopedVars;
// Filled in by the Scenario runner
events?: PanelData[];
res?: PanelData;
queryCalledWith?: DataQueryRequest;
runner: PanelQueryRunner;
}
type ScenarioFn = (ctx: ScenarioContext) => void;
function describeQueryRunnerScenario(description: string, scenarioFn: ScenarioFn, panelConfig?: DataConfigSource) {
describe(description, () => {
let setupFn = () => {};
const defaultPanelConfig: DataConfigSource = {
getFieldOverrideOptions: () => undefined,
getTransformations: () => undefined,
};
const ctx: ScenarioContext = {
maxDataPoints: 200,
scopedVars: {
server: { text: 'Server1', value: 'server-1' },
},
runner: new PanelQueryRunner(panelConfig || defaultPanelConfig),
setup: (fn: () => void) => {
setupFn = fn;
},
};
const response: any = {
data: [
{
target: 'hello',
datapoints: [
[1, 1000],
[2, 2000],
],
},
],
};
beforeEach(async () => {
setEchoSrv(new Echo());
setupFn();
const datasource: any = {
name: 'TestDB',
interval: ctx.dsInterval,
QueryProcessing: Observable query interface and RxJS for query & stream processing (#18899) * I needed to learn some rxjs and understand this more, so just playing around * Updated * Removed all the complete calls * Refactoring * StreamHandler -> observable start * progress * simple singal works * Handle update time range * added error handling * wrap old function * minor changes * handle data format in the subscribe function * Use replay subject to return last value to subscribers * Set loading state after no response in 50ms * added missing file * updated comment * Added cancelation of network requests * runRequest: Added unit test scenario framework * Progress on tests * minor refactor of unit tests * updated test * removed some old code * Shared queries work again, and also became so much simplier * unified query and observe methods * implict any fix * Fixed closed subject issue * removed comment * Use last returned data for loading state * WIP: Explore to runRequest makover step1 * Minor progress * Minor progress on explore and runRequest * minor progress * Things are starting to work in explore * Updated prometheus to use new observable query response, greatly simplified code * Revert refId change * Found better solution for key/refId/requestId problem * use observable with loki * tests compile * fix loki query prep * Explore: correct first response handling * Refactorings * Refactoring * Explore: Fixes LoadingState and GraphResults between runs (#18986) * Refactor: Adds state to DataQueryResponse * Fix: Fixes so we do not empty results before new data arrives Fixes: #17409 * Transformations work * observable test data * remove single() from loki promise * Fixed comment * Explore: Fixes failing Loki and Prometheus unit tests (#18995) * Tests: Makes datasource tests work again * Fix: Fixes loki datasource so highligthing works * Chore: Runs Prettier * Fixed query runner tests * Delay loading state indication to 200ms * Fixed test * fixed unit tests * Clear cached calcs * Fixed bug getProcesedDataFrames * Fix the correct test is a better idea * Fix: Fixes so queries in Explore are only run if Graph/Table is shown (#19000) * Fix: Fixes so queries in Explore are only run if Graph/Table is shown Fixes: #18618 * Refactor: Removes unnecessary condition * PanelData: provide legacy data only when needed (#19018) * no legacy * invert logic... now compiles * merge getQueryResponseData and getDataRaw * update comment about query editor * use single getData() function * only send legacy when it is used in explore * pre process rather than post process * pre process rather than post process * Minor refactoring * Add missing tags to test datasource response * MixedDatasource: Adds query observable pattern to MixedDatasource (#19037) * start mixed datasource * Refactor: Refactors into observable parttern * Tests: Fixes tests * Tests: Removes console.log * Refactor: Adds unique requestId
6 years ago
query: (options: DataQueryRequest) => {
ctx.queryCalledWith = options;
return Promise.resolve(response);
},
testDatasource: jest.fn(),
};
const args: any = {
datasource,
scopedVars: ctx.scopedVars,
minInterval: ctx.minInterval,
maxDataPoints: ctx.maxDataPoints,
timeRange: {
from: grafanaData.dateTime().subtract(1, 'days'),
to: grafanaData.dateTime(),
raw: { from: '1d', to: 'now' },
},
panelId: 1,
queries: [{ refId: 'A', test: 1 }],
};
ctx.runner = new PanelQueryRunner(panelConfig || defaultPanelConfig);
ctx.runner.getData({ withTransforms: true, withFieldConfig: true }).subscribe({
next: (data: PanelData) => {
QueryProcessing: Observable query interface and RxJS for query & stream processing (#18899) * I needed to learn some rxjs and understand this more, so just playing around * Updated * Removed all the complete calls * Refactoring * StreamHandler -> observable start * progress * simple singal works * Handle update time range * added error handling * wrap old function * minor changes * handle data format in the subscribe function * Use replay subject to return last value to subscribers * Set loading state after no response in 50ms * added missing file * updated comment * Added cancelation of network requests * runRequest: Added unit test scenario framework * Progress on tests * minor refactor of unit tests * updated test * removed some old code * Shared queries work again, and also became so much simplier * unified query and observe methods * implict any fix * Fixed closed subject issue * removed comment * Use last returned data for loading state * WIP: Explore to runRequest makover step1 * Minor progress * Minor progress on explore and runRequest * minor progress * Things are starting to work in explore * Updated prometheus to use new observable query response, greatly simplified code * Revert refId change * Found better solution for key/refId/requestId problem * use observable with loki * tests compile * fix loki query prep * Explore: correct first response handling * Refactorings * Refactoring * Explore: Fixes LoadingState and GraphResults between runs (#18986) * Refactor: Adds state to DataQueryResponse * Fix: Fixes so we do not empty results before new data arrives Fixes: #17409 * Transformations work * observable test data * remove single() from loki promise * Fixed comment * Explore: Fixes failing Loki and Prometheus unit tests (#18995) * Tests: Makes datasource tests work again * Fix: Fixes loki datasource so highligthing works * Chore: Runs Prettier * Fixed query runner tests * Delay loading state indication to 200ms * Fixed test * fixed unit tests * Clear cached calcs * Fixed bug getProcesedDataFrames * Fix the correct test is a better idea * Fix: Fixes so queries in Explore are only run if Graph/Table is shown (#19000) * Fix: Fixes so queries in Explore are only run if Graph/Table is shown Fixes: #18618 * Refactor: Removes unnecessary condition * PanelData: provide legacy data only when needed (#19018) * no legacy * invert logic... now compiles * merge getQueryResponseData and getDataRaw * update comment about query editor * use single getData() function * only send legacy when it is used in explore * pre process rather than post process * pre process rather than post process * Minor refactoring * Add missing tags to test datasource response * MixedDatasource: Adds query observable pattern to MixedDatasource (#19037) * start mixed datasource * Refactor: Refactors into observable parttern * Tests: Fixes tests * Tests: Removes console.log * Refactor: Adds unique requestId
6 years ago
ctx.res = data;
ctx.events?.push(data);
},
});
ctx.events = [];
QueryProcessing: Observable query interface and RxJS for query & stream processing (#18899) * I needed to learn some rxjs and understand this more, so just playing around * Updated * Removed all the complete calls * Refactoring * StreamHandler -> observable start * progress * simple singal works * Handle update time range * added error handling * wrap old function * minor changes * handle data format in the subscribe function * Use replay subject to return last value to subscribers * Set loading state after no response in 50ms * added missing file * updated comment * Added cancelation of network requests * runRequest: Added unit test scenario framework * Progress on tests * minor refactor of unit tests * updated test * removed some old code * Shared queries work again, and also became so much simplier * unified query and observe methods * implict any fix * Fixed closed subject issue * removed comment * Use last returned data for loading state * WIP: Explore to runRequest makover step1 * Minor progress * Minor progress on explore and runRequest * minor progress * Things are starting to work in explore * Updated prometheus to use new observable query response, greatly simplified code * Revert refId change * Found better solution for key/refId/requestId problem * use observable with loki * tests compile * fix loki query prep * Explore: correct first response handling * Refactorings * Refactoring * Explore: Fixes LoadingState and GraphResults between runs (#18986) * Refactor: Adds state to DataQueryResponse * Fix: Fixes so we do not empty results before new data arrives Fixes: #17409 * Transformations work * observable test data * remove single() from loki promise * Fixed comment * Explore: Fixes failing Loki and Prometheus unit tests (#18995) * Tests: Makes datasource tests work again * Fix: Fixes loki datasource so highligthing works * Chore: Runs Prettier * Fixed query runner tests * Delay loading state indication to 200ms * Fixed test * fixed unit tests * Clear cached calcs * Fixed bug getProcesedDataFrames * Fix the correct test is a better idea * Fix: Fixes so queries in Explore are only run if Graph/Table is shown (#19000) * Fix: Fixes so queries in Explore are only run if Graph/Table is shown Fixes: #18618 * Refactor: Removes unnecessary condition * PanelData: provide legacy data only when needed (#19018) * no legacy * invert logic... now compiles * merge getQueryResponseData and getDataRaw * update comment about query editor * use single getData() function * only send legacy when it is used in explore * pre process rather than post process * pre process rather than post process * Minor refactoring * Add missing tags to test datasource response * MixedDatasource: Adds query observable pattern to MixedDatasource (#19037) * start mixed datasource * Refactor: Refactors into observable parttern * Tests: Fixes tests * Tests: Removes console.log * Refactor: Adds unique requestId
6 years ago
ctx.runner.run(args);
});
scenarioFn(ctx);
});
}
describe('PanelQueryRunner', () => {
describeQueryRunnerScenario('simple scenario', ctx => {
it('should set requestId on request', async () => {
expect(ctx.queryCalledWith?.requestId).toBe('Q100');
});
it('should set datasource name on request', async () => {
expect(ctx.queryCalledWith?.targets[0].datasource).toBe('TestDB');
});
it('should pass scopedVars to datasource with interval props', async () => {
expect(ctx.queryCalledWith?.scopedVars.server.text).toBe('Server1');
expect(ctx.queryCalledWith?.scopedVars.__interval.text).toBe('5m');
expect(ctx.queryCalledWith?.scopedVars.__interval_ms.text).toBe('300000');
});
});
describeQueryRunnerScenario('with maxDataPoints', ctx => {
ctx.setup(() => {
ctx.maxDataPoints = 200;
});
it('should return data', async () => {
expect(ctx.res?.error).toBeUndefined();
expect(ctx.res?.series.length).toBe(1);
});
it('should use widthPixels as maxDataPoints', async () => {
expect(ctx.queryCalledWith?.maxDataPoints).toBe(200);
});
it('should calculate interval based on width', async () => {
expect(ctx.queryCalledWith?.interval).toBe('5m');
});
it('fast query should only publish 1 data events', async () => {
expect(ctx.events?.length).toBe(1);
});
});
describeQueryRunnerScenario('with no panel min interval but datasource min interval', ctx => {
ctx.setup(() => {
ctx.maxDataPoints = 20000;
ctx.dsInterval = '15s';
});
it('should limit interval to data source min interval', async () => {
expect(ctx.queryCalledWith?.interval).toBe('15s');
});
});
describeQueryRunnerScenario('with panel min interval and data source min interval', ctx => {
ctx.setup(() => {
ctx.maxDataPoints = 20000;
ctx.dsInterval = '15s';
ctx.minInterval = '30s';
});
it('should limit interval to panel min interval', async () => {
expect(ctx.queryCalledWith?.interval).toBe('30s');
});
});
describeQueryRunnerScenario('with maxDataPoints', ctx => {
ctx.setup(() => {
ctx.maxDataPoints = 10;
});
it('should pass maxDataPoints if specified', async () => {
expect(ctx.queryCalledWith?.maxDataPoints).toBe(10);
});
it('should use instead of width to calculate interval', async () => {
expect(ctx.queryCalledWith?.interval).toBe('2h');
});
});
describeQueryRunnerScenario(
'field overrides',
ctx => {
it('should apply when field override options are set', async () => {
const spy = jest.spyOn(grafanaData, 'applyFieldOverrides');
ctx.runner.getData({ withTransforms: true, withFieldConfig: true }).subscribe({
next: (data: PanelData) => {
return data;
},
});
expect(spy).toBeCalled();
});
},
{
getFieldOverrideOptions: () => ({
fieldConfig: {
defaults: {
unit: 'm/s',
},
// @ts-ignore
overrides: [],
},
replaceVariables: v => v,
theme: {} as GrafanaTheme,
}),
getTransformations: () => undefined,
}
);
describeQueryRunnerScenario(
'transformations',
ctx => {
it('should apply when transformations are set', async () => {
const spy = jest.spyOn(grafanaData, 'transformDataFrame');
spy.mockClear();
ctx.runner.getData({ withTransforms: true, withFieldConfig: true }).subscribe({
next: (data: PanelData) => {
return data;
},
});
expect(spy).toBeCalled();
});
},
{
getFieldOverrideOptions: () => undefined,
// @ts-ignore
getTransformations: () => [{}],
}
);
describeQueryRunnerScenario(
'getData',
ctx => {
it('should not apply transformations when transform option is false', async () => {
const spy = jest.spyOn(grafanaData, 'transformDataFrame');
spy.mockClear();
ctx.runner.getData({ withTransforms: false, withFieldConfig: true }).subscribe({
next: (data: PanelData) => {
return data;
},
});
expect(spy).not.toBeCalled();
});
it('should not apply field config when applyFieldConfig option is false', async () => {
const spy = jest.spyOn(grafanaData, 'applyFieldOverrides');
spy.mockClear();
ctx.runner.getData({ withFieldConfig: false, withTransforms: true }).subscribe({
next: (data: PanelData) => {
return data;
},
});
expect(spy).not.toBeCalled();
});
},
{
getFieldOverrideOptions: () => ({
fieldConfig: {
defaults: {
unit: 'm/s',
},
// @ts-ignore
overrides: [],
},
replaceVariables: v => v,
theme: {} as GrafanaTheme,
}),
// @ts-ignore
getTransformations: () => [{}],
}
);
});