The open and composable observability and data visualization platform. Visualize metrics, logs, and traces from multiple sources like Prometheus, Loki, Elasticsearch, InfluxDB, Postgres and many more.
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
grafana/public/app/plugins/datasource/cloud-monitoring/functions.ts

159 lines
5.3 KiB

import { chunk, initial, startCase, uniqBy } from 'lodash';
import { rangeUtil } from '@grafana/data';
import { getTemplateSrv, TemplateSrv } from '@grafana/runtime';
import { AGGREGATIONS, ALIGNMENTS, SYSTEM_LABELS } from './constants';
import CloudMonitoringDatasource from './datasource';
import { AlignmentTypes, PreprocessorType, TimeSeriesList, MetricKind, ValueTypes } from './types/query';
import { CustomMetaData, MetricDescriptor } from './types/types';
Stackdriver: Project selector (#22447) * clean PR #17366 * udpate vendor * [WIP] Implement projects management for stackdriver * [WIP] Implement projects management for stackdriver * [WIP] Implement projects management for stackdriver * Implement projects management for stackdriver * [WIP][Tests] Fix errors * clean anonymous struct * remove await * don't store project list * Add default project on query editor * gofmt * Fix tests * Move test data source to backend * Use segment instead of dropdown. remove ensure default project since it's not being used anymore. * Fix broken annotation editor * Load gceDefaultAccount only once when in the config page * Reset error message on auth type change * Add metric find query for projects * Remove debug code * Fix broken tests * Fix typings * Fix lint error * Slightly different approach - now having a distiction between config page default project, and project that is selectable from the dropdown in the query editor. * Fix broken tests * Attempt to fix strict ts errors * Prevent state from being set multiple times * Remove noOptionsMessage since it seems to be obosolete in react select * One more attempt to solve ts strict error * Interpolate project template variable. Make sure its loaded correctly when opening variable query editor first time * Implicit any fix * fix: typescript strict null check fixes * Return empty array in case project endpoint fails * Rename project to projectName to prevent clashing with legacy query prop * Fix broken test * fix: Stackdriver - template replace on filter label should have a regex format as that escapes the dots in the label name which is not valid. Co-authored-by: Labesse Kévin <kevin@labesse.me> Co-authored-by: Elias Cédric Laouiti <elias@abtasty.com> Co-authored-by: Daniel Lee <dan.limerick@gmail.com>
5 years ago
export const extractServicesFromMetricDescriptors = (metricDescriptors: MetricDescriptor[]) =>
uniqBy(metricDescriptors, 'service');
Stackdriver: Project selector (#22447) * clean PR #17366 * udpate vendor * [WIP] Implement projects management for stackdriver * [WIP] Implement projects management for stackdriver * [WIP] Implement projects management for stackdriver * Implement projects management for stackdriver * [WIP][Tests] Fix errors * clean anonymous struct * remove await * don't store project list * Add default project on query editor * gofmt * Fix tests * Move test data source to backend * Use segment instead of dropdown. remove ensure default project since it's not being used anymore. * Fix broken annotation editor * Load gceDefaultAccount only once when in the config page * Reset error message on auth type change * Add metric find query for projects * Remove debug code * Fix broken tests * Fix typings * Fix lint error * Slightly different approach - now having a distiction between config page default project, and project that is selectable from the dropdown in the query editor. * Fix broken tests * Attempt to fix strict ts errors * Prevent state from being set multiple times * Remove noOptionsMessage since it seems to be obosolete in react select * One more attempt to solve ts strict error * Interpolate project template variable. Make sure its loaded correctly when opening variable query editor first time * Implicit any fix * fix: typescript strict null check fixes * Return empty array in case project endpoint fails * Rename project to projectName to prevent clashing with legacy query prop * Fix broken test * fix: Stackdriver - template replace on filter label should have a regex format as that escapes the dots in the label name which is not valid. Co-authored-by: Labesse Kévin <kevin@labesse.me> Co-authored-by: Elias Cédric Laouiti <elias@abtasty.com> Co-authored-by: Daniel Lee <dan.limerick@gmail.com>
5 years ago
export const getMetricTypesByService = (metricDescriptors: MetricDescriptor[], service: string) =>
metricDescriptors.filter((m: MetricDescriptor) => m.service === service);
export const getMetricTypes = (
Stackdriver: Project selector (#22447) * clean PR #17366 * udpate vendor * [WIP] Implement projects management for stackdriver * [WIP] Implement projects management for stackdriver * [WIP] Implement projects management for stackdriver * Implement projects management for stackdriver * [WIP][Tests] Fix errors * clean anonymous struct * remove await * don't store project list * Add default project on query editor * gofmt * Fix tests * Move test data source to backend * Use segment instead of dropdown. remove ensure default project since it's not being used anymore. * Fix broken annotation editor * Load gceDefaultAccount only once when in the config page * Reset error message on auth type change * Add metric find query for projects * Remove debug code * Fix broken tests * Fix typings * Fix lint error * Slightly different approach - now having a distiction between config page default project, and project that is selectable from the dropdown in the query editor. * Fix broken tests * Attempt to fix strict ts errors * Prevent state from being set multiple times * Remove noOptionsMessage since it seems to be obosolete in react select * One more attempt to solve ts strict error * Interpolate project template variable. Make sure its loaded correctly when opening variable query editor first time * Implicit any fix * fix: typescript strict null check fixes * Return empty array in case project endpoint fails * Rename project to projectName to prevent clashing with legacy query prop * Fix broken test * fix: Stackdriver - template replace on filter label should have a regex format as that escapes the dots in the label name which is not valid. Co-authored-by: Labesse Kévin <kevin@labesse.me> Co-authored-by: Elias Cédric Laouiti <elias@abtasty.com> Co-authored-by: Daniel Lee <dan.limerick@gmail.com>
5 years ago
metricDescriptors: MetricDescriptor[],
metricType: string,
Stackdriver: Project selector (#22447) * clean PR #17366 * udpate vendor * [WIP] Implement projects management for stackdriver * [WIP] Implement projects management for stackdriver * [WIP] Implement projects management for stackdriver * Implement projects management for stackdriver * [WIP][Tests] Fix errors * clean anonymous struct * remove await * don't store project list * Add default project on query editor * gofmt * Fix tests * Move test data source to backend * Use segment instead of dropdown. remove ensure default project since it's not being used anymore. * Fix broken annotation editor * Load gceDefaultAccount only once when in the config page * Reset error message on auth type change * Add metric find query for projects * Remove debug code * Fix broken tests * Fix typings * Fix lint error * Slightly different approach - now having a distiction between config page default project, and project that is selectable from the dropdown in the query editor. * Fix broken tests * Attempt to fix strict ts errors * Prevent state from being set multiple times * Remove noOptionsMessage since it seems to be obosolete in react select * One more attempt to solve ts strict error * Interpolate project template variable. Make sure its loaded correctly when opening variable query editor first time * Implicit any fix * fix: typescript strict null check fixes * Return empty array in case project endpoint fails * Rename project to projectName to prevent clashing with legacy query prop * Fix broken test * fix: Stackdriver - template replace on filter label should have a regex format as that escapes the dots in the label name which is not valid. Co-authored-by: Labesse Kévin <kevin@labesse.me> Co-authored-by: Elias Cédric Laouiti <elias@abtasty.com> Co-authored-by: Daniel Lee <dan.limerick@gmail.com>
5 years ago
interpolatedMetricType: string,
selectedService: string
) => {
const metricTypes = getMetricTypesByService(metricDescriptors, selectedService).map((m) => ({
value: m.type,
name: m.displayName,
}));
Stackdriver: Project selector (#22447) * clean PR #17366 * udpate vendor * [WIP] Implement projects management for stackdriver * [WIP] Implement projects management for stackdriver * [WIP] Implement projects management for stackdriver * Implement projects management for stackdriver * [WIP][Tests] Fix errors * clean anonymous struct * remove await * don't store project list * Add default project on query editor * gofmt * Fix tests * Move test data source to backend * Use segment instead of dropdown. remove ensure default project since it's not being used anymore. * Fix broken annotation editor * Load gceDefaultAccount only once when in the config page * Reset error message on auth type change * Add metric find query for projects * Remove debug code * Fix broken tests * Fix typings * Fix lint error * Slightly different approach - now having a distiction between config page default project, and project that is selectable from the dropdown in the query editor. * Fix broken tests * Attempt to fix strict ts errors * Prevent state from being set multiple times * Remove noOptionsMessage since it seems to be obosolete in react select * One more attempt to solve ts strict error * Interpolate project template variable. Make sure its loaded correctly when opening variable query editor first time * Implicit any fix * fix: typescript strict null check fixes * Return empty array in case project endpoint fails * Rename project to projectName to prevent clashing with legacy query prop * Fix broken test * fix: Stackdriver - template replace on filter label should have a regex format as that escapes the dots in the label name which is not valid. Co-authored-by: Labesse Kévin <kevin@labesse.me> Co-authored-by: Elias Cédric Laouiti <elias@abtasty.com> Co-authored-by: Daniel Lee <dan.limerick@gmail.com>
5 years ago
const metricTypeExistInArray = metricTypes.some(
(m: { value: string; name: string }) => m.value === interpolatedMetricType
);
const metricTypeByService = metricTypes.length ? metricTypes[0].value : '';
const selectedMetricType = metricTypeExistInArray ? metricType : metricTypeByService;
return {
metricTypes,
selectedMetricType,
};
};
export const getAlignmentOptionsByMetric = (
metricValueType?: string,
metricKind?: string,
preprocessor?: PreprocessorType
) => {
if (preprocessor && preprocessor === PreprocessorType.Rate) {
metricKind = MetricKind.GAUGE;
}
return !metricValueType
? []
: ALIGNMENTS.filter((i) => {
Stackdriver: Project selector (#22447) * clean PR #17366 * udpate vendor * [WIP] Implement projects management for stackdriver * [WIP] Implement projects management for stackdriver * [WIP] Implement projects management for stackdriver * Implement projects management for stackdriver * [WIP][Tests] Fix errors * clean anonymous struct * remove await * don't store project list * Add default project on query editor * gofmt * Fix tests * Move test data source to backend * Use segment instead of dropdown. remove ensure default project since it's not being used anymore. * Fix broken annotation editor * Load gceDefaultAccount only once when in the config page * Reset error message on auth type change * Add metric find query for projects * Remove debug code * Fix broken tests * Fix typings * Fix lint error * Slightly different approach - now having a distiction between config page default project, and project that is selectable from the dropdown in the query editor. * Fix broken tests * Attempt to fix strict ts errors * Prevent state from being set multiple times * Remove noOptionsMessage since it seems to be obosolete in react select * One more attempt to solve ts strict error * Interpolate project template variable. Make sure its loaded correctly when opening variable query editor first time * Implicit any fix * fix: typescript strict null check fixes * Return empty array in case project endpoint fails * Rename project to projectName to prevent clashing with legacy query prop * Fix broken test * fix: Stackdriver - template replace on filter label should have a regex format as that escapes the dots in the label name which is not valid. Co-authored-by: Labesse Kévin <kevin@labesse.me> Co-authored-by: Elias Cédric Laouiti <elias@abtasty.com> Co-authored-by: Daniel Lee <dan.limerick@gmail.com>
5 years ago
return (
i.valueTypes.indexOf(metricValueType as ValueTypes) !== -1 &&
i.metricKinds.indexOf(metricKind as MetricKind) !== -1
);
});
};
export const getAggregationOptionsByMetric = (valueType: ValueTypes, metricKind: MetricKind) => {
return !metricKind
? []
: AGGREGATIONS.filter((i) => {
return i.valueTypes.indexOf(valueType) !== -1 && i.metricKinds.indexOf(metricKind) !== -1;
});
};
Stackdriver: Project selector (#22447) * clean PR #17366 * udpate vendor * [WIP] Implement projects management for stackdriver * [WIP] Implement projects management for stackdriver * [WIP] Implement projects management for stackdriver * Implement projects management for stackdriver * [WIP][Tests] Fix errors * clean anonymous struct * remove await * don't store project list * Add default project on query editor * gofmt * Fix tests * Move test data source to backend * Use segment instead of dropdown. remove ensure default project since it's not being used anymore. * Fix broken annotation editor * Load gceDefaultAccount only once when in the config page * Reset error message on auth type change * Add metric find query for projects * Remove debug code * Fix broken tests * Fix typings * Fix lint error * Slightly different approach - now having a distiction between config page default project, and project that is selectable from the dropdown in the query editor. * Fix broken tests * Attempt to fix strict ts errors * Prevent state from being set multiple times * Remove noOptionsMessage since it seems to be obosolete in react select * One more attempt to solve ts strict error * Interpolate project template variable. Make sure its loaded correctly when opening variable query editor first time * Implicit any fix * fix: typescript strict null check fixes * Return empty array in case project endpoint fails * Rename project to projectName to prevent clashing with legacy query prop * Fix broken test * fix: Stackdriver - template replace on filter label should have a regex format as that escapes the dots in the label name which is not valid. Co-authored-by: Labesse Kévin <kevin@labesse.me> Co-authored-by: Elias Cédric Laouiti <elias@abtasty.com> Co-authored-by: Daniel Lee <dan.limerick@gmail.com>
5 years ago
export const getLabelKeys = async (
datasource: CloudMonitoringDatasource,
Stackdriver: Project selector (#22447) * clean PR #17366 * udpate vendor * [WIP] Implement projects management for stackdriver * [WIP] Implement projects management for stackdriver * [WIP] Implement projects management for stackdriver * Implement projects management for stackdriver * [WIP][Tests] Fix errors * clean anonymous struct * remove await * don't store project list * Add default project on query editor * gofmt * Fix tests * Move test data source to backend * Use segment instead of dropdown. remove ensure default project since it's not being used anymore. * Fix broken annotation editor * Load gceDefaultAccount only once when in the config page * Reset error message on auth type change * Add metric find query for projects * Remove debug code * Fix broken tests * Fix typings * Fix lint error * Slightly different approach - now having a distiction between config page default project, and project that is selectable from the dropdown in the query editor. * Fix broken tests * Attempt to fix strict ts errors * Prevent state from being set multiple times * Remove noOptionsMessage since it seems to be obosolete in react select * One more attempt to solve ts strict error * Interpolate project template variable. Make sure its loaded correctly when opening variable query editor first time * Implicit any fix * fix: typescript strict null check fixes * Return empty array in case project endpoint fails * Rename project to projectName to prevent clashing with legacy query prop * Fix broken test * fix: Stackdriver - template replace on filter label should have a regex format as that escapes the dots in the label name which is not valid. Co-authored-by: Labesse Kévin <kevin@labesse.me> Co-authored-by: Elias Cédric Laouiti <elias@abtasty.com> Co-authored-by: Daniel Lee <dan.limerick@gmail.com>
5 years ago
selectedMetricType: string,
projectName: string
) => {
const refId = 'handleLabelKeysQuery';
Stackdriver: Project selector (#22447) * clean PR #17366 * udpate vendor * [WIP] Implement projects management for stackdriver * [WIP] Implement projects management for stackdriver * [WIP] Implement projects management for stackdriver * Implement projects management for stackdriver * [WIP][Tests] Fix errors * clean anonymous struct * remove await * don't store project list * Add default project on query editor * gofmt * Fix tests * Move test data source to backend * Use segment instead of dropdown. remove ensure default project since it's not being used anymore. * Fix broken annotation editor * Load gceDefaultAccount only once when in the config page * Reset error message on auth type change * Add metric find query for projects * Remove debug code * Fix broken tests * Fix typings * Fix lint error * Slightly different approach - now having a distiction between config page default project, and project that is selectable from the dropdown in the query editor. * Fix broken tests * Attempt to fix strict ts errors * Prevent state from being set multiple times * Remove noOptionsMessage since it seems to be obosolete in react select * One more attempt to solve ts strict error * Interpolate project template variable. Make sure its loaded correctly when opening variable query editor first time * Implicit any fix * fix: typescript strict null check fixes * Return empty array in case project endpoint fails * Rename project to projectName to prevent clashing with legacy query prop * Fix broken test * fix: Stackdriver - template replace on filter label should have a regex format as that escapes the dots in the label name which is not valid. Co-authored-by: Labesse Kévin <kevin@labesse.me> Co-authored-by: Elias Cédric Laouiti <elias@abtasty.com> Co-authored-by: Daniel Lee <dan.limerick@gmail.com>
5 years ago
const labels = await datasource.getLabels(selectedMetricType, refId, projectName);
return [...Object.keys(labels), ...SYSTEM_LABELS];
};
export const getAlignmentPickerData = (
valueType: string | undefined = ValueTypes.DOUBLE,
metricKind: string | undefined = MetricKind.GAUGE,
perSeriesAligner: string | undefined = AlignmentTypes.ALIGN_MEAN,
preprocessor?: PreprocessorType
) => {
const templateSrv: TemplateSrv = getTemplateSrv();
const alignOptions = getAlignmentOptionsByMetric(valueType, metricKind, preprocessor).map((option) => ({
...option,
label: option.text,
}));
if (!alignOptions.some((o: { value: string }) => o.value === templateSrv.replace(perSeriesAligner))) {
perSeriesAligner = alignOptions.length > 0 ? alignOptions[0].value : AlignmentTypes.ALIGN_MEAN;
}
return { alignOptions, perSeriesAligner };
};
export const labelsToGroupedOptions = (groupBys: string[]) => {
const groups = groupBys.reduce<
Record<
string,
Array<{
value: string;
label: string;
}>
>
>((acc, curr) => {
const arr = curr.split('.').map(startCase);
const group = (arr.length === 2 ? arr : initial(arr)).join(' ');
const option = {
value: curr,
label: curr,
};
if (acc[group]) {
acc[group] = [...acc[group], option];
} else {
acc[group] = [option];
}
return acc;
}, {});
return Object.entries(groups).map(([label, options]) => ({ label, options, expanded: true }), []);
};
Stackdriver: Support for SLO queries (#22917) * wip: add slo support * Export DataSourcePlugin * wip: break out metric query editor into its own component * wip: refactor frontend - keep SLO and Metric query in differnt objects * wip - load services and slos * Fix broken test * Add interactive slo expression builder * Change order of dropdowns * Refactoring backend model. slo unit testing in progress * Unit test migration and SLOs * Cleanup SLO editor * Simplify alias by component * Support alias by for slos * Support slos in variable queries * Fix broken last query error * Update Help section to include SLO aliases * streamline datasource resource cache * Break out api specific stuff in datasource to its own file * Move get projects call to frontend * Refactor api caching * Unit test api service * Fix lint go issue * Fix typescript strict errors * Fix test datasource * Use budget fraction selector instead of budget * Reset SLO when service is changed * Handle error in case resource call returned no data * Show real SLI display name * Use unsafe prefix on will mount hook * Store goal in query model since it will be used as soon as graph panel supports adding a threshold * Add comment to describe why componentWillMount is used * Interpolate sloid * Break out SLO aggregation into its own func * Also test group bys for metricquery test * Remove not used type fields * Remove annoying stackdriver prefix from error message * Default view param to FULL * Add part about SLO query builder in docs * Use new images * Fixes after feedback * Add one more group by test * Make stackdriver types internal * Update docs/sources/features/datasources/stackdriver.md Co-Authored-By: Diana Payton <52059945+oddlittlebird@users.noreply.github.com> * Update docs/sources/features/datasources/stackdriver.md Co-Authored-By: Diana Payton <52059945+oddlittlebird@users.noreply.github.com> * Update docs/sources/features/datasources/stackdriver.md Co-Authored-By: Diana Payton <52059945+oddlittlebird@users.noreply.github.com> * Updates after PR feedback * add test for when no alias by defined * fix infinite loop when newVariables feature flag is on onChange being called in componentDidUpdate produces an infinite loop when using the new React template variable implementation. Also fixes a spelling mistake * implements feedback for documentation changes * more doc changes Co-authored-by: Diana Payton <52059945+oddlittlebird@users.noreply.github.com> Co-authored-by: Daniel Lee <dan.limerick@gmail.com>
5 years ago
export const stringArrayToFilters = (filterArray: string[]) =>
chunk(filterArray, 4).map(([key, operator, value, condition = 'AND']) => ({
Stackdriver: Support for SLO queries (#22917) * wip: add slo support * Export DataSourcePlugin * wip: break out metric query editor into its own component * wip: refactor frontend - keep SLO and Metric query in differnt objects * wip - load services and slos * Fix broken test * Add interactive slo expression builder * Change order of dropdowns * Refactoring backend model. slo unit testing in progress * Unit test migration and SLOs * Cleanup SLO editor * Simplify alias by component * Support alias by for slos * Support slos in variable queries * Fix broken last query error * Update Help section to include SLO aliases * streamline datasource resource cache * Break out api specific stuff in datasource to its own file * Move get projects call to frontend * Refactor api caching * Unit test api service * Fix lint go issue * Fix typescript strict errors * Fix test datasource * Use budget fraction selector instead of budget * Reset SLO when service is changed * Handle error in case resource call returned no data * Show real SLI display name * Use unsafe prefix on will mount hook * Store goal in query model since it will be used as soon as graph panel supports adding a threshold * Add comment to describe why componentWillMount is used * Interpolate sloid * Break out SLO aggregation into its own func * Also test group bys for metricquery test * Remove not used type fields * Remove annoying stackdriver prefix from error message * Default view param to FULL * Add part about SLO query builder in docs * Use new images * Fixes after feedback * Add one more group by test * Make stackdriver types internal * Update docs/sources/features/datasources/stackdriver.md Co-Authored-By: Diana Payton <52059945+oddlittlebird@users.noreply.github.com> * Update docs/sources/features/datasources/stackdriver.md Co-Authored-By: Diana Payton <52059945+oddlittlebird@users.noreply.github.com> * Update docs/sources/features/datasources/stackdriver.md Co-Authored-By: Diana Payton <52059945+oddlittlebird@users.noreply.github.com> * Updates after PR feedback * add test for when no alias by defined * fix infinite loop when newVariables feature flag is on onChange being called in componentDidUpdate produces an infinite loop when using the new React template variable implementation. Also fixes a spelling mistake * implements feedback for documentation changes * more doc changes Co-authored-by: Diana Payton <52059945+oddlittlebird@users.noreply.github.com> Co-authored-by: Daniel Lee <dan.limerick@gmail.com>
5 years ago
key,
operator,
value,
condition,
}));
export const alignmentPeriodLabel = (customMetaData: CustomMetaData, datasource: CloudMonitoringDatasource) => {
const { perSeriesAligner, alignmentPeriod } = customMetaData;
if (!alignmentPeriod || !perSeriesAligner) {
return '';
}
const alignment = ALIGNMENTS.find((ap) => ap.value === datasource.templateSrv.replace(perSeriesAligner));
const seconds = parseInt(alignmentPeriod, 10);
const hms = rangeUtil.secondsToHms(seconds);
return `${hms} interval (${alignment?.text ?? ''})`;
};
export const getMetricType = (query?: TimeSeriesList) => {
const metricTypeKey = query?.filters?.findIndex((f) => f === 'metric.type')!;
// filters are in the format [key, operator, value] so we need to add 2 to get the value
const metricType = query?.filters?.[metricTypeKey + 2];
return metricType || '';
};
export const setMetricType = (query: TimeSeriesList, metricType: string) => {
if (!query.filters) {
query.filters = ['metric.type', '=', metricType];
return query;
}
const metricTypeKey = query?.filters?.findIndex((f) => f === 'metric.type')!;
if (metricTypeKey === -1) {
query.filters.push('metric.type', '=', metricType);
} else {
// filters are in the format [key, operator, value] so we need to add 2 to get the value
query.filters![metricTypeKey + 2] = metricType;
}
return query;
};