The open and composable observability and data visualization platform. Visualize metrics, logs, and traces from multiple sources like Prometheus, Loki, Elasticsearch, InfluxDB, Postgres and many more.
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
grafana/public/app/plugins/datasource/loki/metricTimeSplitting.test.ts

23 lines
914 B

import { splitTimeRange } from './metricTimeSplitting';
Loki Query Splitting: Split queries into sub-queries with smaller time interval (#62767) * Range splitting: range splitting function * Range splitting: experiment with 1 hour splits * Range splitting: reorganize code * Range splitting: improve code readability and meaning * Range splitting: add partition limit to prevent infinite loops * Range splitting: add error handling * Range splitting: disable for logs queries * Range splitting: support any arbitrary time splitting + respect original from/to in the partition * Chore: remove console logs * Chore: delete unused import * Range splitting: actually send requests in sequence * Range splitting: do not split when > 1 query * Range splitting: combine frames * Chore: rename function * split in reverse * polished reversing * keep reference to the right frame in the result * Range splitting: change request state to Streaming * Range splitting: fix moving only 1 unit of time instead of the provided one * Chore: change default parameter to timeShift = 1 * Range splitting: do not split for range queqries * Range splitting: add initial support for log queries * Range splitting: do not use MutableDataFrame It has bad performance and it's not required * Chore: remove unused export * Query Splitting: move to module * loki: split: fix off-by-one error (#62966) loki: split: fix off-by-one loop * Range splitting: disable for logs volume queries * Range splitting: combine any number of fields, not just hardcoded 2 * Range splitting: optimize frame-combining function * Range splitting: further optimize * Range splitting: combine frame length * Range splitting: combine stats * Range splitting: combine stats without assuming the same order * Query splitting: catch and raise errors * Range splitting: create feature flag * Range splitting: implement feature flag * Range splitting: add unit test for datasource query * Range splitting: add basic test for runPartitionedQuery * Range splitting: add unit test for resultLimitReached * Range splitting: test frame merging * Chore: fix unit test --------- Co-authored-by: Sven Grossmann <svennergr@gmail.com> Co-authored-by: Gábor Farkas <gabor.farkas@gmail.com>
2 years ago
describe('metric splitTimeRange', () => {
Loki Query Splitting: Split queries into sub-queries with smaller time interval (#62767) * Range splitting: range splitting function * Range splitting: experiment with 1 hour splits * Range splitting: reorganize code * Range splitting: improve code readability and meaning * Range splitting: add partition limit to prevent infinite loops * Range splitting: add error handling * Range splitting: disable for logs queries * Range splitting: support any arbitrary time splitting + respect original from/to in the partition * Chore: remove console logs * Chore: delete unused import * Range splitting: actually send requests in sequence * Range splitting: do not split when > 1 query * Range splitting: combine frames * Chore: rename function * split in reverse * polished reversing * keep reference to the right frame in the result * Range splitting: change request state to Streaming * Range splitting: fix moving only 1 unit of time instead of the provided one * Chore: change default parameter to timeShift = 1 * Range splitting: do not split for range queqries * Range splitting: add initial support for log queries * Range splitting: do not use MutableDataFrame It has bad performance and it's not required * Chore: remove unused export * Query Splitting: move to module * loki: split: fix off-by-one error (#62966) loki: split: fix off-by-one loop * Range splitting: disable for logs volume queries * Range splitting: combine any number of fields, not just hardcoded 2 * Range splitting: optimize frame-combining function * Range splitting: further optimize * Range splitting: combine frame length * Range splitting: combine stats * Range splitting: combine stats without assuming the same order * Query splitting: catch and raise errors * Range splitting: create feature flag * Range splitting: implement feature flag * Range splitting: add unit test for datasource query * Range splitting: add basic test for runPartitionedQuery * Range splitting: add unit test for resultLimitReached * Range splitting: test frame merging * Chore: fix unit test --------- Co-authored-by: Sven Grossmann <svennergr@gmail.com> Co-authored-by: Gábor Farkas <gabor.farkas@gmail.com>
2 years ago
it('should split time range into chunks', () => {
const start = Date.parse('2022-02-06T14:10:03');
const end = Date.parse('2022-02-06T14:11:03');
const step = 10 * 1000;
expect(splitTimeRange(start, end, step, 25000)).toStrictEqual([
Loki Query Splitting: Split queries into sub-queries with smaller time interval (#62767) * Range splitting: range splitting function * Range splitting: experiment with 1 hour splits * Range splitting: reorganize code * Range splitting: improve code readability and meaning * Range splitting: add partition limit to prevent infinite loops * Range splitting: add error handling * Range splitting: disable for logs queries * Range splitting: support any arbitrary time splitting + respect original from/to in the partition * Chore: remove console logs * Chore: delete unused import * Range splitting: actually send requests in sequence * Range splitting: do not split when > 1 query * Range splitting: combine frames * Chore: rename function * split in reverse * polished reversing * keep reference to the right frame in the result * Range splitting: change request state to Streaming * Range splitting: fix moving only 1 unit of time instead of the provided one * Chore: change default parameter to timeShift = 1 * Range splitting: do not split for range queqries * Range splitting: add initial support for log queries * Range splitting: do not use MutableDataFrame It has bad performance and it's not required * Chore: remove unused export * Query Splitting: move to module * loki: split: fix off-by-one error (#62966) loki: split: fix off-by-one loop * Range splitting: disable for logs volume queries * Range splitting: combine any number of fields, not just hardcoded 2 * Range splitting: optimize frame-combining function * Range splitting: further optimize * Range splitting: combine frame length * Range splitting: combine stats * Range splitting: combine stats without assuming the same order * Query splitting: catch and raise errors * Range splitting: create feature flag * Range splitting: implement feature flag * Range splitting: add unit test for datasource query * Range splitting: add basic test for runPartitionedQuery * Range splitting: add unit test for resultLimitReached * Range splitting: test frame merging * Chore: fix unit test --------- Co-authored-by: Sven Grossmann <svennergr@gmail.com> Co-authored-by: Gábor Farkas <gabor.farkas@gmail.com>
2 years ago
[Date.parse('2022-02-06T14:10:00'), Date.parse('2022-02-06T14:10:10')],
[Date.parse('2022-02-06T14:10:20'), Date.parse('2022-02-06T14:10:40')],
[Date.parse('2022-02-06T14:10:50'), Date.parse('2022-02-06T14:11:10')],
]);
});
it('should return the original interval if requested duration is smaller than step', () => {
Loki Query Splitting: Split queries into sub-queries with smaller time interval (#62767) * Range splitting: range splitting function * Range splitting: experiment with 1 hour splits * Range splitting: reorganize code * Range splitting: improve code readability and meaning * Range splitting: add partition limit to prevent infinite loops * Range splitting: add error handling * Range splitting: disable for logs queries * Range splitting: support any arbitrary time splitting + respect original from/to in the partition * Chore: remove console logs * Chore: delete unused import * Range splitting: actually send requests in sequence * Range splitting: do not split when > 1 query * Range splitting: combine frames * Chore: rename function * split in reverse * polished reversing * keep reference to the right frame in the result * Range splitting: change request state to Streaming * Range splitting: fix moving only 1 unit of time instead of the provided one * Chore: change default parameter to timeShift = 1 * Range splitting: do not split for range queqries * Range splitting: add initial support for log queries * Range splitting: do not use MutableDataFrame It has bad performance and it's not required * Chore: remove unused export * Query Splitting: move to module * loki: split: fix off-by-one error (#62966) loki: split: fix off-by-one loop * Range splitting: disable for logs volume queries * Range splitting: combine any number of fields, not just hardcoded 2 * Range splitting: optimize frame-combining function * Range splitting: further optimize * Range splitting: combine frame length * Range splitting: combine stats * Range splitting: combine stats without assuming the same order * Query splitting: catch and raise errors * Range splitting: create feature flag * Range splitting: implement feature flag * Range splitting: add unit test for datasource query * Range splitting: add basic test for runPartitionedQuery * Range splitting: add unit test for resultLimitReached * Range splitting: test frame merging * Chore: fix unit test --------- Co-authored-by: Sven Grossmann <svennergr@gmail.com> Co-authored-by: Gábor Farkas <gabor.farkas@gmail.com>
2 years ago
const start = Date.parse('2022-02-06T14:10:03');
const end = Date.parse('2022-02-06T14:10:33');
const step = 10 * 1000;
expect(splitTimeRange(start, end, step, 1000)).toEqual([[start, end]]);
Loki Query Splitting: Split queries into sub-queries with smaller time interval (#62767) * Range splitting: range splitting function * Range splitting: experiment with 1 hour splits * Range splitting: reorganize code * Range splitting: improve code readability and meaning * Range splitting: add partition limit to prevent infinite loops * Range splitting: add error handling * Range splitting: disable for logs queries * Range splitting: support any arbitrary time splitting + respect original from/to in the partition * Chore: remove console logs * Chore: delete unused import * Range splitting: actually send requests in sequence * Range splitting: do not split when > 1 query * Range splitting: combine frames * Chore: rename function * split in reverse * polished reversing * keep reference to the right frame in the result * Range splitting: change request state to Streaming * Range splitting: fix moving only 1 unit of time instead of the provided one * Chore: change default parameter to timeShift = 1 * Range splitting: do not split for range queqries * Range splitting: add initial support for log queries * Range splitting: do not use MutableDataFrame It has bad performance and it's not required * Chore: remove unused export * Query Splitting: move to module * loki: split: fix off-by-one error (#62966) loki: split: fix off-by-one loop * Range splitting: disable for logs volume queries * Range splitting: combine any number of fields, not just hardcoded 2 * Range splitting: optimize frame-combining function * Range splitting: further optimize * Range splitting: combine frame length * Range splitting: combine stats * Range splitting: combine stats without assuming the same order * Query splitting: catch and raise errors * Range splitting: create feature flag * Range splitting: implement feature flag * Range splitting: add unit test for datasource query * Range splitting: add basic test for runPartitionedQuery * Range splitting: add unit test for resultLimitReached * Range splitting: test frame merging * Chore: fix unit test --------- Co-authored-by: Sven Grossmann <svennergr@gmail.com> Co-authored-by: Gábor Farkas <gabor.farkas@gmail.com>
2 years ago
});
});