17 Commits (6e2b7d25898f321bfd6f94de05d6ce934c785da9)

Author	SHA1	Message	Date
Christian Haudum	95b885691d	Preserve interval parameter when splitting queries by time (#5622) ... * Preserve interval parameter when splitting queries by time Signed-off-by: Christian Haudum <christian.haudum@gmail.com> * Add tests for interval param on Loki downstream requests Signed-off-by: Christian Haudum <christian.haudum@gmail.com> * Add changelog entry for #5622 Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	4 years ago
Cyril Tovena	9b6f17f168	Add the total number of subqueries to query statistics (#5397) ... * Add subqueries count Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes tests Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	4 years ago
Owen Diehl	91d837e79c	Refactor/remove global splitby (#5243) ... * cleans up internal limits struct, removes unnecessary logic * moves querier.split-queries-by-interval to limits, simplifies resulting code. * docs, changelog * flag supports model.duration instead of time.duration * lint and adds new error for deprecated field * reregisters queryrange flags * comment lint * ensures we validate queryrange config and simplifies embedded struct validate() * 0 still disables * simplify cache key generation and protect against divide by zero	4 years ago
Callum Styan	f24b84c15a	[querier] Abort a query in limitedRoundTripper if parallelism is 0 (#5190) ... * Abort a query in limitedRoundTripper if parallelism is 0. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Fix failing tests related to fakeLimits change. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Change limitedRoundTripper parallelism error message. Signed-off-by: Callum Styan <callumstyan@gmail.com>	4 years ago
Sandeep Sukhani	cd1537d06e	step align start and end time of the original query while splitting it (#5217) ... * step align start and end time of the original query while splitting it * also step align queries with step > split interval	4 years ago
Sandeep Sukhani	dad7fca2fd	align metric queries by step and other queries by split interval (#5181)	4 years ago
Cyril Tovena	1a7614ff70	Avoid splitting large range vector aggregation. (#5172) ... * Reduce split by for large range vector Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Update pkg/querier/queryrange/split_by_interval.go Co-authored-by: Christian Haudum <christian.haudum@gmail.com> * improve fn name. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> Co-authored-by: Christian Haudum <christian.haudum@gmail.com>	4 years ago
Dylan Guedes	0e3df1938f	Copy querier subpackages dependency into Loki (#5068) ... * Fork cortex `querier/series`. * Fork cortex `querier/chunkstore`. * Fork cortex `querier/lazyquery`. * Use Loki forked cortex code. * Copy cortex `querier/queryrange` as `querier/queryrange/queryrangebase`. * Fork cortex `pkg/querier/queryrange` to `pkg/querier/queryrange/queryrangebase`. * Tell protoc to use gogo.types instead of pb.Any. * Use `pkg/querier/queryrange/queryrangebase`. * Remove cortex `queryrange` vendoring. * Fix lint. * Use `_ = ...` syntax instead of comment to fix lint.	4 years ago
Sandeep Sukhani	b865afc0d7	fix splitting of queries when step is larger than split interval (#5006)	4 years ago
Owen Diehl	1f6453fb81	exposes loki codec (#3758)	5 years ago
Cyril Tovena	93f2b2d89b	Fixes split interval for metrics queries. (#3459) ... The step needs to be taken into account: - to be removed on each split end. - to not split queries that have a step lower than the split interval. The code is taken from Cortex but adjusted to keep the nanoseconds precision. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago
Aditya C S	5b8d9b4ed6	Support series query in query-frontend (#2123)	6 years ago
Owen Diehl	89d80a6fc4	Feature/querysharding ii (#1927) ... * [wip] sharding evaluator/ast * [wip] continues experimenting with ast mapping * refactoring in preparation for binops * evaluators can pass state to other evaluators * compiler alignment * Evaluator method renamed to StepEvaluator * chained evaluator impl * tidying up sharding code * handling for ConcatSampleExpr * downstream iterator * structure for downstreaming asts * outlines sharding optimizations * work on sharding mapper * ast sharding optimizations * test for different logrange positions * shard mapper tests * stronger ast sharding & tests * shardmapper tests for string->string * removes sharding evaluator code * removes unused ctx arg * Revert "removes sharding evaluator code" This reverts commit 55d41b9519. * interfaces for downstreaming, type conversions * sharding plumbing on frontend * type alignment in queryrange to downstream sharded queriers * downstreaming support for sharding incl storage code * removes chainedevaluator * comment alignment * storage shard injection * speccing out testware for sharding equivalence * [wip] shared engine refactor * sorting streams, sharding eval fixes * downstream evaluator embeds defaultevaluator * other pkgs adopt logql changes * metrics & logs use same middleware instantiation process * wires up shardingware * middleware per metrics/logfilter * empty step populating StepEvaluator promql.Matrix adapter * sharding metrics * log/span injection into sharded engine * sharding metrics avoids multiple instantiation * downstreamhandler tracing * sharding parameterized libsonnet * removes querier replicas * default 32 concurrency for workers * jsonnet correct level override * unquote true in yaml * lowercase error + downstreamEvaluator defaults to embedded defaultEvaluator * makes shardRecorder private * logs query on failed parse * refactors engine to be multi-use, minimizes logger injection, generalizes Query methods, removes Engine interface * basic tests for querysharding mware * [wip] concurrent evaluator * integrates stat propagation into sharding evaluator * splitby histogram * extends le bounds for bytes processed * byte throughput histogram buckets to 40gb * chunk duration mixin * fixes merge w/ field rename * derives logger in sharded engine via ctx & logs some downstream evaluators * moves sharded engine to top, adds comments * logs failed merge results in stats ctx * snapshotting stats merge logic is done more effectively * per query concurrency controlled via downstreamer * unexports decodereq * queryrange testware * downstreamer tests * pr requests	6 years ago
Aditya C S	f9ff2a4b02	Add goimport local-prefixes configuration to .golangci.yml (#1710)	6 years ago
Owen Diehl	3e075497cd	Feature/per tenant splitby (#1565) ... * split by overrides by tenant * tenant aware metric query splitting * consistent split_queries_by_interval naming * updates readme * fixes rebase conflict * refactor to support limit based splitby intervals in all locations * simplifies limits cachesplitter * fixes comment	6 years ago
Owen Diehl	a1d090e161	Fix/deadlock frontend queue (#1490) ... * buffers splitby interval channels to prevent deadlocks * improves codec merge test requiring merging from multiple intervals * bound concurrency by min(parallelism,jobs) * approximates goroutine test	6 years ago
Cyril Tovena	4153740a12	Loki Query Frontend (#1442) ... * Adds frontend to Loki. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Improves tests. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes sneaky bug in entries sorting. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes the split by interval. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Tweak jsonnet deployments and add a way to lint/fmt. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * lint. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * fix timezone issue. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Improve tests and rollback change in loghttp package. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes a flaky test that might run one more goroutine. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes windows build. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Improve tracing in the split by interval. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add test stream to proto conversion. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes flappy retry test. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Remove err shadowing in stopQueryFrontend as it was confusing. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Refactor grpc message size in the libsonnet config file. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Don't check auth header for GRPC TransferChunks. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Query frontend (#3) * frontend codec merging optimizations * codec benchmarks * removes unused bounds code in queryrange ordering * [wip] splitby uses channels instead of sub batching intervals * splitBy channel limit test * single allocation for merging entries from a single stream * skip merging loki responses when limit is already hit * removes checks for unlimited queries in queryrange * removes splitByInterval{,.interval} spans * removes interval_batch_size from jsonnet lib * moves benchmark utils to own file * renames markers -> entries * priority queue comments * Removes unused logRequest. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Sets the cache interval to the same split interval. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Missing import libsonnet for the frontend. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Frontend should not be a cluster IP. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> Co-authored-by: Owen Diehl <ow.diehl@gmail.com>	6 years ago