prometheus

Commit Graph

Author	SHA1	Message	Date
Thomas Jackson	a6dace8829	Check for timeout in each iteration of matrixSelector (#4300 ) Signed-off-by: Thomas Jackson <jacksontj.89@gmail.com> Fixes #4288	8 years ago
Thomas Jackson	630f42fcf1	Timeout if populating iterators takes too long (#4291 ) Right now promql won't time out a request if populating the iterators takes a long time. Signed-off-by: Thomas Jackson <jacksontj.89@gmail.com> Fixes #4289	8 years ago
Alin Sinpalean	91ce63a140	Log the line when failing a PromQL test. (#4272 ) Signed-off-by: Alin Sinpalean <alin.sinpalean@gmail.com>	8 years ago
Thomas Jackson	404abe0f1c	Bubble up errors to promql from populating iterators (#4136 ) This changes the Walk/Inspect API inside the promql package to bubble up errors. This is done by having the inspector return an error (instead of a bool) and then bubbling that up in the Walk. This way if any error is encountered in the Walk() the walk will stop and return the error. This avoids issues where errors from the Querier where being ignored (causing incorrect promql evaluation). Signed-off-by: Thomas Jackson <jacksontj.89@gmail.com> Fixes #4136	8 years ago
Mario Trangoni	0e2aa35771	promql: fix unconvert issues (#4040 ) See, $ gometalinter --vendor --disable-all --enable=unconvert --deadline 6m ./... promql/engine.go:1396:26⚠️ unnecessary conversion (unconvert) promql/engine.go:1396:40⚠️ unnecessary conversion (unconvert) promql/engine.go:1398:26⚠️ unnecessary conversion (unconvert) promql/engine.go:1398:40⚠️ unnecessary conversion (unconvert) promql/engine.go:1427:26⚠️ unnecessary conversion (unconvert) promql/engine.go:1427:40⚠️ unnecessary conversion (unconvert) promql/engine.go:1429:26⚠️ unnecessary conversion (unconvert) promql/engine.go:1429:40⚠️ unnecessary conversion (unconvert) promql/engine.go:1505:50⚠️ unnecessary conversion (unconvert) promql/engine.go:1573:46⚠️ unnecessary conversion (unconvert) promql/engine.go:1578:46⚠️ unnecessary conversion (unconvert) promql/engine.go:1591:80⚠️ unnecessary conversion (unconvert) promql/engine.go:1602:94⚠️ unnecessary conversion (unconvert) promql/engine.go:1630:18⚠️ unnecessary conversion (unconvert) promql/engine.go:1631:24⚠️ unnecessary conversion (unconvert) promql/engine.go:1634:18⚠️ unnecessary conversion (unconvert) promql/engine.go:1635:34⚠️ unnecessary conversion (unconvert) promql/functions.go:302:42⚠️ unnecessary conversion (unconvert) promql/functions.go:315:42⚠️ unnecessary conversion (unconvert) promql/functions.go:334:26⚠️ unnecessary conversion (unconvert) promql/functions.go:395:31⚠️ unnecessary conversion (unconvert) promql/functions.go:406:31⚠️ unnecessary conversion (unconvert) promql/functions.go:454:27⚠️ unnecessary conversion (unconvert) promql/functions.go:701:46⚠️ unnecessary conversion (unconvert) promql/functions.go:701:78⚠️ unnecessary conversion (unconvert) promql/functions.go:730:43⚠️ unnecessary conversion (unconvert) promql/functions.go:1220:23⚠️ unnecessary conversion (unconvert) promql/functions.go:1249:23⚠️ unnecessary conversion (unconvert) promql/quantile.go:107:54⚠️ unnecessary conversion (unconvert) promql/quantile.go:182:16⚠️ unnecessary conversion (unconvert) promql/quantile.go:182:64⚠️ unnecessary conversion (unconvert) Signed-off-by: Mario Trangoni <mjtrangoni@gmail.com>	8 years ago
Brian Brazil	dd6781add2	Optimise PromQL (#3966 ) * Move range logic to 'eval' Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Make aggregegate range aware Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * PromQL is statically typed, so don't eval to find the type. Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Extend rangewrapper to multiple exprs Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Start making function evaluation ranged Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Make instant queries a special case of range queries Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Eliminate evalString Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Evaluate range vector functions one series at a time Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Make unary operators range aware Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Make binops range aware Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Pass time to range-aware functions. Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Make simple _over_time functions range aware Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Reduce allocs when working with matrix selectors Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Add basic benchmark for range evaluation Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Reuse objects for function arguments Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Do dropmetricname and allocating output vector only once. Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Add range-aware support for range vector functions with params Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Optimise holt_winters, cut cpu and allocs by ~25% Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Make rate&friends range aware Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Make more functions range aware. Document calling convention. Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Make date functions range aware Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Make simple math functions range aware Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Convert more functions to be range aware Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Make more functions range aware Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Specialcase timestamp() with vector selector arg for range awareness Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Remove transition code for functions Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Remove the rest of the engine transition code Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Remove more obselete code Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Remove the last uses of the eval* functions Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Remove engine finalizers to prevent corruption The finalizers set by matrixSelector were being called just before the value they were retruning to the pool was then being provided to the caller. Thus a concurrent query could corrupt the data that the user has just been returned. Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Add new benchmark suite for range functinos Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Migrate existing benchmarks to new system Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Expand promql benchmarks Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Simply test by removing unused range code Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * When testing instant queries, check range queries too. To protect against subsequent steps in a range query being affected by the previous steps, add a test that evaluates an instant query that we know works again as a range query with the tiimestamp we care about not being the first step. Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Reuse ring for matrix iters. Put query results back in pool. Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Reuse buffer when iterating over matrix selectors Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Unary minus should remove metric name Cut down benchmarks for faster runs. Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Reduce repetition in benchmark test cases Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Work series by series when doing normal vectorSelectors Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Optimise benchmark setup, cuts time by 60% Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Have rangeWrapper use an evalNodeHelper to cache across steps Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Use evalNodeHelper with functions Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Cache dropMetricName within a node evaluation. This saves both the calculations and allocs done by dropMetricName across steps. Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Reuse input vectors in rangewrapper Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Reuse the point slices in the matrixes input/output by rangeWrapper Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Make benchmark setup faster using AddFast Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Simplify benchmark code. Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Add caching in VectorBinop Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Use xor to have one-level resultMetric hash key Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Add more benchmarks Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Call Query.Close in apiv1 This allows point slices allocated for the response data to be reused by later queries, saving allocations. Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Optimise histogram_quantile It's now 5-10% faster with 97% less garbage generated for 1k steps Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Make the input collection in rangeVector linear rather than quadratic Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Optimise label_replace, for 1k steps 15x fewer allocs and 3x faster Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Optimise label_join, 1.8x faster and 11x less memory for 1k steps Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Expand benchmarks, cleanup comments, simplify numSteps logic. Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Address Fabian's comments Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Comments from Alin. Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Address jrv's comments Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Remove dead code Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Address Simon's comments. Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Rename populateIterators, pre-init some sizes Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Handle case where function has non-matrix args first Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Split rangeWrapper out to rangeEval function, improve comments Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Cleanup and make things more consistent Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Make EvalNodeHelper public Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Fabian's comments. Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>	8 years ago
Henri DF	986674a790	Make some lexing errors more informative (#4167 ) Signed-off-by: Henri DF <henridf@gmail.com>	8 years ago
Elif T. Kuş	57dcdfb15f	Rewrote tests with testutil for several test files (#4086 ) * promql: Rewrote tests with testutil for functions_test Signed-off-by: Elif T. Kuş <elifkus@gmail.com> * pkg/relabel: Rewrote tests with testutil for relabel_test Signed-off-by: Elif T. Kuş <elifkus@gmail.com> * discovery/consul: Rewrote tests with testutil for consul_test Signed-off-by: Elif T. Kuş <elifkus@gmail.com> * scrape: Rewrote tests with testutil for manager_test Signed-off-by: Elif T. Kuş <elifkus@gmail.com>	8 years ago
Karsten Weiss	d79d573f71	Fix spelling mistakes found by codespell (#4065 ) Signed-off-by: Karsten Weiss <knweiss@gmail.com>	8 years ago
David King	6286c10df0	Fix OOM when a large K is used in topk queries (#4087 ) This attempts to close #3973. Handles cases where the length of the input vector to an aggregate topk / bottomk function is less than the K paramater. The change updates Prometheus to allocate a result vector the same length as the input vector in these cases. Previously Prometheus would out-of-memory panic for large K values. This change makes that unlikely unless the size of the input vector is equally large. Signed-off-by: David King <dave@davbo.org>	8 years ago
Tony Lee	7cd56f56df	add queue_time slice to query_duration_seconds (#4050 )	8 years ago
Warren Fernandes	d49a3df55b	Parser test cleanup (#3977 ) * parser test cleanup - Test against the exported package functions instead of the private functions. * Improves readability of TestParseSeries - Moves package function closer to parser function	8 years ago
Anton Tereshchenkov	18bbec050c	promql: propagate storage errors	8 years ago
Brian Brazil	bf7d87aed2	Cleanup storage from all tests. Fixed #3299	8 years ago
Brian Brazil	c0ce35d2d3	Only show debug output on test failure	8 years ago
Brian Brazil	e6ea146c81	Make benchmark tests pass A new query object is needed for each evaulation, as the iterators would otherwise be shared across evaluations.	8 years ago
Nikunj Aggarwal	998dfcbac6	Expose itemtype outside the package (#3933 )	8 years ago
ferhat elmas	ffa673f7d8	General simplifications (#3887 ) Another try as in #1516	8 years ago
Fabian Reinartz	7ccd4b39b8	*: implement query params This adds a parameter to the storage selection interface which allows query engine(s) to pass information about the operations surrounding a data selection. This can for example be used by remote storage backends to infer the correct downsampling aggregates that need to be provided.	8 years ago
Krasi Georgiev	a53d4ed197	drop metric name for bool modifier (#3821 ) fixes #3820	8 years ago
Krasi Georgiev	4801573b64	time() return milliseconds (#3811 )	8 years ago
Julius Volz	953af2c089	promql: Make printer formatting less vintage (#3721 ) - lower-case modifiers - reverse order of aggregation modifiers and aggregated expression - remove spacing before modifier parentheses	8 years ago
Julius Volz	1e943fc10a	promql: Fix printing of empty without() (#3719 ) * promql: Fix printing of empty without() Fixes https://github.com/prometheus/prometheus/issues/3704 * Test cleanup fixup	8 years ago
Brian Brazil	b418063d1a	Add tests for negative selectors. (#3616 ) https://github.com/prometheus/prometheus/issues/3575	8 years ago
Fabian Reinartz	f8fccc73d8	promql: remove global metrics	8 years ago
Fabian Reinartz	83cd270ea4	*: adapt to storage interface changes	8 years ago
David Kaltschmidt	87c46ea6c3	Renamed TotalEvalTime to EvalTotalTime * TotalFoo suggested a comprehensive timing, but TotalEvalTime was part of the Exec timings, together with Queue timings * The other option was to rename ExecTotalTime to TotalExecTime, but there was already ExecQueueTime, suggesting Exec to be some sort of group	8 years ago
David Kaltschmidt	c93e54d240	Adds execution timer stats to the range query API consumers should be able to get insight into the query run times. The UI currently measures total roundtrip times. This PR allows for more fine grained metrics to be exposed. * adds new timer for total execution time (queue + eval) * expose new timer, queue timer, and eval timer in stats field of the range query response: ```json { "status": "success", "data": { "resultType": "matrix", "result": [], "stats": { "execQueueTimeNs": 4683, "execTotalTimeNs": 2086587, "totalEvalTimeNs": 2077851 } } } ``` * stats field is optional, only set when query parameter `stats` is not empty Try it via ```sh curl 'http://localhost:9090/api/v1/query_range?query=up&start=1486480279&end=1486483879&step=14000&stats=true' ``` Review feedback * moved query stats json generation to query_stats.go * use seconds for all query timers * expose all timers available * Changed ExecTotalTime string representation from Exec queue total time to Exec total time	8 years ago
Julius Volz	099df0c5f0	Migrate "golang.org/x/net/context" -> "context" (#3333 ) In some places, where ctxhttp or gRPC are concerned, we still need to use the old contexts.	8 years ago
Brian Brazil	7158675aa8	Add back continue. Accidentally removed in `15a931dbdb`	8 years ago
Brian Brazil	99905f82a6	Remove keep_common modifier. See #3060	8 years ago
Brian Brazil	b2ac3d2d86	Remove count_scalar and drop_common_labels. For #3060	8 years ago
Brian Brazil	67274f0794	Remove 4 interval staleness heuristic. (#3244 ) This means that if there is no stale marker, only the usual staleness delta (5m) applies. It has occured to me that there is an oddity in the heurestic. It works fine as long as you have 2 points within the last 5m, but breaks down when the time window advances to the point where you have just 1 point. Consider you had points at t=0 and t=10. With the heurestic it goes stale at t=51, up until t=300. However from t=301 until t=310 we only see the t=10 point and the series comes back to life. That is not desirable. I don't see a way to keep this form of heurestic working given this issue, so thus I'm removing it.	8 years ago
Julius Volz	f7e8348a88	Re-add contexts to storage.Storage.Querier() (#3230 ) * Re-add contexts to storage.Storage.Querier() These are needed when replacing the storage by a multi-tenant implementation where the tenant is stored in the context. The 1.x query interfaces already had contexts, but they got lost in 2.x. * Convert promql.Engine to use native contexts	8 years ago
Fabian Reinartz	d21f149745	*: migrate to go-kit/log	8 years ago
Brian Brazil	2354c2544b	Set timestamp for date functions (#3070 )	8 years ago
Brian Brazil	4c8173acac	Use timestamp of a sample in deriv() to avoid FP issues (#2958 ) With the squaring of the timestamp, we run into the limitations of the 53bit mantissa for a 64bit float. By subtracting away a timestamp of one of the samples (which is how the intercept is used) we avoid this issue in practice as it's unlikely that it is used over a very long time range. Fixes #2674	9 years ago
Alexey Palazhchenko	695ec0b981	Fix few typos. (#2962 )	9 years ago
Goutham Veeramachaneni	4194d2ac79	Call At() only if Next() is true Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	9 years ago
Tom Wilkie	835eb8c653	Add _test.go suffix to promql/{bench.go, test.go} to prevent importing the testing package in a normal binary.	9 years ago
Goutham Veeramachaneni	b7eddbcd98	textparse: Add fuzzing and fix bug caught See https://github.com/cznic/golex/issues/11 for info on the bug Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	9 years ago
Goutham Veeramachaneni	d407bd150c	Consolidate the duration params in CLI * All CLI params moved to model.Duration Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	9 years ago
Brian Brazil	6f5d952132	Check for invalid utf-8 in lexer strings. This protects against invalid utf-8 sneaking in via label_replace.	9 years ago
Harsh Agarwal	16867c89a7	implement label_join issue 1147 (#2806 ) Replace OptionalArgs int with Variadic int.	9 years ago
Goutham Veeramachaneni	507790a357	Rework logging to use explicitly passed logger Mostly cleaned up the global logger use. Still some uses in discovery package. Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	9 years ago
Goutham Veeramachaneni	baf5b0f0fc	Fix error where we look into the future. (#2829 ) * Fix error where we look into the future. So currently we are adding values that are in the future for an older timestamp. For example, if we have [(1, 1), (150, 2)] we will end up showing [(1, 1), (2,2)]. Further it is not advisable to call .At() after Next() returns false. Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in> * Retuen early if done Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in> * Handle Seek() where we reach the end of iterator Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in> * Simplify code Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	9 years ago
Brian Brazil	220e78b9c3	Consider a series stale after 4.1 intervals with no data. To cover the cases where stale markers may not be available, we need to infer the interval and mark series stale based on that. As we're lacking stale markers this is less accurate, however it should be good enough for these cases. We need 4 intervals as if say we had data at t=0 and t=10, coming via federation. The next data point should be at t=20 however it could take up to t=30 for it actually to be ingested, t=40 for it to be scraped via federation and t=50 for it to be ingested. We then add 10% on to that for slack, as we do elsewhere.	9 years ago
Brian Brazil	c02c25d5ba	Allow peeking back further in buffer.	9 years ago
Brian Brazil	a5cf25743c	Move stalness check into a function	9 years ago
Brian Brazil	80b40e6d91	Add initial staleness handing to promql. For instant vectors, if "stale" is the newest sample ignore the timeseries. For range vectors, filter out "stale" samples. Make it possible to inject "stale" samples in promql tests.	9 years ago

1 2 3 4 5 ...

326 Commits (9fe8bcf4be00a7ebd73932df0cf39cc837cbb993)