prometheus

Commit Graph

Author	SHA1	Message	Date
Fabian Reinartz	057a5ae2b1	Address comments Signed-off-by: Fabian Reinartz <freinartz@google.com>	7 years ago
Fabian Reinartz	ad4c33c1ff	scrape,api: provide per-target metric metadata This adds a per-target cache of scraped metadata. The metadata is only available for the lifecycle of the attached target. An API endpoint allows to select metadata by metric name and a label selection of targets. Signed-off-by: Fabian Reinartz <freinartz@google.com>	7 years ago
Brian Brazil	dd6781add2	Optimise PromQL (#3966 ) * Move range logic to 'eval' Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Make aggregegate range aware Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * PromQL is statically typed, so don't eval to find the type. Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Extend rangewrapper to multiple exprs Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Start making function evaluation ranged Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Make instant queries a special case of range queries Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Eliminate evalString Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Evaluate range vector functions one series at a time Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Make unary operators range aware Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Make binops range aware Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Pass time to range-aware functions. Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Make simple _over_time functions range aware Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Reduce allocs when working with matrix selectors Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Add basic benchmark for range evaluation Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Reuse objects for function arguments Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Do dropmetricname and allocating output vector only once. Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Add range-aware support for range vector functions with params Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Optimise holt_winters, cut cpu and allocs by ~25% Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Make rate&friends range aware Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Make more functions range aware. Document calling convention. Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Make date functions range aware Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Make simple math functions range aware Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Convert more functions to be range aware Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Make more functions range aware Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Specialcase timestamp() with vector selector arg for range awareness Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Remove transition code for functions Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Remove the rest of the engine transition code Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Remove more obselete code Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Remove the last uses of the eval* functions Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Remove engine finalizers to prevent corruption The finalizers set by matrixSelector were being called just before the value they were retruning to the pool was then being provided to the caller. Thus a concurrent query could corrupt the data that the user has just been returned. Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Add new benchmark suite for range functinos Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Migrate existing benchmarks to new system Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Expand promql benchmarks Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Simply test by removing unused range code Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * When testing instant queries, check range queries too. To protect against subsequent steps in a range query being affected by the previous steps, add a test that evaluates an instant query that we know works again as a range query with the tiimestamp we care about not being the first step. Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Reuse ring for matrix iters. Put query results back in pool. Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Reuse buffer when iterating over matrix selectors Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Unary minus should remove metric name Cut down benchmarks for faster runs. Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Reduce repetition in benchmark test cases Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Work series by series when doing normal vectorSelectors Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Optimise benchmark setup, cuts time by 60% Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Have rangeWrapper use an evalNodeHelper to cache across steps Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Use evalNodeHelper with functions Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Cache dropMetricName within a node evaluation. This saves both the calculations and allocs done by dropMetricName across steps. Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Reuse input vectors in rangewrapper Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Reuse the point slices in the matrixes input/output by rangeWrapper Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Make benchmark setup faster using AddFast Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Simplify benchmark code. Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Add caching in VectorBinop Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Use xor to have one-level resultMetric hash key Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Add more benchmarks Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Call Query.Close in apiv1 This allows point slices allocated for the response data to be reused by later queries, saving allocations. Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Optimise histogram_quantile It's now 5-10% faster with 97% less garbage generated for 1k steps Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Make the input collection in rangeVector linear rather than quadratic Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Optimise label_replace, for 1k steps 15x fewer allocs and 3x faster Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Optimise label_join, 1.8x faster and 11x less memory for 1k steps Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Expand benchmarks, cleanup comments, simplify numSteps logic. Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Address Fabian's comments Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Comments from Alin. Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Address jrv's comments Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Remove dead code Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Address Simon's comments. Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Rename populateIterators, pre-init some sizes Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Handle case where function has non-matrix args first Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Split rangeWrapper out to rangeEval function, improve comments Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Cleanup and make things more consistent Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Make EvalNodeHelper public Signed-off-by: Brian Brazil <brian.brazil@robustperception.io> * Fabian's comments. Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>	7 years ago
Krasi Georgiev	ddd46de6f4	Races/3994 (#4005 ) Fix race by properly locking access to scrape pools. Use separate mutex for information needed by UI so that UI isn't blocked when targets are being updated.	7 years ago
Brian Brazil	cc39021b2b	Provide custom marshalling for Point Point has a non-standard marshalling, and is also where the vast majority of CPU time is spent so it is worth optimising.	7 years ago
Brian Brazil	299b78a887	Switch to json-iterator for v1 api. This makes queries ~15% faster and cuts cpu time spent on json encoding by ~40%.	7 years ago
Simon Pasquier	83325c8d82	web: replace deprecated InstrumentHandler() (#3862 ) * web: replace deprecated InstrumentHandler() This change replaces the deprecated InstrumentHandler function by the equivalent functions from the promhttp package. The following metrics are removed: * http_request_duration_microseconds (Summary). * http_request_size_bytes (Summary). * http_requests_total (Counter). And the following metrics are added instead: * prometheus_http_request_duration_seconds (Histogram). * prometheus_http_response_size_bytes (Histogram). * promhttp_metric_handler_requests_in_flight (Gauge). * promhttp_metric_handler_requests_total (Counter). * Update github.com/prometheus/common/route package * web: refactor using the new prometheus/common/route package	7 years ago
Fabian Reinartz	3e6c890aea	api: add flag to skip head on snapshots	7 years ago
Conor Broderick	99006d3baf	Added dropped targets API to targets endpoint (#3870 )	7 years ago
Conor Broderick	1fd20fc954	Add dropped alertmanagers to alertmanagers API (#3865 )	7 years ago
Bartek Plotka	93a63ac5fd	api: Added v1/status/flags endpoint. (#3864 ) Endpoint URL: /api/v1/status/flags Example Output: ```json { "status": "success", "data": { "alertmanager.notification-queue-capacity": "10000", "alertmanager.timeout": "10s", "completion-bash": "false", "completion-script-bash": "false", "completion-script-zsh": "false", "config.file": "my_cool_prometheus.yaml", "help": "false", "help-long": "false", "help-man": "false", "log.level": "info", "query.lookback-delta": "5m", "query.max-concurrency": "20", "query.timeout": "2m", "storage.tsdb.max-block-duration": "36h", "storage.tsdb.min-block-duration": "2h", "storage.tsdb.no-lockfile": "false", "storage.tsdb.path": "data/", "storage.tsdb.retention": "15d", "version": "false", "web.console.libraries": "console_libraries", "web.console.templates": "consoles", "web.enable-admin-api": "false", "web.enable-lifecycle": "false", "web.external-url": "", "web.listen-address": "0.0.0.0:9090", "web.max-connections": "512", "web.read-timeout": "5m", "web.route-prefix": "/", "web.user-assets": "" } } ``` Signed-off-by: Bartek Plotka <bwplotka@gmail.com>	7 years ago
Fabian Reinartz	7ccd4b39b8	*: implement query params This adds a parameter to the storage selection interface which allows query engine(s) to pass information about the operations surrounding a data selection. This can for example be used by remote storage backends to infer the correct downsampling aggregates that need to be provided.	7 years ago
Krasi Georgiev	b75428ec19	rename package retrieve to scrape no fucnctinal changes just renaming retrieval to scrape	8 years ago
Goutham Veeramachaneni	bee6864c14	Make the date returned by snapshot script friendly Fixes #3568 Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	8 years ago
Ed Schouten	bb724f1bef	Deprecate DeduplicateSeriesSet() in favor of NewMergeSeriesSet(). Federation makes use of dedupedSeriesSet to merge SeriesSets for every query into one output stream. If many match[] arguments are provided, many dedupedSeriesSet objects will get chained. This has the downside of causing a potential O(nk) running time, where n is the number of series and k the number of match[] arguments. In the mean time, the storage package provides a mergeSeriesSet that accomplishes the same with an O(nlog(k)) running time by making use of a binary heap. Let's just get rid of dedupedSeriesSet and change all existing callers to use mergeSeriesSet.	8 years ago
Goutham Veeramachaneni	f0599d4dbf	Incorporate review-feedback Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	8 years ago
Julius Volz	ab11a457e8	Remove obsolete TODO in API code In https://github.com/prometheus/prometheus/pull/3230/files, contexts were added to the Querier() method instead, and Cortex is fine with that.	8 years ago
Goutham Veeramachaneni	d8515b2580	Move Admin APIs to v1 Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	8 years ago
Brian Brazil	d7b3df5ae1	Fix staticcheck errors	8 years ago
Fabian Reinartz	83cd270ea4	*: adapt to storage interface changes	8 years ago
David Kaltschmidt	af75ce02c1	Review feedback * renamed MakeQueryStats * added stats to query() as well * gofmt	8 years ago
David Kaltschmidt	c93e54d240	Adds execution timer stats to the range query API consumers should be able to get insight into the query run times. The UI currently measures total roundtrip times. This PR allows for more fine grained metrics to be exposed. * adds new timer for total execution time (queue + eval) * expose new timer, queue timer, and eval timer in stats field of the range query response: ```json { "status": "success", "data": { "resultType": "matrix", "result": [], "stats": { "execQueueTimeNs": 4683, "execTotalTimeNs": 2086587, "totalEvalTimeNs": 2077851 } } } ``` * stats field is optional, only set when query parameter `stats` is not empty Try it via ```sh curl 'http://localhost:9090/api/v1/query_range?query=up&start=1486480279&end=1486483879&step=14000&stats=true' ``` Review feedback * moved query stats json generation to query_stats.go * use seconds for all query timers * expose all timers available * Changed ExecTotalTime string representation from Exec queue total time to Exec total time	8 years ago
Alexey Miroshkin	8c681f4a6c	Provide POST endpoint for query+query_range (#3322 ) This PR fixes #3072 by providing POST endpoints for `query` and `query_range`. POST request must be made with `Content-Type: application/x-www-form-urlencoded` header.	8 years ago
Tom Wilkie	746752b946	Merge external labels in order.	8 years ago
Tom Wilkie	ee011d906d	Port remote read server to 2.0.	8 years ago
Julius Volz	099df0c5f0	Migrate "golang.org/x/net/context" -> "context" (#3333 ) In some places, where ctxhttp or gRPC are concerned, we still need to use the old contexts.	8 years ago
Brian Brazil	3a7c51ab70	Remote read endpoint should handle matchers for external labels. (#3325 ) If the other Prometheus has an external label that matches that of the Prometheus being read from, then we need to remove that matcher from the request as it's not actually stored in the database - it's only added for alerts, federation and on the output of the remote read endpoint. Instead we check for that label being empty, in case there is a time series with a different label value for that external label.	8 years ago
Brian Brazil	c25f7c600b	Apply external labels to remote read endpoint (#3263 ) Fixes #3261	8 years ago
beorn7	f20e6a0ae4	Only respond to API requests once the server is ready	8 years ago
Julius Volz	f7e8348a88	Re-add contexts to storage.Storage.Querier() (#3230 ) * Re-add contexts to storage.Storage.Querier() These are needed when replacing the storage by a multi-tenant implementation where the tenant is stored in the context. The 1.x query interfaces already had contexts, but they got lost in 2.x. * Convert promql.Engine to use native contexts	8 years ago
Tom Wilkie	758d64ffd9	s/EncodReadResponse/EncodeReadResponse/	8 years ago
Tom Wilkie	febed48703	Implement remote read server in Prometheus.	8 years ago
Max Leonard Inden	1c96fbb992	Expose current Prometheus config via /status/config This PR adds the `/status/config` endpoint which exposes the currently loaded Prometheus config. This is the same config that is displayed on `/config` in the UI in YAML format. The response payload looks like such: ``` { "status": "success", "data": { "yaml": <CONFIG> } } ```	8 years ago
Fabian Reinartz	ccf9e62972	*: add admin grpc API	8 years ago
Tom Wilkie	4d9b917d11	Instrument Prometheus with OpenTracing (#2554 ) * Use request.Context() instead of a global map of contexts. * Add some basic opentracing instrumentation on the query path. * Remove tracehandler endpoint.	8 years ago
David Symonds	04ad889751	Preserve Alertmanager URLs as *url.URL. Render a nicer link in the web UI.	8 years ago
Tom Wilkie	f0e8a5f37c	Add promql.ErrStorage, which is interpreted by the API as a 500.	8 years ago
Fabian Reinartz	bbcf20ba01	web: deduplicate series in federation	8 years ago
Fabian Reinartz	f56644e3ae	api/v1: deduplicate selected series	8 years ago
Bplotka	1823ae8bc4	Fixed int64 overflow for timestamp in v1/api parseDuration and parseTime (#2501 ) * Fixed int64 overflow for timestamp in v1/api parseDuration and parseTime This led to unexpected results on wrong query with "(...)&start=148966367200.372&end=1489667272.372" That query is wrong because of `start > end` but actually internal int64 overflow caused start to be something around MinInt64 (huge negative value) and was passing validation. BTW: Not sure if negative timestamp makes sense even.. But model.Earliest is actually MinInt64, can someone explain me why? Signed-off-by: Bartek Plotka <bwplotka@gmail.com> * Added missing trailing periods on comments. Signed-off-by: Bartek Plotka <bwplotka@gmail.com> * MOved to only `<` and `>`. Removed equal. Signed-off-by: Bartek Plotka <bwplotka@gmail.com>	8 years ago
Goutham Veeramachaneni	4b0270290b	Fix comments to match convention	8 years ago
Goutham Veeramachaneni	c6b329c55b	Support Custom Timeouts for Queries	8 years ago
Goutham Veeramachaneni	6634984a38	Comments and Typo Fixes	8 years ago
Fabian Reinartz	d80a3de235	pkg/textparse: add documentation	9 years ago
Fabian Reinartz	157e698958	web/api: fix min/max timestamps to valid range	9 years ago
Frederic Branczyk	bd92571bdd	web/api: make target and alertmanager api responses consistent	9 years ago
Frederic Branczyk	389c6d0043	web/api: add alertmanager api	9 years ago
beorn7	767c0709b1	Retrieval: Avoid copying Target retreival.Target contains a mutex. It was copied in the Targets() call. This potentially can wreak a lot of havoc. It might even have caused the issues reported as #2266 and #2262 .	9 years ago
Fabian Reinartz	bc20d93f0a	storage: rename iterator value getters to At()	9 years ago
Fabian Reinartz	28f547bcc7	api/v1: fix tests, restore series queries	9 years ago

1 2

90 Commits (a155b6d29dd7becae7e08bda1680ec6290d4911e)