84 Commits (main)

Author	SHA1	Message	Date
Fabian Reinartz	83cd270ea4	*: adapt to storage interface changes	8 years ago
Julius Volz	f7e8348a88	Re-add contexts to storage.Storage.Querier() (#3230) ... * Re-add contexts to storage.Storage.Querier() These are needed when replacing the storage by a multi-tenant implementation where the tenant is stored in the context. The 1.x query interfaces already had contexts, but they got lost in 2.x. * Convert promql.Engine to use native contexts	8 years ago
Fabian Reinartz	d21f149745	*: migrate to go-kit/log	8 years ago
Max Leonard Inden	1c96fbb992	Expose current Prometheus config via /status/config ... This PR adds the `/status/config` endpoint which exposes the currently loaded Prometheus config. This is the same config that is displayed on `/config` in the UI in YAML format. The response payload looks like such: ``` { "status": "success", "data": { "yaml": <CONFIG> } } ```	8 years ago
Goutham Veeramachaneni	d407bd150c	Consolidate the duration params in CLI ... * All CLI params moved to model.Duration Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	9 years ago
Brian Brazil	e5f94145b8	Drop series for federation if latest sample is stale.	9 years ago
Brian Brazil	220e78b9c3	Consider a series stale after 4.1 intervals with no data. ... To cover the cases where stale markers may not be available, we need to infer the interval and mark series stale based on that. As we're lacking stale markers this is less accurate, however it should be good enough for these cases. We need 4 intervals as if say we had data at t=0 and t=10, coming via federation. The next data point should be at t=20 however it could take up to t=30 for it actually to be ingested, t=40 for it to be scraped via federation and t=50 for it to be ingested. We then add 10% on to that for slack, as we do elsewhere.	9 years ago
Brian Brazil	c02c25d5ba	Allow peeking back further in buffer.	9 years ago
Fabian Reinartz	8c768f2ca3	web: Fix federation for instance label	9 years ago
Fabian Reinartz	bbcf20ba01	web: deduplicate series in federation	9 years ago
Brian Brazil	8cd5aff8fe	Send instance="" with federation if instance not set. ... This is needed for federating non-instance level metrics, so they don't end up with the instance label of the prometheus target. Also sort external labels, so label output order is consistent.	9 years ago
Brian Brazil	d42e01b07c	Sort labelnames for federation. ... This makes unittests with multiple labels possible, and may be needed for performance with the new ingestion text parser.	9 years ago
Fabian Reinartz	bc20d93f0a	storage: rename iterator value getters to At()	9 years ago
Fabian Reinartz	61bd698143	web: implement federation for new storage	9 years ago
Fabian Reinartz	f8fc1f5bb2	*: migrate ingestion to new batch Appender	9 years ago
Fabian Reinartz	5817cb5bde	*: migrate from model.* to promql.* types	9 years ago
Fabian Reinartz	cef2e04aa3	web: add error counter for federation responses	9 years ago
Fabian Reinartz	9d68e81b32	web: don't return federation errors over HTTP ... We are writing federation responses streaming. So after the first byte we wrote, the status header is fixed. We cannot return an HTTP error for intermediate error but should just abort and log instead.	9 years ago
Fabian Reinartz	8fa18d564a	storage: enhance Querier interface usage ... This extracts Querier as an instantiateable and closeable object rather than just defining extending methods of the storage interface. This improves composability and allows abstracting query transactions, which can be useful for transaction-level caches, consistent data views, and encapsulating teardown.	9 years ago
Julius Volz	c187308366	storage: Contextify storage interfaces. ... This is based on https://github.com/prometheus/prometheus/pull/1997. This adds contexts to the relevant Storage methods and already passes PromQL's new per-query context into the storage's query methods. The immediate motivation supporting multi-tenancy in Frankenstein, but this could also be used by Prometheus's normal local storage to support cancellations and timeouts at some point.	9 years ago
beorn7	39c4915401	federation: Collapse time series of the same name ... This will avoid duplicate MetricFamilies, thereby shrinking the size of the federation payload and also creating legal text format. Also, add unit tests for federation. They were also needed for the previous state of the code, but were missing.	9 years ago
Julius Volz	3bfec97d46	Make the storage interface higher-level. ... See discussion in https://groups.google.com/forum/#!topic/prometheus-developers/bkuGbVlvQ9g The main idea is that the user of a storage shouldn't have to deal with fingerprints anymore, and should not need to do an individual preload call for each metric. The storage interface needs to be made more high-level to not expose these details. This also makes it easier to reuse the same storage interface for remote storages later, as fewer roundtrips are required and the fingerprint concept doesn't work well across the network. NOTE: this deliberately gets rid of a small optimization in the old query Analyzer, where we dedupe instants and ranges for the same series. This should have a minor impact, as most queries do not have multiple selectors loading the same series (and at the same offset).	10 years ago
beorn7	836f1db04c	Improve MetricsForLabelMatchers ... WIP: This needs more tests. It now gets a from and through value, which it may opportunistically use to optimize the retrieval. With possible future range indices, this could be used in a very efficient way. This change merely applies some easy checks, which should nevertheless solve the use case of heavy rule evaluations on servers with a lot of series churn. Idea is the following: - Only archive series that are at least as old as the headChunkTimeout (which was already extremely unlikely to happen). - Then maintain a high watermark for the last archival, i.e. no archived series has a sample more recent than that watermark. - Any query that doesn't reach to a time before that watermark doesn't have to touch the archive index at all. (A production server at Soundcloud with the aforementioned series churn and heavy rule evaluations spends 50% of its CPU time in archive index lookups. Since rule evaluations usually only touch very recent values, most of those lookup should disappear with this change.) - Federation with a very broad label matcher will profit from this, too. As a byproduct, the un-needed MetricForFingerprint method was removed from the Storage interface.	10 years ago
beorn7	1e13f89039	Return SamplePair istead of *SamplePair consistently ... Formalize ZeroSamplePair as return value for non-existing samples. Change LastSamplePairForFingerprint to return a SamplePair (and not a pointer to it), which saves allocations in a potentially extremely frequent call.	10 years ago
Fabian Reinartz	b35e25a082	Don't federate stale samples	10 years ago
Julius Volz	dac26cef71	Rename global "labels" config option to "external_labels".	10 years ago
beorn7	ca63c05b4e	Fix creation of protobuf fields	10 years ago
Fabian Reinartz	9bbd9264e2	Add global labels to federation	10 years ago
Julius Volz	995d3b831d	Fix most golint warnings. ... This is with `golint -min_confidence=0.5`. I left several lint warnings untouched because they were either incorrect or I felt it was better not to change them at the moment.	10 years ago
Fabian Reinartz	d6b8da8d43	Switch promql types to common/model	10 years ago
Fabian Reinartz	d6d88f8950	Add missing license headers	10 years ago
Fabian Reinartz	47aa0d536c	Use common/expfmt in federation	10 years ago
Fabian Reinartz	306e8468a0	Switch from client_golang/model to common/model	10 years ago
Fabian Reinartz	39a8254963	web: add basic federation support. ... This commit adds a federation handler on /federate. It accepts `match[]` query parameters containing vector selectors. Their intersection determines the in-memory metrics that are returned in the same way as the /metrics endpoint does (modulo sorting).	11 years ago