132 Commits (3d8826a3d42566684283a9b7f7e812e412c24407)

Author	SHA1	Message	Date
Julien Pivotto	4e5b1722b3	Move away from testutil, refactor imports (#8087) ... Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>	5 years ago
Dustin Hooten	916dbd4c8a	React UI: Implement missing TSDB head stats section (#7876) ... * React UI: Implement missing TSDB head stats section Signed-off-by: Dustin Hooten <dhooten@splunk.com> * Add break Signed-off-by: Dustin Hooten <dhooten@splunk.com>	5 years ago
showuon	dfdc358a5b	Fix the duplicated results issue from /api/v1/series (#7862) ... * Fix the duplicated results issue from /api/v1/series Signed-off-by: Luke Chen <showuon@gmail.com>	5 years ago
Li Zhijian	6f6ef1451e	Cleanup tempfiles (#7696) ... * cleanup tempfiles for web_test Signed-off-by: Li Zhijian <lizhijian@cn.fujitsu.com> * cleanup tempfiles for api_test Signed-off-by: Li Zhijian <lizhijian@cn.fujitsu.com> * cleanup tempfiles for file_test Signed-off-by: Li Zhijian <lizhijian@cn.fujitsu.com>	5 years ago
Bartlomiej Plotka	e6d7cc5fa4	tsdb: Added ChunkQueryable implementations to db; unified MergeSeriesSets and vertical to single struct. (#7069) ... * tsdb: Added ChunkQueryable implementations to db; unified compactor, querier and fanout block iterating. Chained to https://github.com/prometheus/prometheus/pull/7059 * NewMerge(Chunk)Querier now takies multiple primaries allowing tsdb DB code to use it. * Added single SeriesEntry / ChunkEntry for all series implementations. * Unified all vertical, and non vertical for compact and querying to single merge series / chunk sets by reusing VerticalSeriesMergeFunc for overlapping algorithm (same logic as before) * Added block (Base/Chunk/)Querier for block querying. We then use populateAndTomb(Base/Chunk/) to iterate over chunks or samples. * Refactored endpoint tests and querier tests to include subtests. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> * Addressed comments from Brian and Beorn. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> * Fixed snapshot test and added chunk iterator support for DBReadOnly. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> * Fixed race when iterating over Ats first. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> * Fixed tests. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> * Fixed populate block tests. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> * Fixed endpoints test. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> * Fixed test. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> * Added test & fixed case of head open chunk. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> * Fixed DBReadOnly tests and bug producing 1 sample chunks. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> * Added cases for partial block overlap for multiple full chunks. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> * Added extra tests for chunk meta after compaction. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> * Fixed small vertical merge bug and added more tests for that. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>	5 years ago
Bartlomiej Plotka	b788986717	storage: Adjusted fully storage layer support for chunk iterators: Remote read client, readyStorage, fanout. (#7059) ... * Fixed nits introduced by https://github.com/prometheus/prometheus/pull/7334 * Added ChunkQueryable implementation to fanout and readyStorage. * Added more comments. * Changed NewVerticalChunkSeriesMerger to CompactingChunkSeriesMerger, removed tiny interface by reusing VerticalSeriesMergeFunc for overlapping algorithm for both chunks and series, for both querying and compacting (!) + made sure duplicates are merged. * Added ErrChunkSeriesSet * Added Samples interface for seamless []promb.Sample to []tsdbutil.Sample conversion. * Deprecating non chunks serieset based StreamChunkedReadResponses, added chunk one. * Improved tests. * Split remote client into Write (old storage) and read. * Queryable client is now SampleAndChunkQueryable. Since we cannot use nice QueryableFunc I moved all config based options to sampleAndChunkQueryableClient to aboid boilerplate. In next commit: Changes for TSDB. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>	5 years ago
Marco Pracucci	153f859b74	Fixed returned API status code on error (#7435) ... * Fixed returned API status code on error Signed-off-by: Marco Pracucci <marco@pracucci.com> * Fixed linter Signed-off-by: Marco Pracucci <marco@pracucci.com> * Simplified code Signed-off-by: Marco Pracucci <marco@pracucci.com>	5 years ago
Kemal Akkoyun	66dfb951c4	*: Consistent Error/Warning handling for SeriesSet iterator: Allowing Async Select (#7251) ... * Add errors and Warnings to SeriesSet Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com> * Change Querier interface and refactor accordingly Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com> * Refactor promql/engine to propagate warnings at eval stage Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com> * Address review issues Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com> * Make sure all the series from all Selects are pre-advanced Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com> * Address review issues Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com> * Separate merge series sets Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com> * Clean Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com> * Refactor merge querier failure handling Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com> * Refactored and simplified fanout with improvements from incoming chunk iterator PRs. * Secondary logic is hidden, instead of weird failed series set logic we had. * Fanout is well commented * Fanout closing record all errors * MergeQuerier improved API (clearer) * deferredGenericMergeSeriesSet is not needed as we return no samples anyway for failed series sets (next = false). Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> * Fix formatting Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com> * Fix CI issues Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com> * Added final tests for error handling. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> * Addressed Brian's comments. * Moved hints in populate to be allocated only when needed. * Used sync.Once in secondary Querier to achieve all-or-nothing partial response logic. * Select after first Next is done will panic. NOTE: in lazySeriesSet in theory we could just panic, I think however we can totally just return error, it will panic in expand anyway. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> * Utilize errWithWarnings Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com> * Fix recently introduced expansion issue Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com> * Add tests for secondary querier error handling Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com> * Implement lazy merge Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com> * Add name to test cases Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com> * Reorganize Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com> * Address review comments Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com> * Address review comments Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com> * Remove redundant warnings Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com> * Fix rebase mistake Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com> Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>	5 years ago
Jess G	fdc49fae5b	Added time range parameters to labelNames API (#7288) ... * add time range params to labelNames api Signed-off-by: jessicagreben <Jessica.greben1+github@gmail.com> * evaluate min/max time range when reading labels from the head Signed-off-by: jessicagreben <Jessica.greben1+github@gmail.com> * add time range params to labelValues api Signed-off-by: jessicagreben <Jessica.greben1+github@gmail.com> * fix test, add docs Signed-off-by: jessicagreben <Jessica.greben1+github@gmail.com> * add a test for head min max range Signed-off-by: jessicagreben <Jessica.greben1+github@gmail.com> * fix test to match comment Signed-off-by: jessicagreben <Jessica.greben1+github@gmail.com> * address CR comments Signed-off-by: jessicagreben <Jessica.greben1+github@gmail.com> * combine vars only used once Signed-off-by: jessicagreben <Jessica.greben1+github@gmail.com> * add time range params to labelNames api Signed-off-by: jessicagreben <Jessica.greben1+github@gmail.com> * evaluate min/max time range when reading labels from the head Signed-off-by: jessicagreben <Jessica.greben1+github@gmail.com> * add time range params to labelValues api Signed-off-by: jessicagreben <Jessica.greben1+github@gmail.com> * fix test, add docs Signed-off-by: jessicagreben <Jessica.greben1+github@gmail.com> * add a test for head min max range Signed-off-by: jessicagreben <Jessica.greben1+github@gmail.com> * fix test to match comment Signed-off-by: jessicagreben <Jessica.greben1+github@gmail.com> * address CR comments Signed-off-by: jessicagreben <Jessica.greben1+github@gmail.com> * combine vars only used once Signed-off-by: jessicagreben <Jessica.greben1+github@gmail.com> * fix test Signed-off-by: jessicagreben <Jessica.greben1+github@gmail.com> * restart ci Signed-off-by: jessicagreben <Jessica.greben1+github@gmail.com> * use range expectedLabelNames instead of range actualLabelNames in test Signed-off-by: jessicagreben <Jessica.greben1+github@gmail.com>	5 years ago
Ganesh Vernekar	1c99adb9fd	Callbacks for lifecycle of series in TSDB (#7159) ... * Callbacks for lifecycle of series in TSDB Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Add more comments Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>	5 years ago
gotjosh	bfcd9282a8	API: Allow AlertmanagerRetriever and RulesRetriever to receive a Context (#7256) ... * API: Allow AlertmanagerRetriever and RulesRetriever to receive a Context Signed-off-by: gotjosh <josue@grafana.com> * Use single line functions where possible Signed-off-by: gotjosh <josue@grafana.com>	5 years ago
Ganesh Vernekar	d4b9fe801f	M-map full chunks of Head from disk (#6679) ... When appending to the head and a chunk is full it is flushed to the disk and m-mapped (memory mapped) to free up memory Prom startup now happens in these stages - Iterate the m-maped chunks from disk and keep a map of series reference to its slice of mmapped chunks. - Iterate the WAL as usual. Whenever we create a new series, look for it's mmapped chunks in the map created before and add it to that series. If a head chunk is corrupted the currpted one and all chunks after that are deleted and the data after the corruption is recovered from the existing WAL which means that a corruption in m-mapped files results in NO data loss. [Mmaped chunks format](https://github.com/prometheus/prometheus/blob/master/tsdb/docs/format/head_chunks.md) - main difference is that the chunk for mmaping now also includes series reference because there is no index for mapping series to chunks. [The block chunks](https://github.com/prometheus/prometheus/blob/master/tsdb/docs/format/chunks.md) are accessed from the index which includes the offsets for the chunks in the chunks file - example - chunks of series ID have offsets 200, 500 etc in the chunk files. In case of mmaped chunks, the offsets are stored in memory and accessed from that. During WAL replay, these offsets are restored by iterating all m-mapped chunks as stated above by matching the series id present in the chunk header and offset of that chunk in that file. **Prombench results** _WAL Replay_ 1h Wal reply time 30% less wal reply time - 4m31 vs 3m36 2h Wal reply time 20% less wal reply time - 8m16 vs 7m _Memory During WAL Replay_ High Churn: 10-15% less RAM - 32gb vs 28gb 20% less RAM after compaction 34gb vs 27gb No Churn: 20-30% less RAM - 23gb vs 18gb 40% less RAM after compaction 32.5gb vs 20gb Screenshots are in [this comment](https://github.com/prometheus/prometheus/pull/6679#issuecomment-621678932) Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>	5 years ago
Ben Ye	1e4e37144d	Fixed wrongly handled not ready TSDB on web and API. (#7182) ... * fix federate endpoint panic Signed-off-by: yeya24 <yb532204897@gmail.com> * Fixed all cases of not ready TSDB being wrongly handled. * Fixed issue for federation. * Ensured this will never happen again thanks to interfaces * Fixes same issue for stats. * Added tests for readiness. * Fixed bug in stats. It was: status.MaxTime = db.Head().MaxTime() status.MinTime = db.Head().MaxTime() Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> * Addressed Brian's comments. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> * Addressed Brian's comments. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>	5 years ago
gotjosh	24af5049bb	API: Allow `TargetRetriever` to receive a Context (#7125) ... Fixes #7103 Signed-off-by: gotjosh <josue@grafana.com>	5 years ago
Bartlomiej Plotka	c4eefd1b3a	storage: Removed SelectSorted method; Simplified interface; Added requirement for remote read to sort response. ... This is technically BREAKING CHANGE, but it was like this from the beginning: I just notice that we rely in Prometheus on remote read being sorted. This is because we use selected data from remote reads in MergeSeriesSet which rely on sorting. I found during work on https://github.com/prometheus/prometheus/pull/5882 that we do so many repetitions because of this, for not good reason. I think I found a good balance between convenience and readability with just one method. Smaller the interface = better. Also I don't know what TestSelectSorted was testing, but now it's testing sorting. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>	5 years ago
Bartlomiej Plotka	fe802f29c9	storage: Removed SelectSorted method; Simplified interface; Added requirement for remote read to sort response. ... This is technically BREAKING CHANGE, but it was like this from the beginning: I just notice that we rely in Prometheus on remote read being sorted. This is because we use selected data from remote reads in MergeSeriesSet which rely on sorting. I found during work on https://github.com/prometheus/prometheus/pull/5882 that we do so many repetitions because of this, for not good reason. I think I found a good balance between convenience and readability with just one method. Smaller the interface = better. Also I don't know what TestSelectSorted was testing, but now it's testing sorting. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>	5 years ago
Boyko	84b00564f4	API_V1: Extract time param parsing logic (#6860) ... * extract API time param parsing logic in a func Signed-off-by: blalov <boiskila@gmail.com>	5 years ago
Bartlomiej Plotka	fb79f515fc	Fixed second bug. ... Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>	5 years ago
Harkishen Singh	489a9aa7b9	Adds normalization of localhost urls in targets page react (#6794) ... * support for globalurls in targets page react Signed-off-by: Harkishen Singh <harkishensingh@hotmail.com> * fixed tests Signed-off-by: Harkishen Singh <harkishensingh@hotmail.com> * removed fmts Signed-off-by: Harkishen Singh <harkishensingh@hotmail.com> * implemented suggestions Signed-off-by: Harkishen Singh <harkishensingh@hotmail.com> * formatted Signed-off-by: Harkishen Singh <harkishensingh@hotmail.com> * implemented suggestions. fixed tests. Signed-off-by: Harkishen Singh <harkishensingh@hotmail.com> * formated go code Signed-off-by: Harkishen Singh <harkishensingh@hotmail.com> * implemented suggestions Signed-off-by: Harkishen Singh <harkishensingh@hotmail.com>	5 years ago
Tobias Guggenmos	6c00f2ffcb	Comment fixes ... Signed-off-by: Tobias Guggenmos <tguggenm@redhat.com>	5 years ago
Julien Pivotto	135cc30063	rules: Make deleted rule series as stale after a reload (#6745) ... * rules: Make deleted rule series as stale after a reload Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>	5 years ago
Thor	17d8c49919	made stripe size configurable (#6644) ... Signed-off-by: Thor <thansen@digitalocean.com>	5 years ago
gotjosh	8b49c9285d	scrape: Add metrics to track bytes and entries in the metadata cache (#6675) ... Signed-off-by: gotjosh <josue@grafana.com>	5 years ago
Julien Pivotto	9adad8ad30	Remove MaxConcurrent from the PromQL engine opts (#6712) ... Since we use ActiveQueryTracker to check for concurrency in d992c36b3a it does not make sense to keep the MaxConcurrent value as an option of the PromQL engine. This pull request removes it from the PromQL engine options, sets the max concurrent metric to -1 if there is no active query tracker, and use the value of the active query tracker otherwise. It removes dead code and also will inform people who import the promql package that we made that change, as it breaks the EngineOpts struct. Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>	5 years ago
Julien Pivotto	9d9bc524e5	Add query log (#6520) ... * Add query log, make stats logged in JSON like in the API Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>	6 years ago
gotjosh	e1f2da74a3	api: provide metadata for a single metric ... This adds support for a new query param on the new `api/v1/metadata` endpoint that provides metadata for a specified metric via the V1 API. It collapses metadata that is equal across all targets, and aggregates under the same metric name the ones that differ. Signed-off-by: gotjosh <josue@grafana.com>	6 years ago
gotjosh	4a13c844b7	Add periods to comments on tests. ... Signed-off-by: gotjosh <josue@grafana.com>	6 years ago
gotjosh	0a0a228db3	api: provide per metric metadata (#6420) ... * api: provide per metric metadata This adds a new endpoint that provides per metric metadata via the V1 API. It collapses metadata that is equal across all targets, and aggregates under the same metric name the ones that differ. * Allow tests to be asserted on response length Some tests e.g. limit on API responses, don't require an assertion on equality. This allows us to assert against response length instead of equality. Signed-off-by: gotjosh <josue@grafana.com>	6 years ago
Boyko	1c66aea992	React UI: Implement alerts page (#6402) ... * url filter rules param Signed-off-by: blalov <boiskila@gmail.com> Signed-off-by: Boyko Lalov <boiskila@gmail.com> Signed-off-by: blalov <boiskila@gmail.com> * address review changes Signed-off-by: blalov <boiskila@gmail.com> Signed-off-by: Boyko Lalov <boiskila@gmail.com> Signed-off-by: blalov <boiskila@gmail.com> * ui initial commit Signed-off-by: blalov <boiskila@gmail.com> Signed-off-by: Boyko Lalov <boiskila@gmail.com> Signed-off-by: blalov <boiskila@gmail.com> * improve ui Signed-off-by: blalov <boiskila@gmail.com> Signed-off-by: Boyko Lalov <boiskila@gmail.com> Signed-off-by: blalov <boiskila@gmail.com> * fix typo in component name Signed-off-by: Boyko Lalov <boiskila@gmail.com> Signed-off-by: blalov <boiskila@gmail.com> * create query link + ui enhancements Signed-off-by: Boyko Lalov <boiskila@gmail.com> Signed-off-by: blalov <boiskila@gmail.com> * add count to state labels Signed-off-by: blalov <boiskila@gmail.com> * put alerts table render in the right place Signed-off-by: blalov <boiskila@gmail.com> * refactoring Signed-off-by: blalov <boiskila@gmail.com> * fix rules endpoint test Signed-off-by: blalov <boiskila@gmail.com> * lint fixes Signed-off-by: blalov <boiskila@gmail.com> * test query params Signed-off-by: blalov <boiskila@gmail.com> * refactoring Signed-off-by: blalov <boiskila@gmail.com> * review changes Signed-off-by: blalov <boiskila@gmail.com> * adding down arrow as click indicator in Alert Signed-off-by: blalov <boiskila@gmail.com> * add period at the end of the comment Signed-off-by: blalov <boiskila@gmail.com> * review changes Signed-off-by: blalov <boiskila@gmail.com> * remove left-over css Signed-off-by: blalov <boiskila@gmail.com> * adding expand/collapse arrows on Alert Signed-off-by: blalov <boiskila@gmail.com> * create proper expression for alert name Signed-off-by: blalov <boiskila@gmail.com>	6 years ago
gotjosh	7bb73a9abd	fix: flaky test for api/v1/targets/metadata. (#6436) ... * Allows sorting of responses from the API in tests Fixes flaky test for api/v1/targets/metadata. Allows sorting of responses from the API. For our tests to be deterministic, we need to ensure the response from the API follows an order. This structure allows us to define one. Fixes #6431 Signed-off-by: gotjosh <josue@grafana.com>	6 years ago
gotjosh	428089f83f	api: tests for /target/metadata API endpoint ... This commit introduces several test cases for the current /targets/metadata API endpoint. To achieve so, we use a mock of the metadataStore and inject it to the targets under test. Currently, three success cases are covered: with a metric name, with a target matcher, and with both. As for the failure scenario, the one where we couldn't match against a particular metric is covered. Signed-off-by: gotjosh <josue@grafana.com>	6 years ago
gotjosh	4bf9c6bb82	Allow targets to be injected as arguments when creating a testTargetRetriever ... Previously, the struct `testTargetRetriever` had hardcoded active and dropped targets. This made it difficult to change the target information depending on the test case. This change introduces a way to define them as arguments and pass it to a constructor for building. It lays a foundation for dynamically defining targets with various set of arguments to test different scenarios. Signed-off-by: gotjosh <josue@grafana.com>	6 years ago
Tom Wilkie	de0a772b8e	Port tsdb to use pkg/labels. (#6326) ... * Port tsdb to use pkg/labels. Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com> * Get tests passing. Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com> * Remove useless cast. Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com> * Appease linters. Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com> * Fix review comments Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>	6 years ago
Sharad Gaur	a85e7aac0e	Adding TSDB Stats Page in React UI (#6281) ... Signed-off-by: Sharad Gaur <sgaur@splunk.com>	6 years ago
Dustin Hooten	ca60bf298c	React UI: Implement /targets page (#6276) ... * Add LastScrapeDuration to targets endpoint Signed-off-by: Dustin Hooten <dhooten@splunk.com> * Add Scrape job name to targets endpoint Signed-off-by: Dustin Hooten <dhooten@splunk.com> * Implement the /targets page in react Signed-off-by: Dustin Hooten <dhooten@splunk.com> * Add state query param to targets endpoint Signed-off-by: Dustin Hooten <dhooten@splunk.com> * Use state filter in api call Signed-off-by: Dustin Hooten <dhooten@splunk.com> * api feedback Signed-off-by: Dustin Hooten <dhooten@splunk.com> * pr feedback frontend Signed-off-by: Dustin Hooten <dhooten@splunk.com> * Implement and use localstorage hook Signed-off-by: Dustin Hooten <dhooten@splunk.com> * PR feedback Signed-off-by: Dustin Hooten <dhooten@splunk.com>	6 years ago
Yao Zengzeng	8744afdd1e	cleanup redundant code of `TestEndpoints` (#6022) ... Signed-off-by: YaoZengzeng <yaozengzeng@huawei.com>	6 years ago
Yao Zengzeng	21c9789083	multiple queries test for StreamRead (#5969) ... Signed-off-by: YaoZengzeng <yaozengzeng@zju.edu.cn>	6 years ago
Bartek Płotka	48b2c9c8ea	remote-read: streamed chunked server side; Extended protobuf; Added chunked, checksumed reader (#5703) ... Part of: https://github.com/prometheus/prometheus/issues/4517 and https://github.com/improbable-eng/thanos/issues/488 Changes: * Extended protobuf for chunked remote read and negotation. * Added checksumed, chunked Writer/Reader. * Added Server side implementation for chunked streamed remote-read. Signed-off-by: Bartek Plotka <bwplotka@gmail.com>	6 years ago
Ganesh Vernekar	5ecef3542d	Cleanup after merging tsdb into prometheus ... Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>	6 years ago
Chris Marchbanks	0685eb5395	Refactor testutil.NewStorage into a new package ... This avoids a circular dependency between the testutil and storage packages. Signed-off-by: Chris Marchbanks <csmarchbanks@gmail.com>	6 years ago
@aifsair	0f00737308	Fix log config setup (#5807) ... The return value of Set() was not checked. Hence, the typo ("al" instead of "af") wasn't catched. Signed-off-by: François (fser) <fser@code-libre.org>	6 years ago
Thomas Jackson	fef150f1b5	Add tests to ensure we can marshal and unmarshal our min/max times (#5734) ... * Add tests to ensure we can marshal and unmarshal our min/max times Related to https://github.com/prometheus/client_golang/issues/614 Instead of implementing all the time parsing, we can special-case handle these 2 times. This means if times in this format show up that time.Parse can't handle they will still error, but we can marshal/parse our own min/max time Signed-off-by: Thomas Jackson <jacksontj.89@gmail.com>	6 years ago
Simon Pasquier	45506841e6	*: enable all default linters (#5504) ... Signed-off-by: Simon Pasquier <spasquie@redhat.com>	6 years ago
Bjoern Rabenstein	38d518c0fe	Rework #5009 after comments ... Signed-off-by: Bjoern Rabenstein <bjoern@rabenste.in>	6 years ago
Bob Shannon	8c8bb82d04	Add support for POSTing to /series endpoint (#5422) ... * Add support for POSTing to /series endpoint * Document query API POST support Signed-off-by: Bob Shannon <bob.m.shannon@gmail.com>	6 years ago
Tariq Ibrahim	8fdfa8abea	refine error handling in prometheus (#5388) ... i) Uses the more idiomatic Wrap and Wrapf methods for creating nested errors. ii) Fixes some incorrect usages of fmt.Errorf where the error messages don't have any formatting directives. iii) Does away with the use of fmt package for errors in favour of pkg/errors Signed-off-by: tariqibrahim <tariq181290@gmail.com>	6 years ago
Tom Wilkie	c7b3535997	Use pkg/relabelling in remote write. ... - Unmarshall external_labels config as labels.Labels, add tests. - Convert some more uses of model.LabelSet to labels.Labels. - Remove old relabel pkg (fixes #3647). - Validate external label names. Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	6 years ago
Callum Styan	6f69e31398	Tail the TSDB WAL for remote_write ... This change switches the remote_write API to use the TSDB WAL. This should reduce memory usage and prevent sample loss when the remote end point is down. We use the new LiveReader from TSDB to tail WAL segments. Logic for finding the tracking segment is included in this PR. The WAL is tailed once for each remote_write endpoint specified. Reading from the segment is based on a ticker rather than relying on fsnotify write events, which were found to be complicated and unreliable in early prototypes. Enqueuing a sample for sending via remote_write can now block, to provide back pressure. Queues are still required to acheive parallelism and batching. We have updated the queue config based on new defaults for queue capacity and pending samples values - much smaller values are now possible. The remote_write resharding code has been updated to prevent deadlocks, and extra tests have been added for these cases. As part of this change, we attempt to guarantee that samples are not lost; however this initial version doesn't guarantee this across Prometheus restarts or non-retryable errors from the remote end (eg 400s). This changes also includes the following optimisations: - only marshal the proto request once, not once per retry - maintain a single copy of the labels for given series to reduce GC pressure Other minor tweaks: - only reshard if we've also successfully sent recently - add pending samples, latest sent timestamp, WAL events processed metrics Co-authored-by: Chris Marchbanks <csmarchbanks.com> (initial prototype) Co-authored-by: Tom Wilkie <tom.wilkie@gmail.com> (sharding changes) Signed-off-by: Callum Styan <callumstyan@gmail.com>	6 years ago
Hrishikesh Barman	a1f34bec2e	Added CORS Origin flag (#5011) ... Signed-off-by: Hrishikesh Barman <hrishikeshbman@gmail.com>	7 years ago
Matt Layher	302148fd69	*: apply gofmt -s ... Signed-off-by: Matt Layher <mdlayher@gmail.com>	7 years ago