203 Commits (c0e888e82b6bab94b15094de0aa40f14257ebc07)

Author	SHA1	Message	Date
Matt Bostock	bc97c6c6be	Misc fixes (#285) ... * Fix typo in head.go pralellize -> paralellize * Remove commented out code It's dead code, remove it. * Correct reference to sample buffer	8 years ago
Matt Bostock	50211e89ad	Fix typos in comments (#254) ... a the -> the timestmap -> timestamp badded -> padded its -> it is callers -> caller's	8 years ago
Fabian Reinartz	67f0ca8f0e	Move index and chunk encoders to own packages	8 years ago
Goutham Veeramachaneni	3158b03e6c	Select series with label unset for != and !~ ... Fixes https://github.com/prometheus/prometheus/issues/3575 Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in> Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com>	8 years ago
Goutham Veeramachaneni	05d62ca842	Make sure gc'ed chunks are handled properly ... Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com>	8 years ago
Fabian Reinartz	f1512a368a	Expose ChunkSeriesSet and lookups methods.	8 years ago
Fabian Reinartz	3ef4326114	Refactor tombstone reader types	8 years ago
Fabian Reinartz	e5ce2bef43	Add explicit error to Querier.Select ... This has been a frequent source of debugging pain since errors are potentially delayed to a much later point. They bubble up in an unrelated execution path.	8 years ago
Julius Volz	1dad3370fd	Close WAL when closing the DB ... Also, the `wal` field of the `DB` was not used anywhere, so this removes it.	8 years ago
Oren Shomron	6ca5e52b69	Typo in prometheus_tsdb_head_samples_appended_total description (#188)	8 years ago
Fabian Reinartz	82796db37b	Ensure near-empty chunks end at correct boundary ... We were determining a chunk's end time once it was one quarter full to compute it so all chunks have uniform number of samples. This accidentally skipped the case where series started near the end of a chunk range/block and never reached that threshold. As a result they got persisted but were continued across the range. This resulted in corrupted persisted data.	8 years ago
Fabian Reinartz	9749aa2a3e	head: limit WAL sample processing batch size	8 years ago
Fabian Reinartz	d17104f1f0	Prefix all metrics with `prometheus_*`	8 years ago
Fabian Reinartz	ea817e169b	Return nop iterator for invalid chunk references	8 years ago
Fabian Reinartz	6dcca97755	Fix race in symbol table re-creation	8 years ago
Fabian Reinartz	065f42f58c	head: track number of series not found errors in metric	8 years ago
Fabian Reinartz	88305e7612	Access chunk time range while holding lock	8 years ago
Fabian Reinartz	106eaf39d1	Ensure workers terminated fully before reading unknownRefs	8 years ago
Fabian Reinartz	665955da48	Clarify postings index semantics, handle staleness ... The postings list index may point to series that no longer exist during garbage collection. This clarifies that this is valid behavior. It would be possible, though more complex, to always keep them in sync. However, series existance means nothing in itself as the queried time range defines whether there's actual data. Thus our definition is sane overall as long as drift is kept small.	8 years ago
Fabian Reinartz	fb9da52b11	Add more verbose error handling for closing, reduce locking ... This commit introduces error returns in various places and is explicit about closing persisted blocks. {Index,Chunk,Tombstone}Readers are more consistent about their Close() method. Whenever a reader is retrieved, the corresponding close method must eventually be called. We use this to track pending readers against persisted blocks. Querier's against the DB no longer hold a read lock for their entire lifecycle. This avoids long running queriers to starve new ones when we have to acquire a write lock when reloading blocks.	8 years ago
Fabian Reinartz	7efb830d70	wal: parallelize sample processing	8 years ago
Simon Pasquier	e858c0826c	Fix innocuous typo in variable names ... This change fixes the variable names holding the tsdb_head_max_time and tsdb_head_min_time metrics. It is a cosmetic change to improve the code readability as the metric values are taken from the correct variables.	8 years ago
Fabian Reinartz	d3682d701c	wal: decode and process in separate threads.	8 years ago
Fabian Reinartz	cd2e26b7fc	Load postings in batch on startup ... This allows to insert IDs to postings out of order until a trigger function is called. This avoids the insertion sort we usually do which can be very costly since WAL entries are more out of order than regular adds.	8 years ago
Goutham Veeramachaneni	203012169a	snapshot: Remove truncation check to restore func. ... Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	8 years ago
Goutham Veeramachaneni	c35d3a65bd	Add levels to all log lines. ... Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	8 years ago
Fabian Reinartz	1e88ba06b4	Use boolean function instead of postings to drop WAL series ... There is not guarantee or requirement for WAL writers to only add series entries in increasing order of IDs. A postings list cannot look back and thus unordered WAL entries would skip over IDs to not truncate from the WAL. We replace it with a simple boolean check function that does not require order.	8 years ago
Fabian Reinartz	6ee254e353	Ensure postings are always sorted ... IDs for new series are handed out before the postings are locked. Thus series are not indexed in order of their IDs, which could result in only partially sorted postings list. Iterating over those silently skipped elements as the sort invariant was violated.	8 years ago
Fabian Reinartz	162a48e4f2	Create series with ID recorded in WAL when reading it back	8 years ago
Fabian Reinartz	7ada9cd805	Simplify series create logic in head	8 years ago
Fabian Reinartz	ab8d9b9706	Add missing unlock on early return	8 years ago
Fabian Reinartz	f904cd385f	Do not build a superflous 'all' postings	8 years ago
Fabian Reinartz	6892fc6dcb	Finish old WAL segment async, default to no fsync ... We were still fsyncing while holding the write lock when we cut a new segment. Given we cannot do anything but logging errors, we might just as well complete segments asynchronously. There's not realistic use case where one would fsync after every WAL entry, thus make the default of a flush interval of 0 to never fsync which is a much more likely use case.	8 years ago
Fabian Reinartz	1d5f85817d	Fix various races	8 years ago
Fabian Reinartz	0db4c227b7	Fix min/max time handling and concurrent crc32 usage	8 years ago
Fabian Reinartz	81222849bc	Filter WAL data in Head, misc fixes	8 years ago
Fabian Reinartz	33e9bdf403	WAL refactoring and truncation fixes and test	8 years ago
Fabian Reinartz	c36d574290	Replace single head lock with granular locks ... This adds various new locks to replace the single big lock on the head. All parts now must be COW as they may be held by clients after initial retrieval. Series by ID and hashes are now held in a stripe lock to reduce contention and total holding time during GC. This should reduce starvation of readers.	8 years ago
Fabian Reinartz	1ddedf2b30	Change series ID from uint32 to uint64	8 years ago
Goutham Veeramachaneni	1698c516ad	[WIP]: WAL implementation ... Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	8 years ago
Fabian Reinartz	893b6ec506	Add tests for GC and chunk truncation	8 years ago
Fabian Reinartz	4f037da462	Remove defer statement in hot path	8 years ago
Fabian Reinartz	5cf2662074	Refactor WAL into Head and misc improvements	8 years ago
Fabian Reinartz	8209e3ec23	Add various metrics	8 years ago
Fabian Reinartz	3901b6e70b	Remove multiple heads ... This changes the structure to a single WAL backed by a single head block. Parts of the head block can be compacted. This relieves us from any head amangement and greatly simplifies any consistency and isolation concerns by just having a single head.	8 years ago
Goutham Veeramachaneni	7438ed7035	Expose Intervals type for use by TombstoneReader. ... TombstoneReader is exposed but Intervals is not. Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	8 years ago
Fabian Reinartz	905af27cf9	Refactor compactor	9 years ago
Fabian Reinartz	66ff7b12e9	Pool Chunk objects during compaction	9 years ago
Fabian Reinartz	2644c8665c	Don't allocate ChunkMetas, reuse postings slices	9 years ago
Fabian Reinartz	96d7f540d4	Persist series without allocating the full set ... Change index persistence for series to not be accumulated in memory before being written as one large batch. `Labels` and `ChunkMeta` objects are reused. This cuts down memory spikes during compaction of multiple blocks significantly. As part of the the Index{Reader,Writer} now have an explicit notion of symbols and series must be inserted in order.	9 years ago