257 Commits (4dd740e4cc2d6fb490a94deec899bb4ddb191eb4)

Author	SHA1	Message	Date
Fabian Reinartz	f8e88bfdb7	Close previous block queriers on error ... This ensures we close all previously opened queriers if on of the block querier fails to open. Also swap in new blocks before closing old ones to avoid the situation in general. Make read locking of blocks more conservative to avoid unnecessary retries by clients, e.g. when blocks are getting closed before we can successfully instantiate querier against them.	8 years ago
Fabian Reinartz	d17104f1f0	Prefix all metrics with `prometheus_*`	8 years ago
Fabian Reinartz	fb9da52b11	Add more verbose error handling for closing, reduce locking ... This commit introduces error returns in various places and is explicit about closing persisted blocks. {Index,Chunk,Tombstone}Readers are more consistent about their Close() method. Whenever a reader is retrieved, the corresponding close method must eventually be called. We use this to track pending readers against persisted blocks. Querier's against the DB no longer hold a read lock for their entire lifecycle. This avoids long running queriers to starve new ones when we have to acquire a write lock when reloading blocks.	8 years ago
Bas Harenslak	5e1c258a98	Instrument WAL fsync	8 years ago
Fabian Reinartz	bbe72dccb9	Remove dependency on etcd/pkg/fileutil	8 years ago
Dimitrij Denissenko	83c984d9a3	Use configurable WAL flush inteval	8 years ago
Bas Harenslak	a41dad30fb	Move printing db.blocks to main.go	8 years ago
Bas Harenslak	9945a67bff	Replace columnize by stdlib tabwriter	8 years ago
Bas Harenslak	e190c7c78d	Add list blocks command to CLI	8 years ago
Goutham Veeramachaneni	c35d3a65bd	Add levels to all log lines. ... Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	8 years ago
Thibault Chataigner	0ff8f91ec8	Add db.Head() method ... Signed-off-by: Thibault Chataigner <t.chataigner@criteo.com>	8 years ago
Thibault Chataigner	5ca18837e9	Make DB blocks list public. ... This enable computing the first timestamp in the DB only accessing public interfaces. Signed-off-by: Thibault Chataigner <t.chataigner@criteo.com>	8 years ago
Goutham Veeramachaneni	8919baef03	Expose NewIndexReader() and cleanups ... Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	8 years ago
Fabian Reinartz	f904cd385f	Do not build a superflous 'all' postings	8 years ago
Fabian Reinartz	81222849bc	Filter WAL data in Head, misc fixes	8 years ago
Fabian Reinartz	c36d574290	Replace single head lock with granular locks ... This adds various new locks to replace the single big lock on the head. All parts now must be COW as they may be held by clients after initial retrieval. Series by ID and hashes are now held in a stripe lock to reduce contention and total holding time during GC. This should reduce starvation of readers.	8 years ago
Fabian Reinartz	af2c2f9674	Re-enable head block delet test	8 years ago
Goutham Veeramachaneni	1698c516ad	[WIP]: WAL implementation ... Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	8 years ago
Fabian Reinartz	893b6ec506	Add tests for GC and chunk truncation	8 years ago
Fabian Reinartz	4cc37eecab	Refactor and add tests for compactor	8 years ago
Fabian Reinartz	5cf2662074	Refactor WAL into Head and misc improvements	8 years ago
Fabian Reinartz	8209e3ec23	Add various metrics	8 years ago
Fabian Reinartz	3901b6e70b	Remove multiple heads ... This changes the structure to a single WAL backed by a single head block. Parts of the head block can be compacted. This relieves us from any head amangement and greatly simplifies any consistency and isolation concerns by just having a single head.	8 years ago
Andreas Kohn	4003da0fdd	Log the directory when the WAL cannot be opened ... Fixes #125	8 years ago
Fabian Reinartz	905af27cf9	Refactor compactor	9 years ago
Fabian Reinartz	66ff7b12e9	Pool Chunk objects during compaction	9 years ago
Fabian Reinartz	fa04455cd7	Properly detect compactable heads ... The fullness of the wrong head was being tracked, causing wrong lists of compactable heads.	9 years ago
tianwenjie01	b64054365d	Fix DefaultOptions	9 years ago
Fabian Reinartz	74f67e8271	Fix compacting disable/enable ... Enabling and disabling compaction no longer blocks are potentially causes panics.	9 years ago
Fabian Reinartz	3065be97d8	Fix and document locking order for DB	9 years ago
Fabian Reinartz	9c4235532e	Fix compaction selection after creating new heads ... This fixes the case where between block creations no compaction plans are ran. We were not compacting anything in these cases since the on creation the most recent head block always had a high timestamp of 0.	9 years ago
Goutham Veeramachaneni	9a2ab732b5	Choose ranges by retention and min-block-duration ... Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	9 years ago
Goutham Veeramachaneni	f1ae239c20	Persist the right MaxTime when snapshotting ... This is because we cut a new block from where the snapshotted block ends if we restore from backups and highTimestamp would be where we should be starting from. Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	9 years ago
Goutham Veeramachaneni	ce23adb579	Initial compaction implementation for block-ranges ... Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	9 years ago
Fabian Reinartz	1e74c155eb	Return empty string to signal non-caching	9 years ago
Fabian Reinartz	3410559c1b	Compact head block early ... Let older head blocks be compacted once the newest once has samples at 50% of its total range. This allows the memory of the compacted blocks to be released and garbage collected before a new head block gets created. Thereby the number of head blocks is 1 or 2 instead of 2 or 3 and memory spikes are reduced.	9 years ago
Fabian Reinartz	03f95ee096	Fix directory file descriptor leak.	9 years ago
Fabian Reinartz	d4cb579443	Add DB.Dir method	9 years ago
Goutham Veeramachaneni	ff4ccb6eb0	Remove unnecessary error from ToggleCompaction fns ... Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	9 years ago
Goutham Veeramachaneni	261cd9f393	Incorporate feedback. ... Move back to {Enable, Disable}Compactions. Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	9 years ago
Goutham Veeramachaneni	a110a64abd	Add full Snapshot support ... Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	9 years ago
Goutham Veeramachaneni	a1c8425357	Initial implementation of HeadBlock Snapshots ... Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	9 years ago
Fabian Reinartz	b3b9245ad6	Add and fix metrics	9 years ago
Goutham Veeramachaneni	6febabeb28	Final delete fixes. ... * Make sure no reads happen on the block when delete is in progress. * Fix bugs in compaction. Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	9 years ago
Goutham Veeramachaneni	9bf7aa9af1	Misc. fixes incorporating feedback. ... Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	9 years ago
Goutham Veeramachaneni	0b70333ef6	Add tests for tombstones. ... Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	9 years ago
Goutham Veeramachaneni	d32eb25662	Synchronise deletes and compactions. ... Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	9 years ago
Goutham Veeramachaneni	3dbb400bef	Initial implementation of Delete on DB. ... Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	9 years ago
Fabian Reinartz	debfe9b1e5	Simplify AddFast and avoid errgroup Commits in general	9 years ago
Fabian Reinartz	fb5c5535fc	Misc cleanup	9 years ago