117 Commits (4dd740e4cc2d6fb490a94deec899bb4ddb191eb4)

Author	SHA1	Message	Date
Bartek Płotka	047b1b1357	compact: Verify for chunks outside of compacted time range. Added unit test for populateBlocs. (#349) ... * compact: Verify for chunks outside of compacted time range. Unit test for populateBlocs. Signed-off-by: Bartek Plotka <bwplotka@gmail.com> Co-authored-by: Krasi Georgiev <kgeorgie@redhat.com>	7 years ago
Goutham Veeramachaneni	9c8ca47399	Fix filehandling for windows (#392) ... * Fix filehandling for windows Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com> * Fix more windows filehandling issues Windows: Close files before deleting Checkpoints. Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com> Windows: Close writers in case of errors so they can be deleted Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com> Windows: Close block so that it can be deleted. Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com> Windows: Close file to delete it Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com> Windows: Close dir so that it can be deleted. Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com> Windows: close files so that they can be deleted. Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com> * Review feedback Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com>	7 years ago
Julius Volz	5ae6c60d39	Handle a bunch of unchecked errors (#365) ... As discovered by "gosec". Signed-off-by: Julius Volz <julius.volz@gmail.com>	7 years ago
beorn7	3bc6c670fa	Revert "Remove `prometheus_` prefix from metrics" ... This reverts commit 98fe30438c. After some discussion, it was concluded that we want the full `prometheus_tsdb_...` prefix hardcoded in the library. Signed-off-by: beorn7 <beorn@soundcloud.com>	7 years ago
beorn7	98fe30438c	Remove `prometheus_` prefix from metrics ... This can now be added by users of the library as needed with the new https://godoc.org/github.com/prometheus/client_golang/prometheus#WrapRegistererWithPrefix Signed-off-by: beorn7 <beorn@soundcloud.com>	7 years ago
Ganesh Vernekar	2945db18ca	Changes in series names (and types) exposed (#376) ... Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>	7 years ago
Benoît Knecht	1e1b2e163d	Make interval overlap comparisons more explicit ... Blocks are half-open intervals [a, b), while all other intervals (chunks, head, ...) are closed intervals [a, b]. Make that distinction explicit by defining `OverlapsClosedInterval()` methods for blocks and chunks, and using them in place of the more generic `intervalOverlap()` function. This change also fixes `db.Querier()` and `db.Delete()`, which could previously return one extraneous block at the end of the specified interval. Signed-off-by: Benoît Knecht <benoit.knecht@fsfe.org>	8 years ago
Fabian Reinartz	af9003dcef	Add resilience to crashes during deletion ... Signed-off-by: Fabian Reinartz <freinartz@google.com>	8 years ago
codwu	84a45cb79a	add rwmutex to prevent concurrent map read when delete series ... Signed-off-by: codwu <wuhan9087@163.com>	8 years ago
Bartek Plotka	fada85a83c	repair + compact: Improved logging for easier future debug purposes. ... This is based on my experience while debugging https://github.com/prometheus/prometheus/issues/3943. I needed to deduct few things, and all that would be just bit easier with these two logs: - new block's ULID on each compaction. - actual list of Blocks (ulid + time range) on Prometheus startup (easy to log that while repairing blocks). We don't really need blocks that takes part in compaction - that can be deducted easily based on time ranges of blocks we have currently in system. What do you think? Signed-off-by: Bartek Plotka <bwplotka@gmail.com>	8 years ago
Bartek Plotka	328c0ff5b0	Simplified the flow and tests. ... Signed-off-by: Bartek Plotka <bwplotka@gmail.com>	8 years ago
Bartek Plotka	483da43660	compact: Exclude last block with shortest range instead of newest one by ULID. ... Fixes https://github.com/prometheus/prometheus/issues/3943 issue. Added tests. Signed-off-by: Bartek Plotka <bwplotka@gmail.com>	8 years ago
Fabian Reinartz	8cd32d8fd4	Improve usability of snapshots	8 years ago
Fabian Reinartz	d09283f60a	Add repair of bad index version	8 years ago
Goutham Veeramachaneni	6c62437101	Fix metric unit (#275) ... Fixes #274 Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	8 years ago
Simon Pasquier	5d47710f8a	Fix panic on nil index writer	8 years ago
Shubheksha Jalan	44052bc937	Make version for index format configurable	8 years ago
Fabian Reinartz	1e55b7987f	Improve comments, handle allPostingsKey properly	8 years ago
Fabian Reinartz	7087f68647	Return new block ULID in compactor	8 years ago
Fabian Reinartz	67f0ca8f0e	Move index and chunk encoders to own packages	8 years ago
Fabian Reinartz	cf7f6108fd	Order postings lists in index file by key ... Aligning postings list for similar keys close to each other improves page cache hit rates in typical queries that select postings for multiple label pairs with the same name.	8 years ago
Goutham Veeramachaneni	1fc94a02d1	Don't retry failed compactions. ... Fixes prometheus/prometheus#3487 Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	8 years ago
Goutham Veeramachaneni	1627a47640	A copy is being assigned newChunk not the original ... Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	8 years ago
Fabian Reinartz	a031cf7424	Return ULID in Compactor	8 years ago
Fabian Reinartz	f1512a368a	Expose ChunkSeriesSet and lookups methods.	8 years ago
Fabian Reinartz	3ef4326114	Refactor tombstone reader types	8 years ago
ranbochen	a27cf34a36	fix bugs on platform windows to pass all test case. (#192) ... * fix bugs on platform windows to pass all test case. * fix bugs on platform windows to pass all test case * clean up codes	8 years ago
ranbo chen	b82b2d0c1b	fix bugs on platform windows to pass all test case.	8 years ago
Fabian Reinartz	d17104f1f0	Prefix all metrics with `prometheus_*`	8 years ago
Fabian Reinartz	fb9da52b11	Add more verbose error handling for closing, reduce locking ... This commit introduces error returns in various places and is explicit about closing persisted blocks. {Index,Chunk,Tombstone}Readers are more consistent about their Close() method. Whenever a reader is retrieved, the corresponding close method must eventually be called. We use this to track pending readers against persisted blocks. Querier's against the DB no longer hold a read lock for their entire lifecycle. This avoids long running queriers to starve new ones when we have to acquire a write lock when reloading blocks.	8 years ago
Fabian Reinartz	f04ec031eb	compact: sync temporary directory	8 years ago
Fabian Reinartz	bbe72dccb9	Remove dependency on etcd/pkg/fileutil	8 years ago
Goutham Veeramachaneni	c35d3a65bd	Add levels to all log lines. ... Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	8 years ago
Fabian Reinartz	6ee254e353	Ensure postings are always sorted ... IDs for new series are handed out before the postings are locked. Thus series are not indexed in order of their IDs, which could result in only partially sorted postings list. Iterating over those silently skipped elements as the sort invariant was violated.	8 years ago
Fabian Reinartz	f904cd385f	Do not build a superflous 'all' postings	8 years ago
Fabian Reinartz	c36d574290	Replace single head lock with granular locks ... This adds various new locks to replace the single big lock on the head. All parts now must be COW as they may be held by clients after initial retrieval. Series by ID and hashes are now held in a stripe lock to reduce contention and total holding time during GC. This should reduce starvation of readers.	8 years ago
Fabian Reinartz	1ddedf2b30	Change series ID from uint32 to uint64	8 years ago
Fabian Reinartz	9f41d9fd3c	Instrument chunks on level 1 compactions	8 years ago
Fabian Reinartz	4cc37eecab	Refactor and add tests for compactor	8 years ago
Fabian Reinartz	3901b6e70b	Remove multiple heads ... This changes the structure to a single WAL backed by a single head block. Parts of the head block can be compacted. This relieves us from any head amangement and greatly simplifies any consistency and isolation concerns by just having a single head.	8 years ago
Goutham Veeramachaneni	7438ed7035	Expose Intervals type for use by TombstoneReader. ... TombstoneReader is exposed but Intervals is not. Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	8 years ago
Goutham Veeramachaneni	178f840295	Fix panic caused by 0 division ... Introduced in #108 Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	8 years ago
Fabian Reinartz	905af27cf9	Refactor compactor	9 years ago
Fabian Reinartz	66ff7b12e9	Pool Chunk objects during compaction	9 years ago
Fabian Reinartz	2644c8665c	Don't allocate ChunkMetas, reuse postings slices	9 years ago
Fabian Reinartz	96d7f540d4	Persist series without allocating the full set ... Change index persistence for series to not be accumulated in memory before being written as one large batch. `Labels` and `ChunkMeta` objects are reused. This cuts down memory spikes during compaction of multiple blocks significantly. As part of the the Index{Reader,Writer} now have an explicit notion of symbols and series must be inserted in order.	9 years ago
Fabian Reinartz	3951d8cc29	Fix selection of compactable blocks	9 years ago
Fabian Reinartz	5d7b5994d6	Fix compaction range selection	9 years ago
Goutham Veeramachaneni	401bdfd4a6	Make Chunks return the number of samples inside ... Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	9 years ago
Goutham Veeramachaneni	1263a68875	Recompact blocks with large number of tombstones ... Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	9 years ago