58 Commits (6f9bbc7253cb1cd550eeba2c3cc4c3015abbc91c)

Author	SHA1	Message	Date
Yao Zengzeng	bff3ef3db4	minor enhancement: return multi error when closeAll() (#663) ... Signed-off-by: YaoZengzeng <yaozengzeng@zju.edu.cn>	7 years ago
Ganesh Vernekar	3bc1ea3d75	Reuse byte buffer in WriteChunks and writeHash (#653) ... * Re-use byte buffer in WriteChunks and writeHash Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Add CHANGELOG entry Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Fix review comments Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Remove old comment Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>	7 years ago
johncming	84d71073f1	chunks: correct the interface name ... Signed-off-by: johncming <johncming@yahoo.com>	7 years ago
Ganesh Vernekar	b1cd829030	Reuse Chunk Iterator (#642) ... * Reset method for chunkenc.Iterator Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Reset method only for XORIterator Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Use Reset(...) in querier.go Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Reuse deletedIterator Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Another way of reusing chunk iterators Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Unexport xorIterator Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Fix memSeries.iterator(...) Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Add some comments Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>	7 years ago
Krasi Georgiev	31f7990d1d	Re-encode chunks that are still being appended to when snapshoti… (#641) ... * re encode all head chunks outside the time range. Signed-off-by: Krasi Georgiev <8903888+krasi-georgiev@users.noreply.github.com>	7 years ago
Krasi Georgiev	8eeb70fee1	remove Fsync workaround for macos. (#574) ... since golang 1.12 no special handling is required for file.Sync() @pborzenkov thanks for the pointer. Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>	7 years ago
Pavel Borzenkov	4f204dcbc1	chunks: close all opened MmapFiles on NewDirReader error path (#575) ... Signed-off-by: Pavel Borzenkov <pavel.borzenkov@gmail.com>	7 years ago
zhulongcheng	e11e01f68d	Fix exported function comments (#566) ... Signed-off-by: zhulongcheng <zhulongcheng.me@gmail.com>	7 years ago
zhulongcheng	4d03c70800	Verify chunk format version (#544) ... Verify chunk format version.	7 years ago
zhulongcheng	b7cac504c9	Remove unused NewReader function (#547) ... The `NewReader` function is not used anywhere.	7 years ago
Alec	0861a9b14f	[bugfix] update 'last' variable in chunks.MergeOverlappingChunks() (#539) ... * update Signed-off-by: naivewong <867245430@qq.com> * update comment Signed-off-by: naivewong <867245430@qq.com> * Update failing test. Signed-off-by: naivewong <867245430@qq.com> * Update comment Signed-off-by: naivewong <867245430@qq.com> * Add changelog entry. Signed-off-by: naivewong <867245430@qq.com> * Update CHANGELOG.md Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>	7 years ago
Ganesh Vernekar	c59ed492b2	Vertical query merging and compaction (#370) ... * Vertical series iterator Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Select overlapped blocks first in compactor Plan() Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Added vertical compaction Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Code cleanup and comments Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Fix review comments Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Fix tests Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Add benchmark for compaction Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Perform vertical compaction only when blocks are overlapping. Actions for vertical compaction: * Sorting chunk metas * Calling chunks.MergeOverlappingChunks on the chunks Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Benchmark for vertical compaction * BenchmarkNormalCompaction => BenchmarkCompaction * Moved the benchmark from db_test.go to compact_test.go Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Benchmark for query iterator and seek for non overlapping blocks Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Vertical query merge only for overlapping blocks Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Simplify logging in Compact(...) Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Updated CHANGELOG.md Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Calculate overlapping inside populateBlock Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * MinTime and MaxTime for BlockReader. Using this to find overlapping blocks in populateBlock() Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Sort blocks w.r.t. MinTime in reload() Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Log about overlapping in LeveledCompactor.write() instead of returning bool Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Log about overlapping inside LeveledCompactor.populateBlock() Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Fix review comments Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Refactor createBlock to take optional []Series Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * review1 Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com> * Updated CHANGELOG and minor nits Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * nits Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Updated CHANGELOG Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Refactor iterator and seek benchmarks for Querier. Also has as overlapping blocks. Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Additional test case Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * genSeries takes optional labels. Updated BenchmarkQueryIterator and BenchmarkQuerySeek. Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Split genSeries into genSeries and populateSeries Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Check error in benchmark Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Fix review comments Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in> * Warn about overlapping blocks in reload() Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>	7 years ago
Krasi Georgiev	bf2239079d	refactor multi errors ... Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>	7 years ago
Krasi Georgiev	ee99718ff6	rename chunk reader vars to make it easier to follow. (#508) ... * rename chunk reader vars to make it easyer to understand. Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>	7 years ago
Alec	051a7ae1a7	Missing the length of the encoding byte when calling b.Range ... Signed-off-by: naivewong <867245430@qq.com>	7 years ago
mknapphrt	ebf5d74325	Added storage size based retention method and new metrics (#343) ... Added methods needed to retain data based on a byte limitation rather than time. Limitation is only applied if the flag is set (defaults to 0). Both blocks that are older than the retention period and the blocks that make the size of the storage too large are removed. 2 new metrics for keeping track of the size of the local storage folder and the amount of times data has been deleted because the size restriction was exceeded. Signed-off-by: Mark Knapp <mknapp@hudson-trading.com>	7 years ago
naivewong	bff5aa4d21	Missing the len of crc32 when calculating maxLen in WriteChunks (#494) ... Signed-off-by: naivewong <867245430@qq.com>	7 years ago
Simon Pasquier	d5d7a097e1	Update Makefile.common ... This change also uses the latest staticcheck version which comes with new verifications, hence some clean up in the code. Signed-off-by: Simon Pasquier <spasquie@redhat.com>	7 years ago
Benoît Knecht	1e1b2e163d	Make interval overlap comparisons more explicit ... Blocks are half-open intervals [a, b), while all other intervals (chunks, head, ...) are closed intervals [a, b]. Make that distinction explicit by defining `OverlapsClosedInterval()` methods for blocks and chunks, and using them in place of the more generic `intervalOverlap()` function. This change also fixes `db.Querier()` and `db.Delete()`, which could previously return one extraneous block at the end of the specified interval. Signed-off-by: Benoît Knecht <benoit.knecht@fsfe.org>	8 years ago
Simon Pasquier	f55ccd4ecb	Add unit tests ... Signed-off-by: Simon Pasquier <spasquie@redhat.com>	8 years ago
Simon Pasquier	ee5fe8ea9f	chunks: fix potential "index out of range" error ... When binary.Uvarint() fails, the returned number of bytes is less than or equal to zero. Signed-off-by: Simon Pasquier <spasquie@redhat.com>	8 years ago
Mario Trangoni	c0e888e82b	fix megacheck issues: os.SEEK_SET is deprecated: Use io.SeekStart, io.SeekCurrent, and io.SeekEnd.	8 years ago
Mario Trangoni	09142e4dd1	fix unconvert issues: unnecessary conversion	8 years ago
Fabian Reinartz	1e55b7987f	Improve comments, handle allPostingsKey properly	8 years ago
Fabian Reinartz	67f0ca8f0e	Move index and chunk encoders to own packages	8 years ago
Callum Styan	ff0b0ac4b6	use test utils in chunk_test	8 years ago
Daniel Swarbrick	0585ab3fcd	Use Go 1.9 math/bits implementation for counting leading / trailing zeros	8 years ago
Fabian Reinartz	ea817e169b	Return nop iterator for invalid chunk references	8 years ago
Fabian Reinartz	905af27cf9	Refactor compactor	9 years ago
Fabian Reinartz	66ff7b12e9	Pool Chunk objects during compaction	9 years ago
Fabian Reinartz	96d7f540d4	Persist series without allocating the full set ... Change index persistence for series to not be accumulated in memory before being written as one large batch. `Labels` and `ChunkMeta` objects are reused. This cuts down memory spikes during compaction of multiple blocks significantly. As part of the the Index{Reader,Writer} now have an explicit notion of symbols and series must be inserted in order.	9 years ago
Goutham Veeramachaneni	401bdfd4a6	Make Chunks return the number of samples inside ... Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	9 years ago
Julius Volz	0e2f8917ed	Also add original license info to bstream.go ... Plus, extend the messages a bit to mention the origin repo.	9 years ago
Fabian Reinartz	87676f4a26	Add dgryski/go-tsz LICENSE to header	9 years ago
Julius Volz	8d1fb4fa01	Minor comment fixes and additions.	9 years ago
Fabian Reinartz	778103b450	Add liecence file and headers	9 years ago
Goutham Veeramachaneni	761e4768f3	Lint and Vet Fixes	9 years ago
Fabian Reinartz	91b65b55e7	Run persistence in separate goroutine	9 years ago
Fabian Reinartz	62b8ded0a5	Rename iterator value getters to At()	9 years ago
Fabian Reinartz	5d75a3dc7b	Add basic compaction ... This adds a basic compactor that will merge two persisted blocks into one. It simply fully rewrites the index and concatenates the chunk lists. It just writes into the current working dir and doesn't properly handle which blocks to compact for now.	9 years ago
Fabian Reinartz	675f0886f0	Append to chunks cannot error	9 years ago
Fabian Reinartz	ce7f4106c2	Reda correct label number, fix buffered iterator panic	9 years ago
Fabian Reinartz	70a0224f19	Change chunk sample number to BigEndian	9 years ago
Fabian Reinartz	40a451694f	Refactor persistence into interfaces	9 years ago
Fabian Reinartz	b845f8d3a1	Reduce test data allocations	9 years ago
Fabian Reinartz	c5945177fb	chunks: helper for bit range	9 years ago
Fabian Reinartz	9b459458d0	Docs and interface definitions	9 years ago
Fabian Reinartz	0b6d621471	chunks: bring back lead/trail reuse, truncate incomplete sample	9 years ago
Fabian Reinartz	8c48dc2ca5	chunks: varint encoding in first full 64bit numbers ... This saves about 7 bytes per chunk	9 years ago
Fabian Reinartz	5e76fd3126	chunks: drop shift, remove branch ... This removes the need for the shift value and just calculates everything on the fly through count. Removes a branch condition.	9 years ago