26 Commits (4dd740e4cc2d6fb490a94deec899bb4ddb191eb4)

Author	SHA1	Message	Date
Kautilya Tripathi	3fd6d2f920	more meaningful names for serializedStringTuples and stringTuples (#377) ... more meaningful names for serializedStringTuples and stringTuples structs Signed-off-by: knrt10 <tripathi.kautilya@gmail.com> Co-authored-by: Krasi Georgiev <kgeorgie@redhat.com>	7 years ago
Julius Volz	5ae6c60d39	Handle a bunch of unchecked errors (#365) ... As discovered by "gosec". Signed-off-by: Julius Volz <julius.volz@gmail.com>	7 years ago
Simon Pasquier	e8fc6c8774	index: fix another Uvarint() return check ... Signed-off-by: Simon Pasquier <spasquie@redhat.com>	8 years ago
Mario Trangoni	09142e4dd1	fix unconvert issues: unnecessary conversion	8 years ago
Fabian Reinartz	3f06121dd0	Add Version getter for index reader	8 years ago
Fabian Reinartz	a345231a0e	Add test for block meta version and comments	8 years ago
Fabian Reinartz	d09283f60a	Add repair of bad index version	8 years ago
Goutham Veeramachaneni	bb0e74b343	Fix reader for large index files. ... Currently the offsets are cast into uint32 even though the index can grow larger than 4GiB. Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	8 years ago
Callum Styan	cd67584817	we don't want to write to index file in older version formats	8 years ago
Callum Styan	4792d2bbd1	handle v1 vs v2 for symbol offset vs index changes	8 years ago
Callum Styan	a515b16299	use an incrementing index rather than an offset for symbols table	8 years ago
Goutham Veeramachaneni	2c3400ab4e	Add block ULID to index reader errors ... Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	8 years ago
Shubheksha Jalan	618bceadb4	move version check into newReader()	8 years ago
Shubheksha Jalan	faecfac2fa	update docs, rename docs folder, add error condition	8 years ago
Shubheksha Jalan	7b27dc2109	address review comments, fix index tests	8 years ago
Shubheksha Jalan	510dc17066	fix tests, set version when initializing the index writer	8 years ago
Shubheksha Jalan	44052bc937	Make version for index format configurable	8 years ago
Shubheksha Jalan	129773b41a	merge BlockMeta and blockMeta struct, implement v1 and v2 index readers	8 years ago
Shubheksha Jalan	77eb9fb56e	add padding for series offsets	8 years ago
Fabian Reinartz	b945098e3a	index: extract decoder type	8 years ago
Fabian Reinartz	6dd2b83a7a	index: don't use concatenated string for posting table	8 years ago
Fabian Reinartz	67f0ca8f0e	Move index and chunk encoders to own packages	8 years ago
Fabian Reinartz	ac5bd71d8f	Doc fixes	8 years ago
Fabian Reinartz	b7c3cfecbf	index: abstract ByteSlice and adjust indexReader ... This replaces the builtin byte slice with an interface for the index reader. This allows the complex decoding of the index file format to be used against more generalized implementations.	8 years ago
Fabian Reinartz	c354d6bd59	index: simplify checksum validation	8 years ago
Nipun Talukdar	791a2dda4d	Fixed a problem of adding padding of 4 zero bytes in some cases (#194) ... * Fixed a problem of adding padding of 4 zero bytes in some cases * Incorporated review comments	8 years ago
ranbochen	a27cf34a36	fix bugs on platform windows to pass all test case. (#192) ... * fix bugs on platform windows to pass all test case. * fix bugs on platform windows to pass all test case * clean up codes	8 years ago
ranbo chen	b82b2d0c1b	fix bugs on platform windows to pass all test case.	8 years ago
Sunny Klair	aecac3b521	Resuse single CRC for index checksum validation	8 years ago
Sunny Klair	b65dd43c5b	Validate index series/postings/symbol table checksums on read	8 years ago
Sunny Klair	ab02ea4de4	Validate index offset table checksums on read	8 years ago
Sunny Klair	4fdf9b195c	Validate index TOC checksum on read	8 years ago
Fabian Reinartz	665955da48	Clarify postings index semantics, handle staleness ... The postings list index may point to series that no longer exist during garbage collection. This clarifies that this is valid behavior. It would be possible, though more complex, to always keep them in sync. However, series existance means nothing in itself as the queried time range defines whether there's actual data. Thus our definition is sane overall as long as drift is kept small.	8 years ago
Fabian Reinartz	fb9da52b11	Add more verbose error handling for closing, reduce locking ... This commit introduces error returns in various places and is explicit about closing persisted blocks. {Index,Chunk,Tombstone}Readers are more consistent about their Close() method. Whenever a reader is retrieved, the corresponding close method must eventually be called. We use this to track pending readers against persisted blocks. Querier's against the DB no longer hold a read lock for their entire lifecycle. This avoids long running queriers to starve new ones when we have to acquire a write lock when reloading blocks.	8 years ago
Fabian Reinartz	bbe72dccb9	Remove dependency on etcd/pkg/fileutil	8 years ago
Fabian Reinartz	78df406dac	Allocate and cache strings for persisted blocks ... This change loads the full symbol table when we open a persisted block and allocates a string for each. This ensures that strings retrieved through the index can be used after the block was closed. Before we backed the strings by the mmap'd byte regions which would segfault in this case. Also remove an inconsistency in the disk format and move both offset tables to the end (breaking change).	8 years ago
Goutham Veeramachaneni	8919baef03	Expose NewIndexReader() and cleanups ... Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	8 years ago
Fabian Reinartz	b09d90c79c	Add decoding method to retrieve unsafe strings ... When decoding data from mmaped blocks, we would like to retrieve a string backed by the mmaped region. As the underlying byte slice never changes, this is safe.	8 years ago
Goutham Veeramachaneni	afaf12fe45	Compress the series chunk details in index. ... Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	8 years ago
Fabian Reinartz	1ddedf2b30	Change series ID from uint32 to uint64	8 years ago
Matt Layher	78b15c3434	Add newCRC32 function to simplify hash initialization	8 years ago
Fabian Reinartz	2644c8665c	Don't allocate ChunkMetas, reuse postings slices	9 years ago
Fabian Reinartz	96d7f540d4	Persist series without allocating the full set ... Change index persistence for series to not be accumulated in memory before being written as one large batch. `Labels` and `ChunkMeta` objects are reused. This cuts down memory spikes during compaction of multiple blocks significantly. As part of the the Index{Reader,Writer} now have an explicit notion of symbols and series must be inserted in order.	9 years ago
Goutham Veeramachaneni	a110a64abd	Add full Snapshot support ... Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	9 years ago
Goutham Veeramachaneni	34a86af3c6	Move tombstones to their own thing. ... Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	9 years ago
Goutham Veeramachaneni	4f1d857590	Implement Delete on HeadBlock ... Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	9 years ago
Goutham Veeramachaneni	5579efbd5b	Initial implentation of Deletes on persistedBlock ... Very much a WIP Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>	9 years ago
Fabian Reinartz	2032a11d98	Add padding between fixed-sized index sections	9 years ago
Fabian Reinartz	34ba92eeeb	Move CRC back to chunks file, alignment for fixed-sized ints	9 years ago
Fabian Reinartz	a54f46d5e7	Migrate last IndexWriter pieces to decbuf	9 years ago