1 Commits (6ac955cc7d395874ce60516d182678542415bca3)

Author	SHA1	Message	Date
Sandeep Sukhani	6ac955cc7d	make boltdb-shipper use the generic indexshipper for managing index in object storage (#6226)	3 years ago
Cyril Tovena	8f02495453	Refactoring of the Loki storage (#5833) ... * Extract index interface from the series store * moving package around * Moving more things around Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Moving more things around Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * moving package around * Moving more things around again Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Moving more things around again Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * improve composite store for more generics index store * fixes first round of tests and ensure coverage is still good * implement series method * Fixes storage tests Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add test for GetSeries Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Make Loki compile Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * lint the whole project Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes tests Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * tsdb util package usage Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add more information of why SetChunkFilterer exists for the index Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes ruler/base package Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes o2 Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	3 years ago
Sandeep Sukhani	27a027824f	incorporate feedback on merged pr #5830 (#5831)	3 years ago
Sandeep Sukhani	67ba342684	improve mutex locking when writing index in boltdb-shipper (#5830)	3 years ago
Karsten Jeschkies	46956d0bf6	Use dskit/tenant intead of loki/pkg/tenant. (#5724)	3 years ago
Sandeep Sukhani	9f301cf87d	use the defined querypages callback in all the types implementing QueryPages function from IndexClient interface (#5463)	3 years ago
Kaviraj Kanagaraj	f083aab09d	Remove `cortexpb` dependency from Loki packages. Use only `logproto` now. (#5264) ... * Getting rid of `cortexpb` Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Get rid of `cortex/pkg/tenants` Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Fix linter errors Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * go mod vendor Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Rename some import alias Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com>	3 years ago
Sandeep Sukhani	d87f2a1c8a	per user index changes on the write path (#5193) ... * per user index changes on the write path - add a flag to enable building per user index from the ingesters - when enabled, ingesters would build boltdb files with buckets per tenant	3 years ago
Arve Knudsen	a2dd731586	Chore: Add pkg/util/log package (#5187) ... * Chore: Add pkg/util/log package Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Add changelog entry Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix documentation of loki_log_messages_total Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	3 years ago
Sandeep Sukhani	ff47d7419a	compactor changes for building per user index files in boltdb shipper (#5026) ... * compactor changes for building per user index files in boltdb shipper * update comments * lint and changes suggested from PR review * fix broken code after merging main	3 years ago
Danny Kopping	cff8324f24	Replacing go-kit/kit/log with go-kit/log (#4484) ... * Replacing go-kit/kit/log with go-kit/log Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Appeasing the f&^$ing linter Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Adding depguard Cleaning go mod Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Updating weaveworks after https://github.com/weaveworks/common/pull/226 was merged Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Appeasing the mod checker Signed-off-by: Danny Kopping <danny.kopping@grafana.com>	4 years ago
Sandeep Sukhani	8ef3d5fee9	add a storage client for boltdb-shipper which would do all the object key management for storage operations (#4128) ... * add a storage client for boltdb-shipper which would do all the object key management for storage operations * changes suggested from PR review	4 years ago
Cyril Tovena	32e55c3043	Fork cortex chunk storage into Loki. (#3842) ... The chunk package has been forked from this commit for future reference and licensing. 09fe17c0e6	4 years ago
Sandeep Sukhani	1ed19f7380	add a metric for counting number of failures in opening existing files while loading tables from active index in uploads package (#3977)	4 years ago
Sandeep Sukhani	0cd8f31831	log name of the file failed to open during startup by ingester (#3970)	4 years ago
Sandeep Sukhani	43c16ed8a1	remove boltdb files from ingesters on startup which do not have a index bucket (#3929)	4 years ago
Sandeep Sukhani	ba25b0fc80	increase db retain period in ingesters to cover index cache validity period as well (#3300) ... refactor ingester store query config to simplify the code	4 years ago
Sandeep Sukhani	3ff088173f	dedupe index on all the queries for a table instead of query batches (#3338) ... * dedupe index on all the queries for a table instead query batches * convert sync.Mutex to sync.RWMutex in deduper to allow concurrent checks for isSeen	4 years ago
Jacob Lisi	923671a17e	chore: update cortex to latest and fix refs (#3295) ... * chore: update cortex to latest and fix refs Signed-off-by: Jacob Lisi <jacob.t.lisi@gmail.com> * tidy mod file Signed-off-by: Jacob Lisi <jacob.t.lisi@gmail.com>	4 years ago
Ed Welch	6d442673b7	adding a function to safely open boltdb files and recover from any panics thrown by boltdb when opening a file. (#2983)	5 years ago
Sandeep Sukhani	f60554ce91	Boltdb shipper query performance improvements (#2770) ... * boltdb-shipper query with cursor to do multiple queries in a single transaction * index deduper * use fifo cache in ingesters * snapshot dbs in ingester for reads * retain fewer files in ingesters for queries * set timeout in cache config instead of index read cache config which saves us from unmarshalling the object for checking the expiry * add comment about fifo cache invalidation * tweaking a few comments Co-authored-by: Edward Welch <edward.welch@grafana.com>	5 years ago
Sandeep Sukhani	79c027550c	run multiple queries per table at once with boltdb-shipper (#2656)	5 years ago
Sandeep Sukhani	dab393a62d	fix a flaky test in boltdb shipper (#2550)	5 years ago
Sandeep Sukhani	97dfb297a0	compress boltdb files to gzip while uploading from shipper (#2507)	5 years ago
Sandeep Sukhani	9b0740c0ed	upload boltdb files from shipper only when they are not expected to be modified or during shutdown (#2487) ... * upload boltdb files from shipper only when they are not expected to be modified or during shutdown * updated comment * add one more test condition to make sure files are not re-uploaded Co-authored-by: Edward Welch <edward.welch@grafana.com>	5 years ago
Sandeep Sukhani	048e8bdb9c	minor fixes and improvements for boltdb shipper (#2530) ... 1. Continue uploading of other tables during failures. 2. Remove .temp files while loading dbs during startup. 3. Remove empty table folders while loading tables during startup.	5 years ago
Sandeep Sukhani	e59adccfa0	create smaller unique files from boltdb shipper and other code refactorings (#2261) ... * rename package local to shipper * reorganized files to separate out uploads and downloads * create smaller unique files from boltdb shipper and other improvements * cancellation of downloading of initial files when service is being stopped * migration of existing files to dedicated folders and cleanup of index from ingesters * some more improvements to uploads and downloads code * tests for uploads and downloads package in shipper Signed-off-by: Sandeep Sukhani <sandeep.d.sukhani@gmail.com> * test cleanup fix and some other minor changes * refactor getting uploader from object key * better naming * changes suggested from PR review * Fixes goroutine loop. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Simplify getoradd table. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes loop in upload manager. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * some more tests, logs and changes suggested from PR review * move ready state check for downloaded table inside query function * add info log for successfully finishing uploading of a table * fix merge conflict * fix ingester name and mode config Co-authored-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago