33 Commits (9f301cf87d39bc6876448441854ec75d70f5a3d6)

Author	SHA1	Message	Date
Sandeep Sukhani	9f301cf87d	use the defined querypages callback in all the types implementing QueryPages function from IndexClient interface (#5463)	3 years ago
Kaviraj Kanagaraj	f083aab09d	Remove `cortexpb` dependency from Loki packages. Use only `logproto` now. (#5264) ... * Getting rid of `cortexpb` Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Get rid of `cortex/pkg/tenants` Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Fix linter errors Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * go mod vendor Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Rename some import alias Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com>	3 years ago
Eng Zer Jun	392fde1e27	refactor: use `T.TempDir` to create temporary test directory (#5253) ... * refactor: use `T.TempDir` to create temporary test directory The directory created by `T.TempDir` is automatically removed when the test and all its subtests complete. Reference: https://pkg.go.dev/testing#T.TempDir Signed-off-by: Eng Zer Jun <engzerjun@gmail.com> * docs: update CHANGELOG.md Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>	3 years ago
Sandeep Sukhani	0426870f3f	fix broken boltdb-shipper test (#5222)	3 years ago
Sandeep Sukhani	d87f2a1c8a	per user index changes on the write path (#5193) ... * per user index changes on the write path - add a flag to enable building per user index from the ingesters - when enabled, ingesters would build boltdb files with buckets per tenant	3 years ago
Arve Knudsen	a2dd731586	Chore: Add pkg/util/log package (#5187) ... * Chore: Add pkg/util/log package Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Add changelog entry Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix documentation of loki_log_messages_total Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	3 years ago
Sandeep Sukhani	88b5a767bc	changes on read path for supporting per user index with boltdb-shipper (#5073)	3 years ago
Sandeep Sukhani	ff47d7419a	compactor changes for building per user index files in boltdb shipper (#5026) ... * compactor changes for building per user index files in boltdb shipper * update comments * lint and changes suggested from PR review * fix broken code after merging main	3 years ago
Cyril Tovena	54dc737583	Reduce tracing spans (#4842) ... Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	4 years ago
Danny Kopping	cff8324f24	Replacing go-kit/kit/log with go-kit/log (#4484) ... * Replacing go-kit/kit/log with go-kit/log Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Appeasing the f&^$ing linter Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Adding depguard Cleaning go mod Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Updating weaveworks after https://github.com/weaveworks/common/pull/226 was merged Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Appeasing the mod checker Signed-off-by: Danny Kopping <danny.kopping@grafana.com>	4 years ago
Sandeep Sukhani	8ef3d5fee9	add a storage client for boltdb-shipper which would do all the object key management for storage operations (#4128) ... * add a storage client for boltdb-shipper which would do all the object key management for storage operations * changes suggested from PR review	4 years ago
Cyril Tovena	32e55c3043	Fork cortex chunk storage into Loki. (#3842) ... The chunk package has been forked from this commit for future reference and licensing. 09fe17c0e6	4 years ago
Sandeep Sukhani	1ed19f7380	add a metric for counting number of failures in opening existing files while loading tables from active index in uploads package (#3977)	4 years ago
Sandeep Sukhani	0cd8f31831	log name of the file failed to open during startup by ingester (#3970)	4 years ago
Sandeep Sukhani	43c16ed8a1	remove boltdb files from ingesters on startup which do not have a index bucket (#3929)	4 years ago
Sandeep Sukhani	ba25b0fc80	increase db retain period in ingesters to cover index cache validity period as well (#3300) ... refactor ingester store query config to simplify the code	4 years ago
Sandeep Sukhani	3ff088173f	dedupe index on all the queries for a table instead of query batches (#3338) ... * dedupe index on all the queries for a table instead query batches * convert sync.Mutex to sync.RWMutex in deduper to allow concurrent checks for isSeen	4 years ago
Jacob Lisi	923671a17e	chore: update cortex to latest and fix refs (#3295) ... * chore: update cortex to latest and fix refs Signed-off-by: Jacob Lisi <jacob.t.lisi@gmail.com> * tidy mod file Signed-off-by: Jacob Lisi <jacob.t.lisi@gmail.com>	4 years ago
Ed Welch	6d442673b7	adding a function to safely open boltdb files and recover from any panics thrown by boltdb when opening a file. (#2983)	5 years ago
Sandeep Sukhani	fd451d97f4	fix flaky load tables test for boltdb-shipper uploads table-manager (#2891)	5 years ago
Sandeep Sukhani	feb66d62cf	fix flaky tests in boltdb-shipper (#2831)	5 years ago
Sandeep Sukhani	f60554ce91	Boltdb shipper query performance improvements (#2770) ... * boltdb-shipper query with cursor to do multiple queries in a single transaction * index deduper * use fifo cache in ingesters * snapshot dbs in ingester for reads * retain fewer files in ingesters for queries * set timeout in cache config instead of index read cache config which saves us from unmarshalling the object for checking the expiry * add comment about fifo cache invalidation * tweaking a few comments Co-authored-by: Edward Welch <edward.welch@grafana.com>	5 years ago
Cyril Tovena	62272cdaa1	Fixes all lint errors. (#2768) ... Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago
Davor Kapsa	fffcfa6e81	fix some misspells (#2725)	5 years ago
Sandeep Sukhani	79c027550c	run multiple queries per table at once with boltdb-shipper (#2656)	5 years ago
Lars Lehtonen	334cc1e59a	pkg/storage/stores/shipper/uploads: fix test error (#2620) ... Signed-off-by: Lars Lehtonen <lars.lehtonen@gmail.com>	5 years ago
Sandeep Sukhani	dab393a62d	fix a flaky test in boltdb shipper (#2550)	5 years ago
Sandeep Sukhani	b489130828	compactor for compacting boltdb files uploaded by shipper (#2526)	5 years ago
Sandeep Sukhani	97dfb297a0	compress boltdb files to gzip while uploading from shipper (#2507)	5 years ago
Sandeep Sukhani	9b0740c0ed	upload boltdb files from shipper only when they are not expected to be modified or during shutdown (#2487) ... * upload boltdb files from shipper only when they are not expected to be modified or during shutdown * updated comment * add one more test condition to make sure files are not re-uploaded Co-authored-by: Edward Welch <edward.welch@grafana.com>	5 years ago
Sandeep Sukhani	048e8bdb9c	minor fixes and improvements for boltdb shipper (#2530) ... 1. Continue uploading of other tables during failures. 2. Remove .temp files while loading dbs during startup. 3. Remove empty table folders while loading tables during startup.	5 years ago
Sandeep Sukhani	cf64b70db3	add some spans for boltdb shipper index queries (#2495)	5 years ago
Sandeep Sukhani	e59adccfa0	create smaller unique files from boltdb shipper and other code refactorings (#2261) ... * rename package local to shipper * reorganized files to separate out uploads and downloads * create smaller unique files from boltdb shipper and other improvements * cancellation of downloading of initial files when service is being stopped * migration of existing files to dedicated folders and cleanup of index from ingesters * some more improvements to uploads and downloads code * tests for uploads and downloads package in shipper Signed-off-by: Sandeep Sukhani <sandeep.d.sukhani@gmail.com> * test cleanup fix and some other minor changes * refactor getting uploader from object key * better naming * changes suggested from PR review * Fixes goroutine loop. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Simplify getoradd table. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes loop in upload manager. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * some more tests, logs and changes suggested from PR review * move ready state check for downloaded table inside query function * add info log for successfully finishing uploading of a table * fix merge conflict * fix ingester name and mode config Co-authored-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago