13 Commits (6e2b7d25898f321bfd6f94de05d6ce934c785da9)

Author	SHA1	Message	Date
Owen Diehl	094f8f3b20	[TSDB] fingerprint offsets is now correctly 16 byte aligned (#6630) ... * simpler shard factor calculations, disallows identity factor of 1 Signed-off-by: Owen Diehl <ow.diehl@gmail.com> * log interval span as soon as it finishes Signed-off-by: Owen Diehl <ow.diehl@gmail.com> * another shard bounds test Signed-off-by: Owen Diehl <ow.diehl@gmail.com> * indexclient span logging Signed-off-by: Owen Diehl <ow.diehl@gmail.com> * more index span logging Signed-off-by: Owen Diehl <ow.diehl@gmail.com> * correctly match single shard Signed-off-by: Owen Diehl <ow.diehl@gmail.com> * shardedPostings test Signed-off-by: Owen Diehl <ow.diehl@gmail.com> * shardedPostings maxOffset is properly exclusive Signed-off-by: Owen Diehl <ow.diehl@gmail.com> * minor posting idiomatic-ness Signed-off-by: Owen Diehl <ow.diehl@gmail.com> * store 16 byte aligned offsets for fingerprint table Signed-off-by: Owen Diehl <ow.diehl@gmail.com>	4 years ago
Owen Diehl	4903df6ae5	Tsdb/inverted index wiring (#6252) ... * multi inverted index * delete in inverted index signature * multi inverted index * inverted index testing + skip nil indices * wires up period config aware multi-inverted-idx through ingesters * better inverted index validation * apply shipper defaults to last relevant period config, not current one. * more verbose error	4 years ago
Owen Diehl	c032da9639	Tsdb/inverted index (#6233) ... * begins adding a factor-of-two dynamically sharded inverted index * bit prefix inverted index tests * goimports	4 years ago
Cyril Tovena	8f02495453	Refactoring of the Loki storage (#5833) ... * Extract index interface from the series store * moving package around * Moving more things around Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Moving more things around Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * moving package around * Moving more things around again Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Moving more things around again Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * improve composite store for more generics index store * fixes first round of tests and ensure coverage is still good * implement series method * Fixes storage tests Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add test for GetSeries Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Make Loki compile Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * lint the whole project Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes tests Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * tsdb util package usage Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add more information of why SetChunkFilterer exists for the index Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes ruler/base package Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes o2 Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	4 years ago
Kaviraj Kanagaraj	f5d570177a	Fork `cortex/pkg/util` into Loki (#5257) ... * Fork `cortex/pkg/util/validation` Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Fork `pkg/cortex/util/limiter` Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * fork util/extract Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Spanlogger add more helpers Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * getting rid of `cortex/pkg/util` Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * With lesser linter errors Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Make linter happy by removing unused method Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Remove missed util_log Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Fix gofmt and goimports Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Do not touch operator Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Do not touch generated go files Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Format Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Make linters happy Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Fix lambda promtail tools Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Upgrade Dockerfile of lambda promtail Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Add tests for `pkg/util` Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com>	4 years ago
Dylan Guedes	3f496e7336	Copy `cortex/pkg/cortexpb` package dependency into Loki (#5151) ... * Fork cortexpb into Cortex `pkg/logproto`. * Use `pkg/logproto` instead of Cortex `pkg/cortexpb`. * Rename `cortexpb` usage to `logproto`. * Fix encoding/decoding test. * Replace cortexpb usage in the ruler/base package. * Remove exemplars from logproto.	4 years ago
Dylan Guedes	d5783e86a3	Copy `cortex/pkg/querier/astmapper` package dependency into Loki (#4982) ... * Fork cortex `pkg/querier/asmapper` into Loki. * Move away from cortex `pkg/querier/astmapper`. * Fix lint issues. - Use _ in argument of method `pkg/querier/astmapper/subtree_folder.go` since it is not being used - Remove unnecessary cast from `subtree_folder.go`	4 years ago
Marco Pracucci	c53457feb9	Upgrade Cortex, Prometheus and Thanos (#4830) ... * Upgrade Cortex, Prometheus and Thanos Signed-off-by: Marco Pracucci <marco@pracucci.com> * Fixed imported ordering Signed-off-by: Marco Pracucci <marco@pracucci.com> * Fixed data type used for series ref when recovering from checkpoint/WAL Signed-off-by: Marco Pracucci <marco@pracucci.com> * Upgraded Thanos to fix arm build Signed-off-by: Marco Pracucci <marco@pracucci.com> * Run go mod tidy Signed-off-by: Marco Pracucci <marco@pracucci.com>	4 years ago
Hiroki Sakamoto	0e21f02d76	Respect shard number in series api (#4577) ... * Respect shard number in series api * Refactor codes * remove duplicate logic in instance * format with goimport * Improve complexity of getting all fingerprints	4 years ago
Owen Diehl	92a4e57448	ingester.index-shards config (#4111) ... * consistent shard mapping test & better error msg * ingester.index-shards config * loki fails to start if the ingester & schema factors arent compatible. * lint * docs * update changelog	4 years ago
Cyril Tovena	10bd165996	Fixes unaligned shards between ingesters and storage. (#4087) ... * Fixes unaligned shards between ingesters and storage. Currently shards computation are not correctly aligned between ingester and storage causing bad metric results. - Fixed the hash caculation which was missing the metric name `logs`. - Revert some changes from 5f0e2453fe which was not keeping the alignement of shards. For example if the shard is 9 of 16 total, then it should also be 9 and 25 of 32. The previous commit was trying to use 10 and 11 of 32. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * better tests. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	4 years ago
Cyril Tovena	32e55c3043	Fork cortex chunk storage into Loki. (#3842) ... The chunk package has been forked from this commit for future reference and licensing. 09fe17c0e6	5 years ago
Cyril Tovena	97912b6f4c	Shard ingester queries. (#3852) ... * Shard ingester queries. This is still experimental but already yield from 2x to 6x faster for short period queries. I'm still playing with it but I want to share how I do it early. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add notice of the code origin. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Update pkg/ingester/index/index.go Co-authored-by: Owen Diehl <ow.diehl@gmail.com> * Update pkg/ingester/index/index_test.go Co-authored-by: Owen Diehl <ow.diehl@gmail.com> * Align shards from ingester and storage. Utimately we should have a storage that relies on fingerprint, but that's harder to change. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Remove comment Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes delete index func Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Test reverting. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes a bug causing non deterministic hash. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Update pkg/ingester/index/index.go Co-authored-by: Owen Diehl <ow.diehl@gmail.com> * Update pkg/ingester/index/index.go Co-authored-by: Owen Diehl <ow.diehl@gmail.com> * Update pkg/ingester/index/index.go Co-authored-by: Owen Diehl <ow.diehl@gmail.com> * Update pkg/ingester/index/index_test.go Co-authored-by: Owen Diehl <ow.diehl@gmail.com> * Fixes build. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * got linted :( Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> Co-authored-by: Owen Diehl <ow.diehl@gmail.com>	5 years ago