apitech/loki - loki - Gitea: Git with a cup of tea

Commit Graph

Author	SHA1	Message	Date
Owen Diehl	e823a5037f	better defaults for flush queue parallelism (#5091 ) * better defaults for flush queue length * changelog	4 years ago
Arve Knudsen	d87dbf9f8a	Chore: Upgrade to latest Cortex (#5085 ) * Chore: Upgrade to latest Cortex Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Chore: Upgrade to Prometheus@692a54649ed7 Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix build after upgrading dskit Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Add changelog entry Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	4 years ago
JordanRushing	291360ddcb	Simpler new chunk key v12 (#5054 ) * starts hacking new chunk paths * Create new chunk key for S3; update parsing and add basic test and benchmark Signed-off-by: Jordan Rushing <jordan.rushing@grafana.com> Fix `TestGrpcStore` to work with new chunk key structure Signed-off-by: Jordan Rushing <jordan.rushing@grafana.com> Add a function to SchemaConfig that returns the correct PeriodConfig for a given time. Signed-off-by: Callum Styan <callumstyan@gmail.com> Add schema config to object client so it we can use the right external key function based on the schema version. Signed-off-by: Callum Styan <callumstyan@gmail.com> Fix failing compactor tests by passing SchemaConfig to chunkClient Remove empty conditional from SchemaForTime Define schema v12; wire-in ChunkPathShardFactor and ChunkPathPeriod as configurable values Add PeriodConfig test steps for new values ChunkPathShardFactor and ChunkPathPeriod in v12 Update test for chunk.NewExternalKey(); remove completed TODO Set defaults for new ChunkPathPeriod and ChunkPathShardFactor SchemaConfig values; change FSObjectClient to use IdentityEncoder instead of Base64Encoder Use IdentityEncoder everywhere we use FSObjectClient for Chunks Add ExternalKey() function to SchemaConfig; update ObjectClient for Chunks * Finish plumbing through the chunk.ExternalKey -> schemaConfig.ExternalKey change. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Clean up lint failures Signed-off-by: Jordan Rushing <jordan.rushing@grafana.com> * Clean up chunk.go and fix tests Signed-off-by: Jordan Rushing <jordan.rushing@grafana.com> * Break SchemaConfig.ExternalKey() into smaller functions Signed-off-by: Jordan Rushing <jordan.rushing@grafana.com> * Quickly fix variable name in SchemaConfig.newerExternalKey() Signed-off-by: Jordan Rushing <jordan.rushing@grafana.com> * Clean up ExternalKey conditional logic; add better comments; add tests Signed-off-by: Jordan Rushing <jordan.rushing@grafana.com> * Fix a bug where we are prepending userID to legacy Chunk keys but never parsing it; refactor key tests and benchmarks Signed-off-by: Jordan Rushing <jordan.rushing@grafana.com> * Add small SchemaConfig test Signed-off-by: Jordan Rushing <jordan.rushing@grafana.com> * Update docs and CHANGELOG.md Signed-off-by: Jordan Rushing <jordan.rushing@grafana.com> * Correctly return an error when failing to parse a chunk external key Signed-off-by: Jordan Rushing <jordan.rushing@grafana.com> * Revert IdentityEncoder to Base64Encoder after testing Signed-off-by: Jordan Rushing <jordan.rushing@grafana.com> * Fix assignment in test for linter Signed-off-by: Jordan Rushing <jordan.rushing@grafana.com> * Remove leftover comments from development Signed-off-by: Jordan Rushing <jordan.rushing@grafana.com> * Change v12 version comment format; remove redundant login in `ParseExternalKey()` Co-authored-by: Owen Diehl <ow.diehl@gmail.com> * Change remaining v12 comment style Signed-off-by: Jordan Rushing <jordan.rushing@grafana.com> * Pass chunk.SchemaConfig to new parallel chunk client * Simplify chunk external key prefixes for schema v12 * Fix broken benchmark; add benchmarks for old parsing conditional Signed-off-by: Jordan Rushing <jordan.rushing@grafana.com> * Remove unneeded lines from upgrading doc * Add benchmarks for root chunk external key parsing functions Signed-off-by: Jordan Rushing <jordan.rushing@grafana.com> * memoizes schema number calculations * Resolve linter issue with PeriodConfig receiver name Signed-off-by: Jordan Rushing <jordan.rushing@grafana.com> Co-authored-by: Owen Diehl <ow.diehl@gmail.com> Co-authored-by: Callum Styan <callumstyan@gmail.com>	4 years ago
Karsten Jeschkies	7bbed2be49	Define Ingester interface to enable wrapping. (#4988 )	4 years ago
Kaviraj	38c42a26c6	Move `cortex/pkg/tenant` dependency to Loki. (#4944 )	4 years ago
Michel Hollands	e69ceefd84	Add LabelFilterer and Store wrapper (#4818 ) * Add LabelFilterer and Store wrapper Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Address review comments Signed-off-by: Michel Hollands <michel.hollands@grafana.com>	4 years ago
Marco Pracucci	c53457feb9	Upgrade Cortex, Prometheus and Thanos (#4830 ) * Upgrade Cortex, Prometheus and Thanos Signed-off-by: Marco Pracucci <marco@pracucci.com> * Fixed imported ordering Signed-off-by: Marco Pracucci <marco@pracucci.com> * Fixed data type used for series ref when recovering from checkpoint/WAL Signed-off-by: Marco Pracucci <marco@pracucci.com> * Upgraded Thanos to fix arm build Signed-off-by: Marco Pracucci <marco@pracucci.com> * Run go mod tidy Signed-off-by: Marco Pracucci <marco@pracucci.com>	4 years ago
Dylan Guedes	130878f3f6	Enrich WAL folder creation error message (#4705 ) * Enrich WAL folder creation error message. * Apply suggestions from code review Use %q and try absolute path. Co-authored-by: Mateusz Gozdek <mateusz@kinvolk.io> * Include file/path package. Co-authored-by: Mateusz Gozdek <mateusz@kinvolk.io>	4 years ago
Cyril Tovena	d81f757ecb	Refactor and simplify stats package. (#4712 ) * Refactor and simplify stats package. This PR refactors the stats package and make it smaller and simpler to work with. - It doesn't expose the context which can causes race when mutating it. - Uses concurrency primitive to ensure correctness - Replace trailers with in-batch statistics to propagate ingesters stats. Fixes #3366 Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * fix tests and lint issues. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes moar tests. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * mod-check Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * More fixes. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	4 years ago
Trevor Whitney	bfc77118a1	Update local and docker config for simple scalable deployment (#4660 ) * first draft of local config * remove a few more defaults * interface names fix means we do not need lifecycler ip * update rings to use collectors/ prefix fix replication factor chunks-retain-period default to 0s fix a bug introduced with a Go upgrade? on printing the schema date. * simplify the config for the ruler, synthesize the local directory * simplify loki-docker-config.yaml * change common filesystem config to have chunks and rules directory * now that we are calling register flags on the common ring config we set more values and some of the tests needed to be tweaked. updated a few paths as well. * fix tests Co-authored-by: Edward Welch <edward.welch@grafana.com>	4 years ago
Susana Ferreira	2d24e2ea64	Migration to dskit/ring (#4641 )	4 years ago
Trevor Whitney	ed025e0cd6	Change a few default config values and improve application of common storage config (#4543 ) * further config simplifications and better defaults * default ruler api to on * register defaults for storage configs in the common config (ie. signature version for S3) * enable ingester wal by default * disable chunk retries by default * allow common path prefix to include or omit trailing slash * allow partial storage config to be defined for overriding things like bucket names per component * fix tests * add changelog and upgrade docs * explain impact of wal change in upgrading.md Co-authored-by: Owen Diehl <ow.diehl@gmail.com> * rename add WithPrefix to RegisterFlags * fix (frontend OR worker) and scheduler boot order when both are running (#4545) Co-authored-by: Owen Diehl <ow.diehl@gmail.com>	4 years ago
Cyril Tovena	47336cdbe6	Fixes a panic in the labels API when no parameters are supplied. (#4554 ) * Fixes the a panic in the labels API when no parameters are supplied. We were supporting this before. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Remove bad test Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	4 years ago
Danny Kopping	cff8324f24	Replacing go-kit/kit/log with go-kit/log (#4484 ) * Replacing go-kit/kit/log with go-kit/log Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Appeasing the f&^$ing linter Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Adding depguard Cleaning go mod Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Updating weaveworks after https://github.com/weaveworks/common/pull/226 was merged Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Appeasing the mod checker Signed-off-by: Danny Kopping <danny.kopping@grafana.com>	4 years ago
Trevor Whitney	de0c469161	default chunk target size to ~1MB~ 1.5MB (#4388 ) * default chunk target size to 1MB Signed-off-by: Trevor Whitney <trevorjwhitney@gmail.com> * bump up to 1.5MB Signed-off-by: Trevor Whitney <trevorjwhitney@gmail.com>	4 years ago
Nick Pillitteri	79a7bdbabc	Chore: Use services and modules from grafana/dskit (#4196 ) * Update Cortex vendor to bcf15611345da65e03604a08a0213aaaaac49e54 Signed-off-by: Nick Pillitteri <nick.pillitteri@grafana.com> * Use services and modules packages from grafana/dskit Signed-off-by: Nick Pillitteri <nick.pillitteri@grafana.com> * Update promtail test to set default weaveworks server network type Signed-off-by: Nick Pillitteri <nick.pillitteri@grafana.com> * Punctuation Signed-off-by: Nick Pillitteri <nick.pillitteri@grafana.com>	4 years ago
Owen Diehl	9cdb1295b5	adds loki_ingester_wal_replay_active metric and records this more acc… (#4193 ) * adds loki_ingester_wal_replay_active metric and records this more accurately * adds WAL to msg	4 years ago
Owen Diehl	ff2ced09ea	Cleanup/unordered writes ingester config (#4192 ) * handle replaying unordered WAL into ordered configs. * ingester limiter metric, testware, correctly replayed unordered WAL into ordered config	4 years ago
Owen Diehl	92a4e57448	ingester.index-shards config (#4111 ) * consistent shard mapping test & better error msg * ingester.index-shards config * loki fails to start if the ingester & schema factors arent compatible. * lint * docs * update changelog	4 years ago
Cyril Tovena	32e55c3043	Fork cortex chunk storage into Loki. (#3842 ) The chunk package has been forked from this commit for future reference and licensing. `09fe17c0e6`	4 years ago
Owen Diehl	56256bfd5b	Unordered chunks (#4001 ) * merge feature/unordered-replay * interoperable head chunks * memchunk block interop * retain ordered memchunk optimizations when possible * tests+bench for unordered chunk reads * reorder on chunk close * [wip] ingester stream unorderd * unordered writes default in testware config, fixes OOO bug & removes unused lastChunkTimestamp var * validity window is 1/2 of max age & fixes old transfer test * more consistent headblock checking/creation * more cohesive encoding tests * unordered stream test with validity bounds * compat - unordered * reinstates memchunk defaults when rebounding & updates storage test compatibility * lint * reorder across blocks doesnt overflow * respect chunk configs during rebounding when possible * only sync checks on ordered writes	4 years ago
Vincent Miszczak	d6dc8b991c	Add ingester.autoforget-unhealthy-timeout opt-in feature (#3919 ) * Add ingester.autoforget-unhealthy-timeout opt-in feature * Update docs/sources/configuration/_index.md Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> Co-authored-by: Owen Diehl <ow.diehl@gmail.com>	4 years ago
Michel Hollands	78b073e022	Add a chunk filterer field to the config (#3979 ) Signed-off-by: Michel Hollands <michel.hollands@grafana.com>	4 years ago
Nick Pillitteri	b830d65bdd	Use the Cortex wrapper for getting tenant ID from a context (#3973 ) This change replaces all uses of the weaveworks/common method for getting tenant/org ID with the Cortex wrapper which performs some extra validation. Signed-off-by: Nick Pillitteri <nick.pillitteri@grafana.com>	5 years ago
Ed Welch	6e67d1ea5d	Loki/Promtail: Client Refactor (#3623 ) * move docker driver * move fluent-bit output plugin * move fluentd plugin * move logstash plugin * move promtail * match stage uses simple logql label matcher and filter expressions only * remove remaining logql imports * remove helpers package and move contents to util package * move validation and runtime out of util package * fix imports * move build and cfg packages into util * move ParseRequest function to util package * create a new package for logql models and move marshal/unmarshal code to util folder * move ErrorLabel to logqlmodel * move stats package to logqlmodel * decouple loghttp from logql by moving validation to logql package * move gcplog to tools * move PackedEntryKey constant to logqlmodel * fix stats proto import and rebuild protos * fix windows promtail tests * update drone to new dockerfile locations	5 years ago
Cyril Tovena	72dcda165a	Adds chunk filter hook for ingesters. (#3603 ) * Adds chunk filter hook for ingesters. Follow up on https://github.com/grafana/loki/pull/3569 this is the missing piece. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * That interface is not required to change. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago
Ed Welch	8012362674	Loki: Per Tenant Runtime Configs (#3460 ) * Add per tenant configs mechanism * Add per tenant logging of stream creation * fix tests * enable per tenant configs on push request logging. * fixing up the stream log, changing log levels to debug	5 years ago
Cyril Tovena	f4f1239e55	Fixes ingester stats. (#3350 ) This correclty use the stats context. Introduced by https://github.com/grafana/loki/pull/1837/files#diff-d48cc90b5e10796f4e5256f2c7f4a7009c7c6799b514d7699676f73ae1be09c3L224 Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago
Owen Diehl	862cfa4e1f	adds Start() method to WAL interface to delay checkpointing until after replay (#3339 )	5 years ago
Owen Diehl	8a9b94a8bc	removes recover flag (#3326 )	5 years ago
Jacob Lisi	923671a17e	chore: update cortex to latest and fix refs (#3295 ) * chore: update cortex to latest and fix refs Signed-off-by: Jacob Lisi <jacob.t.lisi@gmail.com> * tidy mod file Signed-off-by: Jacob Lisi <jacob.t.lisi@gmail.com>	5 years ago
Owen Diehl	224e68a50b	WAL backpressure (#3218 ) * adds WAL replay backpressure * backpressure fn doesnt lock during passed fn for concurrency * replayController * replayController supports a RemoveFlushedChunks fn * custom wal flusher * encoding test determinism * standardizes recoveredBytesTotal to replayControllers ownership * linting * default wal memory threshold to 4gb, adds docs * fix test after v3 chunk schema change * more lenient expectCheckpoint function * replayController protects against flush race condition * adds replay_memory_ceiling to wal jsonnet * replay ceiling help msg	5 years ago
Owen Diehl	ed649eec37	Don't fail writes due to full WAL disk (#3136 ) * dont fail writes on full wal disk * wal full failure will cause flush on shutdown * logs the first full WAL failure	5 years ago
Cyril Tovena	cdf4a73238	Improve tailer matching by using the index. (#3090 ) * Improve tailer matching by using the index. ``` ❯ benchcmp before.txt after benchmark old ns/op new ns/op delta Benchmark_instance_addNewTailer-16 537731 1479 -99.72% benchmark old allocs new allocs delta Benchmark_instance_addNewTailer-16 244 3 -98.77% benchmark old bytes new bytes delta Benchmark_instance_addNewTailer-16 22098 121 -99.45% ``` Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Review feedback. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago
Owen Diehl	ad803af9dc	Wal/recover corruption (#3117 ) * simplifies recovery code and continues after errors * does not exit on wal recovery failure * log line parity * less scary error msgs * more descriptive errors * Update pkg/ingester/ingester.go Co-authored-by: Ed Welch <ed@oqqer.com> * Update pkg/ingester/ingester.go Co-authored-by: Ed Welch <ed@oqqer.com> Co-authored-by: Ed Welch <ed@oqqer.com>	5 years ago
Owen Diehl	58987553a1	Disables the stream limiter until wal has recovered (#3114 ) * disables the stream limiter until wal has recovered * limiter disable/enable methods	5 years ago
Owen Diehl	4d9865acd4	Adds WAL support (experimental) (#2981 ) * marshalable chunks * wal record types custom serialization * proto types for wal checkpoints * byteswith output unaffected by buffer * wal & record pool ifcs * wal record can hold entries from multiple series * entry pool * ingester uses noopWal * removes duplicate argument passing in ingester code. adds ingester config validation & derives chunk encoding. * segment writing * [WIP] wal recovery from segments * replay uses sync.Maps & preserves WAL fingerprints * in memory wal recovery * wal segment recovery * ingester metrics struct * wal replay locks streamsMtx in instances, adds checkpoint codec * ingester metrics * checkpointer * WAL checkpoint writer * checkpointwriter can write multiple checkpoints * reorgs checkpointing * wires up checkpointwriter to wal * ingester SeriesIter impl * wires up ingesterRecoverer to consume checkpoints * generic recovery fn * generic recovery fn * recover from both wal types * cleans up old tmp checkpoints & allows aborting in flight checkpoints * wires up wal checkpointing * more granular wal logging * fixes off by 1 wal truncation & removes double logging * adds userID to wal records correctly * wire chunk encoding tests * more granular wal metrics * checkpoint encoding test * ignores debug bins * segment replay ignores out of orders * fixes bug between WAL reading []byte validity and proto unmarshalling refs * conf validations, removes comments * flush on shutdown config * POST /ingester/shutdown * renames flush on shutdown * wal & checkpoint use same segment size * writes entries to wal regardless of tailers * makes wal checkpoing duration default to 5m * recovery metrics * encodes headchunks separately for wal purposes * merge upstream * linting * addresses pr feedback uses entry pool in stream push/tailer removes unnecessary pool interaction checkpointbytes comment fillchunk helper, record resetting in tests via pool redundant comment defers wg done in recovery s/num/count/ checkpoint wal uses a logger encodeWithTypeHeader now creates its own []byte removes pool from decodeEntries wal stop can error * prevent shared access bug with tailers and entry pool * removes stream push entry pool optimization	5 years ago
Sandeep Sukhani	7f4d6d5912	fix panic in ingester when not running with boltdb shipper while queriers does (#2830 )	5 years ago
Sandeep Sukhani	2f7e0e50c2	rpc for querying ingesters to get chunk ids from its store (#2601 ) * rpc for querying ingesters to get chunk ids from its store * fix broken tests * separate out code from querier to query ingester to a new type called IngesterQuerier * add AsyncStore which helps with querying and merging results from both chunk store and ingesters * query ingesters local store for labels when active index type is boltdb-shipper or query store is true * better naming * add some tests * add test case for duplicate chunk ids from ingesters * add some logs and spans * check for nil fetcher * use require.Same to assert that two pointers are pointing to same object * some more improvements 1. query chunk-refs parallely from store and ingesters. 2. limit look back for chunk refs and labels from ingesters to start of schema when using boltdb-shipper. * fixes suggested from PR review * fix mode not being set before cortex_storage.NewStore is being called * add IngesterQuerier as a dependency for store when target is either ingester or querier * do not cache index from ingester * rename tests * fix broken tests * update upgrade guide to rollout ingesters before queriers * upgrade guide changes suggested in PR review * runtime check for whether active index store is boltdb-shipper * consider previous schema as well for boltdb-shipper for calculating max look back	5 years ago
Sandeep Sukhani	f1d00807eb	fix a bug in ingester doing store queries when using boltdb shipper (#2493 )	5 years ago
Aditya C S	0885a49370	configuration doc improvements (#2318 )	6 years ago
Cyril Tovena	0be64fcb34	Improve metric queries by computing samples at the edges. (#2293 ) * First pass breaking the code appart. Wondering how we're going to achieve fast mutation of labels. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Work in progress. I realize I need hash for deduping lines. going to benchmark somes. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Tested some hash and decided which one to use. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Wip Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Starting working on ingester. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Trying to find a better hash function. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * More hash testing we have a winner. xxhash it is. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Settle on xxhash Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Better params interfacing. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add interface for queryparams for things that exist in both type of params. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add storage sample iterator implementations. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixing tests and verifying we don't get collions for the hashing method. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixing ingesters tests and refactoring utility function/tests. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixing and testing that stats are still well computed. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixing more tests. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * More engine tests finished. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes sharding evaluator. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes more engine tests. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fix error tests in the engine. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Finish fixing all tests. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes a bug where extractor was not passed in correctly. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add notes about upgrade. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Renamed and fix a bug. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add memchunk tests and starting test for sampleIterator. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Test heap sample iterator. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * working on test. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Finishing testing all new iterators. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Making sure all store functions are tested. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Benchmark and verify everything is working well. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Make the linter happy. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * use xxhash v2. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fix a flaky test because of map. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * go.mod. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> Co-authored-by: Edward Welch <edward.welch@grafana.com>	6 years ago
Owen Diehl	268b7a3d03	bumps cortex & fixes conflicts (#2204 ) * bumps cortex & fixes conflicts * go mod tidy	6 years ago
Ed Welch	9803eab947	Loki: Allow configuring query_store_max_look_back_period when running a filesystem store and boltdb-shipper (#2073 ) * Allow configuring query_store_max_look_back_period when running a filesystem store, which allows for boltdb-shipper to horizontally scale filesystem store. Signed-off-by: Ed Welch <edward.welch@grafana.com> * Update docs/operations/storage/boltdb-shipper.md Co-authored-by: Sandeep Sukhani <sandeep.d.sukhani@gmail.com> * add some more precision to the timestamp used to make the name unique just to make sure there are not collisions if for some reason the ID is empty (which can happen if the hostname doesn't resolve for some reason) Signed-off-by: Ed Welch <edward.welch@grafana.com> * Allow the ingester to query the store for labels if query store is enabled. Signed-off-by: Ed Welch <edward.welch@grafana.com> * make queriers aware of the fact that ingesters may be querying the store so they don't duplicate query efforts Signed-off-by: Ed Welch <edward.welch@grafana.com> * improve the docs Signed-off-by: Ed Welch <edward.welch@grafana.com> * add tests to the calculateMaxLookBack Signed-off-by: Ed Welch <edward.welch@grafana.com> Co-authored-by: Sandeep Sukhani <sandeep.d.sukhani@gmail.com>	6 years ago
Cyril Tovena	577d8eb168	Ensure trace propagation in our logs. (#1977 ) * Ensure trace propagation in our logs. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add missing import. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes a test. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fix function usage. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	6 years ago
Peter Štibraný	e60164bef6	Convert Loki modules to services (#1804 ) * Loki now uses module services to start and stop its work. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Use services methods to initialize some components. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Use Cortex' NewModuleService. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Converted server to a service. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Converted distributor to service. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Use table manager service Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * querier service Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * query-frontend service Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Merged stopping method into shutdown Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Converted ingester to a service. It now starts all background tasks in Starting state. Stopping needs little work, as does reacting on lifecycler errors. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Loki Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * If lifecycler fails, ingester fails too. It now doesn't call os.Exit, but shuts down gracefully and enters Failed state. That triggers Loki to shutdown completely. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Ignore ErrStopProcess errors from services This is a signal that Loki should stop. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Use single /ready handler It checks the state of all services, and asks ingester for its own check as well. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Removed unused value. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Lint Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Fix test. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Go mod tidy, vendor Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Tailers, not trailers. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Adds return for the healtcheck in case of error. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> Co-authored-by: Cyril Tovena <cyril.tovena@gmail.com>	6 years ago
Sandeep Sukhani	fad3b61be7	flush boltdb to object store (#1837 ) * flush boltdb to object store files are stored in folder per periodic table and are named after ingester flushed every 15 mins to make index available to other services files are also flushed before ingester stops to avoid any data loss new stores can be implemented easily ingester to also query store when using boltdb * persisting uploader name across restarts, detecting objectstore type from periodic config, other refactorings * updated cli flag for active directory * add tests for boltdb shipper and update vendor * syncing boltdb files to disk during update * sleep in tests to let mtime of boltdb file be changed * add documentation for boltdb shipper and remove unwanted config	6 years ago
Sandeep Sukhani	49f04ef4bc	update cortex to latest master (#1956 )	6 years ago
Ed Welch	b5b3e78424	Loki Update the cut block size counter when creating a memchunk from byte slice (#1946 ) * Update the cut block size counter when creating a memchunk from byte slice * Refactor how we create memchunks to require specifying the block size and target size	6 years ago
Peter Štibraný	ca19683511	loki: update Cortex to master (#1799 ) * Update Cortex to master Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Integrate latest Cortex master changes into loki - Frontend worker now needs to be started explicitly - Lifecycler no longer exits on error. Distributor and Ingester do that now Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Lint Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * go mod tidy Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Update Cortex to master. After 0.7 release, it is now showing as v0.7.1-... Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>	6 years ago

1 2

93 Commits (e823a5037f90237e3497da6ec34b25fe70a2c86e)