apitech/loki - loki - Gitea: Git with a cup of tea

Commit Graph

Author	SHA1	Message	Date
Cyril Tovena	e74eb549ba	Reverts flush buffer pooling. (#3210 ) Currently this requires more work, as the cache is asynchronously using the buffer when flushing chunks. We still benefits from other improvements like computing the right size for a chunk. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	4 years ago
Cyril Tovena	ea488f9940	Improve ingester flush memory usage. (#3200 ) * Add basic benchmark. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Improves memory usage of checkpointer series iterator. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * make lint. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add benchmark for flush loop. ``` goos: darwin goarch: amd64 pkg: github.com/grafana/loki/pkg/ingester Benchmark_FlushLoop-16 16 70040359 ns/op 241901661 B/op 1112 allocs/op PASS ok github.com/grafana/loki/pkg/ingester 4.829s ``` Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Improve benchmark. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Re-use buffer when flushing chunks. benchcmp ``` ❯ benchcmp before.txt after.txt benchmark old ns/op new ns/op delta Benchmark_FlushLoop-16 104723243 9233780 -91.18% benchmark old allocs new allocs delta Benchmark_FlushLoop-16 1115 568 -49.06% benchmark old bytes new bytes delta Benchmark_FlushLoop-16 241857243 989590 -99.59% ``` Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * better size computation. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes test ordering flakyness. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * lint. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	4 years ago
Cyril Tovena	07ece2bc2a	Improve checkpoint series iterator. (#3193 ) * Add basic benchmark. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Improves memory usage of checkpointer series iterator. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * make lint. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * better size computation. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes test ordering flakyness. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	4 years ago
Owen Diehl	ed649eec37	Don't fail writes due to full WAL disk (#3136 ) * dont fail writes on full wal disk * wal full failure will cause flush on shutdown * logs the first full WAL failure	4 years ago
Owen Diehl	8c1fe88409	Introduces per stream chunks mutex (#3000 ) * marshalable chunks * wal record types custom serialization * proto types for wal checkpoints * byteswith output unaffected by buffer * wal & record pool ifcs * wal record can hold entries from multiple series * entry pool * ingester uses noopWal * removes duplicate argument passing in ingester code. adds ingester config validation & derives chunk encoding. * segment writing * [WIP] wal recovery from segments * replay uses sync.Maps & preserves WAL fingerprints * in memory wal recovery * wal segment recovery * ingester metrics struct * wal replay locks streamsMtx in instances, adds checkpoint codec * ingester metrics * checkpointer * WAL checkpoint writer * checkpointwriter can write multiple checkpoints * reorgs checkpointing * wires up checkpointwriter to wal * ingester SeriesIter impl * wires up ingesterRecoverer to consume checkpoints * generic recovery fn * generic recovery fn * recover from both wal types * cleans up old tmp checkpoints & allows aborting in flight checkpoints * wires up wal checkpointing * more granular wal logging * fixes off by 1 wal truncation & removes double logging * adds userID to wal records correctly * wire chunk encoding tests * more granular wal metrics * checkpoint encoding test * ignores debug bins * segment replay ignores out of orders * fixes bug between WAL reading []byte validity and proto unmarshalling refs * conf validations, removes comments * flush on shutdown config * POST /ingester/shutdown * renames flush on shutdown * wal & checkpoint use same segment size * writes entries to wal regardless of tailers * makes wal checkpoing duration default to 5m * recovery metrics * encodes headchunks separately for wal purposes * merge upstream * linting * adds chunkMtx to ingester streams * all flush locking locks streams -> chunks in order to prevent deadlocks * wal integration tests * addresses pr feedback uses entry pool in stream push/tailer removes unnecessary pool interaction checkpointbytes comment fillchunk helper, record resetting in tests via pool redundant comment defers wg done in recovery s/num/count/ checkpoint wal uses a logger encodeWithTypeHeader now creates its own []byte removes pool from decodeEntries wal stop can error * cleans up merge * lint * inline functions to simplify unlocking	5 years ago
Cyril Tovena	095b4a1cb7	Fixes a panic with the logql.NoopPipeline. (#2901 ) Totally forgot that now we can't share this across multiple queries since they hold a map. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago
Cyril Tovena	53f4aa4ac3	Labels computation LogQLv2 (#2875 ) * Adding a benchmark to compare before and after. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Improves labels management in logql v2. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Cache grouped result when no changes has occured. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Removes unused methods. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Adds docs and tests. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Uncomment tests. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago
Cyril Tovena	fd5cb96693	LogQL: Labels and Metrics Extraction (#2769 ) * Adds logfmt, regexp and json logql parser Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * hook the ast with parsers. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * hook parser with memchunk. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * hook parser with the storage. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * hook parser with ingesters Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * fixes all tests Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Refactor to pipeline and implement ast parsing. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes the lexer for duration and range Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes all tests and add some for label filters Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add label and line format. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add tests for fmt label and line with validations. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Polishing parsers and add some more test cases Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Finish the unwrap parser, still need to add more tests Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Indent this hell. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Moar tests and it works. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add more tests which lead me to find a bug in the lexer Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add more tests and fix all engine tests Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes match stage in promtail pipelines. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Hook Pipeline into ingester, tailer and storage. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Correctly setup sharding for logqlv2 Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes precedences issue with label filters and add moar tests ✌️ * Adds quantile_over_time, grouping for non associate range aggregation parsing and moar tests * Extract with grouping * Adds parsing duration on unwrap * Improve the lexer to support more common identifier as functions. Also add duration convertion for unwrap. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes the frontend logs to include org_id. The auth middleware was happening after the stats one and so org_id was not set 🤦. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Support byte sizes in label filters. This patch extends the duration label filter with support for byte sizes such as `1kB` and `42MiB`. * Wip on error handling. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes json parser with prometheus label name rules. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * fixup! Support byte sizes in label filters. * Wip error handling, commit before big refactoring. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Refactoring in progress. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Work in progress. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Got something that builds and throw __error__ labels properly now. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add error handling + fixes groupins and post filtering. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * 400 on pipeline errors. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes a races in the log pipeline. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Unsure the key is parsable and valid. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Cleanup and code documentation. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Lint. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Lint. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes frontend handler. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes old test. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fix go1.15 local failing test. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> Co-authored-by: Karsten Jeschkies <k@jeschkies.xyz>	5 years ago
Sandeep Sukhani	2f7e0e50c2	rpc for querying ingesters to get chunk ids from its store (#2601 ) * rpc for querying ingesters to get chunk ids from its store * fix broken tests * separate out code from querier to query ingester to a new type called IngesterQuerier * add AsyncStore which helps with querying and merging results from both chunk store and ingesters * query ingesters local store for labels when active index type is boltdb-shipper or query store is true * better naming * add some tests * add test case for duplicate chunk ids from ingesters * add some logs and spans * check for nil fetcher * use require.Same to assert that two pointers are pointing to same object * some more improvements 1. query chunk-refs parallely from store and ingesters. 2. limit look back for chunk refs and labels from ingesters to start of schema when using boltdb-shipper. * fixes suggested from PR review * fix mode not being set before cortex_storage.NewStore is being called * add IngesterQuerier as a dependency for store when target is either ingester or querier * do not cache index from ingester * rename tests * fix broken tests * update upgrade guide to rollout ingesters before queriers * upgrade guide changes suggested in PR review * runtime check for whether active index store is boltdb-shipper * consider previous schema as well for boltdb-shipper for calculating max look back	5 years ago
Cyril Tovena	0be64fcb34	Improve metric queries by computing samples at the edges. (#2293 ) * First pass breaking the code appart. Wondering how we're going to achieve fast mutation of labels. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Work in progress. I realize I need hash for deduping lines. going to benchmark somes. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Tested some hash and decided which one to use. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Wip Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Starting working on ingester. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Trying to find a better hash function. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * More hash testing we have a winner. xxhash it is. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Settle on xxhash Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Better params interfacing. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add interface for queryparams for things that exist in both type of params. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add storage sample iterator implementations. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixing tests and verifying we don't get collions for the hashing method. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixing ingesters tests and refactoring utility function/tests. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixing and testing that stats are still well computed. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixing more tests. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * More engine tests finished. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes sharding evaluator. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes more engine tests. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fix error tests in the engine. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Finish fixing all tests. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes a bug where extractor was not passed in correctly. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add notes about upgrade. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Renamed and fix a bug. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add memchunk tests and starting test for sampleIterator. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Test heap sample iterator. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * working on test. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Finishing testing all new iterators. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Making sure all store functions are tested. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Benchmark and verify everything is working well. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Make the linter happy. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * use xxhash v2. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fix a flaky test because of map. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * go.mod. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> Co-authored-by: Edward Welch <edward.welch@grafana.com>	5 years ago
Owen Diehl	268b7a3d03	bumps cortex & fixes conflicts (#2204 ) * bumps cortex & fixes conflicts * go mod tidy	5 years ago
Cyril Tovena	9988ce4b9f	Improve protobuf serialization (#2031 ) * Generate custom proto type for Streams. This is the first step to improve protobuf serialization. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Improve protobuf serialization allocations. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Saves allocations by stringifying labels only once. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago
Ed Welch	b89a1f81dd	Update cortex vendoring to latest master (#1975 ) * Update cortex vendoring to latest master checksum `36496a074bc4530fb1d7c74136cab1566fbd9583` Signed-off-by: Ed Welch <edward.welch@grafana.com> * handle all the breaking changes from cortes Signed-off-by: Sandeep Sukhani <sandeep.d.sukhani@gmail.com> * rebasing on master, re-runing go mod vendor seemed to bring in some more changes... Signed-off-by: Ed Welch <edward.welch@grafana.com> Co-authored-by: Sandeep Sukhani <sandeep.d.sukhani@gmail.com>	5 years ago
Peter Štibraný	e60164bef6	Convert Loki modules to services (#1804 ) * Loki now uses module services to start and stop its work. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Use services methods to initialize some components. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Use Cortex' NewModuleService. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Converted server to a service. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Converted distributor to service. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Use table manager service Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * querier service Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * query-frontend service Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Merged stopping method into shutdown Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Converted ingester to a service. It now starts all background tasks in Starting state. Stopping needs little work, as does reacting on lifecycler errors. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Loki Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * If lifecycler fails, ingester fails too. It now doesn't call os.Exit, but shuts down gracefully and enters Failed state. That triggers Loki to shutdown completely. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Ignore ErrStopProcess errors from services This is a signal that Loki should stop. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Use single /ready handler It checks the state of all services, and asks ingester for its own check as well. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Removed unused value. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Lint Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Fix test. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Go mod tidy, vendor Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Tailers, not trailers. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Adds return for the healtcheck in case of error. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> Co-authored-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago
Sandeep Sukhani	fad3b61be7	flush boltdb to object store (#1837 ) * flush boltdb to object store files are stored in folder per periodic table and are named after ingester flushed every 15 mins to make index available to other services files are also flushed before ingester stops to avoid any data loss new stores can be implemented easily ingester to also query store when using boltdb * persisting uploader name across restarts, detecting objectstore type from periodic config, other refactorings * updated cli flag for active directory * add tests for boltdb shipper and update vendor * syncing boltdb files to disk during update * sleep in tests to let mtime of boltdb file be changed * add documentation for boltdb shipper and remove unwanted config	5 years ago
Peter Štibraný	5ae6ea71f8	Update Cortex to master (#1785 ) * Update Cortex to v0.7.0-rc.0 Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Fixed errors after updating to latest master. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Fixed errors after updating to latest master. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Fixed errors after updating to latest master. queryrange.NewRetryMiddleware in NewLogFilterTripperware is now created without metrics, as that would collide with metrics in NewMetricTripperware. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Make sure to start client.Pool before using it. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Use shared singleton memberlist KV. Fix queryrange metrics. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Make lint happy Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * go mod tidy Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Return err early. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>	5 years ago
Cyril Tovena	3f3152427a	Update cortex vendoring to include frontend status code improvement. (#1746 ) * Update cortex vendoring to include frontend status code improvement. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes signature changes in runtimeconfig.NewRuntimeConfigManager. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Forgot to git add . :facelpalm: Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago
Owen Diehl	e0a77656fd	removes duplicate logRangeExprExt grammar (#1651 ) * removes duplicate logRangeExprExt grammar * linting errors	5 years ago
Peter Štibraný	d4cae667ae	loki: use new runtimeconfig package from Cortex (#1484 ) * loki: use new runtimeconfig package from Cortex runtimeconfig is a generalization of previous "overrides" mechanism, and can be used not for per-tenant limits, but other parts of Loki too (only one additional usage for now is "multi" KV client). Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * go mod tidy, go mod vendor Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Removed RuntimeConfig from Ingester dependencies. Ingester already depends on RuntimeConfig via Store and Overrides mechanism. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Removed extra new line. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Remove unused method. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Fix broken code after rebasing. Removed incorrect comment. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Updated docs with section about runtime configuration. Mostly copied from Cortex. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * go mod tidy, go mod vendor Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Updated Cortex to current master. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Go mod tidy, vendor Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>	5 years ago
Owen Diehl	5026dfea57	ingester.max-chunk-age (#1558 ) * ingester.max-chunk-age * bumps helm & changelog * adjusts max-chunk-age test * bumps wait period to let ci catch up * updates changelog	5 years ago
Cyril Tovena	fb76416b27	Add decompression tracing instrumentation. (#1445 ) * Add decompression tracing instrumentation. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fix time conversion. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Use span logger so we also get a log line. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Improve reusability and add fetched chunks. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	6 years ago
Peter Štibraný	e0ec61f986	pkg/ingester: handle labels mapping to the same fast fingerprint. (#1247 ) * pkg/ingester: handle labels mapping to the same fast fingerprint. Uses slightly adapted fpMapper code from Cortex. Fixes issue #898 Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * empty commit to force new build Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Removed empty lines in imports. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Added test that pushes entries concurrently. To be run with -race. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Stream now keeps both original and mapped fingerprint Mapped fingerprint is used in streams map. Original ("rawFP") is preserved when saving chunks. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Vendoring Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Reverted previous commit (keep the test, with updated checks). Preserving raw fingerprints could lead to data loss when doing chunk deduplication while storing chunks. We don't want that. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * go mod vendor Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>	6 years ago
Sandeep Sukhani	ec5bc7054d	limits: limits implementation for loki (#948 ) * limits implementation for loki Added following new limits: Length of query Number of active streams Number of streams matcher per query Ingestion rate * periodically reload ingestion rate limits to apply latest rates from overrides	6 years ago
Sandeep Sukhani	b687ec6e5b	vendoring: update cortex to latest master (#938 ) * update cortex to latest master using v3.4.0-rc.1 for go.etcd.io/etcd v0.0.2 for github.com/prometheus/procfs v1.0.0 for github.com/prometheus/client_golang latest master for github.com/weaveworks/common with changes from https://github.com/weaveworks/common/pull/153 * fixed failing tests * use large instance for test and lint jobs in CircleCI * running only 6 test binaries in parallel * removed resource type change for CircleCI * changed GOGC to 10 for lint makefile target	6 years ago
Robert Fratto	d34b1f7450	ingester: support chunk transfers on ingester shutdown. This commit introduces chunk transfers, borrowing the mechanism from Cortex's implementation: when an ingester is shut down with claim on rollout enabled, the ingester will find a pending ingester and transfer all of its chunks to it.	6 years ago
Cyril Tovena	3346ce1b40	Storage memory improvement (#713 ) * add benchmark for storage queries * improve iterator to load only on next * fix memory retained by lazy chunks * reverse backward lazy iterator	6 years ago
Sandeep Sukhani	3d3c05717a	Update cortex vendor (#610 ) * Update cortex vendor Use query max look back from cortex Update config changes from cortex Fixed breaking code due to updating cortex vendor * fixed linter error	6 years ago
Tom Wilkie	beb0052ad0	Remove unnecessarily long wait times in tests. Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	6 years ago
Cyril Tovena	fa4f936fcb	improves memory usage of golangci-linter	6 years ago
Cyril Tovena	8d7efddde2	switch to golangci-lint linter (#539 ) * switch to golangci-lint linter * fixes for review comments	6 years ago
Goutham Veeramachaneni	7763e8f3dd	Flush idle chunks properly (#209 ) * Flush idle chunks properly Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com> * Review feedback Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com> * Review feedback. Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com> * Check the correct chunk is full before appending a new chunk. Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com> * Add the config to helm and ksonnet too. Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	6 years ago

31 Commits (e8cce09de29dc58c38b333e72dfe11613d49fbb3)