apitech/loki - loki - Gitea: Git with a cup of tea

Commit Graph

Author	SHA1	Message	Date
Kaviraj	38c42a26c6	Move `cortex/pkg/tenant` dependency to Loki. (#4944 )	3 years ago
Sandy Chen	fc2bc69eaa	feat(): add ManagedIdentity in Azure Blob Storage (#4858 ) * feat(): add ManagedIdentity in Azure * fix(): fix merge error * fix(): fix a problem during conflict resolving * fix(): fix the problems based on reviews * feat(): add deps * Update pkg/storage/chunk/azure/blob_storage_client.go * chore(): adjust mi logics Co-authored-by: Cyril Tovena <cyril.tovena@gmail.com>	3 years ago
Oleg Kovalov	e893e2696c	Update hedgedhttp to v0.7.0 (#4892 ) * Update hedgedhttp to v0.7.0 * Update CHANGELOG * Fix returned results	4 years ago
Cyril Tovena	c33a651645	Add rate limiting and metrics to hedging (#4860 ) * Add rate limiting and metrics to hedging Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add changelog entry Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Allow to pass a custom registerer Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	4 years ago
Christian Haudum	e573a4d1a2	Change error responses from plain text to JSON (#4845 ) * Return HTTP errors consistently as JSON response The API returned errors in plain text where the error message was the response body. However, the content type header was set to `application/json`. To make the API responses consistent, errors are now returned as JSON as well. ``` { "message": string, "status": string, "code": int } ``` Signed-off-by: Christian Haudum <christian.haudum@gmail.com> * fixup! Return HTTP errors consistently as JSON response Signed-off-by: Christian Haudum <christian.haudum@gmail.com> * Return 404 Not Found responses as JSON Signed-off-by: Christian Haudum <christian.haudum@gmail.com> * fixup! Return 404 Not Found responses as JSON Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	4 years ago
Sandeep Sukhani	3ac9818f82	recreate compacted boltdb files from compactor when they are more than 12 hours old (#4853 )	4 years ago
Cyril Tovena	b27894fbf4	Adds the ability to hedge storage requests. (#4826 ) * Adds the ability to hedge request for all backends Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Remove race from tests Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Remove the race Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Remove the race Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Testing Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * More testing Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Setup credentials to avoid auth Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * gomod Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * improve tests Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * gomod Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * changelog Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Group the configuration Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	4 years ago
Cyril Tovena	54dc737583	Reduce tracing spans (#4842 ) Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	4 years ago
Marco Pracucci	c53457feb9	Upgrade Cortex, Prometheus and Thanos (#4830 ) * Upgrade Cortex, Prometheus and Thanos Signed-off-by: Marco Pracucci <marco@pracucci.com> * Fixed imported ordering Signed-off-by: Marco Pracucci <marco@pracucci.com> * Fixed data type used for series ref when recovering from checkpoint/WAL Signed-off-by: Marco Pracucci <marco@pracucci.com> * Upgraded Thanos to fix arm build Signed-off-by: Marco Pracucci <marco@pracucci.com> * Run go mod tidy Signed-off-by: Marco Pracucci <marco@pracucci.com>	4 years ago
Cyril Tovena	44d32c5ad4	Removes all GRPC warning. (#4757 ) This works for logcli, promtail and Loki. Cherry on top, removes a lot of dependencies. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	4 years ago
Sandeep Sukhani	a707ced67f	index cleanup fixes while applying retention (#4741 ) * Add metric name to the label if missing for building index entries for deletion * fix calculation of table interval from table name * add changelog * add test for series cleanup without __name__ label	4 years ago
Sandeep Sukhani	02d88462f3	allow applying retention at different interval than compaction with a config (#4736 ) * allow applying retention at different interval than compaction with a config * add changelog entry	4 years ago
Cyril Tovena	d81f757ecb	Refactor and simplify stats package. (#4712 ) * Refactor and simplify stats package. This PR refactors the stats package and make it smaller and simpler to work with. - It doesn't expose the context which can causes race when mutating it. - Uses concurrency primitive to ensure correctness - Replace trailers with in-batch statistics to propagate ingesters stats. Fixes #3366 Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * fix tests and lint issues. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes moar tests. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * mod-check Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * More fixes. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	4 years ago
Sandeep Sukhani	20726e6ba1	fix flaky retention tests (#4697 )	4 years ago
Trevor Whitney	bfc77118a1	Update local and docker config for simple scalable deployment (#4660 ) * first draft of local config * remove a few more defaults * interface names fix means we do not need lifecycler ip * update rings to use collectors/ prefix fix replication factor chunks-retain-period default to 0s fix a bug introduced with a Go upgrade? on printing the schema date. * simplify the config for the ruler, synthesize the local directory * simplify loki-docker-config.yaml * change common filesystem config to have chunks and rules directory * now that we are calling register flags on the common ring config we set more values and some of the tests needed to be tweaked. updated a few paths as well. * fix tests Co-authored-by: Edward Welch <edward.welch@grafana.com>	4 years ago
Callum Styan	678495a972	Instrument s3 client DeleteObject requests. (#4649 ) Signed-off-by: Callum Styan <callumstyan@gmail.com>	4 years ago
Susana Ferreira	2d24e2ea64	Migration to dskit/ring (#4641 )	4 years ago
Trevor Whitney	07ea1ed99c	Add common ring configuration (#4617 ) * add ring section to common config Co-Authored-By: Dylan Guedes <djmgguedes@gmail.com> * fix compactor test to append loopback interface it ifnames in test if present * add loopback interface to compactor ring iface names * clean-up from PR review Co-authored-by: Dylan Guedes <djmgguedes@gmail.com> * only appy ingester config to rings with no non-default properties Co-authored-by: Dylan Guedes <djmgguedes@gmail.com>	4 years ago
Sandeep Sukhani	2564a810c5	Logs deletion fixes (#4625 ) * check all tables when we have delete requests to process since chunks could span multiple tables * upload modified index when the it is modified without dropping the chunk * fix broken test * lint * Fix modified check Co-authored-by: Cyril Tovena <cyril.tovena@gmail.com> * suggested changes from PR review * add changelog Co-authored-by: Cyril Tovena <cyril.tovena@gmail.com> Co-authored-by: Owen Diehl <ow.diehl@gmail.com>	4 years ago
Owen Diehl	308af871cf	uses ring.Write instead of ring.WriteNoExtend for compactor ring checks (#4615 )	4 years ago
Ed Welch	0c9beadaaa	Loki: Add a ring to the compactor used to control concurrency when not running standalone (#4574 ) * add a ring to the compactor * wiring up the start/stop functionality * add some monitoring for multiple compactors running at the same time. * Add compactor to `read` target * fix tests/validation requirements * Update pkg/storage/stores/shipper/compactor/compactor_ring.go Co-authored-by: Owen Diehl <ow.diehl@gmail.com> * Update pkg/storage/stores/shipper/compactor/compactor_ring.go Co-authored-by: Owen Diehl <ow.diehl@gmail.com> * Update pkg/storage/stores/shipper/compactor/compactor_ring.go Co-authored-by: Owen Diehl <ow.diehl@gmail.com> * used a shared ring_config between scheduler and compactor to avoid some duplication * fix lint * better handling of the tokens file with a way to easily tell all the rings to persist tokens * fix query scheduler replication factor * removing "scheduler" from log line which may be called by components other than a scheduler in the future. * update changelog Co-authored-by: Owen Diehl <ow.diehl@gmail.com>	4 years ago
Trevor Whitney	ed025e0cd6	Change a few default config values and improve application of common storage config (#4543 ) * further config simplifications and better defaults * default ruler api to on * register defaults for storage configs in the common config (ie. signature version for S3) * enable ingester wal by default * disable chunk retries by default * allow common path prefix to include or omit trailing slash * allow partial storage config to be defined for overriding things like bucket names per component * fix tests * add changelog and upgrade docs * explain impact of wal change in upgrading.md Co-authored-by: Owen Diehl <ow.diehl@gmail.com> * rename add WithPrefix to RegisterFlags * fix (frontend OR worker) and scheduler boot order when both are running (#4545) Co-authored-by: Owen Diehl <ow.diehl@gmail.com>	4 years ago
Cyril Tovena	165ff31e2d	Fixes the Series function to hanlde properly sharding. (#4563 ) Before they would try to match the injected label but this label is only used by the store and not returned in the chunks causing all chunks to be filtered out. Fixes #4547 Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	4 years ago
Cyril Tovena	ea8e0987b3	Fixes an edge case in the batch chunk iterator. (#4550 ) * no-cache Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * found a bug to fix. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes a edge case in the batch chunk iterator. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	4 years ago
李国忠	6fcd02bad4	[querier] s3: add getObject retry (#4453 ) * [querier] s3: add getObject retry * [querier] s3: add getObject retry * [querier] s3: add getObject retry #4452 * [querier] s3: add getObject retry * [querier] s3: add getObject retry #4453 * [querier] s3: add getObject retry * [querier] s3: add getObject retry #4453 * [querier] s3: add getObject retry #4453 * [querier] s3: add getObject retry #4453 * [querier] s3: add getObject retry Co-authored-by: Owen Diehl <ow.diehl@gmail.com>	4 years ago
Dylan Guedes	77ea11724b	Loki: Enable FIFO cache by default (#4519 ) * Set default FIFO cache with reasonable defaults. Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com> * Remove FIFO cache feature flag. - We noticed that we always want that behavior, so the logic is better implemented in the `ApplyDinamicConfig` method - Update documentation * Move FIFO cache default behavior. - Move FIFO cache default behavior implementation to ApplyDinamicConfig * Add tests for the new FIFO cache default behavior. - The tests tests the behavior of the four existing caches (query results, query index, write dedupe, and chunk results) - Add changelog entry - Fix lint issues regarding import order - Revert moving of EnableFifoCache check * Apply suggestions from code review - Change `EnableFifoCache` description to mention that it is only applied for chunk and query results cache Co-authored-by: Owen Diehl <ow.diehl@gmail.com> Co-authored-by: Mauro Stettler <mauro.stettler@gmail.com> Co-authored-by: Owen Diehl <ow.diehl@gmail.com>	4 years ago
JordanRushing	ca67292b54	Introduce `overrides-exporter` module to Loki (#4520 ) * Introduce overrides-exporter module from cortex into loki; add some test configs for local dev * Fix panic by adding server dependency to overrides-exporter module * Add tests for limits exporter * Remove dummy configs for testing * Add more limits to export * Add more limits; move mockTenantLimits to exporter_test.go * Introduce overrides-exporter module from cortex into loki; add some test configs for local dev * Fix panic by adding server dependency to overrides-exporter module * Add tests for limits exporter * Remove dummy configs for testing * Add more limits to export * Add more limits; move mockTenantLimits to exporter_test.go * Use reflection; add more limits to export * Fix import to satisfy linter * refactors TenantLimits for clarity & efficiency. Also fixes tenantLimits nil check * Fix breaking test with nil check in runtime_config Co-authored-by: Owen Diehl <ow.diehl@gmail.com>	4 years ago
Trevor Whitney	2427fab32d	Configuration: add a common config section for object storage (#4473 ) * add common config for object storage Signed-off-by: Trevor Whitney <trevorjwhitney@gmail.com> * wip Signed-off-by: Trevor Whitney <trevorjwhitney@gmail.com> * add test to compactor config override Signed-off-by: Trevor Whitney <trevorjwhitney@gmail.com> * update changelog Signed-off-by: Trevor Whitney <trevorjwhitney@gmail.com> * changes from PR review * rename object_store -> storage * add local/filesytem support to common object storage config	4 years ago
Danny Kopping	cff8324f24	Replacing go-kit/kit/log with go-kit/log (#4484 ) * Replacing go-kit/kit/log with go-kit/log Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Appeasing the f&^$ing linter Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Adding depguard Cleaning go mod Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Updating weaveworks after https://github.com/weaveworks/common/pull/226 was merged Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Appeasing the mod checker Signed-off-by: Danny Kopping <danny.kopping@grafana.com>	4 years ago
Mauro Stettler	4b59556342	Update cortex to newer version (#4478 ) * update cortex and thanos to newer version * pin k8s.io/apimachinery * fixes for newer libs * add missing method to mock wal data collector * fixing test by using different gokit level pkg * make check-mod	4 years ago
Arve Knudsen	cdde2cf050	Chore: Use dskit/grpcclient (#4312 )	4 years ago
Danny Kopping	d5b6d05557	CI: Fixing linter deprecations (#4353 ) * Fixing CI deprecations Replacing deprecated golint with revive (https://github.com/golangci/golangci-lint/pull/1965) Removing interfacer (https://github.com/golangci/golangci-lint/pull/1755) Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Further suggestions from Marco in https://github.com/grafana/loki/issues/1002 Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Clarify instruction Signed-off-by: Danny Kopping <danny.kopping@grafana.com>	4 years ago
Arve Knudsen	f48e867002	Upgrade dskit (#4313 ) Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> Use middleware package from dskit Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	4 years ago
Cyril Tovena	e81ca28c80	Fixes a bug in the block cache code. (#4302 ) * Fixes a bug in the block cache code. Because chunks can overlap and that can causes multiple processing of the same chunk within batches, we decided to cache the evaluation of it. However there was a bug here causing incorrect query result, specially for metric queries. The bug would occur when we would only partially consume a block (overlaping block between multiple batches) and then try to replay it. In which case it wouldn't actually replay it but keep consuming the underlaying iterator. This took us 2 days to figure out, thank Danny for helping along. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Missing warning when closing fails. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * got linted. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Review feedback. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Forgot to rename usage. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	4 years ago
Christian Haudum	8d561f6caf	Chore: Update Cortex and use kv package from grafana/dskit (#4282 )	4 years ago
Christian Haudum	0f229f471b	cli: Register configuration option `store.max-look-back-period` as CLI argument (#4242 ) * Register CLI flag for ChunkStoreConfig.MaxLookBackPeriod Even though the flag is already deprecated, registering the flag is necessary for generating the config reference documentation. * Update pkg/storage/store.go Co-authored-by: Owen Diehl <ow.diehl@gmail.com>	4 years ago
Arve Knudsen	cfb4fc1f55	Use flagext from dskit (#4225 ) Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	4 years ago
Nick Pillitteri	79a7bdbabc	Chore: Use services and modules from grafana/dskit (#4196 ) * Update Cortex vendor to bcf15611345da65e03604a08a0213aaaaac49e54 Signed-off-by: Nick Pillitteri <nick.pillitteri@grafana.com> * Use services and modules packages from grafana/dskit Signed-off-by: Nick Pillitteri <nick.pillitteri@grafana.com> * Update promtail test to set default weaveworks server network type Signed-off-by: Nick Pillitteri <nick.pillitteri@grafana.com> * Punctuation Signed-off-by: Nick Pillitteri <nick.pillitteri@grafana.com>	4 years ago
Benoît Knecht	488a21bff7	pkg/storage/chunk/aws: Add s3.http.ca-file option (#4211 ) * pkg/storage/chunk/aws: Fix insecure-skip-verify documentation The documentation claimed that the `s3.http.insecure-skip-verify` option needed to be set to `false` in order to skip verifying certificates, but it needs to be set to `true`. Signed-off-by: Benoît Knecht <bknecht@protonmail.ch> * pkg/storage/chunk/aws: Add s3.http.ca-file option This option lets users set a custom CA file for connections to S3, in case they use a local S3 instance with an internal PKI. Signed-off-by: Benoît Knecht <bknecht@protonmail.ch>	4 years ago
Arve Knudsen	10c92a901e	Chore: Upgrade to latest Cortex (#4188 ) * Chore: Upgrade to latest Cortex Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix formatting Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	4 years ago
Sandeep Sukhani	29b07d190d	Retention speedup (#4172 ) * drop the multi table chunk entries from index when the whole table is out of retention * speed up cleaning of deleted series from index * mark partially deleted chunk for deletion when all its tables are processed	4 years ago
Sandeep Sukhani	8ef3d5fee9	add a storage client for boltdb-shipper which would do all the object key management for storage operations (#4128 ) * add a storage client for boltdb-shipper which would do all the object key management for storage operations * changes suggested from PR review	4 years ago
Sandeep Sukhani	67fe2d68a3	fix finding tables which would have out of retention data (#4107 )	4 years ago
Sandeep Sukhani	138ca71dd5	add a method to check whether error is due to a missing object in storage while reading or deleting the object (#4081 )	4 years ago
Cyril Tovena	ac6ace8dbd	Make `ParseExternalKey` from the chunk package alloc free. (#4075 ) ``` ~/go/src/github.com/grafana/loki main* ⇣ ❯ benchcmp before.txt after.txt2 benchmark old ns/op new ns/op delta Benchmark_ParseExternalKey-16 569 415 -27.03% benchmark old allocs new allocs delta Benchmark_ParseExternalKey-16 2 0 -100.00% benchmark old bytes new bytes delta Benchmark_ParseExternalKey-16 96 0 -100.00% ``` This is a very hot function in Loki being used for every chunk reference we get from the index. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	4 years ago
Sandeep Sukhani	19aec6e6a3	fix broken TestCachingStorageClientStoreQueries (#4074 )	4 years ago
Sandeep Sukhani	d31f8676fd	Optimize index queries and caching (#4070 ) * Use RangeValuePrefix for building queries for Series IDs * optimize index store queries and caching by not doing broad queries	4 years ago
Cyril Tovena	32e55c3043	Fork cortex chunk storage into Loki. (#3842 ) The chunk package has been forked from this commit for future reference and licensing. `09fe17c0e6`	4 years ago
Owen Diehl	56256bfd5b	Unordered chunks (#4001 ) * merge feature/unordered-replay * interoperable head chunks * memchunk block interop * retain ordered memchunk optimizations when possible * tests+bench for unordered chunk reads * reorder on chunk close * [wip] ingester stream unorderd * unordered writes default in testware config, fixes OOO bug & removes unused lastChunkTimestamp var * validity window is 1/2 of max age & fixes old transfer test * more consistent headblock checking/creation * more cohesive encoding tests * unordered stream test with validity bounds * compat - unordered * reinstates memchunk defaults when rebounding & updates storage test compatibility * lint * reorder across blocks doesnt overflow * respect chunk configs during rebounding when possible * only sync checks on ordered writes	4 years ago
Cyril Tovena	4c2feb32b5	Fixes test helper `newChunk` in the storage pkg. (#4056 ) I wanted to test that overlapping chunks are correctly re-ordered and bump into a rounding issue. Keeping both for future will avoid the same mistake. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	4 years ago

1 2 3 4 5

201 Commits (94fec51fb5b4fa225f7aec4c5c21592eb33050a1)