apitech/loki - loki - Gitea: Git with a cup of tea

Commit Graph

Author	SHA1	Message	Date
RangerCD	87cbe50b59	Lock consistencyMtx outside locking chunkMtx (#5210 ) * Lock consistencyMtx outside locking chunkMtx * Remove unnecessary RLock	4 years ago
RangerCD	4ee686c22c	Remove ErrStreamMissing (#5234 )	4 years ago
Dylan Guedes	3f496e7336	Copy `cortex/pkg/cortexpb` package dependency into Loki (#5151 ) * Fork cortexpb into Cortex `pkg/logproto`. * Use `pkg/logproto` instead of Cortex `pkg/cortexpb`. * Rename `cortexpb` usage to `logproto`. * Fix encoding/decoding test. * Replace cortexpb usage in the ruler/base package. * Remove exemplars from logproto.	4 years ago
RangerCD	b26b73d5be	Remove instance level streams lock (#5179 ) * Feat(ingester): remove instance level streams lock * Fix lint * Solve Load and Delete consistency problem while pushing to ingester	4 years ago
Arve Knudsen	a2dd731586	Chore: Add pkg/util/log package (#5187 ) * Chore: Add pkg/util/log package Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Add changelog entry Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix documentation of loki_log_messages_total Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	4 years ago
Owen Diehl	1b5b8a3109	log headblock conversion error during replay cleanup (#5185 )	4 years ago
Trevor Whitney	3091ccd3eb	improve default config values (#5077 ) * improve default config values * change defaults for upstream query range package * update changelog * remove max_tranfer_retries fix as it is covered in another PR, 4792 * increase query ingesters within by 1h This provides an additional buffer on top of the max_chunk_age. Co-authored-by: Owen Diehl <ow.diehl@gmail.com> * add comment reminder to remove query range config hack Co-authored-by: Owen Diehl <ow.diehl@gmail.com> * rollback change to align_queries_with_step and parallelise_shardable_queries * add upgrading docs * re-enable parallelise_shardable_queries by default * add parallelise_shardable_queries back to upgrading doc Co-authored-by: Owen Diehl <ow.diehl@gmail.com>	4 years ago
Owen Diehl	e823a5037f	better defaults for flush queue parallelism (#5091 ) * better defaults for flush queue length * changelog	4 years ago
Arve Knudsen	d87dbf9f8a	Chore: Upgrade to latest Cortex (#5085 ) * Chore: Upgrade to latest Cortex Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Chore: Upgrade to Prometheus@692a54649ed7 Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix build after upgrading dskit Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Add changelog entry Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	4 years ago
JordanRushing	291360ddcb	Simpler new chunk key v12 (#5054 ) * starts hacking new chunk paths * Create new chunk key for S3; update parsing and add basic test and benchmark Signed-off-by: Jordan Rushing <jordan.rushing@grafana.com> Fix `TestGrpcStore` to work with new chunk key structure Signed-off-by: Jordan Rushing <jordan.rushing@grafana.com> Add a function to SchemaConfig that returns the correct PeriodConfig for a given time. Signed-off-by: Callum Styan <callumstyan@gmail.com> Add schema config to object client so it we can use the right external key function based on the schema version. Signed-off-by: Callum Styan <callumstyan@gmail.com> Fix failing compactor tests by passing SchemaConfig to chunkClient Remove empty conditional from SchemaForTime Define schema v12; wire-in ChunkPathShardFactor and ChunkPathPeriod as configurable values Add PeriodConfig test steps for new values ChunkPathShardFactor and ChunkPathPeriod in v12 Update test for chunk.NewExternalKey(); remove completed TODO Set defaults for new ChunkPathPeriod and ChunkPathShardFactor SchemaConfig values; change FSObjectClient to use IdentityEncoder instead of Base64Encoder Use IdentityEncoder everywhere we use FSObjectClient for Chunks Add ExternalKey() function to SchemaConfig; update ObjectClient for Chunks * Finish plumbing through the chunk.ExternalKey -> schemaConfig.ExternalKey change. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Clean up lint failures Signed-off-by: Jordan Rushing <jordan.rushing@grafana.com> * Clean up chunk.go and fix tests Signed-off-by: Jordan Rushing <jordan.rushing@grafana.com> * Break SchemaConfig.ExternalKey() into smaller functions Signed-off-by: Jordan Rushing <jordan.rushing@grafana.com> * Quickly fix variable name in SchemaConfig.newerExternalKey() Signed-off-by: Jordan Rushing <jordan.rushing@grafana.com> * Clean up ExternalKey conditional logic; add better comments; add tests Signed-off-by: Jordan Rushing <jordan.rushing@grafana.com> * Fix a bug where we are prepending userID to legacy Chunk keys but never parsing it; refactor key tests and benchmarks Signed-off-by: Jordan Rushing <jordan.rushing@grafana.com> * Add small SchemaConfig test Signed-off-by: Jordan Rushing <jordan.rushing@grafana.com> * Update docs and CHANGELOG.md Signed-off-by: Jordan Rushing <jordan.rushing@grafana.com> * Correctly return an error when failing to parse a chunk external key Signed-off-by: Jordan Rushing <jordan.rushing@grafana.com> * Revert IdentityEncoder to Base64Encoder after testing Signed-off-by: Jordan Rushing <jordan.rushing@grafana.com> * Fix assignment in test for linter Signed-off-by: Jordan Rushing <jordan.rushing@grafana.com> * Remove leftover comments from development Signed-off-by: Jordan Rushing <jordan.rushing@grafana.com> * Change v12 version comment format; remove redundant login in `ParseExternalKey()` Co-authored-by: Owen Diehl <ow.diehl@gmail.com> * Change remaining v12 comment style Signed-off-by: Jordan Rushing <jordan.rushing@grafana.com> * Pass chunk.SchemaConfig to new parallel chunk client * Simplify chunk external key prefixes for schema v12 * Fix broken benchmark; add benchmarks for old parsing conditional Signed-off-by: Jordan Rushing <jordan.rushing@grafana.com> * Remove unneeded lines from upgrading doc * Add benchmarks for root chunk external key parsing functions Signed-off-by: Jordan Rushing <jordan.rushing@grafana.com> * memoizes schema number calculations * Resolve linter issue with PeriodConfig receiver name Signed-off-by: Jordan Rushing <jordan.rushing@grafana.com> Co-authored-by: Owen Diehl <ow.diehl@gmail.com> Co-authored-by: Callum Styan <callumstyan@gmail.com>	4 years ago
Karsten Jeschkies	7bbed2be49	Define Ingester interface to enable wrapping. (#4988 )	4 years ago
Kaviraj	14afb13d36	Chore: Copy `cortex/util/math` into Loki (#5036 ) * Copy `cortex/pkg/util/math` into Loki. Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Fix linter Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Move `MinUint32` from `util` package into `util/math` package Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Remove NewEWMARate and related helpers Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Remove unwanted spaces Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Fix spaces in import paths Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com>	4 years ago
Cyril Tovena	57a8a878c1	Remove unnecessary ticker in the tail loop (#5039 ) Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	4 years ago
Dylan Guedes	92c05493e1	Copy `cortex/pkg/distributor` package dependency into Loki (#4983 ) * Fork cortex `pkg/distributor/distributor_ring.go` Copy content from `pkg/distributor/distributor_ring.go` into Loki's `pkg/distributor/distributor_ring.go`. * Fork cortex `pkg/distributor/ingester_client_pool.go`. Copy Cortex `pkg/distributor/ingester_client_pool.go` into Loki's `pkg/distributor/clientpool/ingester_client_pool.go`. We couldn't copy it directly to `pkg/distributor/ingester_client_pool.go` because we distributor imports a package that imports the ingester_client_pool, which leads to import cycle. * Stop using Cortex `pkg/distributor`. Instead, use Loki's `pkg/distributor` and `pkg/distributor/clientpool`. * Remove unused Cortex distributor pkg. * Use random ports for Loki test.	4 years ago
Dylan Guedes	d5783e86a3	Copy `cortex/pkg/querier/astmapper` package dependency into Loki (#4982 ) * Fork cortex `pkg/querier/asmapper` into Loki. * Move away from cortex `pkg/querier/astmapper`. * Fix lint issues. - Use _ in argument of method `pkg/querier/astmapper/subtree_folder.go` since it is not being used - Remove unnecessary cast from `subtree_folder.go`	4 years ago
Karsten Jeschkies	a5c5f16938	Provide optional matchers to `LabelValuesForMetricName`. (#4933 ) * Provide optional matchers to `LabelValuesForMetricName`. * Pass all tests. * Clarify that we are using series IDs. * Use a set for filtering series ids. * Move matcher support to series. * Write changelog line. * Revert renaming. * Validate time range. * Reset base time. * Update pkg/storage/chunk/series_store.go Co-authored-by: Cyril Tovena <cyril.tovena@gmail.com> * Pre-allocate unique strings. Co-authored-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago
Kaviraj	38c42a26c6	Move `cortex/pkg/tenant` dependency to Loki. (#4944 )	5 years ago
Bryan Boreham	b1ae906517	ingester: use consistent set of instances (#4904 )	5 years ago
Michel Hollands	e69ceefd84	Add LabelFilterer and Store wrapper (#4818 ) * Add LabelFilterer and Store wrapper Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Address review comments Signed-off-by: Michel Hollands <michel.hollands@grafana.com>	5 years ago
Marco Pracucci	c53457feb9	Upgrade Cortex, Prometheus and Thanos (#4830 ) * Upgrade Cortex, Prometheus and Thanos Signed-off-by: Marco Pracucci <marco@pracucci.com> * Fixed imported ordering Signed-off-by: Marco Pracucci <marco@pracucci.com> * Fixed data type used for series ref when recovering from checkpoint/WAL Signed-off-by: Marco Pracucci <marco@pracucci.com> * Upgraded Thanos to fix arm build Signed-off-by: Marco Pracucci <marco@pracucci.com> * Run go mod tidy Signed-off-by: Marco Pracucci <marco@pracucci.com>	5 years ago
Karsten Jeschkies	5ece938f3c	Run all benchmarks. (#4742 ) * Run all benchmarks. * Start debugging memchunk benchmark. * Do not iterate when closed. * Pass default tenant config. * Set default limits. * Fix two more benchmarks. * Enable cron trigger again.	5 years ago
Dylan Guedes	130878f3f6	Enrich WAL folder creation error message (#4705 ) * Enrich WAL folder creation error message. * Apply suggestions from code review Use %q and try absolute path. Co-authored-by: Mateusz Gozdek <mateusz@kinvolk.io> * Include file/path package. Co-authored-by: Mateusz Gozdek <mateusz@kinvolk.io>	5 years ago
Cyril Tovena	d81f757ecb	Refactor and simplify stats package. (#4712 ) * Refactor and simplify stats package. This PR refactors the stats package and make it smaller and simpler to work with. - It doesn't expose the context which can causes race when mutating it. - Uses concurrency primitive to ensure correctness - Replace trailers with in-batch statistics to propagate ingesters stats. Fixes #3366 Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * fix tests and lint issues. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes moar tests. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * mod-check Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * More fixes. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago
Trevor Whitney	bfc77118a1	Update local and docker config for simple scalable deployment (#4660 ) * first draft of local config * remove a few more defaults * interface names fix means we do not need lifecycler ip * update rings to use collectors/ prefix fix replication factor chunks-retain-period default to 0s fix a bug introduced with a Go upgrade? on printing the schema date. * simplify the config for the ruler, synthesize the local directory * simplify loki-docker-config.yaml * change common filesystem config to have chunks and rules directory * now that we are calling register flags on the common ring config we set more values and some of the tests needed to be tweaked. updated a few paths as well. * fix tests Co-authored-by: Edward Welch <edward.welch@grafana.com>	5 years ago
Susana Ferreira	2d24e2ea64	Migration to dskit/ring (#4641 )	5 years ago
Hiroki Sakamoto	0e21f02d76	Respect shard number in series api (#4577 ) * Respect shard number in series api * Refactor codes * remove duplicate logic in instance * format with goimport * Improve complexity of getting all fingerprints	5 years ago
Owen Diehl	0c97afadbc	cleans up redundant setting of stream.unorderedWrites=true when replaying wal (#4595 )	5 years ago
Owen Diehl	633b4cab53	ignore validity window during wal replay (#4596 )	5 years ago
Owen Diehl	45113cb375	Enable unordered_writes by default (#4594 ) * enable unordered_writes by default * removes out_of_orders from transfer test * update changelog	5 years ago
Owen Diehl	cd80bc5e7e	introduces "entry too far behind" instrumentation for unordered writes (#4592 )	5 years ago
Trevor Whitney	ed025e0cd6	Change a few default config values and improve application of common storage config (#4543 ) * further config simplifications and better defaults * default ruler api to on * register defaults for storage configs in the common config (ie. signature version for S3) * enable ingester wal by default * disable chunk retries by default * allow common path prefix to include or omit trailing slash * allow partial storage config to be defined for overriding things like bucket names per component * fix tests * add changelog and upgrade docs * explain impact of wal change in upgrading.md Co-authored-by: Owen Diehl <ow.diehl@gmail.com> * rename add WithPrefix to RegisterFlags * fix (frontend OR worker) and scheduler boot order when both are running (#4545) Co-authored-by: Owen Diehl <ow.diehl@gmail.com>	5 years ago
Cyril Tovena	47336cdbe6	Fixes a panic in the labels API when no parameters are supplied. (#4554 ) * Fixes the a panic in the labels API when no parameters are supplied. We were supporting this before. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Remove bad test Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago
Danny Kopping	cff8324f24	Replacing go-kit/kit/log with go-kit/log (#4484 ) * Replacing go-kit/kit/log with go-kit/log Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Appeasing the f&^$ing linter Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Adding depguard Cleaning go mod Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Updating weaveworks after https://github.com/weaveworks/common/pull/226 was merged Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Appeasing the mod checker Signed-off-by: Danny Kopping <danny.kopping@grafana.com>	5 years ago
Arve Knudsen	cdde2cf050	Chore: Use dskit/grpcclient (#4312 )	5 years ago
Owen Diehl	1921c3d428	tests checkpoints immediately and gives more of a time buffer (#4432 )	5 years ago
Callum Styan	25163470b0	Improve error message for stream rate limit. (#4207 ) * Improve error message for stream rate limit. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Move stream rate limit error message to validation package. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Use ByteSize for number values in stream rate limit error so we can print human readable rate limits and log line byte lengths. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Fix lint issues. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Don't need to type assert twice. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Compare errors contents instead of doing error equals comparison directly since the rate limit error is an RPC error. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Don't call Limit on hot path just to log the value in errors. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Make coment about rate limit clearer. Signed-off-by: Callum Styan <callumstyan@gmail.com>	5 years ago
Trevor Whitney	de0c469161	default chunk target size to ~1MB~ 1.5MB (#4388 ) * default chunk target size to 1MB Signed-off-by: Trevor Whitney <trevorjwhitney@gmail.com> * bump up to 1.5MB Signed-off-by: Trevor Whitney <trevorjwhitney@gmail.com>	5 years ago
Hiroki Sakamoto	502750deca	Add 'loki_ingester_memory_chunks' when recovery from wal (#4272 )	5 years ago
Arve Knudsen	f48e867002	Upgrade dskit (#4313 ) Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> Use middleware package from dskit Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	5 years ago
Arve Knudsen	ae424deac1	Upgrade dskit (#4295 ) Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	5 years ago
Christian Haudum	8d561f6caf	Chore: Update Cortex and use kv package from grafana/dskit (#4282 )	5 years ago
Owen Diehl	1c90b9c57f	remove empty streams after wal replay (#4265 )	5 years ago
Owen Diehl	20515a2402	better per stream rate limits configuration options (#4236 ) * better per stream rate limits configuration options * clean up test config * pr feedback	5 years ago
Arve Knudsen	cfb4fc1f55	Use flagext from dskit (#4225 ) Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	5 years ago
Nick Pillitteri	79a7bdbabc	Chore: Use services and modules from grafana/dskit (#4196 ) * Update Cortex vendor to bcf15611345da65e03604a08a0213aaaaac49e54 Signed-off-by: Nick Pillitteri <nick.pillitteri@grafana.com> * Use services and modules packages from grafana/dskit Signed-off-by: Nick Pillitteri <nick.pillitteri@grafana.com> * Update promtail test to set default weaveworks server network type Signed-off-by: Nick Pillitteri <nick.pillitteri@grafana.com> * Punctuation Signed-off-by: Nick Pillitteri <nick.pillitteri@grafana.com>	5 years ago
Owen Diehl	668622c813	Refactor per stream rate limit (#4213 ) * uses new StreamRateLimiter * sets burst before limiter, uses requested ts * creates new non infinite rate limiters when configs change * time/rate testware * less indirection & protect rate limit access from race conditions * removes old comment * always recreate stream limiter after config change	5 years ago
So Koide	b72d8abfbc	Authc/z: Enable grpc_client_config to allow mTLS (#4176 ) * feat: enable gRPC client config * Update pkg/ingester/client/client.go Co-authored-by: Owen Diehl <ow.diehl@gmail.com>	5 years ago
Owen Diehl	abcf4d083f	WAL replay discard metrics (#4212 ) * adds replay discard metrics * lint	5 years ago
Callum Styan	757eeacb88	[ingester/stream]: Add a byte stream rate limit. (#4191 ) * Add a byte stream rate limit. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Track discarded lines due to stream rate limit + use ByteSize for the config value. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Address review comments Signed-off-by: Callum Styan <callumstyan@gmail.com> * Another round of review fixes. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Fix goimports issue. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Fix duplicate flag registration + stream rate limit test. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Fix stream rate/burst limit flags yaml naming. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Fix tests now that we have stream rate limit. Signed-off-by: Callum Styan <callumstyan@gmail.com> * fix some comments Signed-off-by: Callum Styan <callumstyan@gmail.com>	5 years ago
Arve Knudsen	10c92a901e	Chore: Upgrade to latest Cortex (#4188 ) * Chore: Upgrade to latest Cortex Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix formatting Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	5 years ago

1 2 3 4 5 ...

268 Commits (a3fcd3cc759ef8f29be1d6abe8d6e2c8fecc334c)