apitech/loki - loki - Gitea: Git with a cup of tea

Commit Graph

Author	SHA1	Message	Date
Cyril Tovena	d6727398d0	Removes split-by upstream validation (#5087 ) Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	3 years ago
Arve Knudsen	d87dbf9f8a	Chore: Upgrade to latest Cortex (#5085 ) * Chore: Upgrade to latest Cortex Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Chore: Upgrade to Prometheus@692a54649ed7 Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix build after upgrading dskit Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Add changelog entry Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	3 years ago
Cyril Tovena	6ef934b24c	Fix deadlock in disconnecting querier (#5063 ) * Fix deadlock in disconnecting querier Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * lint Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Update pkg/querier/worker/worker_test.go Co-authored-by: Trevor Whitney <trevorjwhitney@gmail.com> * Update pkg/querier/worker/worker_test.go Co-authored-by: Trevor Whitney <trevorjwhitney@gmail.com> * Uses in-memory GRPC server for tests Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> Co-authored-by: Trevor Whitney <trevorjwhitney@gmail.com>	3 years ago
Cyril Tovena	a7795e57b0	Fixes a possible cancellation issue (#5075 ) Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	3 years ago
Karen Miller	625d19cddb	Docs: Revise hedging configuration block descriptions (#5069 ) * Docs: Revise hedging configuration block descriptions * Docs: revise 99th percential description per review	3 years ago
Trevor Whitney	46cc256702	expand ruler ring section to add num_tokens property (#4899 ) * expand ruler ring section to add num_tokens property * fix num-tokens prefix * fix wording in common ring section around heartbeat_timeout * rebase on main	3 years ago
JordanRushing	291360ddcb	Simpler new chunk key v12 (#5054 ) * starts hacking new chunk paths * Create new chunk key for S3; update parsing and add basic test and benchmark Signed-off-by: Jordan Rushing <jordan.rushing@grafana.com> Fix `TestGrpcStore` to work with new chunk key structure Signed-off-by: Jordan Rushing <jordan.rushing@grafana.com> Add a function to SchemaConfig that returns the correct PeriodConfig for a given time. Signed-off-by: Callum Styan <callumstyan@gmail.com> Add schema config to object client so it we can use the right external key function based on the schema version. Signed-off-by: Callum Styan <callumstyan@gmail.com> Fix failing compactor tests by passing SchemaConfig to chunkClient Remove empty conditional from SchemaForTime Define schema v12; wire-in ChunkPathShardFactor and ChunkPathPeriod as configurable values Add PeriodConfig test steps for new values ChunkPathShardFactor and ChunkPathPeriod in v12 Update test for chunk.NewExternalKey(); remove completed TODO Set defaults for new ChunkPathPeriod and ChunkPathShardFactor SchemaConfig values; change FSObjectClient to use IdentityEncoder instead of Base64Encoder Use IdentityEncoder everywhere we use FSObjectClient for Chunks Add ExternalKey() function to SchemaConfig; update ObjectClient for Chunks * Finish plumbing through the chunk.ExternalKey -> schemaConfig.ExternalKey change. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Clean up lint failures Signed-off-by: Jordan Rushing <jordan.rushing@grafana.com> * Clean up chunk.go and fix tests Signed-off-by: Jordan Rushing <jordan.rushing@grafana.com> * Break SchemaConfig.ExternalKey() into smaller functions Signed-off-by: Jordan Rushing <jordan.rushing@grafana.com> * Quickly fix variable name in SchemaConfig.newerExternalKey() Signed-off-by: Jordan Rushing <jordan.rushing@grafana.com> * Clean up ExternalKey conditional logic; add better comments; add tests Signed-off-by: Jordan Rushing <jordan.rushing@grafana.com> * Fix a bug where we are prepending userID to legacy Chunk keys but never parsing it; refactor key tests and benchmarks Signed-off-by: Jordan Rushing <jordan.rushing@grafana.com> * Add small SchemaConfig test Signed-off-by: Jordan Rushing <jordan.rushing@grafana.com> * Update docs and CHANGELOG.md Signed-off-by: Jordan Rushing <jordan.rushing@grafana.com> * Correctly return an error when failing to parse a chunk external key Signed-off-by: Jordan Rushing <jordan.rushing@grafana.com> * Revert IdentityEncoder to Base64Encoder after testing Signed-off-by: Jordan Rushing <jordan.rushing@grafana.com> * Fix assignment in test for linter Signed-off-by: Jordan Rushing <jordan.rushing@grafana.com> * Remove leftover comments from development Signed-off-by: Jordan Rushing <jordan.rushing@grafana.com> * Change v12 version comment format; remove redundant login in `ParseExternalKey()` Co-authored-by: Owen Diehl <ow.diehl@gmail.com> * Change remaining v12 comment style Signed-off-by: Jordan Rushing <jordan.rushing@grafana.com> * Pass chunk.SchemaConfig to new parallel chunk client * Simplify chunk external key prefixes for schema v12 * Fix broken benchmark; add benchmarks for old parsing conditional Signed-off-by: Jordan Rushing <jordan.rushing@grafana.com> * Remove unneeded lines from upgrading doc * Add benchmarks for root chunk external key parsing functions Signed-off-by: Jordan Rushing <jordan.rushing@grafana.com> * memoizes schema number calculations * Resolve linter issue with PeriodConfig receiver name Signed-off-by: Jordan Rushing <jordan.rushing@grafana.com> Co-authored-by: Owen Diehl <ow.diehl@gmail.com> Co-authored-by: Callum Styan <callumstyan@gmail.com>	3 years ago
Sandeep Sukhani	ff47d7419a	compactor changes for building per user index files in boltdb shipper (#5026 ) * compactor changes for building per user index files in boltdb shipper * update comments * lint and changes suggested from PR review * fix broken code after merging main	3 years ago
Callum Styan	a5c544e7ba	Add an egress bytes total metric to the azure client. (#5067 ) * Add an egress bytes total metric to the azure client. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Update changelog entry PR # for egress metric addition. Signed-off-by: Callum Styan <callumstyan@gmail.com>	3 years ago
Karsten Jeschkies	7bbed2be49	Define Ingester interface to enable wrapping. (#4988 )	3 years ago
李国忠	6b377b40d5	[new] promtail: add readline rate limit (#5031 ) * [new] promtail: add readline rate limit * [new] promtail: add readline rate limit #5031 * [new] promtail: add readline rate limit #5031 * [new] promtail: add readline rate limit #5031 * [new] promtail: add readline rate limit #5031 * Update clients/pkg/promtail/limit/config.go Co-authored-by: Cyril Tovena <cyril.tovena@gmail.com> * Update clients/pkg/promtail/limit/config.go Co-authored-by: Cyril Tovena <cyril.tovena@gmail.com> Co-authored-by: Cyril Tovena <cyril.tovena@gmail.com>	3 years ago
Cyril Tovena	9a7f5a22f6	Fix race conditions in frontend_scheduler_worker. (#5060 ) Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	3 years ago
Susana Ferreira	191a8a6f9d	Store metrics queueTime in seconds (#5052 ) * Store metrics queueTime in seconds * Address comments * Helper function to convert seconds to nanoseconds	3 years ago
Karen Miller	c03614efe5	Docs: add an empty subsection for v2.4.2 bug fix items (#5056 )	3 years ago
Kaviraj	cecadf2a18	Chore: Remove `cortex/util/test` dependency (#5050 ) * Remove `cortex/util/test` dependency Copy `util/test.Poll` into Loki Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * go mod vendor Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com>	3 years ago
Kaviraj	ac6e092f3c	Remove `cortex/pkg/util/extract` dependency from Loki (#5049 ) Copy only `extract.MetricNameMatcherFromMatchers`. Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com>	3 years ago
Dylan Guedes	a4f218305d	Copy `cortex/pkg/frontend` package dependency into Loki (#5044 ) * Move `cortex/pkg/frontend` to `loki/pkg/lokifrontend/frontend`. * Update modules.txt with removed Cortex frontend. * Use `pkg/lokifrontend` instead of cortex `pkg/frontend`.	3 years ago
Ed Welch	ba91a0e48a	lock the mutex before working with the `fileEventWatchers` (#5043 ) Signed-off-by: Edward Welch <edward.welch@grafana.com>	3 years ago
Kaviraj	564f833a30	Improve error message if incoming logs timestamp is far too behind. (#5040 ) * Improve error message if incoming logs timestamp is far too behind. This is part of JSON response giving out as the response to HTTP /push endpoint. Old message ``` {"code":400,"status":"error","message":"entry for stream '{foo=\"bar\"}' has timestamp too old: 1970-01-01 01:00:00.5 +0100 CET"} ``` New message ``` {"code":400,"status":"error","message":"entry for stream '{foo=\"bar\"}' has timestamp too old: 2021-12-28 01:48:45.5 +0100 CET, accepts timestamp from: 2021-12-29 09:48:45.737756651 +0100 CET"} ``` main rationale being, hard to know what is the closest timestamp that Loki is expecting without going through the config. Also config is not straight forward (its `latest stream's timestamp - ingester.max-chunk-age/2`) Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Tweaks the error message Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Fix `distributor/validator_test`. 1. Make `time.Now()` mockable. 2. Make time format consistent across error messages. Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Rename `getValidationContextFor` to `getValidationContextForTime` Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com>	3 years ago
Dylan Guedes	c0bec07e0d	Loki: Implement common net interface/instance addr (#4950 ) * Implement common net interface/instance addr. - Implement two new common configurations: instance_interface_names, which is a list of net interfaces used to discover addresses and instance_addr, used to advertise a component address. * Add docs to new common configurations. * Fix lint issue: missing line. * Add missing CHANGELOG entry. * Change default common net interface from `127.0.0.1` to `""`. * Add missing `InstanceAddr` docstring.	3 years ago
Cyril Tovena	305b41aedb	Improve title of the chunk dashboard. (#5041 ) I was looking at the dashboard and a bit confused by the title, so I made them more specifics. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	3 years ago
Kaviraj	14afb13d36	Chore: Copy `cortex/util/math` into Loki (#5036 ) * Copy `cortex/pkg/util/math` into Loki. Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Fix linter Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Move `MinUint32` from `util` package into `util/math` package Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Remove NewEWMARate and related helpers Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Remove unwanted spaces Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Fix spaces in import paths Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com>	3 years ago
Cyril Tovena	57a8a878c1	Remove unnecessary ticker in the tail loop (#5039 ) Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	3 years ago
Ronen Schaffer	595cb7d3a6	Fix typo 'Registring'->'Registering' (#4979 )	3 years ago
Kevin	5030a1f463	Promtail: exclude_user_data description fix (#5038 ) Based on https://github.com/grafana/loki/blob/main/clients/pkg/promtail/scrapeconfig/scrapeconfig.go#L224 the option to exclude user data of windows evens is misspelled in the documentation. This puts the correct exclude_user_data option in the documentation.	3 years ago
Siavash Safi	cd9d9c9b42	Support Redis Cluster Configuration Endpoint (#4947 ) The go-redis options requires multiple redis addresses for a cluster. In some scenarios like AWS ElastiCache, a single configuration endpoint is used to resolve all cluster nodes. This change adds an extra step to resolve all nodes and pass them to go-redis. It remove the need to specify multiple cluster nodes in the configuration. No tests were added for this scenario since it requires patching the Go resolver during runtime. Signed-off-by: Siavash Safi <siavash.safi@gmail.com>	3 years ago
Ronen Schaffer	9d8d91c045	Update hack_operator_make_run.md (#5025 ) Due to the migration of loki-operator to this repo, the crd was renamed `lokistacks.loki.grafana.com` -> `lokistacks.loki.openshift.io`	3 years ago
Susana Ferreira	82f8f3c1e5	Log time in queue per request (#4949 ) * Log time in queue per request * Add querierID to log message * Add unescape request url and tenant_ids to log message * Add query enqueue time to metrics.go * Remove "querier request dequeued" log from scheduler * Address comments * Rename enqueue_time to queue_time * Store queue time in nanoseconds (int64) * Use CanonicalMIMEHeaderKey for setting the httpgrpc header	3 years ago
Cyril Tovena	8b71a44a86	Pre-allocate chunk buffer size (#4884 ) * Pre-allocate chunk buffer size Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * add doc	3 years ago
Karen Miller	9a49ab78fe	Docs: revise block names (ring and filesystem) and correct spelling (#5015 ) * Docs: revise block names (ring and filesystem) and correct spelling * Docs: scheduler_ring and compactor_ring point to ring config Change made per review comment.	3 years ago
Trevor Whitney	acbbe4ddfd	temporary fix for the ruler joining the ingester ring (#4968 )	3 years ago
Timur Olzhabayev	0a6c41763d	feat(docs): auto assign type/docs label to docs project (#5032 )	3 years ago
Kaviraj	fe1d28a545	Chore: use `spanlogger` from `dskit` (#5029 ) * Move `cortex/pkg/spanlogger` into `loki`. It uses spanlogger from `dskit/pkg/spanlogger` with small wrapper created on the `pkg/util/spanlogger` in loki repo. Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Remove `cortex/util/spanlogger` -> `loki/util/spanlogger` Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> * Fix linter (with goimports) and `FromContext` Run goimports with `-local` flag `FromContext` takes util logger to keep the API same as currently used inside loki Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com>	3 years ago
Karen Miller	94d1550e9c	Docs: improve Promtail installation prose (#5017 )	3 years ago
李国忠	a66e148f83	[new feature] logql: extrapolate unwrapped rate function (#5013 ) * [new feature] logql：reimplement metrics rate function * [new feature] logql：reimplement metrics rate function * [new feature] logql：reimplement metrics rate function * [new feature] logql: extrapolate unwrapped rate function #5013	3 years ago
Cyril Tovena	03be8dac1e	Allow to disable http2 for GCS. (#4942 ) * Allow to disable http2 for GCS. This also tweaks our iddle connection pooling accross the board and reduce the insane parallelism when fetching chunks. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * update changelog Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Update pkg/storage/chunk/gcp/gcs_object_client.go Co-authored-by: Danny Kopping <dannykopping@gmail.com> * update comment Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Update docs/sources/configuration/_index.md Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> * Update CHANGELOG.md Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com> * Reduce MaxIdleConns to a more standard value. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Allow to configure max parallelism for fetching chunks Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes tests with no default config Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> Co-authored-by: Danny Kopping <dannykopping@gmail.com> Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com>	3 years ago
Dylan Guedes	98ecfefd89	Add distributor ring page (#4938 ) * Implement new IngestionRateStrat Enum. - This new IngestionRateStrat is used internally to represent the chosen ingestion rate strategy, to avoid using plain strings. * Add a ring to the distributor. As of now, the distributor only has a pointer to its lifecycler. Since the ring HTTP handler isn't implemented for the lifecycler but it is for the ruler, this extends the distributor by also holding a ring instance. * Implement /distributor/ring page. * Reuse existing global strat const. * Add distributor ring HTTP page tests. * Add missing CHANGELOG entry. * Document missing ring status pages. * Fork Cortex util/http.go into source code.	3 years ago
Owen Diehl	7ed7e88173	adds instrumentation to azure object client (#5022 ) * adds instrumentation to azure object client * Update pkg/storage/chunk/azure/blob_storage_client.go nice catch! Co-authored-by: Susana Ferreira <ssncferreira@gmail.com> Co-authored-by: Susana Ferreira <ssncferreira@gmail.com>	3 years ago
Dylan Guedes	07b583bb83	Copy `cortex/pkg/scheduler` package dependency into Loki (#4962 ) * Fork pkg/scheduler from cortex to Loki. * Replace cortex/pkg/scheduler with loki/pkg/scheduler. * Remove unused cortex subpackages from modules.txt. * Run `make format` to fix new loki/pkg/scheduler imports. * Update new scheduler .proto generated files.	3 years ago
Dylan Guedes	d450451c07	Copy `cortex/pkg/ruler/rulestorage` package dependency into Loki (#4978 ) * Fork cortex `pkg/ruler/rulestore` package. * Replace cortex `pkg/rulestore` usage. * Apply formatting.	3 years ago
Dylan Guedes	92c05493e1	Copy `cortex/pkg/distributor` package dependency into Loki (#4983 ) * Fork cortex `pkg/distributor/distributor_ring.go` Copy content from `pkg/distributor/distributor_ring.go` into Loki's `pkg/distributor/distributor_ring.go`. * Fork cortex `pkg/distributor/ingester_client_pool.go`. Copy Cortex `pkg/distributor/ingester_client_pool.go` into Loki's `pkg/distributor/clientpool/ingester_client_pool.go`. We couldn't copy it directly to `pkg/distributor/ingester_client_pool.go` because we distributor imports a package that imports the ingester_client_pool, which leads to import cycle. * Stop using Cortex `pkg/distributor`. Instead, use Loki's `pkg/distributor` and `pkg/distributor/clientpool`. * Remove unused Cortex distributor pkg. * Use random ports for Loki test.	3 years ago
Shardul Srivastava	d8305ecd0a	Add doc for limit param max_concurrent_tail_requests (#5011 )	3 years ago
Dylan Guedes	d5783e86a3	Copy `cortex/pkg/querier/astmapper` package dependency into Loki (#4982 ) * Fork cortex `pkg/querier/asmapper` into Loki. * Move away from cortex `pkg/querier/astmapper`. * Fix lint issues. - Use _ in argument of method `pkg/querier/astmapper/subtree_folder.go` since it is not being used - Remove unnecessary cast from `subtree_folder.go`	3 years ago
Susana Ferreira	2fdd24038b	Move querier.split-queries-by-interval to a per-tenant configuration (#5023 )	3 years ago
Karen Miller	7867451a93	Docs: SSD mode read target runs the compactor (#5016 )	3 years ago
Susana Ferreira	0c2ed01d92	Remove querier.query-ingesters-within check when sharding is enabled (#5024 )	3 years ago
Avtion	1b16bb3d1d	docs: fix yaml grammar mistakes (#5027 )	3 years ago
Sandeep Sukhani	b865afc0d7	fix splitting of queries when step is larger than split interval (#5006 )	3 years ago
Robby Milo	0b93520994	update promtail daemonset docs (#4946 ) * Update installation.md * Update installation.md	3 years ago
Joe Stevens	b5d5123e0b	Docs: Fix parent of wal and wal_cleaner in loki ruler config docs (#4993 ) * fix parent of wal and wal_cleaner in loki ruler config docs * update changelog	3 years ago

1 2 3 4 5 ...

2939 Commits (d6727398d042cc429232120a7127701278249a26) All Branches Search

2939 Commits (d6727398d042cc429232120a7127701278249a26)

All Branches