apitech/loki - loki - Gitea: Git with a cup of tea

Commit Graph

Author	SHA1	Message	Date
na--	724a841f0f	Add log duplication (tee-ing) in the Distributor (#11496 ) This allows us to inject a Tee in the Distributor that can duplicate all of the incoming log stream, before they are replicated.	1 year ago
na--	71e9fcaab7	Refactor Distributor.Push() to be able to Tee log streams (#11488 ) This allows us to duplicate (i.e. "tee") some of the log streams from the `Distributor` by introducing an optional interface in a follow-up PR.	1 year ago
Michel Hollands	e93f5bfd93	Add dashboard changes for migrating from cortex_ metrics to loki_ metrics (#11082 ) What this PR does / why we need it: - add missing namespace param for ruler manager metrics - update the loki mixin to use the renamed metrics so both are supported - created the compiled mixin - make the prefix of the cortex_distributor_ingester_clients metrics configurable Checklist - [X] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) - [ ] If the change is deprecating or removing a configuration option, update the `deprecated-config.yaml` and `deleted-config.yaml` files respectively in the `tools/deprecated-config-checker` directory. [Example PR](`0d4416a4b0`) --------- Signed-off-by: Michel Hollands <michel.hollands@gmail.com>	2 years ago
Shantanu Alshi	fb6f6d2e2f	Stop using global logger in modules (#11051 ) What this PR does / why we need it: Refactors Loki to inject logger from the constructors instead of using global logger variable. It is generally considered a good practice. Which issue(s) this PR fixes: [Fixes #<issue number>](https://github.com/grafana/loki-private/issues/810) Special notes for your reviewer: Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [x] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) - [ ] If the change is deprecating or removing a configuration option, update the `deprecated-config.yaml` and `deleted-config.yaml` files respectively in the `tools/deprecated-config-checker` directory. <!-- TODO(salvacorts): Add example PR -->	2 years ago
Michel Hollands	8628b15624	Use metrics namespace for more metrics (#11025 ) What this PR does / why we need it: Use the metrics namespace setting instead of hardcoding to `cortex`. This is a follow up to (and based on) https://github.com/grafana/loki/pull/11014. Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [X] Tests updated - [x] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) - [ ] If the change is deprecating or removing a configuration option, update the `deprecated-config.yaml` and `deleted-config.yaml` files respectively in the `tools/deprecated-config-checker` directory. <!-- TODO(salvacorts): Add example PR --> --------- Signed-off-by: Michel Hollands <michel.hollands@gmail.com> Co-authored-by: Ashwanth <iamashwanth@gmail.com>	2 years ago
Christian Haudum	2abb472e7e	Unify ring managers across components (#10931 ) What this PR does / why we need it: This PR creates a re-usable `RingManager` implementation that is then used by * Query Scheduler ring * Index Gateway ring * Bloom Gateway ring * Bloom Compactor ring Which issue(s) this PR fixes: Every component had its own implementation of the ring manager which only really differed in the log messages containing the component name. --------- Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
Christian Haudum	b49b3ce6e1	Add basic structure of bloom gateways (#10782 ) ### Summary This pull requests adds the basic structure for the new bloom gateway component. - Adds new `bloom-gateway` target that runs with multiple instances joined by a ring - Adds a querier and client component on the index gateway to filter chunk refs - Adds the gRPC protobuf definitions for commication between index gateways and bloom gateways - Adds a store component used on the bloom gateways to query binary bloom files ``` Querier Query Frontend \| \| ................................... service boundary \| \| +----+------+ \| indexgateway.Gateway \| bloomgateway.BloomQuerier \| bloomgateway.GatewayClient \| logproto.BloomGatewayClient \| ................................... service boundary \| bloomgateway.Gateway \| bloomshipper.Store \| bloomshipper.Shipper \| bloomshipper.BloomFileClient \| ObjectClient \| ................................... service boundary \| object storage not part of this PR ``` This PR still contains a lot of TODOs and possibilities for optimisations, which will be addressed in subsequent pull requests. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
Shantanu Alshi	183d6f2c39	Upgrade prometheus to 0.47.1 and dskit (#10814 )	2 years ago
Ashwanth	91cae03fe1	chore: restructure compactor package (#10775 ) What this PR does / why we need it: This pr moves the compactor pkg from `pkg/storage/stores/shipper/indexshipper/compactor` to `pkg/compactor` - same level as other high level components (querier, distributor...) table compaction code however resides in the respective store pkg closer to other store specific code. `pkg/storage/stores/shipper/indexshipper/boltdb/compactor/table_compactor.go` `pkg/storage/stores/shipper/indexshipper/tsdb/compactor.go` Special notes for your reviewer: Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`)	2 years ago
Christian Haudum	2e775f3f44	chore(storage): Restructure shipper packages (#10729 ) What this PR does / why we need it: This PR restructures the code for the shipper component of the Loki storage layer. New package layout: ```console $ tree -d pkg/storage/stores/shipper pkg/storage/stores/shipper └── indexshipper ├── boltdb │ └── compactor ├── compactor │ ├── client │ │ └── grpc │ ├── deletion │ ├── deletionmode │ ├── generationnumber │ └── retention ├── downloads ├── gatewayclient ├── index ├── indexgateway ├── storage ├── testutil ├── tsdb │ ├── index │ ├── testdata │ └── testutil ├── uploads └── util 23 directories ``` * TSDB and BoltDB specific code is under `./pkg/storage/stores/shipper/indexshipper/tsdb` and `./pkg/storage/stores/shipper/indexshipper/boltdb` respectively. * Common code for both TSDB and BoltDB is directly under `./pkg/storage/stores/shipper/indexshipper` and subdirectories, such as `uploads/`, `downloads/`, `compactor/`, ... Special notes for your reviewer: This PR is identical to https://github.com/grafana/loki/pull/10724 except of the package `pkg/storage/stores/indexshipper` the package is `pkg/storage/stores/shipper/indexshipper` (one level deeper). Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	2 years ago
Danny Kopping	2c84959901	Distributor: add ingester append timeouts error (#10456 ) What this PR does / why we need it: Failing to send samples to ingesters because the request exceeded its timeout is a very clear signal that ingesters are unable to keep up with demand. In an incident today we saw that ingesters' push latencies were increased sharply by an expensive regex query which was starving other goroutines of time on CPU. This new alert `loki_distributor_ingester_append_timeouts_total` will give us a high-signal metric which we can use for alerting.	2 years ago
Charles Korn	bde65667f7	Upgrade dskit to use packages migrated from weaveworks/common (#10164 ) What this PR does / why we need it: This PR upgrades dskit and replaces use of packages from weaveworks/common with their migrated equivalents in dskit. See https://github.com/grafana/dskit/pull/342 for more details. Note that Loki still uses some packages from weaveworks/common that I haven't migrated (`aws` and `test`) - I'll migrate these separately. If this PR needs to be rebuilt, I used `rewrite.sh` ([source](https://gist.github.com/charleskorn/48efe62a09d6d70f3de30327003df5c5#file-rewrite-sh)) to generate most of these changes. Which issue(s) this PR fixes: (none) Special notes for your reviewer: (none) Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [n/a] Documentation added - [n/a] Tests updated - [n/a] `CHANGELOG.md` updated - [n/a] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [n/a] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [n/a] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) Co-authored-by: Kaviraj Kanagaraj <kavirajkanagaraj@gmail.com>	2 years ago
Dylan Guedes	9ed7fa9df4	WriteFailures: Add metrics to increase monitoring (#9904 ) What this PR does / why we need it: Add metrics for our write failures manager. This way we can easily monitor how much logging we're doing and how much we're discarding. Which issue(s) this PR fixes: N/A	2 years ago
Hamish Forbes	d0baa40df3	Only modify identical timestamps when increment-duplicate-timestamps enabled (#9437 ) When `-validation.increment-duplicate-timestamps` is enabled out of order log entries within the same push request can have their timestamps incorrectly mutated. The code is currently just looking for 2 consecutive entries where the timestamp is not increasing. It should only affect entries where the timestamp is a duplicate Test push request payload: ```json { "streams": [ { "stream": { "increment_duplicate_timestamp": "fixed\|true\|false" }, "values": [ [ "1683665111628000000", "1a: 1683665111628000000" ], [ "1683665111628000000", "1b: 1683665111628000000" ], [ "1683665111628000003", "3: 1683665111628000003" ], [ "1683665111628000002", "2: 1683665111628000002" ] ] } ] } ```` As only the first 2 of these timestamps are the same I would expect all but `1b` to have the timestamps recorded as-is and the logs returned by a query in the order: 1a,1b,2,3. Actual Results: ``` > logcli query --forward --limit 100 -z UTC -q --no-labels --since 60m -o jsonl '{increment_duplicate_timestamp="fixed"}' \| jq { "line": "1a: 1683665111628000000", "timestamp": "2023-05-09T20:45:11.628Z" } { "line": "1b: 1683665111628000000", "timestamp": "2023-05-09T20:45:11.628000001Z" } { "line": "2: 1683665111628000002", "timestamp": "2023-05-09T20:45:11.628000002Z" } { "line": "3: 1683665111628000003", "timestamp": "2023-05-09T20:45:11.628000003Z" } > logcli query --forward --limit 100 -z UTC -q --no-labels --since 60m -o jsonl '{increment_duplicate_timestamp="true"}' \| jq { "line": "1a: 1683665111628000000", "timestamp": "2023-05-09T20:45:11.628Z" } { "line": "1b: 1683665111628000000", "timestamp": "2023-05-09T20:45:11.628000001Z" } { "line": "3: 1683665111628000003", "timestamp": "2023-05-09T20:45:11.628000003Z" } { "line": "2: 1683665111628000002", "timestamp": "2023-05-09T20:45:11.628000004Z" } > logcli query --forward --limit 100 -z UTC -q --no-labels --since 60m -o jsonl '{increment_duplicate_timestamp="false"}' \| jq { "line": "1a: 1683665111628000000", "timestamp": "2023-05-09T20:45:11.628Z" } { "line": "1b: 1683665111628000000", "timestamp": "2023-05-09T20:45:11.628Z" } { "line": "2: 1683665111628000002", "timestamp": "2023-05-09T20:45:11.628000002Z" } { "line": "3: 1683665111628000003", "timestamp": "2023-05-09T20:45:11.628000003Z" } ``` `true` and `false` were written to Loki 2.8.2 with the `-validation.increment-duplicate-timestamps` enabled or disabled `fixed` with this patch applied and the flag enabled Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [x] Tests updated - [ ] `CHANGELOG.md` updated - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	2 years ago
Dylan Guedes	1d20fdbf57	LogPushFailures: Log the tenant id (#9819 ) What this PR does / why we need it: Modify the `writefailures.Manager` to - Log the tenant id. Otherwise, it is hard to identify which tenant is responsible for a problematic push - Log stream parsing failures Which issue(s) this PR fixes: N/A	2 years ago
Salva Corts	08e505dd50	Return all errors for a failed push request (#9807 ) What this PR does / why we need it: Distributors enforce limits and validate each push payload. before this PR, if a single push request yields multiple errors, only the last error is forwarded to the client. This PR modifies the distributor push handler so we forward all errors from a single push request to the client. Which issue(s) this PR fixes: Fixes #<issue number> Special notes for your reviewer: - I wonder if the use of [MultiError][1] is well-suited for this. [This test case][2] will return 100 errors with the same content. I wonder if we can leverage the MultiError to group errors that have the same content. Printing for example: `1 error like: 'Error parsing labels...'; 100 errors like: 'Max entry size...'`. This would require extra processing but If we do this processing in the Error() method I think it should be ok-ish? - Edit: I ended up implementing this in [`4b0816f`](`4b0816f67f`) Checklist - [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [ ] Documentation added - [x] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](`d10549e3ec`) [1]: `d4a4728933/pkg/util/errors.go (L32)` [2]: `f544b93c20/pkg/distributor/distributor_test.go (L75)`	2 years ago
Dylan Guedes	c0d9a2b4ba	Log push failures per-entry instead of in batches (#9720 ) What this PR does / why we need it: Modify how we log push errors to do it per-entry instead of accumulating all errors and then logging them all in a single pass. Which issue(s) this PR fixes: N/A	2 years ago
Dylan Guedes	7d67b6306f	Loki: Implement initial phase of `limited_log_push_errors` (#9556 ) What this PR does / why we need it: Adds an initial implementation of `limited_log_push_errors`. This initial implementation encompass: - Runtime per-tenant configuration - Simple per-tenant rate-limiting based on error message size Notable features that will be added in future phases: - Instead of a single error with the final string, give to the manager the list of all entries - Once it supports per-entry error, make the rate-limiting separated per-reason - Hash the entry error and avoid repeating errors by caching seen errors in memory	2 years ago
Travis Patterson	fdc337692d	Stream Sharding amortize push payloads across the observed rate. (#9414 ) This should account for large payloads that come infrequently	2 years ago
Trevor Whitney	50bc1cee71	Rename usagestats to analytics (#9212 ) What this PR does / why we need it: Rename usagestats to analytics. The config section in the yaml is already called analytics, this makes things more consistent. Checklist - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (required) - [x] Documentation added - [ ] Tests updated - [x] `CHANGELOG.md` updated - [x] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`	2 years ago
Dylan Guedes	1bd5a20595	Distributor: Modify `createShard` to create slice with more precise size (#9265 ) What this PR does / why we need it: Modify the `createShard` method to create the entries slice with the number of entries divided by the total number of shards instead of always creating it with len=1024.	2 years ago
Dylan Guedes	24d426ce44	Distributor: Add AutoForget support (#8067 ) What this PR does / why we need it: - Add auto-forget capabilities to distributors - Change the distributor's ring implementation from Lifecycler to BasicLifecycler. That AutoForget delegate requires it.	2 years ago
Dylan Guedes	9159c1dac3	Loki: Improve spans usage (#8927 ) What this PR does / why we need it: - At different places, inherit the span/spanlogger from the given context instead of instantiating a new one from scratch, which fix spans being orphaned on a read/write operation. - At different places, turn spans into events. Events are lighter than spans and by having fewer spans in the trace, trace visualization will be cleaner without losing any details. - Adds new spans/events to places that might be a bottleneck for our writes/reads.	2 years ago
Dylan Guedes	4f94b89fdb	Loki: Add more spans to write path (#8888 ) What this PR does / why we need it: Add new spans to our write path to better determine which operations are holding our write performance.	2 years ago
Callum Styan	9a2a038f43	Allow passing of context to query related limits functions (#8689 ) In this PR we're allowing for passing of a `context.Context` via the Limits interfaces (some of which are new, to clean up hardcoding/embedding of `validation.Overrides`) This is based on work/ideas by @jeschkies . Fixes #8694 --------- Signed-off-by: Callum Styan <callumstyan@gmail.com> Co-authored-by: Karsten Jeschkies <karsten.jeschkies@grafana.com>	2 years ago
Kaviraj Kanagaraj	db3a9c961e	distributor: Fix `loki_mutated_bytes_total` metric increment (#8223 ) Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> Previously this metric was not not incremented for all the log lines in the stream. This PR fixes it.	2 years ago
Travis Patterson	a1e9e4cdd0	Revert "Make some stream shard metrics per-tenant (#7838 )" (#7842 ) Because these metrics will be coming from each distributor, this is going to have an adverse impact on cardinality. Consider doing this with logging	3 years ago
Travis Patterson	96fc2abfff	Make some stream shard metrics per-tenant (#7838 ) This PR just adds tenant information so some of our per-stream-rate-limit metrics.	3 years ago
Travis Patterson	c75b822fc6	Track rates per stream and per tenant (#7564 ) It is possible for two tenants to have streams with the same labels. This PR changes the stream tracking in the `shard_tracker`, `stream_rate_calculator`, and `rate_store` to account for tenant ID when tracking streams.	3 years ago
Travis Patterson	a65da7d78e	Shard streams round-robin rather in batches. (#7560 ) To avoid hot spots, keep track of shard numbers across pushes. When tracking shard numbers across pushes, it's no longer necessary to randomize shards, in the case where the desired number of shards is greater than the number of entries. Instead, we just partition the entries among the shards in round robin fashion. Over time, this should give us a more even distribution of logs in shards, particularly for large streams that are made of small individual pushes.	3 years ago
Travis Patterson	0a654e2357	Ensure labels on sharded streams are sorted before they're sent anywhere (#7556 ) Pushing streams with unsorted labels to ingesters results in a bug where ingesters duplicate chunks for streams. This causes a memory leak and eventual OOM crash when ingesters recover from the WAL because one of the duplicate streams is orphaned. This PR ensures that sharded streams have sorted labels to avoid this bug.	3 years ago
Callum Styan	46b40a5582	Hardcode ring.WriteNoExtend for distributor push operations (#7517 ) With zone awareness enabled we allow restarting multiple ingesters within a single zone at the same time, this means that for many more write requests there's potentially only 2 of the 3 replicas for a stream up at the time the write request is received. This lead to a large increase in 500s during rollouts. My suspicion is that the 500s on rollouts was always related to the following; the replica set ring code has something referred to as "extending the replica set". This essentially is the adding of ingesters in the ring that are not in an ACTIVE state (for example, are in LEAVING) to the set of replicas considered valid for an operation. Adding of the instance in a state other than ACTIVE can be controlled by using a different operation type, which Mimir was using but we were not. Here are the relevant functions, [ring.Get](`8d6d914ef6/ring/ring.go (L335-L403)`) (particularly these lines [1](`8d6d914ef6/ring/ring.go (L381-L389)`) and [2](`8d6d914ef6/ring/ring.go (L394-L397)`)) and [replicationStrategy.Filter](`789ec0ca4a/ring/replication_strategy.go (L22-L71)`) (particularly these lines [1](`789ec0ca4a/ring/replication_strategy.go (L32-L36)`) and [2](`789ec0ca4a/ring/replication_strategy.go (L54-L67)`)) The tl; dr of the above code snippets is that we seem to always have required a minSuccess from replicationFactor/2 + 1 except we overwrite replicationFactor to be the # of instances which is likely always going to be more than just 3 ingester replicas because of the replica set extension. At least for zone aware rollouts this seems to be the case. This is because many ingesters will be in the LEAVING state. We can avoid this by just using `WriteNoExtend` operations instead of `Write` operations ([here](`789ec0ca4a/ring/ring.go (L83-L93)`)) In this PR I've hardcoded our call to the ring in `distributor.Push` to always use `ring.WriteNoExtend`. IMO this is the better option to allowing `ring.WriteNoExtend` to be used optionally and defaulting to `ring.Write`. Extending the replication set on writes feels like a footgun for write outages and there's even some feelings internally that allowing that extension is a bug. For more background information see: https://github.com/grafana/mimir/issues/1854 Signed-off-by: Callum Styan <callumstyan@gmail.com> Signed-off-by: Callum Styan <callumstyan@gmail.com>	3 years ago
Dylan Guedes	28e192f22e	Loki: Implement random entries assignment when sharding (#7427 ) What this PR does / why we need it: Modify sharding to assign random shards when the number of shards is bigger than the number of entries. This helps us to avoid hot-spotting the first shards. By randomizing the entries assignment we help spread the load between all shards.	3 years ago
Travis Patterson	8c84ee8b7a	Fix stream sharding (#7393 ) Promtail doesn't add a stream hash to push requests. As a result, we weren't accounting for the current rate in our sharding calculation and only sharding on individual large pushes. This PR ensures that streams have a hash as they're ingested. It also add a couple of metrics that were useful in debugging this issue.	3 years ago
Travis Patterson	7979cfbe07	Implement Distributor Rate Store (#7349 ) The `RateStore` concurrently queries all ingester `GetStreamRate` APIs and stores the results. `RateStore` deduplicates replicas and combines sharded streams so a caller of `RateFor` gets an unsharded view of a stream's rate.	3 years ago
Dylan Guedes	b1d4efab12	Loki: Per-tenant stream sharding (#7311 ) What this PR does / why we need it: - Move stream sharding configuration to its own package to avoid cyclic imports - Change stream sharding to be a per-tenant configuration - Change ingesters to reject whole streams due to rate-limit based on per-tenant stream sharding - Change stream sharding flags prefix from `distributor.shard-stream` to `shard-stream`	3 years ago
Travis Patterson	e3e4e27c8c	Implement ingester API to return stream rates (#7176 ) This PR introduces an API on the Ingester that reports all current streams in memory and their current rate in bytes/second. The reponse from the API contains a slice of `StreamRate` structs: ```go type StreamRate struct { StreamHash uint64 `protobuf:"varint,1,opt,name=streamHash,proto3" json:"streamHash,omitempty"` StreamHashNoShard uint64 `protobuf:"varint,2,opt,name=streamHashNoShard,proto3" json:"streamHashNoShard,omitempty"` Rate int64 `protobuf:"varint,3,opt,name=rate,proto3" json:"rate,omitempty"` } ``` In the interest of space on the caller and in transmission, only hashes and rates are sent. - `StreamHash` will be used to deduplicate replicas - `StreamHashNoShard` will be used to aggregate a sharded stream's total rate. Co-authored-by: Dylan Guedes <djmgguedes@gmail.com>	3 years ago
Dylan Guedes	da7acb49e5	Loki: Add sharding support for negative/zeroed desired rate (#7342 ) What this PR does / why we need it: Adds the handling of negative/zeroed desired rate to our distributor stream sharding. Without this, invalid desired rates results in a -Inf number of shards. This also adds a log line to report that such a thing happened. Signed-off-by: Danny Kopping <danny.kopping@grafana.com> Co-authored-by: Danny Kopping <danny.kopping@grafana.com>	3 years ago
Danny Kopping	c3e92b876c	Restoring lint checks that were previously disabled (#7340 ) I removed these checks in https://github.com/grafana/loki/pull/7337 to not make https://github.com/grafana/loki/pull/7243 overly complex to review. Refactored code to fix valid lint issues, ignored the irrelevant/dangerous ones. I also removed `unused` and `unparam` as these produce low-quality results that are largely useless.	3 years ago
Dylan Guedes	dcfba366de	Loki: Shard streams based on desired rate (#7199 ) What this PR does / why we need it: Modify our sharding mechanism to be dictated by the desired rate and ingestion rate. Once the number of shards is calculated, the stream entries will be distributed across the shards. If the number of entries is lower than number of shards, we report a metric and use number of entries as the number of shards.	3 years ago
Christian Haudum	67b12f2357	Update distributor push tests (#6641 ) Adds a test case for concurrent pushes (which would have spotted a bug in https://github.com/grafana/loki/pull/5248) and a test case for tolerance regarding replication factor. The PR also replaces the mock ring with an actual in-memory kv store ring. And variables and function names were changed to meet Loki/logs terminology. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>	3 years ago
Dylan Guedes	5d8c48bd10	Loki: Implement stream sharding (#6952 ) * introduce 'StreamSharder' interface to decouple sharding from managing shards * make the streamsharder take a whole stream instead of just and ID * Implement stream sharding usage. * move streamShard to distributor * Implement the stream sharder and shard iter (#6941) * Modify sharding to not duplicate all entries for all shards (#6942) * Modify sharding to not duplicate all entries for all shards. * Rename ShardIter and get rid of NextShardId method. * Get rid of ShardIter and ShardStats. * Implement a StreamSharder mock and use it in tests. * Rewrite stubbing and remove flags. * Add auto-splitting. * Only shard stream when approppriate. * Make sharding mode an enum. * Bench shardStream function. * Modify shard comparison to be <= 1. * Add a guard clause for when lowerbound>upperbound. * Apply suggestions from code review Avoid using `fmt.Sprintf`. Co-authored-by: Danny Kopping <danny.kopping@grafana.com> * Update pkg/distributor/distributor.go Rename metric name. Co-authored-by: Danny Kopping <danny.kopping@grafana.com> * Avoid double-negative by Danny's suggestion. * Move TODO to its right place. * Log scenario where upperbound < lowerbound. * Reuse compiled regex. * Add docstring explaining why we use max(shards2, 2). make sharder use a RWMutex * simplify sharding config * replace doc string * Revert "Add auto-splitting" for standalone PR. This reverts commit `e3886a6210`. * shard loop refactor * move shard creation into it's own function * fix after refactor * make config more explicit and document it * reduce allocations * review feedback Co-authored-by: Travis Patterson <travis.patterson@grafana.com> Co-authored-by: Danny Kopping <danny.kopping@grafana.com>	3 years ago
Dylan Guedes	2d7ef07494	Fix `increment_duplicate_timestamps` for multiple dupped timestamps (#6924 )	3 years ago
Sandeep Sukhani	882010a810	Index shipper move code (#6704 ) * move compactor code to generic indexshipper * move table client to generic indexshipper * move storage package and relevant code for managing files on storage to generic indexshipper * add missing files * lint	3 years ago
Dylan Guedes	2bb2a2c0e1	Loki: Remove ring client from the distributor (#6622 ) Removes the ring client from the distributor. - It was originally added on PR #4938 because it was the only ring implementation with HTTPHandler support (used for the ring page). Now that DSKit has support for the other ring implementations (including the lifecycler) this isn't necessary anymore. - It had an IgnoreUnhealthyInstancesReplicationStrategy attached to it, added by mistake	3 years ago
Karen Miller	b90c460769	Rename fudge_duplicate_timestamp to be increment_duplicate_timestamp (#6120 ) * Rename fudge_duplicate_timestamp to be increment_duplicate_timestamp * run `gofmt -d -w pkg/validation/limits.go` Co-authored-by: Christian Simon <simon@swine.de>	3 years ago
Kaviraj Kanagaraj	24f8b19e91	Remove unused grpc health check endpoints (#6113 ) We have generic `grpc_healthcheck` endpoint that get registered for all the components during initialization stage. Looks like these health checks that are part of actual component's struct itself came from legacy. Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com>	3 years ago
Christian Haudum	18058d2b55	Serve correct status of ingester/distributor in gRPC healthcheck endpoint (#6094 ) * Serve correct status of ingester in gRPC healthcheck endpoint The gRCP healthcheck endpoint is used by clients that determin whether a connection to the service is healthy or not. In case the ingester is not in "running" state and is not "active" in the ring, this healthcheck should return "not serving". Signed-off-by: Christian Haudum <christian.haudum@gmail.com> * Serve correct status of distributor in gRPC healthcheck endpoint	3 years ago
Ed Welch	fc8c4f0592	Loki: Add a configurable ability to fudge incoming timestamps for guaranteed query sort order when receiving entries for the same stream that have duplicate timestamps. (#6042 ) * Add a per tenant ability to fudge duplicate timestamps in the distributor Signed-off-by: Edward Welch <edward.welch@grafana.com> * update docs Signed-off-by: Edward Welch <edward.welch@grafana.com> * lint Signed-off-by: Edward Welch <edward.welch@grafana.com> * update changelog Signed-off-by: Edward Welch <edward.welch@grafana.com>	3 years ago
Karsten Jeschkies	46956d0bf6	Use dskit/tenant intead of loki/pkg/tenant. (#5724 )	3 years ago

1 2 3

124 Commits (e9446a93dea5a5f853d756c4255997992177c8f3)