Like Prometheus, but for logs.

prometheushacktoberfestgrafanacloudnativeloggingloki

You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

Salva Corts 4248825ad2 Bloom-compactor Sharding (#11154) ... **What this PR does / why we need it**: This PR adds tenant and fingerprint (FP) sharding to bloom compactors. Note that the bloom-compactor doesn't yet perform any compaction, but iterates through all tables, tenants, and series checking if the compactor owns the tenant and the series (by the series FP). Actual compaction will be implemented with https://github.com/grafana/loki/pull/11115. A new structure `Job` is added which will carry around all the context for a compaction job such as the tenant ID, the table name, and the series FP. The sharding strategy has two methods: - `OwnsTenant(tenant string)`: Checks if the compactor shard owns the tenant. - `OwnsJob(job Job)`: Checks (again) if the compactor owns the job's tenant. Then, it checks if the compactor owns the job's fingerprint by looking inside the tenant subring. We add a new per-tenant limit: `bloom_compactor_shard_size`. If it's 0, the tenant can use all compactors (i.e. `OwnsTenant` will always return `true`), otherwise, only `bloom_compactor_shard_size` out of the total number of compactors will own the tenant. A given job's FP will be owned by exactly one compactor within the tenant shard. **Special notes for your reviewer**: - Added a bunch of metrics in `metrics.go` - Added a test for the sharding strategy		2 years ago
..
ring.go	Unify ring managers across components (#10931)	2 years ago
ring_config.go	fix: All lifecycler cfgs ref a valid IPv6 addr and port combination (#11121)	2 years ago
ring_test.go	Unify ring managers across components (#10931)	2 years ago
ringmanager.go	Unify ring managers across components (#10931)	2 years ago
sharding.go	Bloom-compactor Sharding (#11154)	2 years ago