Like Prometheus, but for logs.

prometheushacktoberfestgrafanacloudnativeloggingloki

You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

Salva Corts 4248825ad2 Bloom-compactor Sharding (#11154) ... **What this PR does / why we need it**: This PR adds tenant and fingerprint (FP) sharding to bloom compactors. Note that the bloom-compactor doesn't yet perform any compaction, but iterates through all tables, tenants, and series checking if the compactor owns the tenant and the series (by the series FP). Actual compaction will be implemented with https://github.com/grafana/loki/pull/11115. A new structure `Job` is added which will carry around all the context for a compaction job such as the tenant ID, the table name, and the series FP. The sharding strategy has two methods: - `OwnsTenant(tenant string)`: Checks if the compactor shard owns the tenant. - `OwnsJob(job Job)`: Checks (again) if the compactor owns the job's tenant. Then, it checks if the compactor owns the job's fingerprint by looking inside the tenant subring. We add a new per-tenant limit: `bloom_compactor_shard_size`. If it's 0, the tenant can use all compactors (i.e. `OwnsTenant` will always return `true`), otherwise, only `bloom_compactor_shard_size` out of the total number of compactors will own the tenant. A given job's FP will be owned by exactly one compactor within the tenant shard. **Special notes for your reviewer**: - Added a bunch of metrics in `metrics.go` - Added a test for the sharding strategy		2 years ago
..
exporter.go	Allow passing of context to query related limits functions (#8689)	3 years ago
exporter_test.go	only exports tenant limits which differ from defaults and export defa… (#4542)	4 years ago
limits.go	Bloom-compactor Sharding (#11154)	2 years ago
limits_test.go	chore: restructure compactor package (#10775)	2 years ago
rate.go	Refactor per stream rate limit (#4213)	4 years ago
validate.go	Use metrics namespace for more metrics (#11025)	2 years ago