Like Prometheus, but for logs.

prometheushacktoberfestgrafanacloudnativeloggingloki

You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

afayngelerindbx 36109eb09a compactor: prefetch existing compacted user index files in parallel (#6803) ... **What this PR does / why we need it**: Before this change, these files are fetched while [holding a table global lock](a1e75da970/pkg/storage/stores/shipper/index/compactor/table_compactor.go (L195)), serializing fetches. I chose the approach of prefetching index files over attempting to change the locking pattern to allow parallel, on-demand fetches because the complexity of parallelizing on-demand fetches outweighed the benefits of avoiding extra fetches. More specifically, this patch trades off fetching idle, non-compactable index files in favor of avoiding blocking during compactions. The extra fetches are parallelized and so should incur a negligible wall clock penalty even for large numbers of tenants(e.g. 1k idle tenants at 50 threads and 500ms to fetch is ~10s). **Which issue(s) this PR fixes**: This should help with #6775, without completely resolving it. **Checklist** This change does not introduce any settings, metrics etc. It should be transparent from a user perspective. Therefore, I don't believe it merits documentation/changelog updates. - [x] Tests updated		4 years ago
..
index	compactor: prefetch existing compacted user index files in parallel (#6803)	4 years ago
indexgateway	[TSDB] Indexgateway Protobuf refactor (#6841)	4 years ago
testutil	generic index compactor (#6537)	4 years ago
util	Index shipper move code (#6704)	4 years ago
metrics.go	create smaller unique files from boltdb shipper and other code refactorings (#2261)	6 years ago
shipper_index_client.go	make index shipper read path handle range of tables by type of index (#6304)	4 years ago