Support bloom compactor and gateway in SSD mode (#11661)

**What this PR does / why we need it**:

This PR fixes adds support for running the bloom compactor and gateway
in SSD mode.

If `legacy-read-mode` is `true` (default):
- `write` runs: Ingester, Distributor.
- `read` runs: QueryFrontend, Querier, QueryScheduler, Ruler, Compactor,
IndexGateway, **BloomGateway**, **BloomCompactor**.


Otherwise:
- `write` runs: Ingester, Distributor.
- `read`: QueryFrontend, Querier.
- `backend`: QueryScheduler, Ruler, Compactor, IndexGateway,
**BloomGateway**, **BloomCompactor**.

I tested this out with a local SSD deployment and seems to be building
blooms fine (when bloom compactor is enabled).
```
$ tree .data/minio/loki-data/bloom -I "*meta" -I "part*" -L 5
.data/minio/loki-data/bloom
└── index_19733
    └── docker
        ├── blooms
        │   ├── 38dad41ad5f1b79-fe294823bd539109
        │   │   ├── 1704976430461-1704978045667-7e33bf85
        │   │   ├── 1704976430461-1704978922898-dd3be57e
        │   │   ├── 1704976430461-1704979829636-33ae0d37
        │   │   ├── 1704976430461-1704980743484-33ae0d37
        │   │   ├── 1704976430461-1704981642000-33ae0d37
        │   │   └── 1704976430461-1704982541351-33ae0d37
        │   ├── 437383945fca166-f5ebd41d2edc508f
        │   │   ├── 1704976430460-1704978044909-b2767d8c
        │   │   ├── 1704976430460-1704978923151-183db34c
        │   │   ├── 1704976430460-1704979833202-183db34c
        │   │   ├── 1704976430460-1704980745263-183db34c
        │   │   ├── 1704976430460-1704981641237-183db34c
        │   │   └── 1704976430460-1704982542614-183db34c
        │   └── 46938d684d3cf87-fd4f06a77900049a
        │       ├── 1704976430447-1704978045918-ba691f3c
        │       ├── 1704976430447-1704978045918-fe558bee
        │       ├── 1704976430447-1704978921878-de5e8dc0
        │       ├── 1704976430447-1704979833964-de5e8dc0
        │       ├── 1704976430447-1704980744753-de5e8dc0
        │       ├── 1704976430447-1704981640725-de5e8dc0
        │       └── 1704976430447-1704982541858-de5e8dc0
        └── metas
            ├── 38dad41ad5f1b79-fe294823bd539109-1704976430461-1704978045667-1d179deb
            ├── 38dad41ad5f1b79-fe294823bd539109-1704976430461-1704978045667-1f3b9273
            ├── 38dad41ad5f1b79-fe294823bd539109-1704976430461-1704978922898-296c7d14
            ├── 38dad41ad5f1b79-fe294823bd539109-1704976430461-1704979829636-a3833f21
            ├── 38dad41ad5f1b79-fe294823bd539109-1704976430461-1704980743484-8abbc022
            ├── 38dad41ad5f1b79-fe294823bd539109-1704976430461-1704981642000-7d80fe65
            ├── 38dad41ad5f1b79-fe294823bd539109-1704976430461-1704982541351-cfcbb8b6
            ├── 437383945fca166-f5ebd41d2edc508f-1704976430460-1704978044909-bdb8aad0
            ├── 437383945fca166-f5ebd41d2edc508f-1704976430460-1704978044909-e0d7b52c
            ├── 437383945fca166-f5ebd41d2edc508f-1704976430460-1704978923151-692d2033
            ├── 437383945fca166-f5ebd41d2edc508f-1704976430460-1704979833202-d0b8c046
            ├── 437383945fca166-f5ebd41d2edc508f-1704976430460-1704980745263-aa0521f
            ├── 437383945fca166-f5ebd41d2edc508f-1704976430460-1704981641237-87636172
            ├── 437383945fca166-f5ebd41d2edc508f-1704976430460-1704982542614-dba1fd47
            ├── 46938d684d3cf87-fd4f06a77900049a-1704976430447-1704978045918-8fb7a27c
            ├── 46938d684d3cf87-fd4f06a77900049a-1704976430447-1704978045918-e031baad
            ├── 46938d684d3cf87-fd4f06a77900049a-1704976430447-1704978921878-635ccc8f
            ├── 46938d684d3cf87-fd4f06a77900049a-1704976430447-1704979833964-376036ef
            ├── 46938d684d3cf87-fd4f06a77900049a-1704976430447-1704980744753-628b5958
            ├── 46938d684d3cf87-fd4f06a77900049a-1704976430447-1704981640725-7cefe675
            └── 46938d684d3cf87-fd4f06a77900049a-1704976430447-1704982541858-1e4de9d4
```

**Which issue(s) this PR fixes**:
Fixes #<issue number>

**Special notes for your reviewer**:

**Checklist**
- [ ] Reviewed the
[`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md)
guide (**required**)
- [ ] Documentation added
- [ ] Tests updated
- [ ] `CHANGELOG.md` updated
- [ ] If the change is worth mentioning in the release notes, add
`add-to-release-notes` label
- [ ] Changes that require user attention or interaction to upgrade are
documented in `docs/sources/setup/upgrade/_index.md`
- [ ] For Helm chart changes bump the Helm chart version in
`production/helm/loki/Chart.yaml` and update
`production/helm/loki/CHANGELOG.md` and
`production/helm/loki/README.md`. [Example
PR](d10549e3ec)
- [ ] If the change is deprecating or removing a configuration option,
update the `deprecated-config.yaml` and `deleted-config.yaml` files
respectively in the `tools/deprecated-config-checker` directory.
[Example
PR](0d4416a4b0)
pull/11587/head
Salva Corts 1 year ago committed by GitHub
parent c9c8692d21
commit 9287c93dd3
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
  1. 7
      pkg/loki/loki.go

@ -647,7 +647,7 @@ func (t *Loki) setupModuleManager() error {
Read: {QueryFrontend, Querier},
Write: {Ingester, Distributor},
Backend: {QueryScheduler, Ruler, Compactor, IndexGateway},
Backend: {QueryScheduler, Ruler, Compactor, IndexGateway, BloomGateway, BloomCompactor},
All: {QueryScheduler, QueryFrontend, Querier, Ingester, Distributor, Ruler, Compactor},
}
@ -694,13 +694,12 @@ func (t *Loki) setupModuleManager() error {
}
// Add bloom gateway ring in client mode to IndexGateway service dependencies if bloom filtering is enabled.
if t.Cfg.isModuleEnabled(IndexGateway) && t.Cfg.BloomGateway.Enabled {
if t.Cfg.BloomGateway.Enabled {
deps[IndexGateway] = append(deps[IndexGateway], BloomGatewayRing)
}
//TODO(poyzannur) not sure this is needed for BloomCompactor
if t.Cfg.LegacyReadTarget {
deps[Read] = append(deps[Read], QueryScheduler, Ruler, Compactor, IndexGateway, BloomGateway, BloomCompactor)
deps[Read] = append(deps[Read], deps[Backend]...)
}
if t.Cfg.InternalServer.Enable {

Loading…
Cancel
Save