* Add docs for updated deletion api
Signed-off-by: Michel Hollands <michel.hollands@grafana.com>
* Fix parameter name
Signed-off-by: Michel Hollands <michel.hollands@grafana.com>
* Clarify that only the delete requests for the current tenant are shown
Signed-off-by: Michel Hollands <michel.hollands@grafana.com>
* Update docs/sources/api/_index.md
Co-authored-by: Christian Simon <simon@swine.de>
* Update docs/sources/operations/storage/logs-deletion.md
Co-authored-by: Sandeep Sukhani <sandeep.d.sukhani@gmail.com>
* Update docs/sources/operations/storage/logs-deletion.md
Co-authored-by: Sandeep Sukhani <sandeep.d.sukhani@gmail.com>
* Update docs/sources/operations/storage/logs-deletion.md
Co-authored-by: Sandeep Sukhani <sandeep.d.sukhani@gmail.com>
* Specify all delete endpoints use authentication
Signed-off-by: Michel Hollands <michel.hollands@grafana.com>
* Add description of delete modes
Signed-off-by: Michel Hollands <michel.hollands@grafana.com>
* Apply suggestions from code review
Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com>
Co-authored-by: Christian Simon <simon@swine.de>
Co-authored-by: Sandeep Sukhani <sandeep.d.sukhani@gmail.com>
Co-authored-by: Karen Miller <84039272+KMiller-Grafana@users.noreply.github.com>
A [list of clients](../clients) can be found in the clients documentation.
@ -799,10 +802,6 @@ In microservices mode, the `/ingester/flush_shutdown` endpoint is exposed by the
Displays a web page with the distributor hash ring status, including the state, healthy and last heartbeat time of each distributor.
### `GET /compactor/ring`
Displays a web page with the compactor hash ring status, including the state, healthy and last heartbeat time of each compactor.
## `GET /metrics`
`/metrics` exposes Prometheus metrics. See
@ -1109,3 +1108,21 @@ GET /prometheus/api/v1/alerts
Prometheus-compatible rules endpoint to list all active alerts.
For more information, please check out the Prometheus [alerts](https://prometheus.io/docs/prometheus/latest/querying/api/#alerts) documentation.
## Compactor
### `GET /compactor/ring`
Displays a web page with the compactor hash ring status, including the state, health, and last heartbeat time of each compactor.
### `POST /loki/api/v1/delete`
Create a new delete request for the authenticated tenant. More details can be found in the [logs deletion documentation](../operations/storage/logs-deletion.md#request-log-entry-deletion).
### `GET /loki/api/v1/delete`
List the existing delete requests for the authenticated tenant. More details can be found in the [logs deletion documentation](../operations/storage/logs-deletion.md#list-delete-requests).
### `DELETE /loki/api/v1/delete`
Remove a delete request for the authenticated tenant. More details can be found in the [logs deletion documentation](../operations/storage/logs-deletion.md#request-cancellation-of-a-delete-request).
<spanstyle="background-color:#f3f973;">Log entry deletion is experimental. It is only supported for the BoltDB Shipper index store.</span>
Log entry deletion is supported _only_ for the BoltDB Shipper index store.
Grafana Loki supports the deletion of log entries from a specified stream.
Log entries that fall within a specified time window and match an optional line filter are those that will be deleted.
Grafana Loki supports the deletion of log entries from specified streams.
Log entries that fall within a specified time window are those that will be deleted.
The Compactor component exposes REST endpoints that process delete requests.
Hitting the endpoint specifies the streams and the time window.
@ -17,7 +18,12 @@ Log entry deletion relies on configuration of the custom logs retention workflow
## Configuration
Enable log entry deletion by setting `retention_enabled` to true in the Compactor's configuration. See the example in [Retention Configuration](../retention#retention-configuration).
Enable log entry deletion by setting `retention_enabled` to true and `deletion_enabled` to `whole-stream-deletion`, `filter-only`, or `filter-and-delete` in the compactor's configuration. See the example in [Retention configuration](../retention#retention-configuration).
With `whole-stream-deletion`, all the log entries matching the query given in the delete request are removed.
With `filter-only`, log lines matching the query in the delete request are filtered out when querying Loki. They are not removed from the on-disk chunks.
With `filter-and-delete`, log lines matching the query in the delete request are filtered out when querying Loki, and they are also removed from the on-disk chunks.
A delete request may be canceled within a configurable cancellation period. Set the `delete_request_cancel_period` in the Compactor's YAML configuration or on the command line when invoking Loki. Its default value is 24h.
@ -28,24 +34,26 @@ The Compactor exposes endpoints to allow for the deletion of log entries from sp
### Request log entry deletion
```
POST /loki/api/admin/delete
PUT /loki/api/admin/delete
POST /loki/api/v1/delete
PUT /loki/api/v1/delete
```
Query parameters:
* `match[]=<series_selector>`: Repeated label matcher argument that identifies the streams from which to delete. At least one `match[]` argument must be provided.
* `query=<series_selector>`: query argument that identifies the streams from which to delete with optional line filters.
* `start=<rfc3339 | unix_timestamp>`: A timestamp that identifies the start of the time window within which entries will be deleted. If not specified, defaults to 0, the Unix Epoch time.
* `end=<rfc3339 | unix_timestamp>`: A timestamp that identifies the end of the time window within which entries will be deleted. If not specified, defaults to the current time.
A 204 response indicates success.
URL encode the `match[]` parameter. This sample form of a cURL command URL encodes `match[]={foo="bar"}`:
URL encode the `query` parameter. This sample form of a cURL command URL encodes `query={foo="bar"}`:
The query parameter can also include filter operations. For example `query={foo="bar"} |= "other"` will filter out lines that contain the string "other" for the streams matching the stream selector `{foo="bar"}`.
```
### List delete requests
@ -53,14 +61,14 @@ curl -g -X POST \
List the existing delete requests using the following API:
```
GET /loki/api/admin/delete
GET /loki/api/v1/delete
```
Sample form of a cURL command:
```
curl -X GET \
<compactor_addr>/loki/api/admin/delete \
<compactor_addr>/loki/api/v1/delete \
-H 'x-scope-orgid: <orgid>'
```
@ -73,8 +81,7 @@ Loki allows cancellation of delete requests until the requests are picked up for
Cancel a delete request using this Compactor endpoint:
```
POST /loki/api/admin/cancel_delete_request
PUT /loki/api/admin/cancel_delete_request
DELETE /loki/api/v1/delete
```
Query parameters:
@ -86,7 +93,7 @@ A 204 response indicates success.
Michel Hollands
4 years ago
committed by