mirror of https://github.com/grafana/grafana
Unified Storage: Adds back readme to resource store (#91812)
adds back readme to resource storepull/91819/head
parent
5e638b4af0
commit
8fb334cf77
@ -0,0 +1,205 @@ |
||||
|
||||
# Unified Storage |
||||
|
||||
The unified storage projects aims to provide a simple and extensible backend to unify the way we store different objects within the Grafana app platform. |
||||
|
||||
It provides generic storage for k8s objects, and can store data either within dedicated tables in the main Grafana database, or in separate storage. |
||||
|
||||
By default it runs in-process within Grafana, but it can also be run as a standalone GRPC service (`storage-server`). |
||||
|
||||
## Storage Overview |
||||
|
||||
There are 2 main tables, the `resource` table stores a "current" view of the objects, and the `resource_history` table stores a record of each revision of a given object. |
||||
|
||||
## Running Unified Storage |
||||
|
||||
### Baseline configuration |
||||
|
||||
The minimum config settings required are: |
||||
|
||||
```ini |
||||
; need to specify target here for override to work later |
||||
target = all |
||||
|
||||
[server] |
||||
; https is required for kubectl |
||||
protocol = https |
||||
|
||||
[feature_toggles] |
||||
; enable unified storage |
||||
unifiedStorage = true |
||||
; enable k8s apiserver |
||||
grafanaAPIServer = true |
||||
; store playlists in k8s |
||||
kubernetesPlaylists = true |
||||
; store json id token in context |
||||
idForwarding = true |
||||
|
||||
[grafana-apiserver] |
||||
; use unified storage for k8s apiserver |
||||
storage_type = unified |
||||
``` |
||||
|
||||
With this configuration, you can run everything in-process. Run the Grafana backend with: |
||||
|
||||
```sh |
||||
bra run |
||||
``` |
||||
|
||||
or |
||||
|
||||
```sh |
||||
make run |
||||
``` |
||||
|
||||
The default kubeconfig sends requests directly to the apiserver, to authenticate as a grafana user, create `grafana.kubeconfig`: |
||||
```yaml |
||||
apiVersion: v1 |
||||
clusters: |
||||
- cluster: |
||||
insecure-skip-tls-verify: true |
||||
server: https://127.0.0.1:3000 |
||||
name: default-cluster |
||||
contexts: |
||||
- context: |
||||
cluster: default-cluster |
||||
namespace: default |
||||
user: default |
||||
name: default-context |
||||
current-context: default-context |
||||
kind: Config |
||||
preferences: {} |
||||
users: |
||||
- name: default |
||||
user: |
||||
username: <username> |
||||
password: <password> |
||||
``` |
||||
Where `<username>` and `<password>` are credentials for basic auth against Grafana. For example, with the [default credentials](https://github.com/grafana/grafana/blob/HEAD/contribute/developer-guide.md#backend): |
||||
```yaml |
||||
username: admin |
||||
password: admin |
||||
``` |
||||
|
||||
In this mode, you can interact with the k8s api. Make sure you are in the directory where you created `grafana.kubeconfig`. Then run: |
||||
```sh |
||||
kubectl --kubeconfig=./grafana.kubeconfig get playlist |
||||
``` |
||||
|
||||
If this is your first time running the command, a successful response would be: |
||||
```sh |
||||
No resources found in default namespace. |
||||
``` |
||||
|
||||
To create a playlist, create a file `playlist-generate.yaml`: |
||||
```yaml |
||||
apiVersion: playlist.grafana.app/v0alpha1 |
||||
kind: Playlist |
||||
metadata: |
||||
generateName: x # anything is ok here... except yes or true -- they become boolean! |
||||
labels: |
||||
foo: bar |
||||
annotations: |
||||
grafana.app/slug: "slugger" |
||||
grafana.app/updatedBy: "updater" |
||||
spec: |
||||
title: Playlist with auto generated UID |
||||
interval: 5m |
||||
items: |
||||
- type: dashboard_by_tag |
||||
value: panel-tests |
||||
- type: dashboard_by_uid |
||||
value: vmie2cmWz # dashboard from devenv |
||||
``` |
||||
then run: |
||||
```sh |
||||
kubectl --kubeconfig=./grafana.kubeconfig create -f playlist-generate.yaml |
||||
``` |
||||
|
||||
For example, a successful response would be: |
||||
```sh |
||||
playlist.playlist.grafana.app/u394j4d3-s63j-2d74-g8hf-958773jtybf2 created |
||||
``` |
||||
|
||||
When running |
||||
```sh |
||||
kubectl --kubeconfig=./grafana.kubeconfig get playlist |
||||
``` |
||||
you should now see something like: |
||||
```sh |
||||
NAME TITLE INTERVAL CREATED AT |
||||
u394j4d3-s63j-2d74-g8hf-958773jtybf2 Playlist with auto generated UID 5m 2023-12-14T13:53:35Z |
||||
``` |
||||
|
||||
To update the playlist, update the `playlist-generate.yaml` file then run: |
||||
```sh |
||||
kubectl --kubeconfig=./grafana.kubeconfig patch playlist <NAME> --patch-file playlist-generate.yaml |
||||
``` |
||||
|
||||
In the example, `<NAME>` would be `u394j4d3-s63j-2d74-g8hf-958773jtybf2`. |
||||
|
||||
### Use a separate database |
||||
|
||||
By default Unified Storage uses the Grafana database. To run against a separate database, update `custom.ini` by adding the following section to it: |
||||
|
||||
``` |
||||
[resource_api] |
||||
db_type = mysql |
||||
db_host = localhost:3306 |
||||
db_name = grafana |
||||
db_user = <username> |
||||
db_pass = <password> |
||||
``` |
||||
|
||||
MySQL and Postgres are both supported. The `<username>` and `<password>` values can be found in the following devenv docker compose files: [MySQL](https://github.com/grafana/grafana/blob/main/devenv/docker/blocks/mysql/docker-compose.yaml#L6-L7) and [Postgres](https://github.com/grafana/grafana/blob/main/devenv/docker/blocks/postgres/docker-compose.yaml#L4-L5). |
||||
|
||||
Then, run |
||||
```sh |
||||
make devenv sources=<source> |
||||
``` |
||||
where source is either `mysql` or `postgres`. |
||||
|
||||
Finally, run the Grafana backend with |
||||
|
||||
```sh |
||||
bra run |
||||
``` |
||||
or |
||||
```sh |
||||
make run |
||||
``` |
||||
|
||||
### Run as a GRPC service |
||||
|
||||
#### Start GRPC storage-server |
||||
|
||||
This currently only works with a separate database configuration (see previous section). |
||||
|
||||
Start the storage-server with: |
||||
```sh |
||||
GF_DEFAULT_TARGET=storage-server ./bin/grafana server target |
||||
``` |
||||
|
||||
The GRPC service will listen on port 10000 |
||||
|
||||
#### Use GRPC server |
||||
|
||||
To run grafana against the storage-server, override the `storage_type` setting: |
||||
```sh |
||||
GF_GRAFANA_APISERVER_STORAGE_TYPE=unified-grpc ./bin/grafana server |
||||
``` |
||||
|
||||
You can then list the previously-created playlists with: |
||||
```sh |
||||
kubectl --kubeconfig=./grafana.kubeconfig get playlist |
||||
``` |
||||
|
||||
## Changing protobuf interface |
||||
|
||||
- install [protoc](https://grpc.io/docs/protoc-installation/) |
||||
- install the protocol compiler plugin for Go |
||||
```sh |
||||
go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@latest |
||||
``` |
||||
- make changes in `.proto` file |
||||
- to compile all protobuf files in the repository run `make protobuf` at its top level |
||||
Loading…
Reference in new issue