mirror of https://github.com/grafana/grafana
prometheushacktoberfestmetricsmonitoringalertinggrafanagoinfluxdbmysqlpostgresanalyticsdata-visualizationdashboardbusiness-intelligenceelasticsearch
You can not select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
160 lines
5.3 KiB
160 lines
5.3 KiB
|
4 years ago
|
---
|
||
|
|
title: Build a logs data source plugin
|
||
|
2 years ago
|
description: How to build a logs data source plugin.
|
||
|
|
aliases:
|
||
|
|
- ../../../plugins/build-a-logs-data-source-plugin/
|
||
|
|
keywords:
|
||
|
|
- grafana
|
||
|
|
- plugins
|
||
|
|
- plugin
|
||
|
|
- logs
|
||
|
|
- logs data source
|
||
|
|
- datasource
|
||
|
|
weight: 500
|
||
|
4 years ago
|
---
|
||
|
6 years ago
|
|
||
|
6 years ago
|
# Build a logs data source plugin
|
||
|
6 years ago
|
|
||
|
3 years ago
|
Grafana data source plugins support metrics, logs, and other data types. The steps to build a logs data source plugin are largely the same as for a metrics data source, but there are a few differences which we will explain in this guide.
|
||
|
6 years ago
|
|
||
|
3 years ago
|
## Before you begin
|
||
|
|
|
||
|
2 years ago
|
This guide assumes that you're already familiar with how to [Build a data source plugin]({{< relref "./build-a-data-source-plugin" >}}) for metrics. We recommend that you review this material before continuing.
|
||
|
6 years ago
|
|
||
|
|
## Add logs support to your data source
|
||
|
|
|
||
|
|
To add logs support to an existing data source, you need to:
|
||
|
|
|
||
|
3 years ago
|
1. Enable logs support
|
||
|
|
1. Construct the log data
|
||
|
|
|
||
|
|
When these steps are done, then you can improve the user experience with one or more [optional features](#enhance-your-logs-data-source-plugin-with-optional-features).
|
||
|
6 years ago
|
|
||
|
3 years ago
|
### Step 1: Enable logs support
|
||
|
6 years ago
|
|
||
|
2 years ago
|
Tell Grafana that your data source plugin can return log data, by adding `"logs": true` to the [plugin.json]({{< relref "../../metadata.md" >}}) file.
|
||
|
6 years ago
|
|
||
|
|
```json
|
||
|
|
{
|
||
|
|
"logs": true
|
||
|
|
}
|
||
|
|
```
|
||
|
|
|
||
|
3 years ago
|
### Step 2: Construct the log data
|
||
|
6 years ago
|
|
||
|
2 years ago
|
As it does with metrics data, Grafana expects your plugin to return log data as a [data frame]({{< relref "../../introduction-to-plugin-development/data-frames.md" >}}).
|
||
|
6 years ago
|
|
||
|
|
To return log data, return a data frame with at least one time field and one text field from the data source's `query` method.
|
||
|
|
|
||
|
|
**Example:**
|
||
|
|
|
||
|
|
```ts
|
||
|
|
const frame = new MutableDataFrame({
|
||
|
|
refId: query.refId,
|
||
|
|
fields: [
|
||
|
|
{ name: 'time', type: FieldType.time },
|
||
|
|
{ name: 'content', type: FieldType.string },
|
||
|
|
],
|
||
|
|
});
|
||
|
|
|
||
|
|
frame.add({ time: 1589189388597, content: 'user registered' });
|
||
|
|
frame.add({ time: 1589189406480, content: 'user logged in' });
|
||
|
|
```
|
||
|
|
|
||
|
2 years ago
|
That's all you need to start returning log data from your data source. Go ahead and try it out in [Explore]({{< relref "../../../../explore" >}}) or by adding a [Logs panel]({{< relref "../../../../panels-visualizations/visualizations/logs" >}}).
|
||
|
6 years ago
|
|
||
|
|
Congratulations, you just wrote your first logs data source plugin! Next, let's look at a couple of features that can further improve the experience for the user.
|
||
|
|
|
||
|
3 years ago
|
## Enhance your logs data source plugin with optional features
|
||
|
|
|
||
|
|
Add visualization type hints, labels, and other optional features to logs.
|
||
|
|
|
||
|
|
### Add a preferred visualization type hint to the data frame
|
||
|
5 years ago
|
|
||
|
2 years ago
|
To make sure Grafana recognizes data as logs and shows logs visualization automatically in Explore, set `meta.preferredVisualisationType` to `'logs'` in the returned data frame. See [Selecting preferred visualization section]({{< relref "../extend-a-plugin/add-support-for-explore-queries#select-a-preferred-visualization-type" >}})
|
||
|
5 years ago
|
|
||
|
|
**Example:**
|
||
|
|
|
||
|
|
```ts
|
||
|
|
const frame = new MutableDataFrame({
|
||
|
|
refId: query.refId,
|
||
|
|
meta: {
|
||
|
|
preferredVisualisationType: 'logs',
|
||
|
|
},
|
||
|
|
fields: [
|
||
|
|
{ name: 'time', type: FieldType.time },
|
||
|
|
{ name: 'content', type: FieldType.string },
|
||
|
|
],
|
||
|
|
});
|
||
|
|
```
|
||
|
|
|
||
|
3 years ago
|
### Add labels to your logs
|
||
|
6 years ago
|
|
||
|
3 years ago
|
Many log systems let you query logs based on metadata, or _labels_, to help filter log lines.
|
||
|
6 years ago
|
|
||
|
3 years ago
|
Add labels to a stream of logs by setting the `labels` property on the Field.
|
||
|
6 years ago
|
|
||
|
|
**Example**:
|
||
|
|
|
||
|
|
```ts
|
||
|
|
const frame = new MutableDataFrame({
|
||
|
|
refId: query.refId,
|
||
|
|
fields: [
|
||
|
|
{ name: 'time', type: FieldType.time },
|
||
|
|
{ name: 'content', type: FieldType.string, labels: { filename: 'file.txt' } },
|
||
|
|
],
|
||
|
|
});
|
||
|
|
|
||
|
|
frame.add({ time: 1589189388597, content: 'user registered' });
|
||
|
|
frame.add({ time: 1589189406480, content: 'user logged in' });
|
||
|
|
```
|
||
|
|
|
||
|
3 years ago
|
### Extract detected fields from your logs
|
||
|
6 years ago
|
|
||
|
3 years ago
|
Add additional information about each log line by supplying more data frame fields.
|
||
|
6 years ago
|
|
||
|
2 years ago
|
If a data frame has more than one text field, then Grafana assumes the first field in the data frame to be the actual log line. Grafana treats subsequent text fields as detected fields.
|
||
|
6 years ago
|
|
||
|
3 years ago
|
Any number of custom fields can be added to your data frame; Grafana comes with two dedicated fields: `levels` and `id`.
|
||
|
6 years ago
|
|
||
|
3 years ago
|
#### Levels
|
||
|
6 years ago
|
|
||
|
|
To set the level for each log line, add a `level` field.
|
||
|
|
|
||
|
|
**Example:**
|
||
|
|
|
||
|
|
```ts
|
||
|
|
const frame = new MutableDataFrame({
|
||
|
|
refId: query.refId,
|
||
|
|
fields: [
|
||
|
|
{ name: 'time', type: FieldType.time },
|
||
|
|
{ name: 'content', type: FieldType.string, labels: { filename: 'file.txt' } },
|
||
|
|
{ name: 'level', type: FieldType.string },
|
||
|
|
],
|
||
|
|
});
|
||
|
|
|
||
|
|
frame.add({ time: 1589189388597, content: 'user registered', level: 'info' });
|
||
|
|
frame.add({ time: 1589189406480, content: 'unknown error', level: 'error' });
|
||
|
|
```
|
||
|
|
|
||
|
3 years ago
|
#### 'id' for assigning unique identifiers to log lines
|
||
|
6 years ago
|
|
||
|
|
By default, Grafana offers basic support for deduplicating log lines. You can improve the support by adding an `id` field to explicitly assign identifiers to each log line.
|
||
|
|
|
||
|
|
**Example:**
|
||
|
|
|
||
|
|
```ts
|
||
|
|
const frame = new MutableDataFrame({
|
||
|
|
refId: query.refId,
|
||
|
|
fields: [
|
||
|
|
{ name: 'time', type: FieldType.time },
|
||
|
|
{ name: 'content', type: FieldType.string, labels: { filename: 'file.txt' } },
|
||
|
|
{ name: 'level', type: FieldType.string },
|
||
|
|
{ name: 'id', type: FieldType.string },
|
||
|
|
],
|
||
|
|
});
|
||
|
|
|
||
|
|
frame.add({ time: 1589189388597, content: 'user registered', level: 'info', id: 'd3b07384d113edec49eaa6238ad5ff00' });
|
||
|
|
frame.add({ time: 1589189406480, content: 'unknown error', level: 'error', id: 'c157a79031e1c40f85931829bc5fc552' });
|
||
|
|
```
|