**What this PR does / why we need it**:
This adds several new template functions to the logql to be used in
`line_format` and `label_format`
- bytes
- duration (and duration_seconds)
- unixEpochMillis
- unixEpochNanos
- toDateInZone
- b64Enc and b64Dec
`bytes` and `duration` adds parity with the `unwrap` pipeline stage in
logql, allowing you to convert human readable versions of bytes and
times into their respective primitives of bytes and seconds. This makes
it possible to do math and comparisons of these extracted values in
`line_format` and `label_format`
`unixEpochMillis` and `unixEpochNanos` extends the existing `unixEpoch`
function which can only generate seconds level precision to allow having
millisecond and nanosecond level precision.
`toDateInZone` allows for specifying a specific timezone when parsing a
date.
`b64Enc` and `b64Dec` allow base64 encoding and decoding values.
**Which issue(s) this PR fixes**:
Fixes #<issue number>
**Special notes for your reviewer**:
@JStickler I made a lot of changes to the template_functions.md doc page
to try and create some more consistency as well as remove some notices
that really aren't relevant anymore:
* Tried to make sure every function had a `Signature` and `Examples`
block
* Removed a lot of `NOTE` about functions being introduced in a specific
version
* Removed documentation for some deprecated string functions (they still
exist in code but I removed the docs for them, their exist better
replacements for all of them)
**Checklist**
- [x] Reviewed the
[`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md)
guide (**required**)
- [x] Documentation added
- [x] Tests updated
- [ ] `CHANGELOG.md` updated
- [ ] Changes that require user attention or interaction to upgrade are
documented in `docs/sources/upgrading/_index.md`
---------
Signed-off-by: Edward Welch <edward.welch@grafana.com>
* [7906](https://github.com/grafana/loki/pull/7906) **kavirajk**: Add API endpoint that formats LogQL expressions and support new `fmt` subcommand in `logcli` to format LogQL query.
* [6675](https://github.com/grafana/loki/pull/6675) **btaani**: Add logfmt expression parser for selective extraction of labels from logfmt formatted logs
* [8474](https://github.com/grafana/loki/pull/8474) **farodin91**: Add support for short-lived S3 session tokens
* [8774](https://github.com/grafana/loki/pull/8774) **slim-bean**: Add new logql template functions `bytes`, `duration`, `unixEpochMillis`, `unixEpochNanos`, `toDateInZone`, `b64Enc`, and `b64Dec`
This function returns the current log lines timestamp.
Signature:
`timestamp() time.Time`
Signature: `timestamp() time.Time`
```template
"{{ __timestamp__ }}"
@ -56,110 +52,29 @@ Signature:
See the blog: [Parsing and formatting date/time in Go](https://www.pauladamsmith.com/blog/2011/05/go_time.html) for more information.
## ToLower and ToUpper
This function converts the entire string to lowercase or uppercase.
Signatures:
- `ToLower(string) string`
- `ToUpper(string) string`
Examples:
```template
"{{.request_method | ToLower}}"
"{{.request_method | ToUpper}}"
`{{ToUpper "This is a string" | ToLower}}`
```
> **Note:** In Grafana Loki 2.1 you can also use respectively [`lower`](#lower) and [`upper`](#upper) shortcut, e.g `{{.request_method | lower }}`.
## Replace string
> **Note:** In Loki 2.1 [`replace`](#replace) (as opposed to `Replace`) is available with a different signature but easier to chain within pipeline.
Use this function to perform a simple string replacement.
Signature:
`Replace(s, old, new string, n int) string`
It takes four arguments:
- `s` source string
- `old` string to replace
- `new` string to replace with
- `n` the maximun amount of replacement (-1 for all)
Example:
```template
`{{ Replace "This is a string" " " "-" -1 }}`
```
The results in `This-is-a-string`.
## Trim, TrimLeft, TrimRight, and TrimSpace
> **Note:** In Loki 2.1 [trim](#trim), [trimAll](#trimall), [trimSuffix](#trimsuffix) and [trimPrefix](#trimprefix) have been added with a different signature for better pipeline chaining.
`Trim` returns a slice of the string s with all leading and
trailing Unicode code points contained in cutset removed.
Signature: `Trim(value, cutset string) string`
`TrimLeft` and `TrimRight` are the same as `Trim` except that it trims only leading and trailing characters respectively.
```template
`{{ Trim .query ",. " }}`
`{{ TrimLeft .uri ":" }}`
`{{ TrimRight .path "/" }}`
```
`TrimSpace` TrimSpace returns string s with all leading
and trailing white space removed, as defined by Unicode.
Signature: `TrimSpace(value string) string`
```template
{{ TrimSpace .latency }}
```
`TrimPrefix` and `TrimSuffix` will trim respectively the prefix or suffix supplied.
`regexReplaceAll` returns a copy of the input string, replacing matches of the Regexp with the replacement string replacement. Inside string replacement, $ signs are interpreted as in Expand, so for instance $1 represents the text of the first sub-match. See the golang [Regexp.replaceAll documentation](https://golang.org/pkg/regexp/#Regexp.ReplaceAll) for more examples.
`regexReplaceAllLiteral` function returns a copy of the input string and replaces matches of the Regexp with the replacement string replacement. The replacement string is substituted directly, without using Expand.
`date` returns a textual representation of the time value formatted according to the provided [golang datetime layout](https://pkg.go.dev/time#pkg-constants).
Returns a textual representation of the time value formatted according to the provided [golang datetime layout](https://pkg.go.dev/time#pkg-constants).
Signature: `date(fmt string, date interface{}) string`
Example:
```template
{{ date "2006-01-02" now }}
@ -670,10 +593,15 @@ Example of a query to print a newline per queries stored as a json array in the
## unixEpoch
`unixEpoch` returns the number of seconds elapsed since January 1, 1970 UTC.
Returns the number of seconds elapsed since January 1, 1970 UTC.
Ed Welch
2 years ago
committed by