streams. This is helpful to find high cardinality labels.
```
### LogCLI query command reference
### `query` command reference
The output of `logcli help query`:
@ -263,13 +263,14 @@ The "query" command is useful for querying for logs. Logs can be returned in a f
The output of the log can be specified with the "-o" flag, for example, "-o raw" for the raw output format.
The "query" command will output extra information about the query and its results, such as the API URL, set of common labels, and set of excluded labels. This extra information can be
suppressed with the --quiet flag.
The "query" command will output extra information about the query and its results, such as the API URL, set of common labels, and set of
excluded labels. This extra information can be suppressed with the --quiet flag.
By default we look over the last hour of data; use --since to modify or provide specific start and end times with --from and --to respectively.
By default we look over the last hour of data; use --since to modify or provide specific start and end times with --from and --to
respectively.
Notice that when using --from and --to then ensure to use RFC3339Nano time format, but without timezone at the end. The local timezone will be added automatically or if using --timezone
flag.
Notice that when using --from and --to then ensure to use RFC3339Nano time format, but without timezone at the end. The local timezone will be
added automatically or if using --timezone flag.
Example:
@ -282,9 +283,9 @@ Example:
The output is limited to 30 entries by default; use --limit to increase.
While "query" does support metrics queries, its output contains multiple data points between the start and end query time. This output is used to build graphs, similar to what is seen
in the Grafana Explore graph view. If you are querying metrics and just want the most recent data point (like what is seen in the Grafana Explore table view), then you should use the
"instant-query" command instead.
While "query" does support metrics queries, its output contains multiple data points between the start and end query time. This output is used
to build graphs, similar to what is seen in the Grafana Explore graph view. If you are querying metrics and just want the most recent data
point (like what is seen in the Grafana Explore table view), then you should use the "instant-query" command instead.
Parallelization:
@ -312,26 +313,29 @@ Example:
--merge-parts
'my-query'
This example will create a queue of jobs to execute, each being 15 minutes in duration. In this case, that means, for the 10-hour total duration, there will be forty 15-minute jobs.
The --limit flag is ignored.
This example will create a queue of jobs to execute, each being 15 minutes in duration. In this case, that means, for the 10-hour total
duration, there will be forty 15-minute jobs. The --limit flag is ignored.
It will start four workers, and they will each take a job to work on from the queue until all the jobs have been completed.
Each job will save a "part" file to the location specified by the --part-path-prefix. Different prefixes can be used to run multiple queries at the same time. The timestamp of the start and
end of the part is in the file name. While the part is being downloaded, the filename will end in ".part", when it is complete, the file will be renamed to remove this ".part" extension.
By default, if a completed part file is found, that part will not be downloaded again. This can be overridden with the --overwrite-completed-parts flag.
Each job will save a "part" file to the location specified by the --part-path-prefix. Different prefixes can be used to run multiple queries
at the same time. The timestamp of the start and end of the part is in the file name. While the part is being downloaded, the filename will
end in ".part", when it is complete, the file will be renamed to remove this ".part" extension. By default, if a completed part file is found,
that part will not be downloaded again. This can be overridden with the `--overwrite-completed-parts` flag.
Part file example using the previous command, adding --keep-parts so they are not deleted:
Since we don't have the --forward flag, the parts will be downloaded in reverse. Two of the workers have finished their jobs (last two files), and have picked up the next jobs in the queue.
Running ls, this is what we should expect to see.
Since we don't have the --forward flag, the parts will be downloaded in reverse. Two of the workers have finished their jobs (last two files),
and have picked up the next jobs in the queue. Running ls, this is what we should expect to see.
$ ls -1 /tmp/my_query* /tmp/my_query_20210119T183000_20210119T184500.part.tmp /tmp/my_query_20210119T184500_20210119T190000.part.tmp /tmp/my_query_20210119T190000_20210119T191500.part.tmp
If you do not specify the --merge-parts flag, the part files will be downloaded, and logcli will exit, and you can process the files as you wish. With the flag specified, the part files
will be read in order, and the output printed to the terminal. The lines will be printed as soon as the next part is complete, you don't have to wait for all the parts to download before
getting output. The --merge-parts flag will remove the part files when it is done reading each of them. To change this, you can use the --keep-parts flag, and the part files will not be
If you do not specify the `--merge-parts` flag, the part files will be downloaded, and logcli will exit, and you can process the files as you
wish. With the flag specified, the part files will be read in order, and the output printed to the terminal. The lines will be printed as
soon as the next part is complete, you don't have to wait for all the parts to download before getting output. The `--merge-parts` flag will
remove the part files when it is done reading each of them. To change this, you can use the `--keep-parts` flag, and the part files will not be
removed.
Flags:
@ -352,13 +356,16 @@ Flags:
--tls-skip-verify Server certificate TLS skip verify. Can also be set using LOKI_TLS_SKIP_VERIFY env var.
--cert="" Path to the client certificate. Can also be set using LOKI_CLIENT_CERT_PATH env var.
--key="" Path to the client certificate key. Can also be set using LOKI_CLIENT_KEY_PATH env var.
--org-id="" adds X-Scope-OrgID to API requests for representing tenant ID. Useful for requesting tenant data when bypassing an auth gateway. Can also be set using
LOKI_ORG_ID env var.
--query-tags="" adds X-Query-Tags http header to API requests. This header value will be part of `metrics.go` statistics. Useful for tracking the query. Can also be set
using LOKI_QUERY_TAGS env var.
--bearer-token="" adds the Authorization header to API requests for authentication purposes. Can also be set using LOKI_BEARER_TOKEN env var.
--bearer-token-file="" adds the Authorization header to API requests for authentication purposes. Can also be set using LOKI_BEARER_TOKEN_FILE env var.
--retries=0 How many times to retry each query when getting an error response from Loki. Can also be set using LOKI_CLIENT_RETRIES env var.
--org-id="" adds X-Scope-OrgID to API requests for representing tenant ID. Useful for requesting tenant data when
bypassing an auth gateway. Can also be set using LOKI_ORG_ID env var.
--query-tags="" adds X-Query-Tags http header to API requests. This header value will be part of `metrics.go` statistics.
Useful for tracking the query. Can also be set using LOKI_QUERY_TAGS env var.
--bearer-token="" adds the Authorization header to API requests for authentication purposes. Can also be set using
LOKI_BEARER_TOKEN env var.
--bearer-token-file="" adds the Authorization header to API requests for authentication purposes. Can also be set using
LOKI_BEARER_TOKEN_FILE env var.
--retries=0 How many times to retry each query when getting an error response from Loki. Can also be set using
LOKI_CLIENT_RETRIES env var.
--min-backoff=0 Minimum backoff time between retries. Can also be set using LOKI_CLIENT_MIN_BACKOFF env var.
--max-backoff=0 Maximum backoff time between retries. Can also be set using LOKI_CLIENT_MAX_BACKOFF env var.
--auth-header="Authorization"
@ -369,19 +376,24 @@ Flags:
--from=FROM Start looking for logs at this absolute time (inclusive)
--to=TO Stop looking for logs at this absolute time (exclusive)
--step=STEP Query resolution step width, for metric queries. Evaluate the query at the specified step over the time range.
--interval=INTERVAL Query interval, for log queries. Return entries at the specified interval, ignoring those between. **This parameter is experimental, see Issue 1779**
--interval=INTERVAL Query interval, for log queries. Return entries at the specified interval, ignoring those between. **This
parameter is experimental, please see Issue 1779**
--batch=1000 Query batch size to use until 'limit' is reached
--parallel-duration=1h Split the range into jobs of this length to download the logs in parallel. This will result in the logs being out of order. Use --part-path-prefix to create
a file per job to maintain ordering.
--parallel-max-workers=1 Max number of workers to start up for parallel jobs. A value of 1 will not create any parallel workers. When using parallel workers, limit is ignored.
--parallel-duration=1h Split the range into jobs of this length to download the logs in parallel. This will result in the logs being
out of order. Use --part-path-prefix to create a file per job to maintain ordering.
--parallel-max-workers=1 Max number of workers to start up for parallel jobs. A value of 1 will not create any parallel workers.
When using parallel workers, limit is ignored.
--part-path-prefix=PART-PATH-PREFIX
When set, each server response will be saved to a file with this prefix. Creates files in the format: 'prefix-utc_start-utc_end.part'. Intended to be used
with the parallel-* flags so that you can combine the files to maintain ordering based on the filename. Default is to write to stdout.
When set, each server response will be saved to a file with this prefix. Creates files in the format:
'prefix-utc_start-utc_end.part'. Intended to be used with the parallel-* flags so that you can combine the
files to maintain ordering based on the filename. Default is to write to stdout.
--overwrite-completed-parts
Overwrites completed part files. This will download the range again, and replace the original completed part file. Default will skip a range if it's part
file is already downloaded.
--merge-parts Reads the part files in order and writes the output to stdout. Original part files will be deleted with this option.
--keep-parts Overrides the default behaviour of --merge-parts which will delete the part files once all the files have been read. This option will keep the part files.
Overwrites completed part files. This will download the range again, and replace the original completed part
file. Default will skip a range if it's part file is already downloaded.
--merge-parts Reads the part files in order and writes the output to stdout. Original part files will be deleted with this
option.
--keep-parts Overrides the default behaviour of --merge-parts which will delete the part files once all the files have been
read. This option will keep the part files.
--forward Scan forwards through logs.
--no-labels Do not print any labels
--exclude-label=EXCLUDE-LABEL ...
@ -390,7 +402,8 @@ Flags:
Include labels given the provided key during output.
--labels-length=0 Set a fixed padding to labels
--store-config="" Execute the current query using a configured storage from a given Loki configuration file.
--remote-schema Execute the current query using a remote schema retrieved using the configured storage in the given Loki configuration file.
--remote-schema Execute the current query using a remote schema retrieved from the configured -schema-store.
--schema-store="" Store used for retrieving remote schema.
Yoshitaka Fujii
1 year ago
committed by