Like Prometheus, but for logs.
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
loki/pkg/logcli/output/output.go

75 lines
1.6 KiB

package output
import (
"fmt"
"hash/fnv"
Logcli: automatically batch requests (#2482) * first stab at batching logcli requests * fix other calls to printResult * working batching code * refactor the logcli Client into an interface so that it can be mocked for testing. * fixing some bugs * refactoring the output interface to make testing easier * starting to add tests * adding a bunch of tests fixing up how the MockQuerier works. * updating docs * Update docs/sources/getting-started/logcli.md Co-authored-by: Diana Payton <52059945+oddlittlebird@users.noreply.github.com> * Update docs/sources/getting-started/logcli.md Co-authored-by: Diana Payton <52059945+oddlittlebird@users.noreply.github.com> * Update docs/sources/getting-started/logcli.md Co-authored-by: Diana Payton <52059945+oddlittlebird@users.noreply.github.com> * Update docs/sources/getting-started/logcli.md Co-authored-by: Diana Payton <52059945+oddlittlebird@users.noreply.github.com> * Update docs/sources/getting-started/logcli.md Co-authored-by: Diana Payton <52059945+oddlittlebird@users.noreply.github.com> * Update docs/sources/getting-started/logcli.md Co-authored-by: Diana Payton <52059945+oddlittlebird@users.noreply.github.com> * Update docs/sources/getting-started/logcli.md Co-authored-by: Diana Payton <52059945+oddlittlebird@users.noreply.github.com> * Update docs/sources/getting-started/logcli.md Co-authored-by: Diana Payton <52059945+oddlittlebird@users.noreply.github.com> Co-authored-by: Owen Diehl <ow.diehl@gmail.com> Co-authored-by: Diana Payton <52059945+oddlittlebird@users.noreply.github.com>
5 years ago
"io"
"time"
"github.com/fatih/color"
Loki HTTP/JSON Model Layer (#1022) * First pass data model Signed-off-by: Joe Elliott <number101010@gmail.com> * Use prom model b/c we're serializing promql objects Signed-off-by: Joe Elliott <number101010@gmail.com> * Added legacy query support and tests Signed-off-by: Joe Elliott <number101010@gmail.com> * Added legacy label test Signed-off-by: Joe Elliott <number101010@gmail.com> * Added tail response marshalling and tests Signed-off-by: Joe Elliott <number101010@gmail.com> * Fixed marshallers and test Signed-off-by: Joe Elliott <number101010@gmail.com> * Expanded legacy test cases Signed-off-by: Joe Elliott <number101010@gmail.com> * Dropped streams nano test Signed-off-by: Joe Elliott <number101010@gmail.com> * First pass v1 new objects Signed-off-by: Joe Elliott <number101010@gmail.com> * Added failing tail response tests Signed-off-by: Joe Elliott <number101010@gmail.com> * Added failing tailresponse test Signed-off-by: Joe Elliott <number101010@gmail.com> * Partial v1 tests Signed-off-by: Joe Elliott <number101010@gmail.com> * Improved legacy labels test Signed-off-by: Joe Elliott <number101010@gmail.com> * Improved legacy query tests Signed-off-by: Joe Elliott <number101010@gmail.com> * Moved all legacy tests to new method Signed-off-by: Joe Elliott <number101010@gmail.com> * Added v1 tests and fixed stream marshalling bug Signed-off-by: Joe Elliott <number101010@gmail.com> * First pass new Model Signed-off-by: Joe Elliott <number101010@gmail.com> * Added vector test Signed-off-by: Joe Elliott <number101010@gmail.com> * Added matrix tests Signed-off-by: Joe Elliott <number101010@gmail.com> * Added conversions for all things except tailed responses Signed-off-by: Joe Elliott <number101010@gmail.com> * Fixed mixed case issues Signed-off-by: Joe Elliott <number101010@gmail.com> * Fixed tail marshalling Signed-off-by: Joe Elliott <number101010@gmail.com> * Removed unused testStream Signed-off-by: Joe Elliott <number101010@gmail.com> * Moved TailResponse to loghttp Signed-off-by: Joe Elliott <number101010@gmail.com> * Removed legacy tailresponse objects in favor of actual legacy tailresponse objects Signed-off-by: Joe Elliott <number101010@gmail.com> * Updated v1 methods to take legacy tail objects Signed-off-by: Joe Elliott <number101010@gmail.com> * Cleaned up tests. Added some comments Signed-off-by: Joe Elliott <number101010@gmail.com> * Versioned tail endpoint Signed-off-by: Joe Elliott <number101010@gmail.com> * Improved readability on loghttp packages in http.go Signed-off-by: Joe Elliott <number101010@gmail.com> * Removed new as a var name Signed-off-by: Joe Elliott <number101010@gmail.com> * Started all error messages with lowercase alerts Signed-off-by: Joe Elliott <number101010@gmail.com> * new => ret Signed-off-by: Joe Elliott <number101010@gmail.com> * Added comments on exported methods Signed-off-by: Joe Elliott <number101010@gmail.com> * Removed two personal notes Signed-off-by: Joe Elliott <number101010@gmail.com> * Changed legacy package name to loghttp Signed-off-by: Joe Elliott <number101010@gmail.com> * Moved and renamed loghttp v1 package Signed-off-by: Joe Elliott <number101010@gmail.com> * Moved marshalling code out of model Signed-off-by: Joe Elliott <number101010@gmail.com> * Added package comments Signed-off-by: Joe Elliott <number101010@gmail.com> * Added legacy testing Signed-off-by: Joe Elliott <number101010@gmail.com> * Changed DroppedStream slice to value type for consistency Signed-off-by: Joe Elliott <number101010@gmail.com> * gofmt'ed test files Signed-off-by: Joe Elliott <number101010@gmail.com> * Cleaned up linting issues Signed-off-by: Joe Elliott <number101010@gmail.com> * Minor comment cleanup Signed-off-by: Joe Elliott <number101010@gmail.com> * Adjusted GOGC to make CircleCI happy Signed-off-by: Joe Elliott <number101010@gmail.com> * Changed legacy => loghttp for consistency Signed-off-by: Joe Elliott <number101010@gmail.com> * Fixed matrix error message to be correct Signed-off-by: Joe Elliott <number101010@gmail.com> * Moved label query over to loghttp response Signed-off-by: Joe Elliott <number101010@gmail.com> * Added marshal loop tests Signed-off-by: Joe Elliott <number101010@gmail.com> * Added response type test Signed-off-by: Joe Elliott <number101010@gmail.com> * Fixed tail response marshal/unmarshal Signed-off-by: Joe Elliott <number101010@gmail.com> * Passing unmarshal/marshal queryresponse tests Signed-off-by: Joe Elliott <number101010@gmail.com> * Fixed vector and matrix Signed-off-by: Joe Elliott <number101010@gmail.com> * Added output support for streams minus ordering Signed-off-by: Joe Elliott <number101010@gmail.com> * Fixed tailing Signed-off-by: Joe Elliott <number101010@gmail.com> * Fixed output tests Signed-off-by: Joe Elliott <number101010@gmail.com> * Fixed query tests Signed-off-by: Joe Elliott <number101010@gmail.com> * Order log output Signed-off-by: Joe Elliott <number101010@gmail.com> * Use labels instead of stream Signed-off-by: Joe Elliott <number101010@gmail.com> * Lowered parallelization for CircleCI Signed-off-by: Joe Elliott <number101010@gmail.com>
6 years ago
"github.com/grafana/loki/pkg/loghttp"
)
// Blue color is excluded since we are already printing timestamp
// in blue color
var colorList = []*color.Color{
color.New(color.FgHiCyan),
color.New(color.FgCyan),
color.New(color.FgHiGreen),
color.New(color.FgGreen),
color.New(color.FgHiMagenta),
color.New(color.FgMagenta),
color.New(color.FgHiYellow),
color.New(color.FgYellow),
color.New(color.FgHiRed),
color.New(color.FgRed),
}
// LogOutput is the interface any output mode must implement
type LogOutput interface {
Logcli: automatically batch requests (#2482) * first stab at batching logcli requests * fix other calls to printResult * working batching code * refactor the logcli Client into an interface so that it can be mocked for testing. * fixing some bugs * refactoring the output interface to make testing easier * starting to add tests * adding a bunch of tests fixing up how the MockQuerier works. * updating docs * Update docs/sources/getting-started/logcli.md Co-authored-by: Diana Payton <52059945+oddlittlebird@users.noreply.github.com> * Update docs/sources/getting-started/logcli.md Co-authored-by: Diana Payton <52059945+oddlittlebird@users.noreply.github.com> * Update docs/sources/getting-started/logcli.md Co-authored-by: Diana Payton <52059945+oddlittlebird@users.noreply.github.com> * Update docs/sources/getting-started/logcli.md Co-authored-by: Diana Payton <52059945+oddlittlebird@users.noreply.github.com> * Update docs/sources/getting-started/logcli.md Co-authored-by: Diana Payton <52059945+oddlittlebird@users.noreply.github.com> * Update docs/sources/getting-started/logcli.md Co-authored-by: Diana Payton <52059945+oddlittlebird@users.noreply.github.com> * Update docs/sources/getting-started/logcli.md Co-authored-by: Diana Payton <52059945+oddlittlebird@users.noreply.github.com> * Update docs/sources/getting-started/logcli.md Co-authored-by: Diana Payton <52059945+oddlittlebird@users.noreply.github.com> Co-authored-by: Owen Diehl <ow.diehl@gmail.com> Co-authored-by: Diana Payton <52059945+oddlittlebird@users.noreply.github.com>
5 years ago
FormatAndPrintln(ts time.Time, lbls loghttp.LabelSet, maxLabelsLen int, line string)
}
// LogOutputOptions defines options supported by LogOutput
type LogOutputOptions struct {
Timezone *time.Location
NoLabels bool
ColoredOutput bool
}
// NewLogOutput creates a log output based on the input mode and options
Logcli: automatically batch requests (#2482) * first stab at batching logcli requests * fix other calls to printResult * working batching code * refactor the logcli Client into an interface so that it can be mocked for testing. * fixing some bugs * refactoring the output interface to make testing easier * starting to add tests * adding a bunch of tests fixing up how the MockQuerier works. * updating docs * Update docs/sources/getting-started/logcli.md Co-authored-by: Diana Payton <52059945+oddlittlebird@users.noreply.github.com> * Update docs/sources/getting-started/logcli.md Co-authored-by: Diana Payton <52059945+oddlittlebird@users.noreply.github.com> * Update docs/sources/getting-started/logcli.md Co-authored-by: Diana Payton <52059945+oddlittlebird@users.noreply.github.com> * Update docs/sources/getting-started/logcli.md Co-authored-by: Diana Payton <52059945+oddlittlebird@users.noreply.github.com> * Update docs/sources/getting-started/logcli.md Co-authored-by: Diana Payton <52059945+oddlittlebird@users.noreply.github.com> * Update docs/sources/getting-started/logcli.md Co-authored-by: Diana Payton <52059945+oddlittlebird@users.noreply.github.com> * Update docs/sources/getting-started/logcli.md Co-authored-by: Diana Payton <52059945+oddlittlebird@users.noreply.github.com> * Update docs/sources/getting-started/logcli.md Co-authored-by: Diana Payton <52059945+oddlittlebird@users.noreply.github.com> Co-authored-by: Owen Diehl <ow.diehl@gmail.com> Co-authored-by: Diana Payton <52059945+oddlittlebird@users.noreply.github.com>
5 years ago
func NewLogOutput(w io.Writer, mode string, options *LogOutputOptions) (LogOutput, error) {
if options.Timezone == nil {
options.Timezone = time.Local
}
switch mode {
case "default":
return &DefaultOutput{
Logcli: automatically batch requests (#2482) * first stab at batching logcli requests * fix other calls to printResult * working batching code * refactor the logcli Client into an interface so that it can be mocked for testing. * fixing some bugs * refactoring the output interface to make testing easier * starting to add tests * adding a bunch of tests fixing up how the MockQuerier works. * updating docs * Update docs/sources/getting-started/logcli.md Co-authored-by: Diana Payton <52059945+oddlittlebird@users.noreply.github.com> * Update docs/sources/getting-started/logcli.md Co-authored-by: Diana Payton <52059945+oddlittlebird@users.noreply.github.com> * Update docs/sources/getting-started/logcli.md Co-authored-by: Diana Payton <52059945+oddlittlebird@users.noreply.github.com> * Update docs/sources/getting-started/logcli.md Co-authored-by: Diana Payton <52059945+oddlittlebird@users.noreply.github.com> * Update docs/sources/getting-started/logcli.md Co-authored-by: Diana Payton <52059945+oddlittlebird@users.noreply.github.com> * Update docs/sources/getting-started/logcli.md Co-authored-by: Diana Payton <52059945+oddlittlebird@users.noreply.github.com> * Update docs/sources/getting-started/logcli.md Co-authored-by: Diana Payton <52059945+oddlittlebird@users.noreply.github.com> * Update docs/sources/getting-started/logcli.md Co-authored-by: Diana Payton <52059945+oddlittlebird@users.noreply.github.com> Co-authored-by: Owen Diehl <ow.diehl@gmail.com> Co-authored-by: Diana Payton <52059945+oddlittlebird@users.noreply.github.com>
5 years ago
w: w,
options: options,
}, nil
case "jsonl":
return &JSONLOutput{
Logcli: automatically batch requests (#2482) * first stab at batching logcli requests * fix other calls to printResult * working batching code * refactor the logcli Client into an interface so that it can be mocked for testing. * fixing some bugs * refactoring the output interface to make testing easier * starting to add tests * adding a bunch of tests fixing up how the MockQuerier works. * updating docs * Update docs/sources/getting-started/logcli.md Co-authored-by: Diana Payton <52059945+oddlittlebird@users.noreply.github.com> * Update docs/sources/getting-started/logcli.md Co-authored-by: Diana Payton <52059945+oddlittlebird@users.noreply.github.com> * Update docs/sources/getting-started/logcli.md Co-authored-by: Diana Payton <52059945+oddlittlebird@users.noreply.github.com> * Update docs/sources/getting-started/logcli.md Co-authored-by: Diana Payton <52059945+oddlittlebird@users.noreply.github.com> * Update docs/sources/getting-started/logcli.md Co-authored-by: Diana Payton <52059945+oddlittlebird@users.noreply.github.com> * Update docs/sources/getting-started/logcli.md Co-authored-by: Diana Payton <52059945+oddlittlebird@users.noreply.github.com> * Update docs/sources/getting-started/logcli.md Co-authored-by: Diana Payton <52059945+oddlittlebird@users.noreply.github.com> * Update docs/sources/getting-started/logcli.md Co-authored-by: Diana Payton <52059945+oddlittlebird@users.noreply.github.com> Co-authored-by: Owen Diehl <ow.diehl@gmail.com> Co-authored-by: Diana Payton <52059945+oddlittlebird@users.noreply.github.com>
5 years ago
w: w,
options: options,
}, nil
case "raw":
return &RawOutput{
Logcli: automatically batch requests (#2482) * first stab at batching logcli requests * fix other calls to printResult * working batching code * refactor the logcli Client into an interface so that it can be mocked for testing. * fixing some bugs * refactoring the output interface to make testing easier * starting to add tests * adding a bunch of tests fixing up how the MockQuerier works. * updating docs * Update docs/sources/getting-started/logcli.md Co-authored-by: Diana Payton <52059945+oddlittlebird@users.noreply.github.com> * Update docs/sources/getting-started/logcli.md Co-authored-by: Diana Payton <52059945+oddlittlebird@users.noreply.github.com> * Update docs/sources/getting-started/logcli.md Co-authored-by: Diana Payton <52059945+oddlittlebird@users.noreply.github.com> * Update docs/sources/getting-started/logcli.md Co-authored-by: Diana Payton <52059945+oddlittlebird@users.noreply.github.com> * Update docs/sources/getting-started/logcli.md Co-authored-by: Diana Payton <52059945+oddlittlebird@users.noreply.github.com> * Update docs/sources/getting-started/logcli.md Co-authored-by: Diana Payton <52059945+oddlittlebird@users.noreply.github.com> * Update docs/sources/getting-started/logcli.md Co-authored-by: Diana Payton <52059945+oddlittlebird@users.noreply.github.com> * Update docs/sources/getting-started/logcli.md Co-authored-by: Diana Payton <52059945+oddlittlebird@users.noreply.github.com> Co-authored-by: Owen Diehl <ow.diehl@gmail.com> Co-authored-by: Diana Payton <52059945+oddlittlebird@users.noreply.github.com>
5 years ago
w: w,
options: options,
}, nil
default:
return nil, fmt.Errorf("unknown log output mode '%s'", mode)
}
}
func getColor(labels string) *color.Color {
hash := fnv.New32()
_, _ = hash.Write([]byte(labels))
id := hash.Sum32() % uint32(len(colorList))
color := colorList[id]
return color
}