565 Commits (61fc17536120bcbc201d16a0032aad0375cbfca2)

Author	SHA1	Message	Date
Owen Diehl	61fc175361	Fix vector-scalar comparisons (#2189) ... * fmts shards in instance logger * vector-scalar comparisons now filter by default * gofmt	6 years ago
Owen Diehl	1530120c2e	Logql comparison ops (#2182) ... * comparison binops * starts comparison operator docs * implements bool modifier for comparison binops * vector-vector binop merging cases, including bool modifier * logql docs formatting * resets protos * linter fixes * goimports local * gofmt	6 years ago
Cyril Tovena	cb76da6c1d	Fixes path prefix in the querier. (#2178) ... Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	6 years ago
Cyril Tovena	b95db8d1fd	Fix error message for max tail connections. (#2161) ... * Fix error message for max tail connections. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes a test. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	6 years ago
Aditya C S	5b8d9b4ed6	Support series query in query-frontend (#2123)	6 years ago
Cyril Tovena	18f1b0640d	Bytes aggregations (#2150) ... * Add bytes_rate and bytes_over_time. Still need to works on test. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add more tests. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * lint. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * More lint fixes. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * More comments. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add documentation. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Review feedbacks. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fix bad conflic resolution. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	6 years ago
Cyril Tovena	4c8199ec21	Validate local storage config for the logcli (#2126) ... * Allows to pass inlined pipeline stages to the docker driver. Fixes #1296 Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add validation before using storage config for the logcli. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	6 years ago
Peter Štibraný	e4a5360586	Update vendored Cortex to master (1.1+) (#2149) ... * Updated Cortex to master (post 1.1) Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Use new modules system from Cortex. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Treat query errors reported by chunk store as client errors. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Removed obsolete tests. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Fix race for ct variable. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Fix test. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>	6 years ago
Aditya C S	38ee57c7d5	Feature: Replace stage in pipeline (#2060)	6 years ago
Owen Diehl	89d80a6fc4	Feature/querysharding ii (#1927) ... * [wip] sharding evaluator/ast * [wip] continues experimenting with ast mapping * refactoring in preparation for binops * evaluators can pass state to other evaluators * compiler alignment * Evaluator method renamed to StepEvaluator * chained evaluator impl * tidying up sharding code * handling for ConcatSampleExpr * downstream iterator * structure for downstreaming asts * outlines sharding optimizations * work on sharding mapper * ast sharding optimizations * test for different logrange positions * shard mapper tests * stronger ast sharding & tests * shardmapper tests for string->string * removes sharding evaluator code * removes unused ctx arg * Revert "removes sharding evaluator code" This reverts commit 55d41b9519. * interfaces for downstreaming, type conversions * sharding plumbing on frontend * type alignment in queryrange to downstream sharded queriers * downstreaming support for sharding incl storage code * removes chainedevaluator * comment alignment * storage shard injection * speccing out testware for sharding equivalence * [wip] shared engine refactor * sorting streams, sharding eval fixes * downstream evaluator embeds defaultevaluator * other pkgs adopt logql changes * metrics & logs use same middleware instantiation process * wires up shardingware * middleware per metrics/logfilter * empty step populating StepEvaluator promql.Matrix adapter * sharding metrics * log/span injection into sharded engine * sharding metrics avoids multiple instantiation * downstreamhandler tracing * sharding parameterized libsonnet * removes querier replicas * default 32 concurrency for workers * jsonnet correct level override * unquote true in yaml * lowercase error + downstreamEvaluator defaults to embedded defaultEvaluator * makes shardRecorder private * logs query on failed parse * refactors engine to be multi-use, minimizes logger injection, generalizes Query methods, removes Engine interface * basic tests for querysharding mware * [wip] concurrent evaluator * integrates stat propagation into sharding evaluator * splitby histogram * extends le bounds for bytes processed * byte throughput histogram buckets to 40gb * chunk duration mixin * fixes merge w/ field rename * derives logger in sharded engine via ctx & logs some downstream evaluators * moves sharded engine to top, adds comments * logs failed merge results in stats ctx * snapshotting stats merge logic is done more effectively * per query concurrency controlled via downstreamer * unexports decodereq * queryrange testware * downstreamer tests * pr requests	6 years ago
Jérôme Loyet	665712ff18	Add Entry variable to template (#2105) ... The new Entry variable is the log message that could be needed in template. For instance, if I want to convert all my log line to lowercase: - template: source: message template: '{{ ToLower .Entry }}'	6 years ago
Sebastian Widmer	d175fa58c8	Update go-syslog dependency to accept any encoding in message (#2127)	6 years ago
Aditya C S	ff0a4c35f4	Set JournalTarget Priority value to keyword (#2087)	6 years ago
Aditya C S	f984e71192	Fix Promtail journal seeking known position (#2111)	6 years ago
Aditya C S	360d9a7f97	Support querying labels on time range in logcli (#2083)	6 years ago
Cyril Tovena	f9a2c436ac	Adds backtick for the quoted string token lexer. (#2095) ... * Fixes unit in stats request log. (#2093) * Fix unit conversion in stats log. 🤦 Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fix the tests because you know ! you know ? Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> wip Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add backtick for the quoted string token lexer. This allows to pass unescaped string to Loki. For example we now support this : `` {name="cassandra"} |~ `error=\w+` `` `` {name!~`mysql-\d+`} `` As you can see no escape is required when using ` quoted strings. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes newlines. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add back missing test. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	6 years ago
Cyril Tovena	c81be6603e	Fixes unit in stats request log. (#2093) ... * Fix unit conversion in stats log. 🤦 Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fix the tests because you know ! you know ? Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	6 years ago
Ed Welch	0520a0ac1c	allow no encoding/compression on chunks (#2088) ... Signed-off-by: Ed Welch <edward.welch@grafana.com>	6 years ago
Owen Diehl	e0dacd2795	removes yolostring (#2078)	6 years ago
Cyril Tovena	0de1a42406	Fixes a panic in dry-run when using external labels. (#2075) ... Fixes #1972. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	6 years ago
Ed Welch	9803eab947	Loki: Allow configuring query_store_max_look_back_period when running a filesystem store and boltdb-shipper (#2073) ... * Allow configuring query_store_max_look_back_period when running a filesystem store, which allows for boltdb-shipper to horizontally scale filesystem store. Signed-off-by: Ed Welch <edward.welch@grafana.com> * Update docs/operations/storage/boltdb-shipper.md Co-authored-by: Sandeep Sukhani <sandeep.d.sukhani@gmail.com> * add some more precision to the timestamp used to make the name unique just to make sure there are not collisions if for some reason the ID is empty (which can happen if the hostname doesn't resolve for some reason) Signed-off-by: Ed Welch <edward.welch@grafana.com> * Allow the ingester to query the store for labels if query store is enabled. Signed-off-by: Ed Welch <edward.welch@grafana.com> * make queriers aware of the fact that ingesters may be querying the store so they don't duplicate query efforts Signed-off-by: Ed Welch <edward.welch@grafana.com> * improve the docs Signed-off-by: Ed Welch <edward.welch@grafana.com> * add tests to the calculateMaxLookBack Signed-off-by: Ed Welch <edward.welch@grafana.com> Co-authored-by: Sandeep Sukhani <sandeep.d.sukhani@gmail.com>	6 years ago
Ed Welch	7dd109763c	expand the match_all to also optionally allow counting bytes per stream (#2066) ... Signed-off-by: Ed Welch <edward.welch@grafana.com>	6 years ago
Sandeep Sukhani	3a28a17946	some metrics for measuring performance and failures in boltdb shipper (#2034) ... * some metrics for measuring performance and failures in boltdb shipper * counting both failures and success in upload and download operation in boltdb shipper * changes suggested from PR review	6 years ago
Ed Welch	f206165c4f	output a debug log if the requested path in the scrape config doesn't match any files (#2069) ... Signed-off-by: Ed Welch <edward.welch@grafana.com>	6 years ago
Aditya C S	11937d3c3a	Targets not required in promtail config (#2026) ... * Targets not required in promtail config * changing the defaulted value from `dummy` to `localhost`, I think this will be more intuitive for people, also re-worded the docs a bit. Signed-off-by: Ed Welch <edward.welch@grafana.com> Co-authored-by: Ed Welch <edward.welch@grafana.com>	6 years ago
Aditya C S	199e334126	Update loki to cortex master (#2030)	6 years ago
Cyril Tovena	a8b94aec3d	Reverse entry iterator pool (#2064) ... * wi Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add tests. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	6 years ago
Cyril Tovena	9988ce4b9f	Improve protobuf serialization (#2031) ... * Generate custom proto type for Streams. This is the first step to improve protobuf serialization. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Improve protobuf serialization allocations. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Saves allocations by stringifying labels only once. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	6 years ago
Cyril Tovena	bce4470a5b	Recover from panic in http and grpc handlers. (#2059) ... * Recover from panic in http and grpc handlers. I don't see any good reason to crash any component during a bad request. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add alerts to the mixin for panics. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * 😡 gomod Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	6 years ago
Cyril Tovena	fcb4c763dc	Fix a bug in range vector skipping data. (#2058) ... Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	6 years ago
Aditya C S	83873d1c6f	Fix promtail client default values (#2049)	6 years ago
Arve Knudsen	d975b1fdd8	Fix spelling of per second (#2055) ... Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	6 years ago
Arve Knudsen	cc98d10d74	Chore: Correct typo "per seconds" (#2050) ... Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>	6 years ago
Goutham Veeramachaneni	06b50590a7	Fix bug in logql parsing that leads to crash. (#2046) ... * Fix bug in logql parsing that leads to crash. ``` panic: runtime error: invalid memory address or nil pointer dereference [signal SIGSEGV: segmentation violation code=0x1 addr=0x18 pc=0x20ae356] goroutine 39530021 [running]: github.com/grafana/loki/pkg/logql.andFilter.Filter(0x0, 0x0, 0x3beffc0, 0xc01f065bc0, 0xc01aaaec00, 0x13d, 0x200, 0xffffffffffffffff) /src/loki/pkg/logql/filter.go:66 +0x26 github.com/grafana/loki/pkg/chunkenc.(*bufferedIterator).Next(0xc00a48fba0, 0x0) /src/loki/pkg/chunkenc/memchunk.go:616 +0x1cb github.com/grafana/loki/pkg/iter.(*nonOverlappingIterator).Next(0xc00130ae40, 0x0) /src/loki/pkg/iter/iterator.go:438 +0x135 github.com/grafana/loki/pkg/iter.(*timeRangedIterator).Next(0xc00130ae80, 0x0) /src/loki/pkg/iter/iterator.go:497 +0x48 github.com/grafana/loki/pkg/iter.(*reverseIterator).load(0xc00072ac40) /src/loki/pkg/iter/iterator.go:555 +0x70 github.com/grafana/loki/pkg/iter.(*reverseIterator).Next(0xc00072ac40, 0x0) /src/loki/pkg/iter/iterator.go:563 +0x2f github.com/grafana/loki/pkg/iter.(*nonOverlappingIterator).Next(0xc00130af40, 0x10) /src/loki/pkg/iter/iterator.go:438 +0x135 ``` This is caused by people doing `|= ""` which would result in parsing of TrueFilter but would be returned as nil. Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com> * Add tests for the panic Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com>	6 years ago
Cyril Tovena	4fd670dc7a	Support Post request in the frontend queryrange handler. (#2023) ... Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	6 years ago
Peter Štibraný	4906fb2065	logcli: Query needs to be stored into url.RawQuery, and not url.Path (#2027) ... * Query needs to be stored into url.RawQuery, and not url.Path Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * QueryStringBuilder deals with queries, not paths. Removed unused method. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Fixed test. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>	6 years ago
Ed Welch	272a89c13d	more linter fixes (#2024) ... * more linter woes Signed-off-by: Ed Welch <edward.welch@grafana.com> * feedback Signed-off-by: Ed Welch <edward.welch@grafana.com>	6 years ago
Ed Welch	4f9a2fe2d9	I <3 you Linter (#2022) ... Signed-off-by: Ed Welch <edward.welch@grafana.com>	6 years ago
Ed Welch	3dfe89369f	Loki: refactor validation and improve error messages (#2021) ... * copies some of the validation logic from cortex and modifies it to appropriate for Loki, this gives us a single metric we can use for detetecting validation errors (previously we had to check cortex_ and loki_ metrics) as well as lets us use error messages more appropriate for logs than metrics. Signed-off-by: Ed Welch <edward.welch@grafana.com> * fixing tests and lint Signed-off-by: Ed Welch <edward.welch@grafana.com> * updating cause to string type Signed-off-by: Ed Welch <edward.welch@grafana.com> * extracting a function for rate limited Signed-off-by: Ed Welch <edward.welch@grafana.com> * extracting functions for the rest of the error strings Signed-off-by: Ed Welch <edward.welch@grafana.com>	6 years ago
Josh Soref	0d9cf4946a	Spelling (#2014) ... * spelling: access * spelling: actual * spelling: asynchronously * spelling: circleci * spelling: circle * spelling: cognitive * spelling: comment * spelling: computation * spelling: correct * spelling: daemon * spelling: elliott * spelling: enabled * spelling: encbuf * spelling: extracted * spelling: explicitly * spelling: independent * spelling: languages * spelling: loose * spelling: message * spelling: metrics * spelling: mismatch * spelling: nonexistent * spelling: output * spelling: pipeline * spelling: pipe line * spelling: position * spelling: prepend * spelling: query * spelling: receive * spelling: received * spelling: reducible * spelling: registry * spelling: response * spelling: retryable * spelling: scanning * spelling: sourced * spelling: supported * spelling: template * spelling: timeout * spelling: unsupported * spelling: adapter * spelling: deferring * spelling: expirable	6 years ago
Ed Welch	872a998fbb	make `loki_ingester_memory_streams` Gauge per tenant. (#2019) ... Signed-off-by: Ed Welch <edward.welch@grafana.com>	6 years ago
Ed Welch	0ab1b28812	Loki: Improve logging and add metrics to streams dropped by stream limit (#2012) ... * Improve the log message when we drop streams because a user is hitting a stream limit. Increment the dropped samples metrics when this happens also. Signed-off-by: Ed Welch <edward.welch@grafana.com> * improving comments Signed-off-by: Ed Welch <edward.welch@grafana.com>	6 years ago
Cyril Tovena	5e34df8caf	Fix a bad rebase between #1970 and #1965. (#2011) ... * Fix a bad rebase between #1970 and #1965. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * fmt Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	6 years ago
Cyril Tovena	34f9c601c9	Adds config to disable HTTP and GRPC server in Promtail. (#2004) ... Fixes #1991. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	6 years ago
Cyril Tovena	dc5cf8f57c	Allow to aggregate binary operations. (#1970) ... * Allow to aggregate binary operations. For example: `sum by (job) (count_over_time({namespace="tns"}[5m] |= "level=error") / count_over_time({namespace="tns"}[5m]))` would have failed previously. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add missing () rules in yacc. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	6 years ago
Sandeep Sukhani	989fb106a0	table client for boltdb shipper to enforce retention (#2001) ... * table client for boltdb shipper to enforce retention	6 years ago
Cyril Tovena	d70fc0e45f	Improve URL building in the logcli to strip trailing /. (#2000) ... Fixes #1985. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	6 years ago
Cyril Tovena	577d8eb168	Ensure trace propagation in our logs. (#1977) ... * Ensure trace propagation in our logs. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add missing import. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes a test. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fix function usage. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	6 years ago
Ed Welch	c7a1a1fb2e	Loki: Adds an `interval` paramater to query_range queries allowing a sampling of events to be returned based on the provided interval (#1965) ... * Adds an `interval` paramater to query_range queries allowing a sampling of events to be returned based on the provided interval Signed-off-by: Ed Welch <edward.welch@grafana.com> * lint Signed-off-by: Ed Welch <edward.welch@grafana.com> * extract function for parsing parameter Signed-off-by: Ed Welch <edward.welch@grafana.com> * Clarify the experimental nature of interval in the docs. Signed-off-by: Ed Welch <edward.welch@grafana.com> * make min time a constant Signed-off-by: Ed Welch <edward.welch@grafana.com> * Expand the docs around the experimental nature of interval a little more Signed-off-by: Ed Welch <edward.welch@grafana.com>	6 years ago
Sandeep Sukhani	21d4ca41e2	make boltdb shipper singleton and some other minor refactoring (#1995) ... * make boltdb shipper singleton and some other minor refactoring	6 years ago