512 Commits (2374d4891b0879cb77ffb5ef04c408442fcaff63)

Author	SHA1	Message	Date
Ed Welch	b89a1f81dd	Update cortex vendoring to latest master (#1975) ... * Update cortex vendoring to latest master checksum `36496a074bc4530fb1d7c74136cab1566fbd9583` Signed-off-by: Ed Welch <edward.welch@grafana.com> * handle all the breaking changes from cortes Signed-off-by: Sandeep Sukhani <sandeep.d.sukhani@gmail.com> * rebasing on master, re-runing go mod vendor seemed to bring in some more changes... Signed-off-by: Ed Welch <edward.welch@grafana.com> Co-authored-by: Sandeep Sukhani <sandeep.d.sukhani@gmail.com>	5 years ago
Peter Štibraný	e60164bef6	Convert Loki modules to services (#1804) ... * Loki now uses module services to start and stop its work. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Use services methods to initialize some components. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Use Cortex' NewModuleService. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Converted server to a service. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Converted distributor to service. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Use table manager service Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * querier service Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * query-frontend service Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Merged stopping method into shutdown Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Converted ingester to a service. It now starts all background tasks in Starting state. Stopping needs little work, as does reacting on lifecycler errors. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Loki Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * If lifecycler fails, ingester fails too. It now doesn't call os.Exit, but shuts down gracefully and enters Failed state. That triggers Loki to shutdown completely. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Ignore ErrStopProcess errors from services This is a signal that Loki should stop. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Use single /ready handler It checks the state of all services, and asks ingester for its own check as well. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Removed unused value. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Lint Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Fix test. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Go mod tidy, vendor Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Tailers, not trailers. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Adds return for the healtcheck in case of error. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> Co-authored-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago
Owen Diehl	bd4d516262	chunk bounds metric now records 8h range in 1h increments (#1964)	5 years ago
Sandeep Sukhani	fad3b61be7	flush boltdb to object store (#1837) ... * flush boltdb to object store files are stored in folder per periodic table and are named after ingester flushed every 15 mins to make index available to other services files are also flushed before ingester stops to avoid any data loss new stores can be implemented easily ingester to also query store when using boltdb * persisting uploader name across restarts, detecting objectstore type from periodic config, other refactorings * updated cli flag for active directory * add tests for boltdb shipper and update vendor * syncing boltdb files to disk during update * sleep in tests to let mtime of boltdb file be changed * add documentation for boltdb shipper and remove unwanted config	5 years ago
Owen Diehl	8b924a52cb	chunk lifespan histogram (#1958) ... * chunk lifespan histogram * renames to bounds for clarity	5 years ago
Sandeep Sukhani	49f04ef4bc	update cortex to latest master (#1956)	5 years ago
Ed Welch	b5b3e78424	Loki Update the cut block size counter when creating a memchunk from byte slice (#1946) ... * Update the cut block size counter when creating a memchunk from byte slice * Refactor how we create memchunks to require specifying the block size and target size	5 years ago
Cyril Tovena	15c4ebb35f	Fixes case handling in regex simplification. (#1950) ... Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago
Cyril Tovena	658dcb8416	Support stdin target via flag instead of automatic detection. (#1935) ... This seems to causes issue on some environment so let's use a flag instead. Fixes #1921 Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago
Owen Diehl	e9789f7f99	adds config validation, similar to cortex (#1939)	5 years ago
Peter Štibraný	6d5408c100	Update Cortex to latest master (#1869) ... * Update Cortex to master Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Fix compilation errors after Cortex update. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Update to Cortex v1.0.0-rc.0 Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Added missing vendored file. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Vendor Cortex 1.0.0 Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Configuration changes. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Mention Cortex 1.0 changes and link to docs. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Fix jsonnet files, and mention fields that were removed. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>	5 years ago
alexanderGalushka	5e752bf343	feat: tms readiness check bypass implementation ... fix: renamed flag to health_check_target, default to true fix: flattened nested if statements	5 years ago
Owen Diehl	3347871154	only fetches one chunk per series in /series (#1914) ... * only fetches one chunk per series in /series * Update pkg/storage/store.go * chunksBySeries naming * sorts series response, adds tests for chunks < batchsize	5 years ago
Cyril Tovena	8e80160db7	Removes file size check for pipe, not provided by linux. (#1893) ... * Removes file size check for pipe, not provided by linux. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Disable color on windows. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> (cherry picked from commit 75730782a7)	5 years ago
Joe Elliott	383eb6e70b	Addition of a `/suspend` endpoint to Loki Canary (#1891) ... * First pass loki canary pause Signed-off-by: Joe Elliott <number101010@gmail.com> * Write suspending msg to stderr Signed-off-by: Joe Elliott <number101010@gmail.com> * Added resume endpoint Signed-off-by: Joe Elliott <number101010@gmail.com> * Added documentation on new endpoints Signed-off-by: Joe Elliott <number101010@gmail.com> * Pass both signals to the same channel Signed-off-by: Joe Elliott <number101010@gmail.com> * lint Signed-off-by: Joe Elliott <number101010@gmail.com> * Removed return on startCanary Signed-off-by: Joe Elliott <number101010@gmail.com>	5 years ago
Cyril Tovena	75730782a7	Removes file size check for pipe, not provided by linux. (#1893) ... * Removes file size check for pipe, not provided by linux. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Disable color on windows. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago
Cyril Tovena	a252a8fc01	Fixes possible panic in json pipeline stage. (#1894) ... This was reported to me today and it happens because the previous source can be nil. This was only happening when debug log was on. And the problem was only in the json because it was using .String on a nil value. I've added a test and activate debug during tests to cover those path too. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago
Aditya C S	e713a8d83a	Fix flaky promtail test (#1865)	5 years ago
Edward Welch	3c479c8f9c	promtail retry 429 rate limit errors, clarifying risks in the docs of configuring multiple client sections in promtail, also increased the backoff and retry settings in promtail. ... Signed-off-by: Edward Welch <edward.welch@grafana.com> (cherry picked from commit 6841c418ef)	5 years ago
Ben Ye	9a62deb829	Support series API in logcli (#1861) ... * support series API in logcli Signed-off-by: yeya24 <yb532204897@gmail.com> add changelog Signed-off-by: yeya24 <yb532204897@gmail.com> * update logcli docs Signed-off-by: yeya24 <yb532204897@gmail.com>	5 years ago
Owen Diehl	726b8cd6d4	support `match[]` encoding (#1875)	5 years ago
Cyril Tovena	ba51aad551	Logcli remote storage. (#1814) ... * wip Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Start hooking logcli with local storage config. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes org-id and documentation. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Review feedback. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago
Owen Diehl	7effeec642	Sharding optimizations I: AST mapping (#1846) ... * [wip] sharding evaluator/ast * [wip] continues experimenting with ast mapping * refactoring in preparation for binops * evaluators can pass state to other evaluators * compiler alignment * Evaluator method renamed to StepEvaluator * chained evaluator impl * tidying up sharding code * handling for ConcatSampleExpr * downstream iterator * structure for downstreaming asts * outlines sharding optimizations * work on sharding mapper * ast sharding optimizations * test for different logrange positions * shard mapper tests * stronger ast sharding & tests * shardmapper tests for string->string * removes sharding evaluator code * removes unused ctx arg * Update pkg/logql/evaluator.go Co-Authored-By: Cyril Tovena <cyril.tovena@gmail.com> Co-authored-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago
Aditya C S	f41273182c	Support configurable maximum of the limits parameter (#1798)	5 years ago
Christian Schlotter	57023e00cb	logcli client: use OrgID in LiveTail (#1850) ... Signed-off-by: Schlotter, Christian <christi.schlotter@gmail.com>	5 years ago
Edward Welch	6841c418ef	promtail retry 429 rate limit errors, clarifying risks in the docs of configuring multiple client sections in promtail, also increased the backoff and retry settings in promtail. ... Signed-off-by: Edward Welch <edward.welch@grafana.com>	5 years ago
Aditya C S	7f61f111ae	Log error message when source does not exist in extracted values (#1815)	5 years ago
Cyril Tovena	d5f27ee04d	Adds a counter for total flushed chunks per reason. (#1819) ... Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago
Cyril Tovena	3829b707c3	Optimize empty filter queries. (#1810) ... * Optimize filter queries such as `{label="foo"} |~ "" |~ ".*" |= ""` doesn't contains a filter. The query is not really rewritten and the stringer func still returns the correct original query. However the `Filter` func of the `Expr` returns nil which what is being checked when splitting queries in the frontend and also what is being used in the memchunk. see https://github.com/grafana/loki/blob/master/pkg/querier/queryrange/roundtrip.go#L81 and https://github.com/grafana/loki/blob/master/pkg/chunkenc/memchunk.go#L614 Fixes #1792 Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add missing tests with combination of empty or regex. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Review feedback. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago
Cyril Tovena	2a933b7bc3	Test stats memchunk (#1809) ... * Add a test for ensuring stats computed from memchunk are correct. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add some comment Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago
Peter Štibraný	ca19683511	loki: update Cortex to master (#1799) ... * Update Cortex to master Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Integrate latest Cortex master changes into loki - Frontend worker now needs to be started explicitly - Lifecycler no longer exits on error. Distributor and Ingester do that now Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Lint Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * go mod tidy Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Update Cortex to master. After 0.7 release, it is now showing as v0.7.1-... Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>	5 years ago
Ed Welch	3abee7829d	Log the full stats and send to stderr instead of stdout (#1808) ... Signed-off-by: Edward Welch <edward.welch@grafana.com>	5 years ago
Cyril Tovena	1eb139c37c	Add a test for duplicates count in the heap iterator and fixes it. (#1802) ... * Add a tests for duplicate count in the heap iterator and fixes it. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Update pkg/iter/iterator.go	5 years ago
Cyril Tovena	97165fc0a0	Use ingester client GRPC call options from config. (#1797) ... Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago
Aditya C S	89263b09a7	Add /ready endpoint to table-manager (#1736)	5 years ago
johncming	13c9a864c6	pkg/chunkenc: fix leak of pool. (#1793) ... Signed-off-by: johncming <johncming@yahoo.com>	5 years ago
Peter Štibraný	dac8615f39	Convert module names to string. This allows us to remove bit of a code. (#1794) ... Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>	5 years ago
Cyril Tovena	d34b0a9d30	Regexp simplification (#1787) ... * Refactor line filter to support custom parsing for perl regexp. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Support alternate, concat regex simplification Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add benchmark between normal and simplified regex. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Working through all possible concat operation. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes concat operations. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Adds more type of capture group Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Improve test robustness Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Support for nexted concat. git push Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes bug with anychar. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * add more not supported cases. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Refactor the concat alternates function. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Improve documentation and refactor code. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes all code using line filter. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Improve tests to check nil filter. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Ensure benchmark is not optimized by the compilation. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Review feedback. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago
Peter Štibraný	5ae6ea71f8	Update Cortex to master (#1785) ... * Update Cortex to v0.7.0-rc.0 Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Fixed errors after updating to latest master. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Fixed errors after updating to latest master. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Fixed errors after updating to latest master. queryrange.NewRetryMiddleware in NewLogFilterTripperware is now created without metrics, as that would collide with metrics in NewMetricTripperware. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Make sure to start client.Pool before using it. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Use shared singleton memberlist KV. Fix queryrange metrics. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Make lint happy Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * go mod tidy Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Return err early. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>	5 years ago
Cyril Tovena	3f3152427a	Update cortex vendoring to include frontend status code improvement. (#1746) ... * Update cortex vendoring to include frontend status code improvement. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fixes signature changes in runtimeconfig.NewRuntimeConfigManager. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Forgot to git add . :facelpalm: Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago
Ed Welch	f5d79aee7a	Revert "loki: Allow setting a 'step' when querying logs. (#1773)" (#1777) ... This reverts commit 934a0b8d Signed-off-by: Edward Welch <edward.welch@grafana.com>	5 years ago
Ed Welch	934a0b8dc6	loki: Allow setting a 'step' when querying logs. (#1773) ... * Setting a step on log queries will result in similar behavior as metrics, only returning logs at multiples of the step Signed-off-by: Edward Welch <edward.welch@grafana.com> * clear up logic a little to make it easier to follow Signed-off-by: Edward Welch <edward.welch@grafana.com> * try to protect future us from future us Signed-off-by: Edward Welch <edward.welch@grafana.com>	5 years ago
Edward Welch	8726033bfa	remove the read lines counter when the log file stops being tailed ... Signed-off-by: Edward Welch <edward.welch@grafana.com>	5 years ago
Aditya C S	6f3489e661	Fix single job with multiple service discovery elements (#1770)	5 years ago
talham7391	fa8bf2acba	Clearer error response from push endpoint when labels are malformed (#1750) ... * Clearer error response from push endpoint when labels are malformed * Concise error formatting	5 years ago
Cyril Tovena	4da4d74311	Query range should not support date where start == end. ... Since we are always checking against start inclusively and not end, this means this type of query will always returns no result. Fixes #1702, the frontend was not handling this type of query correctly. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago
Aditya C S	75ef5c82c8	Fix error in templating when extracted key has nil value	5 years ago
Cyril Tovena	4fdfbe8e56	Increase discarded samples when line is too long. (#1708) ... * Increase discarded samples when line is too long. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Add comment on the public props Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> Co-authored-by: Joe Elliott <joe.elliott@grafana.com>	5 years ago
Cyril Tovena	6328146876	Fixes querier cancellation error to return 499 instead of 500. (#1745) ... Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>	5 years ago
David Tennander	5139aa1867	Promtail: Ignore dropped entries in subsequent metric-stages in pipelines. (#1743) ... * Adds testcase for promtail that records metrics after a "match-drop". * Makes subsequent metrics stages in promtail ignore dropped entries. Before this change a "metric" stage after a "match" stage with `action: drop` would result in an error. This specifies the behavior in the docs and adds logic to ignore dropped entries.	5 years ago