73 Commits (9faa8bcf1dc819179bfc29d3d5d9a7d0878717c5)

Author	SHA1	Message	Date
Joe Elliott	7794d15b6a	Extended test query to expose broken query. Parsed logql correctly in validate (#1044) ... Signed-off-by: Joe Elliott <number101010@gmail.com>	6 years ago
Joe Elliott	9614b021d1	Loki HTTP/JSON Model Layer (#1022) ... * First pass data model Signed-off-by: Joe Elliott <number101010@gmail.com> * Use prom model b/c we're serializing promql objects Signed-off-by: Joe Elliott <number101010@gmail.com> * Added legacy query support and tests Signed-off-by: Joe Elliott <number101010@gmail.com> * Added legacy label test Signed-off-by: Joe Elliott <number101010@gmail.com> * Added tail response marshalling and tests Signed-off-by: Joe Elliott <number101010@gmail.com> * Fixed marshallers and test Signed-off-by: Joe Elliott <number101010@gmail.com> * Expanded legacy test cases Signed-off-by: Joe Elliott <number101010@gmail.com> * Dropped streams nano test Signed-off-by: Joe Elliott <number101010@gmail.com> * First pass v1 new objects Signed-off-by: Joe Elliott <number101010@gmail.com> * Added failing tail response tests Signed-off-by: Joe Elliott <number101010@gmail.com> * Added failing tailresponse test Signed-off-by: Joe Elliott <number101010@gmail.com> * Partial v1 tests Signed-off-by: Joe Elliott <number101010@gmail.com> * Improved legacy labels test Signed-off-by: Joe Elliott <number101010@gmail.com> * Improved legacy query tests Signed-off-by: Joe Elliott <number101010@gmail.com> * Moved all legacy tests to new method Signed-off-by: Joe Elliott <number101010@gmail.com> * Added v1 tests and fixed stream marshalling bug Signed-off-by: Joe Elliott <number101010@gmail.com> * First pass new Model Signed-off-by: Joe Elliott <number101010@gmail.com> * Added vector test Signed-off-by: Joe Elliott <number101010@gmail.com> * Added matrix tests Signed-off-by: Joe Elliott <number101010@gmail.com> * Added conversions for all things except tailed responses Signed-off-by: Joe Elliott <number101010@gmail.com> * Fixed mixed case issues Signed-off-by: Joe Elliott <number101010@gmail.com> * Fixed tail marshalling Signed-off-by: Joe Elliott <number101010@gmail.com> * Removed unused testStream Signed-off-by: Joe Elliott <number101010@gmail.com> * Moved TailResponse to loghttp Signed-off-by: Joe Elliott <number101010@gmail.com> * Removed legacy tailresponse objects in favor of actual legacy tailresponse objects Signed-off-by: Joe Elliott <number101010@gmail.com> * Updated v1 methods to take legacy tail objects Signed-off-by: Joe Elliott <number101010@gmail.com> * Cleaned up tests. Added some comments Signed-off-by: Joe Elliott <number101010@gmail.com> * Versioned tail endpoint Signed-off-by: Joe Elliott <number101010@gmail.com> * Improved readability on loghttp packages in http.go Signed-off-by: Joe Elliott <number101010@gmail.com> * Removed new as a var name Signed-off-by: Joe Elliott <number101010@gmail.com> * Started all error messages with lowercase alerts Signed-off-by: Joe Elliott <number101010@gmail.com> * new => ret Signed-off-by: Joe Elliott <number101010@gmail.com> * Added comments on exported methods Signed-off-by: Joe Elliott <number101010@gmail.com> * Removed two personal notes Signed-off-by: Joe Elliott <number101010@gmail.com> * Changed legacy package name to loghttp Signed-off-by: Joe Elliott <number101010@gmail.com> * Moved and renamed loghttp v1 package Signed-off-by: Joe Elliott <number101010@gmail.com> * Moved marshalling code out of model Signed-off-by: Joe Elliott <number101010@gmail.com> * Added package comments Signed-off-by: Joe Elliott <number101010@gmail.com> * Added legacy testing Signed-off-by: Joe Elliott <number101010@gmail.com> * Changed DroppedStream slice to value type for consistency Signed-off-by: Joe Elliott <number101010@gmail.com> * gofmt'ed test files Signed-off-by: Joe Elliott <number101010@gmail.com> * Cleaned up linting issues Signed-off-by: Joe Elliott <number101010@gmail.com> * Minor comment cleanup Signed-off-by: Joe Elliott <number101010@gmail.com> * Adjusted GOGC to make CircleCI happy Signed-off-by: Joe Elliott <number101010@gmail.com> * Changed legacy => loghttp for consistency Signed-off-by: Joe Elliott <number101010@gmail.com> * Fixed matrix error message to be correct Signed-off-by: Joe Elliott <number101010@gmail.com> * Moved label query over to loghttp response Signed-off-by: Joe Elliott <number101010@gmail.com> * Added marshal loop tests Signed-off-by: Joe Elliott <number101010@gmail.com> * Added response type test Signed-off-by: Joe Elliott <number101010@gmail.com> * Fixed tail response marshal/unmarshal Signed-off-by: Joe Elliott <number101010@gmail.com> * Passing unmarshal/marshal queryresponse tests Signed-off-by: Joe Elliott <number101010@gmail.com> * Fixed vector and matrix Signed-off-by: Joe Elliott <number101010@gmail.com> * Added output support for streams minus ordering Signed-off-by: Joe Elliott <number101010@gmail.com> * Fixed tailing Signed-off-by: Joe Elliott <number101010@gmail.com> * Fixed output tests Signed-off-by: Joe Elliott <number101010@gmail.com> * Fixed query tests Signed-off-by: Joe Elliott <number101010@gmail.com> * Order log output Signed-off-by: Joe Elliott <number101010@gmail.com> * Use labels instead of stream Signed-off-by: Joe Elliott <number101010@gmail.com> * Lowered parallelization for CircleCI Signed-off-by: Joe Elliott <number101010@gmail.com>	6 years ago
Sandeep Sukhani	ec5bc7054d	limits: limits implementation for loki (#948) ... * limits implementation for loki Added following new limits: Length of query Number of active streams Number of streams matcher per query Ingestion rate * periodically reload ingestion rate limits to apply latest rates from overrides	6 years ago
polar3130	b2cd49a13e	fix typo in error messages (#1030)	6 years ago
Sandeep Sukhani	107dfd3607	Use grpc client config from cortex for Ingester to get more control (#755) ... * Use grpc client config from cortex for Ingester for more control over the connection * moved ingester client config block at right place in production ksonnet	6 years ago
Cyril Tovena	364b5bc4f6	Make Loki HTTP API more similar to Prometheus	6 years ago
Cyril Tovena	8f9cd97880	Improve API compatibility. ... - keep supporting regex for old endpoint. - use reserved keyword for removed property in grpc	6 years ago
Cyril Tovena	8bd9f752aa	Adds more tests for range queries	6 years ago
Cyril Tovena	7d63795350	fix linter & better comments	6 years ago
Cyril Tovena	7f3db9d7f0	LogQL: Vector and Range Vector Aggregation. ... - adds avg,min,max,top,bottomk,stddev,stdvar,count - updates api documentation - adds tests Improve yacc & go lexer to understand duration Remove support for regexp in all queries Clean up querier and logselector	6 years ago
Robert Fratto	9484632d3d	pkg/querier: fix concurrent access to querier tail clients (#944)	6 years ago
Marco Pracucci	e5d6f3c317	Fix panic in tailer when an ingester is removed from the ring while tailing (#897)	6 years ago
Marco Pracucci	b9185324f4	Improved Tailer loop (dropped entries, logic refactoring, introduced tests)	6 years ago
Sandeep Sukhani	2b8a3d9794	live-tailing: preload all the historic entries before query context is cancelled (#862) ... * preload all the historic entries before query context is cancelled * fix lint error	6 years ago
Aditya C S	b5d8c77b99	Readiness probe for querier (#854)	6 years ago
Sandeep Sukhani	f38fa2d184	gofmt file	6 years ago
Sandeep Sukhani	bddf4fc9be	rebased and removed some unwanted code	6 years ago
Sandeep Sukhani	3457f44cf5	Fix querying historic entries while live tailing logs ... Querying logs in backward direction, then fetching all the logs upto the query limit and then iterating over it in reverse order	6 years ago
Cyril Tovena	4d48972939	Query storage by iterating through chunks by batches. ... Also adds more tests for the store	6 years ago
Marco Pracucci	476c3961c2	Added -querier.query_timeout support (#788)	6 years ago
sh0rez	38cb093358	feat(loki): extended tailing (#764) ... * refactor(querier/ingester): TailRequest Lookback window Moves the specifications of the Lookback Window out of the logproto.QueryRequest into it's own type logproto.Lookback. This is required, because the Lookback Window will be used in the TailRequest as well. * feat(querier): parse Lookback from HTTP Request * feat(logcli): send Lookback Window spec with tail request * feat(querier): include historic entries in tail mode Extends tailing by sending a configurable amount of historic entries with before the live entries. This enables a behaviour that is closer to kubectl logs -f and docker logs -f. It is implemented by running a regular Query before subscribing to the ingesters. * fix: adapt tests to Lookback change * feat(querier): check all errors to make the linter happy * fix(logproto): flatten Lookback window spec Flattens the Lookback window spec into the individual queries * fix(ingester): adapt test to Lookback flatten	6 years ago
Sandeep Sukhani	3a2b569684	Added tail length limit to limit duration of live tailing of logs to 1 hour (#756) ... * Added tail length limit to limit duration of live tailing of logs to 1 hour * Some code refactoring suggested from PR review for live tailing duration limit * Fixed error messages in live tailing of logs * Fixed lint errors	6 years ago
Cyril Tovena	3346ce1b40	Storage memory improvement (#713) ... * add benchmark for storage queries * improve iterator to load only on next * fix memory retained by lazy chunks * reverse backward lazy iterator	6 years ago
Cyril Tovena	86bc3c0518	Query label values and names are now fetched from the store. (#521) ... * Query label values and names are now fetched from the store. A time range is now required by the /api/prom/label with a sane default (6 hours from now). * fix http querystring and update doc * update vendor * rebased	6 years ago
Sandeep Sukhani	4c7138231f	Removed test for tailer in querier due to issues with synchronization	6 years ago
Sandeep Sukhani	2a8a03795a	Improvements in live tailing of logs (#541) ... * Improvements in live tailing of logs Instead of polling for new logs, a grpc stream is opened between ingester and querier to get live logs Querier reconnects to disconnected or newly added ingesters * Added more comments to code for live log tailing * Some code refactoring in live log tailing * handling delayfor in logcli, max delayfor to be 5 seconds * some changes in tail response in live tailing * Fixed issue with stopping ingesters gracefully when live tailing is being used * Added tests for tailer in querier * Live tailing made logql compatible, some code refactoring suggested in PR	6 years ago
Cyril Tovena	f939e42afd	add support for RFC3339Nano in query timestamps (#656)	6 years ago
Cyril Tovena	922f1daf5d	Remove label __name__ from store querier (#648)	6 years ago
Sandeep Sukhani	3d3c05717a	Update cortex vendor (#610) ... * Update cortex vendor Use query max look back from cortex Update config changes from cortex Fixed breaking code due to updating cortex vendor * fixed linter error	6 years ago
Cyril Tovena	2ad98007ff	fixes chunks lazy loading (#595)	6 years ago
Sandeep Sukhani	f5c8b22b34	Change tail response for backward compatibilty with future changes (#590) ... Changed it to dict with single key called "streams" and value set to list of logproto.Stream	6 years ago
Sandeep Sukhani	7b584b54f7	Limiting query start time with config (#572) ... * Limiting query start time with config * Fixed maxLookBackPeriod check, added it to existing configs, some other fixes * gofmted file * Changed maxLookBackPeriod to 4 weeks in helm chart * Setting maxLookBackPeriod in helm chart to 0 and bumping up the helm chart version * changed maxLookBackPeriod in yaml in cmd to 0 and some nit	6 years ago
Tom Wilkie	80bbd04778	Add an extra filter step if the user specifies a regex using old API. ... Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	6 years ago
Tom Wilkie	e73024d3a4	Remove RegexpIter, we don't need it. ... Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	6 years ago
Tom Wilkie	bfb34e4070	Wire up LogQL filters in the ingester. ... Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	6 years ago
Tom Wilkie	5e3630815b	Errors and wiring. ... Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	6 years ago
Tom Wilkie	b16da1d7c3	Parse pipelines of regex matches. ... Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	6 years ago
Cyril Tovena	8d7efddde2	switch to golangci-lint linter (#539) ... * switch to golangci-lint linter * fixes for review comments	6 years ago
Goutham Veeramachaneni	27ca58e79f	Lazy load chunks (#435) ... * Lazy load chunks for queries. Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com> * Add context to the lazy chunk calls Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com> * Make sure labels are plumbed through Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com> * Pre-load non-overlapping chunk iterators Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com> * Review feedback Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com>	6 years ago
Sandeep Sukhani	1f1706f467	Move live log tailing behind websocket on server (#439) ... * Move live log tailing behind websocket on server Live log tailing now requires client to just open a websocket and keep listening to it for getting live logs for matching query * Fixed linting errors * Unexported tailQuery querier method * Added a delay in querying of logs to not miss delayed entries * Renamed variable for better readability * Some code refactoring suggested in PR review * Fixed linter errors * Code refactoring requested in PR review * Disabling linting for writeHTTPErrorResponse since same value for an input is being passed from all function calls for now * Removed a function which already existed * Improved printing of log entries using cli and some other code refactoring * Checking no-labels flag before printing log stream while tailing	6 years ago
Tom Wilkie	d6e8788eae	Don't forget to apply regexp filters to chunk entries. (#124) ... Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	7 years ago
Tom Wilkie	9f257c591a	[WIP] Persistence (#44) ... Persistence chunks using the Cortex chunk store.	7 years ago
Tom Wilkie	5f73598389	Rename Tempo to Loki. (#36) ... * Rename Tempo to Loki. Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com> * Use new build image, don't delete generated files on clean. Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	7 years ago
Tom Wilkie	1fef92bc17	Lint stuff (#27) ... * Check in generated files, to make getting started easier. Add gometalinter, and a lint check for changes to the generated files. Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com> * Make lint pass? Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	7 years ago
Tom Wilkie	4635769b4f	Don't register variables twice. ... Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	7 years ago
Tom Wilkie	3b69cf127d	Don't repeat client config. ... Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	7 years ago
Tom Wilkie	0ffb432f30	Update for cortex changes. ... Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	7 years ago
Tom Wilkie	3f62b4ec36	s/logish/tempo/ ... Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	7 years ago
Tom Wilkie	96223322ba	Make queries take nanoseconds since epoch. ... Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	7 years ago
Tom Wilkie	5ec9dfc5d1	Fix logic. ... Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	7 years ago