139 Commits (439e7bb7d66fac44f5c85d5937df057348101824)

Author	SHA1	Message	Date
Edward Welch	4b8eda64f5	PR feedback	6 years ago
Edward Welch	db31d96425	Cleaning up metrics when files are deleted, adding tests to verify the metrics.	6 years ago
Edward Welch	c681c76cc8	periodically clean up positions file, removing any files which no longer exist	6 years ago
Edward Welch	3f8963ce01	Updating hpcloud tail library to forked version modified for our purposes. ... Changes to the library move the handling of symlinks and re-opening to the tail library. fsnotify events are only used now to start new tailers for files not previously seen.	6 years ago
Sandeep Sukhani	1f1706f467	Move live log tailing behind websocket on server (#439) ... * Move live log tailing behind websocket on server Live log tailing now requires client to just open a websocket and keep listening to it for getting live logs for matching query * Fixed linting errors * Unexported tailQuery querier method * Added a delay in querying of logs to not miss delayed entries * Renamed variable for better readability * Some code refactoring suggested in PR review * Fixed linter errors * Code refactoring requested in PR review * Disabling linting for writeHTTPErrorResponse since same value for an input is being passed from all function calls for now * Removed a function which already existed * Improved printing of log entries using cli and some other code refactoring * Checking no-labels flag before printing log stream while tailing	6 years ago
Steven Sheehy	8c0b3d8099	Add a /ready endpoint to promtail ... Signed-off-by: Steven Sheehy <ssheehy@firescope.com>	6 years ago
Xiang Dai	b5e632f62c	Remove useless func (#472) ... Remove useless func No one called this func, keep it only mislead. Signed-off-by: Xiang Dai <764524258@qq.com>	6 years ago
Sandeep Sukhani	bc571c54c9	Fix failing test `TestGlobWithMultipleFiles` in pkg/promtail/targets (#466) ... TestClient used for watching log file events appends logs to a list Due to race condition some of the logs added to it goes missing Adding a Mutex to it to lock messages before adding logs to it	6 years ago
Edward Welch	947428d77e	Adding a check to make sure we don't set tailer offset past the end of the file. ... Fixing lagging alert to also catch when the read_bytes is past the end of the file. Adding an alert to let us know if we are missing tailing of any files.	6 years ago
Edward Welch	c994823369	Change hpcloud tailer library to poll mode to workaround bugs in the event driven code within the library ... Had to rework tests with some waits for messages because polling is slower There are several longer sleeps in the promtail_test which are not ideal and have been marked with FIXME to remove once we have a better solution	6 years ago
Xiang Dai	2c2b42f885	Add comment about external_labels ... Signed-off-by: Xiang Dai <764524258@qq.com>	6 years ago
Tom Wilkie	d495c68ef9	Need to set the number of bytes read, not add them. ... Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	6 years ago
Tom Wilkie	9ab14c9e0a	Add readLines.Inc() call back. ... Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	6 years ago
Tom Wilkie	b2fafdf360	Export position in file, not bytes read. ... Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	6 years ago
Ed	ca1a5be975	Adding metric and alert to notify promtail falling behind (#412) ... * Adding the metric promtail_file_bytes_total which will report on the total size of each file with an active tailer. Added an alert which will fire whenever the total bytes for a file is 100kb or more ahead the read bytes for more than 15mins. Added an additional alert to track client bytes ready to send (encoded bytes) vs actually sent so we can add an alert on promtail send failures and still track stats on encoded bytes being sent. * Moving where we update the total file bytes outside the tailer so it shouldn't get hung up by tailer issues. * Recording file size based on Glob matches rather than what we are tailing to give even better insight	6 years ago
Goutham Veeramachaneni	b8d81ababa	Add tracing to Loki (#328) ... * Add tracing to loki Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com> * Use the op names from common/server We don't need to the tracing middleware again. Further, the op names are not sanitised here. Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com>	6 years ago
Goutham Veeramachaneni	ddddd14726	Update cortex vendoring (#325) ... * Update cortex vendoring Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com> * Fix API breakage Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com>	6 years ago
Edward Welch	972531d3e5	fixed one bug which was probably unrelated to the test failure and broke out the test to avoid some non-deterministic behaviors in querying the ingester	6 years ago
Steven Sheehy	600b6c1aa9	Allow blank log lines for CRI ... Signed-off-by: Steven Sheehy <ssheehy@firescope.com>	6 years ago
Ed	46334ed5da	adding a few more test cases for entry parsing docker format (#369) ... * adding a few more test cases for entry parsing docker format * adding testify assert and require to clean up tests * some tweaking to improve readability	6 years ago
Ed	19260d4425	Adding a configurable timeout for calls to the loki server. (#368) ... Adding a configurable timeout for calls to the loki server. Updating promtail test to set sane defaults for all configurations and then override for testing. Removing .idea folder from .gitignore, we should encourage people (and myself) to use global ignore for IDE configs	6 years ago
Steven Sheehy	fa917b61b2	Address code review items ... Signed-off-by: Steven Sheehy <ssheehy@firescope.com>	6 years ago
Steven Sheehy	9d0a8c1370	Add stream label to CRI ... Signed-off-by: Steven Sheehy <ssheehy@firescope.com>	6 years ago
Steven Sheehy	f13b232de7	Rename to CRI as CRI-O also implements standard ... Signed-off-by: Steven Sheehy <ssheehy@firescope.com>	6 years ago
Steven Sheehy	3d664474ec	Add containerd log format support ... Signed-off-by: Steven Sheehy <ssheehy@firescope.com>	6 years ago
Baokun Lee	e6192a6908	Fix start tailing panic ... Signed-off-by: kun <oiooj@qq.com>	6 years ago
Xiang Dai	351b4c69ac	delete all duplicate empty blanks (#344) ... Signed-off-by: Xiang Dai <764524258@qq.com>	6 years ago
Tom Wilkie	b115835487	Test & fix queries return multiple streams. (#341) ... Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	6 years ago
Ed	0fdbf24b79	adding third party lib for Glob pattern matching which supports /**/ style subdirectory matching (#331) ... * adding third party lib for Glob pattern matching which supports ** refactor promtail_test to be easier to read and include more tests	6 years ago
Ed	f6d0857ddf	Adding a sync() function to filetarget (#293) ... * Adding a sync() function to filetarget which runs on a timer to catch any events missed by fsnotify * moving where we mark the final position to avoid a race condition exposed in testing, we want to stop the tailer before shutting down our tailer loop * Fix changes from PR and refactor to remove circular imports and make config consistent * Adding an "integration" type unit test which tests promtail as a whole * fixing a bunch of imports I munged moved tailer to its own file refactoring filetarget to make synclogic easier to follow and reused as much as possible	6 years ago
Justin Thomas	e1bc7ea6bb	typo in EntryParser comment	6 years ago
Tom Wilkie	40ae03b48a	Include the stream's labels in OOO error responses. (#304) ... * Include the stream's labels in OOO error responses. Also, read the body of error responses and log them. And retries 500s and connection errors. Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com> * Log close errors. Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com> * Update pkg/promtail/client/client.go Co-Authored-By: tomwilkie <tomwilkie@users.noreply.github.com> * Log retries at warn, final errors at error. Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	6 years ago
Anthony Woods	61fc02a002	set currLabels when currEntry is set. (#308) ... * set currLabels when creating new heapIterator fixes #307 * update unit tests to ensure labels are set correctly	6 years ago
Tom Wilkie	e1c5c4be4a	Basic distributor & ingester tests. ... Proves that they can accept multiple samples with the same timestamp. Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	6 years ago
Tom Wilkie	7527487f31	Track the right number of streams, and cancel the push when the request is cancelled. ... This was broken in a422f394bb. Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	6 years ago
Tom Wilkie	961dd3513f	Typo. ... Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	6 years ago
Tom Wilkie	db26fd3691	Update comment. ... Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	6 years ago
Tom Wilkie	44814cc274	Compare lines directly. ... Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	6 years ago
Tom Wilkie	42df286b7e	Factor out mostCommon function, test it, fix it. ... Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	6 years ago
Tom Wilkie	db36b3c74e	Update heap iterator to allow for entries with duplicate timestamps. ... Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	6 years ago
Tom Wilkie	26f3b52096	Relax restriction on timestamp reuse in chunks, add test. ... Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	6 years ago
woodsaj	93fbd40e2f	fix promtail_request_duration metric	6 years ago
Tom Wilkie	2365004683	Explicty encode the chunk. ... I broke this when I updated the vendored Cortex, which includes ac65306519 (diff-42196489d8c6339357699ab3656cb2e1R315). Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	6 years ago
Tom Wilkie	10d49e798b	Remove stream from index when its remove from memory. ... Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	6 years ago
woodsaj	4582c4303b	more nits	6 years ago
woodsaj	ceb9205ad2	address review feedback	6 years ago
woodsaj	329ed9a123	start and stop modules in correct order ... When starting up, start all dependencies first then start the target module. When stopping, reverse the order to ensure we dont try and stop a module while there is still something running that needs it. fixes #105	6 years ago
Tom Wilkie	f6acb87d1f	Correctly propagate errors in the distributor; should now return 4xx on OOO entries. ... Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	6 years ago
Tom Wilkie	68e1a049a0	lint ... Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	6 years ago
Tom Wilkie	9d7691a52e	Move stuff from file -> targets to prevent the stutter; move tests about; update broken rebase. ... Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	6 years ago