113 Commits (72032854d86ef60e8d055a2e6f2d592d28a19766)

Author	SHA1	Message	Date
Xiang Dai	351b4c69ac	delete all duplicate empty blanks (#344) ... Signed-off-by: Xiang Dai <764524258@qq.com>	6 years ago
Tom Wilkie	b115835487	Test & fix queries return multiple streams. (#341) ... Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	6 years ago
Ed	0fdbf24b79	adding third party lib for Glob pattern matching which supports /**/ style subdirectory matching (#331) ... * adding third party lib for Glob pattern matching which supports ** refactor promtail_test to be easier to read and include more tests	6 years ago
Ed	f6d0857ddf	Adding a sync() function to filetarget (#293) ... * Adding a sync() function to filetarget which runs on a timer to catch any events missed by fsnotify * moving where we mark the final position to avoid a race condition exposed in testing, we want to stop the tailer before shutting down our tailer loop * Fix changes from PR and refactor to remove circular imports and make config consistent * Adding an "integration" type unit test which tests promtail as a whole * fixing a bunch of imports I munged moved tailer to its own file refactoring filetarget to make synclogic easier to follow and reused as much as possible	6 years ago
Justin Thomas	e1bc7ea6bb	typo in EntryParser comment	6 years ago
Tom Wilkie	40ae03b48a	Include the stream's labels in OOO error responses. (#304) ... * Include the stream's labels in OOO error responses. Also, read the body of error responses and log them. And retries 500s and connection errors. Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com> * Log close errors. Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com> * Update pkg/promtail/client/client.go Co-Authored-By: tomwilkie <tomwilkie@users.noreply.github.com> * Log retries at warn, final errors at error. Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	6 years ago
Anthony Woods	61fc02a002	set currLabels when currEntry is set. (#308) ... * set currLabels when creating new heapIterator fixes #307 * update unit tests to ensure labels are set correctly	6 years ago
Tom Wilkie	e1c5c4be4a	Basic distributor & ingester tests. ... Proves that they can accept multiple samples with the same timestamp. Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	6 years ago
Tom Wilkie	7527487f31	Track the right number of streams, and cancel the push when the request is cancelled. ... This was broken in a422f394bb. Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	6 years ago
Tom Wilkie	961dd3513f	Typo. ... Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	6 years ago
Tom Wilkie	db26fd3691	Update comment. ... Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	6 years ago
Tom Wilkie	44814cc274	Compare lines directly. ... Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	6 years ago
Tom Wilkie	42df286b7e	Factor out mostCommon function, test it, fix it. ... Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	6 years ago
Tom Wilkie	db36b3c74e	Update heap iterator to allow for entries with duplicate timestamps. ... Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	6 years ago
Tom Wilkie	26f3b52096	Relax restriction on timestamp reuse in chunks, add test. ... Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	6 years ago
woodsaj	93fbd40e2f	fix promtail_request_duration metric	6 years ago
Tom Wilkie	2365004683	Explicty encode the chunk. ... I broke this when I updated the vendored Cortex, which includes ac65306519 (diff-42196489d8c6339357699ab3656cb2e1R315). Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	6 years ago
Tom Wilkie	10d49e798b	Remove stream from index when its remove from memory. ... Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	6 years ago
woodsaj	4582c4303b	more nits	6 years ago
woodsaj	ceb9205ad2	address review feedback	6 years ago
woodsaj	329ed9a123	start and stop modules in correct order ... When starting up, start all dependencies first then start the target module. When stopping, reverse the order to ensure we dont try and stop a module while there is still something running that needs it. fixes #105	6 years ago
Tom Wilkie	f6acb87d1f	Correctly propagate errors in the distributor; should now return 4xx on OOO entries. ... Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	6 years ago
Tom Wilkie	68e1a049a0	lint ... Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	6 years ago
Tom Wilkie	9d7691a52e	Move stuff from file -> targets to prevent the stutter; move tests about; update broken rebase. ... Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	6 years ago
Christian Simon	31789eafc4	Refactor structure of promtail ... This allows easier integration of other targets than files for promtail. Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	6 years ago
Anthony Woods	df84364788	handle symbolic links correctly (#275) ... * handle symbolic links correctly When tailing a file that is rotated we expect that a fsnotify.Rename event will be sent when the old file is renamed and a fsnotif.Create event will be sent when the replacement file is created. This allows us to close and re-open the file. However if the file we are tailing is a symbolic link (and the target is in another directory) we wont get these events. To address this we need to tail the target file directly. fixes #274 * handle symlinks being re-linked When a symbolic link is re-linked we will only get a fsnotify.Create event. No rename or delete event will have been sent to trigger the tailer to be stoped. When this happens we should stop the current tailer for the path and start a new one. This will allow us to perform a new Readink on the path to get the updated target filename.	6 years ago
Goutham Veeramachaneni	7763e8f3dd	Flush idle chunks properly (#209) ... * Flush idle chunks properly Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com> * Review feedback Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com> * Review feedback. Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com> * Check the correct chunk is full before appending a new chunk. Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com> * Add the config to helm and ksonnet too. Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	6 years ago
Tom Wilkie	a422f394bb	Wait for the right number of streams. ... Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	6 years ago
Tom Wilkie	29c2213d1a	Add more metrics to promtail. ... Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	6 years ago
Tom Wilkie	eeac68f183	Log filename with parsing errors. ... Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	6 years ago
Tom Wilkie	f6d547d32d	Don't log request body. ... Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	6 years ago
Tom Wilkie	ab4c5bebab	Validate labels in the distributor. (#251) ... Also, make ErrOutOfOrder a HTTP 4xx error and continue to append even when we encourter an out of order entry. Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	6 years ago
Ed	3e6a75e182	removing incorrect calling of positions.Stop from within a target as it runs single instance within promtail, moved the call to Stop up to promtail.go within the Shutdown() method. Should Fix #249 (#252)	6 years ago
Ed	07f20b0db3	Unit tests for Promtail (#244) ... * Adding a test for target.go Fixing some shutdown issues where tailers were stopped but didn't properly wait for the current position to be saved to file Fixed an issue where an empty directory would lead to nothing being tailed * adding more tests for target.go * renaming quitComplete -> done per feedback making PositionsFile private again, no reason for it to be public removed unnecessary save of empty file on creation of positions file * fixing lint errors * cleaning up some comments removing TODO around directories, added a test to verify behavior some more cleanup per review * fixing imports	6 years ago
Anthony Woods	fed3943596	allow promtail path to be a glob match (#206) ... * allow promtail target path to be a glob match Promtail now expects __path__ to be a glob. All files that match the glob expression will be tailed. If new files are created that match the glob, they will immediately be tailed. * remove entry from positions file when file is deleted * fix syncing the current positions to positions file * handle file renames When a file is renamed a "rename" and a "create" event are sent. We need to treat the rename as a remove. If the new filename matches the path glob, then we will start tailing it when the "create" event is reveived. * lint	6 years ago
Ken Haines	99d0f97a5c	Added call to StrorageConfig.RegisterFlags, in main.go for loki. (#216)	6 years ago
Ye Ben	904c190364	add readLines metric (#214)	6 years ago
Adam Shannon	eb390523e3	all: update dependencies (#211) ... * all: update dependencies * Update some more dependancies and pin the right versions for Prometheus deps. Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	6 years ago
Goutham Veeramachaneni	9b67d6cb7a	Rename instance to tenant to avoid clash with Prom (#164) ... Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com>	7 years ago
Goutham Veeramachaneni	8609fcab74	Cut new blocks once the existing ones are full (#71) ... * Cut new blocks once the existing ones are full Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com> * Address PR comments Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com>	7 years ago
Tom Wilkie	da27baad2c	Support for JSON, not just protos. (#152) ... * Support for JSON, not just protos. Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com> * s/timestamp/ts/ Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	7 years ago
Goutham Veeramachaneni	7bcdf0e0b1	Tenant metrics (#135) ... * Fix stream tracking metric Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com> * Track lines and bytes received per user Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com> * Use instance instead of org Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com> * Nuke dead code Usage has been removed as part of perf-optimisation: 3006e3901d (diff-180f56d9aaf5a9aa079e6fc9cfcf1bc8L365) Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com>	7 years ago
Tom Wilkie	d6e8788eae	Don't forget to apply regexp filters to chunk entries. (#124) ... Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	7 years ago
Egor Kovetskiy	4d78e165dc	Fix duplicating base directory for file create event (#128) ... Signed-off-by: Egor Kovetskiy <e.kovetskiy@gmail.com>	7 years ago
Yeh-lei Wu	251017e41e	Refine ingester locks: eliminate smell & allow concurrent ingester query (#120) ... Signed-off-by: alei <rayingecho@gmail.com> Fix chunk retain period Signed-off-by: alei <rayingecho@gmail.com> Fix undefined symbol Signed-off-by: alei <rayingecho@gmail.com>	7 years ago
Eugen Biegler	1d87a2285e	Add line length to batchSize (#108) ... * Add line length to batchSize Signed-off-by: Eugen Biegler <nupagadi@posteo.de> * Reset batchSize in timer condition too Signed-off-by: Eugen Biegler <nupagadi@posteo.de>	7 years ago
Ye Ben	248c3756a2	fix a typo Eeror -> Error (#66) ... Signed-off-by: yeya24 <ben.ye@daocloud.io>	7 years ago
Tom Wilkie	9f257c591a	[WIP] Persistence (#44) ... Persistence chunks using the Cortex chunk store.	7 years ago
Goutham Veeramachaneni	f2de97c880	Add external labels support (#43) ... Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com>	7 years ago
Tom Wilkie	5f73598389	Rename Tempo to Loki. (#36) ... * Rename Tempo to Loki. Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com> * Use new build image, don't delete generated files on clean. Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	7 years ago