72 Commits (94c252b91fabdafff2e833bf09eaa57766fbafa7)

Author	SHA1	Message	Date
Sandeep Sukhani	2a8a03795a	Improvements in live tailing of logs (#541) ... * Improvements in live tailing of logs Instead of polling for new logs, a grpc stream is opened between ingester and querier to get live logs Querier reconnects to disconnected or newly added ingesters * Added more comments to code for live log tailing * Some code refactoring in live log tailing * handling delayfor in logcli, max delayfor to be 5 seconds * some changes in tail response in live tailing * Fixed issue with stopping ingesters gracefully when live tailing is being used * Added tests for tailer in querier * Live tailing made logql compatible, some code refactoring suggested in PR	7 years ago
Sandeep Sukhani	3d3c05717a	Update cortex vendor (#610) ... * Update cortex vendor Use query max look back from cortex Update config changes from cortex Fixed breaking code due to updating cortex vendor * fixed linter error	7 years ago
Cyril Tovena	ec7d453fef	fixes ingester querier not honoring filters (#594)	7 years ago
Steven Sheehy	c1603363ec	Improve readiness probe output (#556) ... Signed-off-by: Steven Sheehy <ssheehy@firescope.com>	7 years ago
Goutham Veeramachaneni	3d7119d8a7	Add a histogram for chunk encoding time (#565) ... Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com>	7 years ago
Tom Wilkie	beb0052ad0	Remove unnecessarily long wait times in tests. ... Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	7 years ago
Edward Welch	6446503593	Close all chunks before flushing. ... Moved the check for already closed to make sure we honor this check when `immediate` flushing is requested	7 years ago
Tom Wilkie	80bbd04778	Add an extra filter step if the user specifies a regex using old API. ... Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	7 years ago
Tom Wilkie	32d2f64b31	Fix ingester test. ... Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	7 years ago
Tom Wilkie	bfb34e4070	Wire up LogQL filters in the ingester. ... Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	7 years ago
Tom Wilkie	b16da1d7c3	Parse pipelines of regex matches. ... Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	7 years ago
Cyril Tovena	fa4f936fcb	improves memory usage of golangci-linter	7 years ago
Cyril Tovena	8d7efddde2	switch to golangci-lint linter (#539) ... * switch to golangci-lint linter * fixes for review comments	7 years ago
Goutham Veeramachaneni	017bcefb3f	Better buckets for chunk sizes (#529) ... Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com>	7 years ago
Ed	fe53924ff8	using the timestamp from the log file might lead to undesired behavior as it could be way in the past or future depending on the clock for the source system generating the logs. Given that it's compared to the servers time.Now() when looking for idle chunks it's probably best to use the server time when setting the value. (#546)	7 years ago
Steven Sheehy	85a9e82663	Fix #531 chunk_block_size not found ... Signed-off-by: Steven Sheehy <ssheehy@firescope.com>	7 years ago
Xiang Dai	aa4684ee52	support config chunk size (#464) ... * support config chunk size Signed-off-by: Xiang Dai <764524258@qq.com> * fix lint Signed-off-by: Xiang Dai <764524258@qq.com> * review feedback	7 years ago
Goutham Veeramachaneni	c1bc17e778	fix linting (#514) ... Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com>	7 years ago
Goutham Veeramachaneni	e54a1b21c3	vendor: update cortex (#512) ... Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com>	7 years ago
Goutham Veeramachaneni	cfb4c60813	Add stats about chunk sizes and ages (#513) ... Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com>	7 years ago
Goutham Veeramachaneni	27ca58e79f	Lazy load chunks (#435) ... * Lazy load chunks for queries. Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com> * Add context to the lazy chunk calls Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com> * Make sure labels are plumbed through Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com> * Pre-load non-overlapping chunk iterators Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com> * Review feedback Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com>	7 years ago
Goutham Veeramachaneni	ddddd14726	Update cortex vendoring (#325) ... * Update cortex vendoring Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com> * Fix API breakage Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com>	7 years ago
Edward Welch	972531d3e5	fixed one bug which was probably unrelated to the test failure and broke out the test to avoid some non-deterministic behaviors in querying the ingester	7 years ago
Tom Wilkie	b115835487	Test & fix queries return multiple streams. (#341) ... Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	7 years ago
Tom Wilkie	40ae03b48a	Include the stream's labels in OOO error responses. (#304) ... * Include the stream's labels in OOO error responses. Also, read the body of error responses and log them. And retries 500s and connection errors. Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com> * Log close errors. Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com> * Update pkg/promtail/client/client.go Co-Authored-By: tomwilkie <tomwilkie@users.noreply.github.com> * Log retries at warn, final errors at error. Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	7 years ago
Tom Wilkie	e1c5c4be4a	Basic distributor & ingester tests. ... Proves that they can accept multiple samples with the same timestamp. Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	7 years ago
Tom Wilkie	2365004683	Explicty encode the chunk. ... I broke this when I updated the vendored Cortex, which includes ac65306519 (diff-42196489d8c6339357699ab3656cb2e1R315). Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	7 years ago
Tom Wilkie	10d49e798b	Remove stream from index when its remove from memory. ... Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	7 years ago
Goutham Veeramachaneni	7763e8f3dd	Flush idle chunks properly (#209) ... * Flush idle chunks properly Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com> * Review feedback Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com> * Review feedback. Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com> * Check the correct chunk is full before appending a new chunk. Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com> * Add the config to helm and ksonnet too. Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	7 years ago
Tom Wilkie	ab4c5bebab	Validate labels in the distributor. (#251) ... Also, make ErrOutOfOrder a HTTP 4xx error and continue to append even when we encourter an out of order entry. Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	7 years ago
Goutham Veeramachaneni	9b67d6cb7a	Rename instance to tenant to avoid clash with Prom (#164) ... Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com>	8 years ago
Goutham Veeramachaneni	7bcdf0e0b1	Tenant metrics (#135) ... * Fix stream tracking metric Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com> * Track lines and bytes received per user Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com> * Use instance instead of org Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com> * Nuke dead code Usage has been removed as part of perf-optimisation: 3006e3901d (diff-180f56d9aaf5a9aa079e6fc9cfcf1bc8L365) Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com>	8 years ago
Yeh-lei Wu	251017e41e	Refine ingester locks: eliminate smell & allow concurrent ingester query (#120) ... Signed-off-by: alei <rayingecho@gmail.com> Fix chunk retain period Signed-off-by: alei <rayingecho@gmail.com> Fix undefined symbol Signed-off-by: alei <rayingecho@gmail.com>	8 years ago
Tom Wilkie	9f257c591a	[WIP] Persistence (#44) ... Persistence chunks using the Cortex chunk store.	8 years ago
Tom Wilkie	5f73598389	Rename Tempo to Loki. (#36) ... * Rename Tempo to Loki. Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com> * Use new build image, don't delete generated files on clean. Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	8 years ago
Tom Wilkie	600b24101e	Promtail updates (#30) ... * Make promtail read all its config from a file. Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com> * Use gokit logging everywhere. Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com> * Start work to parse normal log lines. Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com> * update common and kuberesolver. Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com> * lint Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com> * Get EntryParser stuff working, so we can tail raw log files. Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com> * Make sure filename label works. Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com> * lint, remove interfacer. Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	8 years ago
Tom Wilkie	1fef92bc17	Lint stuff (#27) ... * Check in generated files, to make getting started easier. Add gometalinter, and a lint check for changes to the generated files. Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com> * Make lint pass? Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	8 years ago
Tom Wilkie	d06ed8fb1f	Remove flagext, use cortex's. Allow flags to be loaded from a file. ... Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	8 years ago
Tom Wilkie	de05515248	Init pool config correctly. ... Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	8 years ago
Tom Wilkie	3b69cf127d	Don't repeat client config. ... Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	8 years ago
Tom Wilkie	0ffb432f30	Update for cortex changes. ... Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	8 years ago
Tom Wilkie	3f62b4ec36	s/logish/tempo/ ... Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	8 years ago
Goutham Veeramachaneni	8f4e12a5ec	Chunking (#10) ... * Add checkenc without serialisation for now. Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in> * Simplify interface and add serialisatio` Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in> * Move away from \xFF magic to something simple Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in> * Add serialisation and Deserialisation Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in> * Modify interface to be closer to logish interface. Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in> * chunkenc: Fix race b/w append and iteration. Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in> * chunkenc: Make iterators honour bounds Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in> * chunkenc: Remove locks as safety is assured externally Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in> * chunkenc: Add checksums Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in> * Add code quotes around block design. * Split headBlock into it's own type. Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in> * Simplify encoding and decoding. Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in> * Expose flags. Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com> * Use the already existing EntryIterator interface Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com> * Use existing Chunk interface. Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com> * Review feedback Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com> * Integrate the compressed chunk and add metrics Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com>	8 years ago
Tom Wilkie	b247a5450d	Command to fetch label names. ... Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	8 years ago
Tom Wilkie	248f9e25e4	Move iterators to own package; check in protos and parser to make CLI go gettable. ... Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	8 years ago
Tom Wilkie	0fa01651ff	Add cli for getting label names, and sort the labels. ... Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	8 years ago
Tom Wilkie	411e4a8ce4	Plumb through label value queries. ... Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	8 years ago
Tom Wilkie	8df1489ddc	Don't infinitely loop deduping. ... Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	8 years ago
Tom Wilkie	544eed1b95	Add some unit tests for the iterators. ... Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	8 years ago
Tom Wilkie	0443ee7f9e	Minor tidy ups. ... Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	8 years ago