80 Commits (f7d30e2cbca9745c152e2ff5be26dcc2e9b280ea)

Author	SHA1	Message	Date
Robert Fratto	e64c7e5faf	ingester: log error if closing client after transfer fails	7 years ago
Robert Fratto	e4e5e3c1f1	ingester: fix feedback from PR review	7 years ago
Robert Fratto	15cbba7242	ingester: clean up chunk transfer code	7 years ago
Robert Fratto	f8155a1109	ingester: Add test for chunk transfers	7 years ago
Robert Fratto	2eb666b0a8	ingester: fix lint issues for chunk transfers	7 years ago
Robert Fratto	d34b1f7450	ingester: support chunk transfers on ingester shutdown. ... This commit introduces chunk transfers, borrowing the mechanism from Cortex's implementation: when an ingester is shut down with claim on rollout enabled, the ingester will find a pending ingester and transfer all of its chunks to it.	7 years ago
Cyril Tovena	4bf77662d3	rounds nanoseconds boundaries to milliseconds (#771) ... * rounds nanoseconds boundaries to milliseconds * convert also store query	7 years ago
Cyril Tovena	3346ce1b40	Storage memory improvement (#713) ... * add benchmark for storage queries * improve iterator to load only on next * fix memory retained by lazy chunks * reverse backward lazy iterator	7 years ago
Sandeep Sukhani	2a8a03795a	Improvements in live tailing of logs (#541) ... * Improvements in live tailing of logs Instead of polling for new logs, a grpc stream is opened between ingester and querier to get live logs Querier reconnects to disconnected or newly added ingesters * Added more comments to code for live log tailing * Some code refactoring in live log tailing * handling delayfor in logcli, max delayfor to be 5 seconds * some changes in tail response in live tailing * Fixed issue with stopping ingesters gracefully when live tailing is being used * Added tests for tailer in querier * Live tailing made logql compatible, some code refactoring suggested in PR	7 years ago
Sandeep Sukhani	3d3c05717a	Update cortex vendor (#610) ... * Update cortex vendor Use query max look back from cortex Update config changes from cortex Fixed breaking code due to updating cortex vendor * fixed linter error	7 years ago
Cyril Tovena	ec7d453fef	fixes ingester querier not honoring filters (#594)	7 years ago
Steven Sheehy	c1603363ec	Improve readiness probe output (#556) ... Signed-off-by: Steven Sheehy <ssheehy@firescope.com>	7 years ago
Goutham Veeramachaneni	3d7119d8a7	Add a histogram for chunk encoding time (#565) ... Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com>	7 years ago
Tom Wilkie	beb0052ad0	Remove unnecessarily long wait times in tests. ... Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	7 years ago
Edward Welch	6446503593	Close all chunks before flushing. ... Moved the check for already closed to make sure we honor this check when `immediate` flushing is requested	7 years ago
Tom Wilkie	80bbd04778	Add an extra filter step if the user specifies a regex using old API. ... Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	7 years ago
Tom Wilkie	32d2f64b31	Fix ingester test. ... Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	7 years ago
Tom Wilkie	bfb34e4070	Wire up LogQL filters in the ingester. ... Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	7 years ago
Tom Wilkie	b16da1d7c3	Parse pipelines of regex matches. ... Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	7 years ago
Cyril Tovena	fa4f936fcb	improves memory usage of golangci-linter	7 years ago
Cyril Tovena	8d7efddde2	switch to golangci-lint linter (#539) ... * switch to golangci-lint linter * fixes for review comments	7 years ago
Goutham Veeramachaneni	017bcefb3f	Better buckets for chunk sizes (#529) ... Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com>	7 years ago
Ed	fe53924ff8	using the timestamp from the log file might lead to undesired behavior as it could be way in the past or future depending on the clock for the source system generating the logs. Given that it's compared to the servers time.Now() when looking for idle chunks it's probably best to use the server time when setting the value. (#546)	7 years ago
Steven Sheehy	85a9e82663	Fix #531 chunk_block_size not found ... Signed-off-by: Steven Sheehy <ssheehy@firescope.com>	7 years ago
Xiang Dai	aa4684ee52	support config chunk size (#464) ... * support config chunk size Signed-off-by: Xiang Dai <764524258@qq.com> * fix lint Signed-off-by: Xiang Dai <764524258@qq.com> * review feedback	7 years ago
Goutham Veeramachaneni	c1bc17e778	fix linting (#514) ... Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com>	7 years ago
Goutham Veeramachaneni	e54a1b21c3	vendor: update cortex (#512) ... Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com>	7 years ago
Goutham Veeramachaneni	cfb4c60813	Add stats about chunk sizes and ages (#513) ... Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com>	7 years ago
Goutham Veeramachaneni	27ca58e79f	Lazy load chunks (#435) ... * Lazy load chunks for queries. Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com> * Add context to the lazy chunk calls Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com> * Make sure labels are plumbed through Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com> * Pre-load non-overlapping chunk iterators Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com> * Review feedback Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com>	7 years ago
Goutham Veeramachaneni	ddddd14726	Update cortex vendoring (#325) ... * Update cortex vendoring Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com> * Fix API breakage Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com>	7 years ago
Edward Welch	972531d3e5	fixed one bug which was probably unrelated to the test failure and broke out the test to avoid some non-deterministic behaviors in querying the ingester	7 years ago
Tom Wilkie	b115835487	Test & fix queries return multiple streams. (#341) ... Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	7 years ago
Tom Wilkie	40ae03b48a	Include the stream's labels in OOO error responses. (#304) ... * Include the stream's labels in OOO error responses. Also, read the body of error responses and log them. And retries 500s and connection errors. Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com> * Log close errors. Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com> * Update pkg/promtail/client/client.go Co-Authored-By: tomwilkie <tomwilkie@users.noreply.github.com> * Log retries at warn, final errors at error. Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	7 years ago
Tom Wilkie	e1c5c4be4a	Basic distributor & ingester tests. ... Proves that they can accept multiple samples with the same timestamp. Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	7 years ago
Tom Wilkie	2365004683	Explicty encode the chunk. ... I broke this when I updated the vendored Cortex, which includes ac65306519 (diff-42196489d8c6339357699ab3656cb2e1R315). Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	7 years ago
Tom Wilkie	10d49e798b	Remove stream from index when its remove from memory. ... Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	7 years ago
Goutham Veeramachaneni	7763e8f3dd	Flush idle chunks properly (#209) ... * Flush idle chunks properly Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com> * Review feedback Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com> * Review feedback. Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com> * Check the correct chunk is full before appending a new chunk. Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com> * Add the config to helm and ksonnet too. Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	7 years ago
Tom Wilkie	ab4c5bebab	Validate labels in the distributor. (#251) ... Also, make ErrOutOfOrder a HTTP 4xx error and continue to append even when we encourter an out of order entry. Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	7 years ago
Goutham Veeramachaneni	9b67d6cb7a	Rename instance to tenant to avoid clash with Prom (#164) ... Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com>	8 years ago
Goutham Veeramachaneni	7bcdf0e0b1	Tenant metrics (#135) ... * Fix stream tracking metric Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com> * Track lines and bytes received per user Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com> * Use instance instead of org Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com> * Nuke dead code Usage has been removed as part of perf-optimisation: 3006e3901d (diff-180f56d9aaf5a9aa079e6fc9cfcf1bc8L365) Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com>	8 years ago
Yeh-lei Wu	251017e41e	Refine ingester locks: eliminate smell & allow concurrent ingester query (#120) ... Signed-off-by: alei <rayingecho@gmail.com> Fix chunk retain period Signed-off-by: alei <rayingecho@gmail.com> Fix undefined symbol Signed-off-by: alei <rayingecho@gmail.com>	8 years ago
Tom Wilkie	9f257c591a	[WIP] Persistence (#44) ... Persistence chunks using the Cortex chunk store.	8 years ago
Tom Wilkie	5f73598389	Rename Tempo to Loki. (#36) ... * Rename Tempo to Loki. Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com> * Use new build image, don't delete generated files on clean. Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	8 years ago
Tom Wilkie	600b24101e	Promtail updates (#30) ... * Make promtail read all its config from a file. Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com> * Use gokit logging everywhere. Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com> * Start work to parse normal log lines. Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com> * update common and kuberesolver. Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com> * lint Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com> * Get EntryParser stuff working, so we can tail raw log files. Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com> * Make sure filename label works. Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com> * lint, remove interfacer. Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	8 years ago
Tom Wilkie	1fef92bc17	Lint stuff (#27) ... * Check in generated files, to make getting started easier. Add gometalinter, and a lint check for changes to the generated files. Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com> * Make lint pass? Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	8 years ago
Tom Wilkie	d06ed8fb1f	Remove flagext, use cortex's. Allow flags to be loaded from a file. ... Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	8 years ago
Tom Wilkie	de05515248	Init pool config correctly. ... Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	8 years ago
Tom Wilkie	3b69cf127d	Don't repeat client config. ... Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	8 years ago
Tom Wilkie	0ffb432f30	Update for cortex changes. ... Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	8 years ago
Tom Wilkie	3f62b4ec36	s/logish/tempo/ ... Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>	8 years ago