watcha-synapse

Commit Graph

Author	SHA1	Message	Date
Richard van der Hoff	15997618e2	Clean up PreserveLoggingContext (#7877 ) This had some dead code and some just plain wrong docstrings.	4 years ago
Patrick Cloke	d7c2df2fa3	Improve per-block CPU and DB usage metrics (#7426 )	5 years ago
Patrick Cloke	fe69fb6263	Add backwards compatibility codepath to LoggingContext. (#7408 )	5 years ago
Richard van der Hoff	0f05fd1530	Reduce the number of calls to `resource.getrusage` (#7183 ) Let's just call `getrusage` once on each logcontext change, rather than twice.	5 years ago
Richard van der Hoff	60adcbed91	Fix "'NoneType' has no attribute start\|stop" logcontext errors (#7181 ) Fixes #7179.	5 years ago
Richard van der Hoff	39230d2171	Clean up some LoggingContext stuff (#7120 ) * Pull Sentinel out of LoggingContext ... and drop a few unnecessary references to it * Factor out LoggingContext.current_context move `current_context` and `set_context` out to top-level functions. Mostly this means that I can more easily trace what's actually referring to LoggingContext, but I think it's generally neater. * move copy-to-parent into `stop` this really just makes `start` and `stop` more symetric. It also means that it behaves correctly if you manually `set_log_context` rather than using the context manager. * Replace `LoggingContext.alive` with `finished` Turn `alive` into `finished` and make it a bit better defined.	5 years ago
Brendan Abolivier	dc6fb56c5f	Hopefully mypy is happy now	5 years ago
Brendan Abolivier	fe593ef990	Attempt at appeasing the gods of mypy	5 years ago
Neil Pilgrim	2bff4457d9	Add type hints to logging/context.py (#6309 ) * Add type hints to logging/context.py Signed-off-by: neiljp (Neil Pilgrim) <github@kepier.clara.net>	5 years ago
Richard van der Hoff	da4e52544e	comment for run_in_background	5 years ago
Erik Johnston	495005360c	Bump version of mypy	5 years ago
Erik Johnston	ffeafade48	Update comment	5 years ago
Erik Johnston	9a2223d4c8	Fix make_deferred_yieldable to work with coroutines	5 years ago
Andrew Morgan	54fef094b3	Remove usage of deprecated logger.warn method from codebase (#6271 ) Replace every instance of `logger.warn` with `logger.warning` as the former is deprecated.	5 years ago
J. Ryan Stinnett	a86a290850	Fix logcontext spam on non-Linux platforms (#6059 ) This checks whether the current platform supports thread resource usage tracking before logging a warning to avoid log spam. Fixes https://github.com/matrix-org/synapse/issues/6055	5 years ago
Amber Brown	7dc398586c	Implement a structured logging output system. (#5680 )	5 years ago
Jorik Schellekens	38a6d3eea7	Add basic opentracing support (#5544 ) * Configure and initialise tracer Includes config options for the tracer and sets up JaegerClient. * Scope manager using LogContexts We piggy-back our tracer scopes by using log context. The current log context gives us the current scope. If new scope is created we create a stack of scopes in the context. * jaeger is a dependency now * Carrier inject and extraction for Twisted Headers * Trace federation requests on the way in and out. The span is created in _started_processing and closed in _finished_processing because we need a meaningful log context. * Create logcontext for new scope. Instead of having a stack of scopes in a logcontext we create a new context for a new scope if the current logcontext already has a scope. * Remove scope from logcontext if logcontext is top level * Disable tracer if not configured * typo * Remove dependence on jaeger internals * bools * Set service name * :Explicitely state that the tracer is disabled * Black is the new black * Newsfile * Code style * Use the new config setup. * Generate config. * Copyright * Rename config to opentracing * Remove user whitelisting * Empty whitelist by default * User ConfigError instead of RuntimeError * Use isinstance * Use tag constants for opentracing. * Remove debug comment and no need to explicitely record error * Two errors a "s(c)entry" * Docstrings! * Remove debugging brainslip * Homeserver Whitlisting * Better opentracing config comment * linting * Inclue worker name in service_name * Make opentracing an optional dependency * Neater config retreival * Clean up dummy tags * Instantiate tracing as object instead of global class * Inlcude opentracing as a homeserver member. * Thread opentracing to the request level * Reference opetnracing through hs * Instantiate dummy opentracin g for tests. * About to revert, just keeping the unfinished changes just in case * Revert back to global state, commit number: 9ce4a3d9067bf9889b86c360c05ac88618b85c4f * Use class level methods in tracerutils * Start and stop requests spans in a place where we have access to the authenticated entity * Seen it, isort it * Make sure to close the active span. * I'm getting black and blue from this. * Logger formatting Co-Authored-By: Erik Johnston <erik@matrix.org> * Outdated comment * Import opentracing at the top * Return a contextmanager * Start tracing client requests from the servlet * Return noop context manager if not tracing * Explicitely say that these are federation requests * Include servlet name in client requests * Use context manager * Move opentracing to logging/ * Seen it, isort it again! * Ignore twisted return exceptions on context exit * Escape the scope * Scopes should be entered to make them useful. * Nicer decorator names * Just one init, init? * Don't need to close something that isn't open * Docs make you smarter	5 years ago
Amber Brown	463b072b12	Move logging utilities out of the side drawer of util/ and into logging/ (#5606 )	5 years ago
Richard van der Hoff	cb8d568cf9	Fix 'utime went backwards' errors on daemonization. (#5609 ) * Fix 'utime went backwards' errors on daemonization. Fixes #5608 * remove spurious debug	5 years ago
Amber Brown	0ee9076ffe	Fix media repo breaking (#5593 )	5 years ago
Richard van der Hoff	dc94773e60	Avoid raising exceptions in metrics Sentry will catch the errors if they happen, so that should be good enough, and woun't make things explode if we hit the error condition.	6 years ago
Amber Brown	32e7c9e7f2	Run Black. (#5482 )	6 years ago
Richard van der Hoff	fe641df770	Sanity-checking for metrics updates Check that our clocks go forward.	6 years ago
Richard van der Hoff	3dcf2feba8	Improve logging for logcontext leaks. (#5288 )	6 years ago
Robert Jacob	2a7f0b8953	Implement workaround for login error. Signed-off-by: Robert Jacob <xperimental@solidproject.de>	6 years ago
Richard van der Hoff	5c445114d3	Correctly account for cpu usage by background threads (#4074 ) Wrap calls to deferToThread() in a thing which uses a child logcontext to attribute CPU usage to the right request. While we're in the area, remove the logcontext_tracer stuff, which is never used, and afaik doesn't work. Fixes #4064	6 years ago
Richard van der Hoff	4a15a3e4d5	Include eventid in log lines when processing incoming federation transactions (#3959 ) when processing incoming transactions, it can be hard to see what's going on, because we process a bunch of stuff in parallel, and because we may end up recursively working our way through a chain of three or four events. This commit creates a way to use logcontexts to add the relevant event ids to the log lines.	6 years ago
Richard van der Hoff	be6527325a	Fix exceptions when a connection is closed before we read the headers This fixes bugs introduced in #3700, by making sure that we behave sanely when an incoming connection is closed before the headers are read.	6 years ago
Richard van der Hoff	55e6bdf287	Robustness fix for logcontext filter Make the logcontext filter not explode if it somehow ends up with a logcontext of None, since that infinite-loops the whole logging system.	6 years ago
Amber Brown	b37c472419	Rename async to async_helpers because `async` is a keyword on Python 3.7 (#3678 )	6 years ago
Amber Brown	95ccb6e2ec	Don't spew errors because we can't save metrics (#3563 )	6 years ago
Richard van der Hoff	c3c29aa196	Attempt to include db threads in cpu usage stats (#3496 ) Let's try to include time spent in the DB threads in the per-request/block cpu usage metrics.	6 years ago
Richard van der Hoff	55370331da	Refactor logcontext resource usage tracking (#3501 ) Factor out the resource usage tracking out to a separate object, which can be passed around and copied independently of the logcontext itself.	6 years ago
Amber Brown	49af402019	run isort	6 years ago
Richard van der Hoff	0495fe0035	Indirect evt_count updates via method call so that we can stub it for the sentinel and not have a billion failing UTs	7 years ago
Richard van der Hoff	b088aafcae	Log number of events fetched from DB When we finish processing a request, log the number of events we fetched from the database to handle it. [I'm trying to figure out which requests are responsible for large amounts of event cache churn. It may turn out to be more helpful to add counts to the prometheus per-request/block metrics, but that is an extension to this code anyway.]	7 years ago
Amber Brown	3ef5cd74a6	update to more consistently use seconds in any metrics or logging	7 years ago
Erik Johnston	7948ecf234	Comment	7 years ago
Erik Johnston	020377a550	Fix logcontext resource usage tracking	7 years ago
Richard van der Hoff	f22e7cda2c	Fix a class of logcontext leaks So, it turns out that if you have a first `Deferred` `D1`, you can add a callback which returns another `Deferred` `D2`, and `D2` must then complete before any further callbacks on `D1` will execute (and later callbacks on `D1` get the result of `D2` rather than `D2` itself). So, `D1` might have `called=True` (as in, it has started running its callbacks), but any new callbacks added to `D1` won't get run until `D2` completes - so if you `yield D1` in an `inlineCallbacks` function, your `yield` will 'block'. In conclusion: some of our assumptions in `logcontext` were invalid. We need to make sure that we don't optimise out the logcontext juggling when this situation happens. Fortunately, it is easy to detect by checking `D1.paused`.	7 years ago
Richard van der Hoff	2a13af23bc	Use run_in_background in preference to preserve_fn While I was going through uses of preserve_fn for other PRs, I converted places which only use the wrapped function once to use run_in_background, to avoid creating the function object.	7 years ago
Richard van der Hoff	13843f771e	Trap exceptions thrown within run_in_background Turn any exceptions that get thrown synchronously within run_in_background into Failures instead.	7 years ago
Richard van der Hoff	9255a6cb17	Improve exception handling for background processes There were a bunch of places where we fire off a process to happen in the background, but don't have any exception handling on it - instead relying on the unhandled error being logged when the relevent deferred gets garbage-collected. This is unsatisfactory for a number of reasons: - logging on garbage collection is best-effort and may happen some time after the error, if at all - it can be hard to figure out where the error actually happened. - it is logged as a scary CRITICAL error which (a) I always forget to grep for and (b) it's not really CRITICAL if a background process we don't care about fails. So this is an attempt to add exception handling to everything we fire off into the background.	7 years ago
Adrian Tschira	f63ff73c7f	add __bool__ alias to __nonzero__ methods Signed-off-by: Adrian Tschira <nota@notafile.com>	7 years ago
Richard van der Hoff	5a6e54264d	Make 'unexpected logging context' into warnings I think we've now fixed enough of these that the rest can be logged at warning.	7 years ago
Richard van der Hoff	20f40348d4	Factor run_in_background out from preserve_fn It annoys me that we create temporary function objects when there's really no need for it. Let's factor the gubbins out of preserve_fn and start using it.	7 years ago
Richard van der Hoff	3a75de923b	Rewrite make_deferred_yieldable avoiding inlineCallbacks ... because (a) it's actually simpler (b) it might be marginally more performant?	7 years ago
Richard van der Hoff	3d12d97415	Track DB scheduling delay per-request For each request, track the amount of time spent waiting for a db connection. This entails adding it to the LoggingContext and we may as well add metrics for it while we are passing.	7 years ago
Richard van der Hoff	6324b65f08	Track db txn time in millisecs ... to reduce the amount of floating-point foo we do.	7 years ago
Richard van der Hoff	44a498418c	Optimise LoggingContext creation and copying It turns out that the only thing we use the __dict__ of LoggingContext for is `request`, and given we create lots of LoggingContexts and then copy them every time we do a db transaction or log line, using the __dict__ seems a bit redundant. Let's try to optimise things by making the request attribute explicit.	7 years ago

18 Commits (5dd73d029eff32668b3ca69b7fb8529fc7c58745)