171 Commits (a19aa8b162edf99673bc9b0600c9ab550967b7e8)

Author	SHA1	Message	Date
Jonathan de Jong	95e47b2e78	[pyupgrade] `synapse/` (#10348) ... This PR is tantamount to running ``` pyupgrade --py36-plus --keep-percent-format `find synapse/ -type f -name "*.py"` ``` Part of #9744	3 years ago
Jonathan de Jong	bf72d10dbf	Use inline type hints in various other places (in `synapse/`) (#10380)	3 years ago
Richard van der Hoff	b2557cbf42	opentracing: use a consistent name for background processes (#10135) ... ... otherwise we tend to get a namespace clash between the bg process and the functions that it calls.	4 years ago
Richard van der Hoff	ed53bf314f	Set opentracing priority before setting other tags (#10092) ... ... because tags on spans which aren't being sampled get thrown away.	4 years ago
Erik Johnston	8771b1337d	Export jemalloc stats to prometheus when used (#9882)	4 years ago
Erik Johnston	1fb9a2d0bf	Limit how often GC happens by time. (#9902) ... Synapse can be quite memory intensive, and unless care is taken to tune the GC thresholds it can end up thrashing, causing noticable performance problems for large servers. We fix this by limiting how often we GC a given generation, regardless of current counts/thresholds. This does not help with the reverse problem where the thresholds are set too high, but that should only happen in situations where they've been manually configured. Adds a `gc_min_seconds_between` config option to override the defaults. Fixes #9890.	4 years ago
Richard van der Hoff	5d281c10dd	Stop BackgroundProcessLoggingContext making new prometheus timeseries (#9854) ... This undoes part of b076bc276e.	4 years ago
Patrick Cloke	b076bc276e	Always use the name as the log ID. (#9829) ... As far as I can tell our logging contexts are meant to log the request ID, or sometimes the request ID followed by a suffix (this is generally stored in the name field of LoggingContext). There's also code to log the name@memory location, but I'm not sure this is ever used. This simplifies the code paths to require every logging context to have a name and use that in logging. For sub-contexts (created via nested_logging_contexts, defer_to_threadpool, Measure) we use the current context's str (which becomes their name or the string "sentinel") and then potentially modify that (e.g. add a suffix).	4 years ago
Jonathan de Jong	4b965c862d	Remove redundant "coding: utf-8" lines (#9786) ... Part of #9744 Removes all redundant `# -*- coding: utf-8 -*-` lines from files, as python 3 automatically reads source code as utf-8 now. `Signed-off-by: Jonathan de Jong <jonathan@automatia.nl>`	4 years ago
Patrick Cloke	48d44ab142	Record more information into structured logs. (#9654) ... Records additional request information into the structured logs, e.g. the requester, IP address, etc.	4 years ago
Andrew Morgan	0d87c6bd12	Don't report anything from GaugeBucketCollector metrics until data is present (#8926) ... This PR modifies `GaugeBucketCollector` to only report data once it has been updated, rather than initially reporting a value of 0. Fixes zero values being reported for some metrics on startup until a background job to update the metric's value runs later.	4 years ago
Patrick Cloke	33a02f0f52	Fix additional type hints from Twisted upgrade. (#9518)	4 years ago
Eric Eastwood	0a00b7ff14	Update black, and run auto formatting over the codebase (#9381) ... - Update black version to the latest - Run black auto formatting over the codebase - Run autoformatting according to [`docs/code_style.md `](80d6dc9783/docs/code_style.md) - Update `code_style.md` docs around installing black to use the correct version	4 years ago
Patrick Cloke	1619802228	Various clean-ups to the logging context code (#8935)	4 years ago
David Teller	f14428b25c	Allow spam-checker modules to be provide async methods. (#8890) ... Spam checker modules can now provide async methods. This is implemented in a backwards-compatible manner.	4 years ago
Erik Johnston	427ede619f	Add metrics for tracking 3PID /requestToken requests. (#8712) ... The main use case is to see how many requests are being made, and how many are second/third/etc attempts. If there are large number of retries then that likely indicates a delivery problem.	4 years ago
Erik Johnston	2b7c180879	Start fewer opentracing spans (#8640) ... #8567 started a span for every background process. This is good as it means all Synapse code that gets run should be in a span (unless in the sentinel logging context), but it means we generate about 15x the number of spans as we did previously. This PR attempts to reduce that number by a) not starting one for send commands to Redis, and b) deferring starting background processes until after we're sure they're necessary. I don't really know how much this will help.	4 years ago
Patrick Cloke	34a5696f93	Fix typos and spelling errors. (#8639)	4 years ago
Erik Johnston	1fcdbeb3ab	Start an opentracing span for background processes. (#8567) ... This should reduce the number of `There was no active span` errors we see. Fixes #8510.	4 years ago
Richard van der Hoff	6d2d42f8fb	Rewrite BucketCollector ... This was a bit unweildy for what I wanted: in particular, I wanted to assign each measurement straight into a bucket, rather than storing an intermediate Counter which didn't do any bucketing at all. I've replaced it with something that is hopefully a bit easier to use. (I'm not entirely sure what the difference between a HistogramMetricFamily and a GaugeHistogramMetricFamily is, but given our counters can go down as well as up the latter *sounds* more accurate?)	4 years ago
Richard van der Hoff	1c8ca2c543	Fix _exposition.py to stop stripping samples ... Our hacked-up `_exposition.py` was stripping out some samples it shouldn't have been. Put them back in, to more closely match the upstream `exposition.py`.	4 years ago
Richard van der Hoff	ceafb5a1c6	Drop support for ancient prometheus_client (#8426) ... Drop compatibility hacks for prometheus-client pre 0.4.0. Debian stretch and Fedora 31 both have newer versions, so hopefully this will be ok.	4 years ago
Patrick Cloke	aec294ee0d	Use slots in attrs classes where possible (#8296) ... slots use less memory (and attribute access is faster) while slightly limiting the flexibility of the class attributes. This focuses on objects which are instantiated "often" and for short periods of time.	4 years ago
Patrick Cloke	c619253db8	Stop sub-classing object (#8249)	4 years ago
Patrick Cloke	d89692ea84	Convert runWithConnection to async. (#8121)	4 years ago
Patrick Cloke	c36228c403	Convert run_as_background_process inner function to async. (#8032)	4 years ago
Richard van der Hoff	8ca39bd2c3	Improve stacktraces from exceptions in background processes (#7808) ... use `Failure()` to fish out the real exception.	4 years ago
Erik Johnston	a99658074d	Add some metrics for inbound and outbound federation processing times (#7755)	4 years ago
Christian Svensson	8bbe87f42d	Set Content-Length for Metrics requests (#7730) ... HTTP requires the response to contain a Content-Length header unless chunked encoding is being used. Prometheus metrics endpoint did not set this, causing software such as prometheus-proxy to not be able to scrape synapse for metrics. Signed-off-by: Christian Svensson <blue@cmd.nu>	4 years ago
Patrick Cloke	bd6dc17221	Replace iteritems/itervalues/iterkeys with native versions. (#7692)	5 years ago
Erik Johnston	f5353eff21	Make inflight background metrics more efficient. (#7597) ... Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>	5 years ago
Ivan Shapovalov	ac481a738e	synapse.metrics: implement detailed memory usage reporting on PyPy (#7536) ... PyPy's gc.get_stats() returns an object containing detailed allocator statistics which could be beneficial to collect as metrics. Signed-off-by: Ivan Shapovalov <intelfx@intelfx.name>	5 years ago
Amber Brown	7cb8b4bc67	Allow configuration of Synapse's cache without using synctl or environment variables (#6391)	5 years ago
Richard van der Hoff	8c75667ad7	Add prometheus metrics for the number of active pushers (#7103)	5 years ago
Patrick Cloke	509e381afa	Clarify list/set/dict/tuple comprehensions and enforce via flake8 (#6957) ... Ensure good comprehension hygiene using flake8-comprehensions.	5 years ago
Amber Brown	864f144543	Fix up some typechecking (#6150) ... * type checking fixes * changelog	5 years ago
Richard van der Hoff	a96318127d	Update comments and docstring	5 years ago
Erik Johnston	367158a609	Add wrap_as_background_process decorator. ... This does the same thing as `run_as_background_process` but means we don't need to create superfluous functions.	5 years ago
Amber Brown	b617864cd9	Fix for structured logging tests stomping on logs (#6023)	5 years ago
Amber Brown	aeb9b2179e	Add a build info metric to Prometheus (#6005)	5 years ago
Amber Brown	7ad1d76356	Support Prometheus_client 0.4.0+ (#5636)	5 years ago
Amber Brown	463b072b12	Move logging utilities out of the side drawer of util/ and into logging/ (#5606)	5 years ago
Amber Brown	071150ce19	Don't log GC 0s at INFO (#5557)	5 years ago
Amber Brown	32e7c9e7f2	Run Black. (#5482)	6 years ago
Erik Johnston	3ed595e327	Prometheus histograms are cumalative	6 years ago
Amber H. Brown	a10c8dae85	fix prometheus rendering error	6 years ago
Amber Brown	6312d6cc7c	Expose statistics on extrems to prometheus (#5384)	6 years ago
Richard van der Hoff	82ca6d1f9f	Add metrics for number of outgoing EDUs, by type (#4695)	6 years ago
Erik Johnston	7c570bff74	Fix exception in background metrics collection ... We attempted to iterate through a list on a separate thread without doing the necessary copying.	6 years ago
Erik Johnston	94ae1dea3c	Add missing logger	6 years ago