watcha-synapse

Commit Graph

Author	SHA1	Message	Date
Richard van der Hoff	3ff2251754	Improved validation for received requests (#9817 ) * Simplify `start_listening` callpath * Correctly check the size of uploaded files	4 years ago
Richard van der Hoff	51a20914a8	Limit the size of HTTP responses read over federation. (#9833 )	4 years ago
Jonathan de Jong	4b965c862d	Remove redundant "coding: utf-8" lines (#9786 ) Part of #9744 Removes all redundant `# -- coding: utf-8 --` lines from files, as python 3 automatically reads source code as utf-8 now. `Signed-off-by: Jonathan de Jong <jonathan@automatia.nl>`	4 years ago
Patrick Cloke	0b3112123d	Use mock from the stdlib. (#9772 )	4 years ago
Jonathan de Jong	2ca4e349e9	Bugbear: Add Mutable Parameter fixes (#9682 ) Part of #9366 Adds in fixes for B006 and B008, both relating to mutable parameter lint errors. Signed-off-by: Jonathan de Jong <jonathan@automatia.nl>	4 years ago
Andrew Morgan	5b268997bd	Allow providing credentials to HTTPS_PROXY (#9657 ) Addresses https://github.com/matrix-org/synapse-dinsic/issues/70 This PR causes `ProxyAgent` to attempt to extract credentials from an `HTTPS_PROXY` env var. If credentials are found, a `Proxy-Authorization` header ([details](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Proxy-Authorization)) is sent to the proxy server to authenticate against it. The headers are not passed to the remote server. Also added some type hints.	4 years ago
Patrick Cloke	e55bd0e110	Add tests for blacklisting reactor/agent. (#9563 )	4 years ago
Patrick Cloke	16ec8c3272	(Hopefully) stop leaking file descriptors in media repo. (#9497 ) By consuming the response if the headers imply that the content is too large.	4 years ago
Tim Leung	ddb240293a	Add support for no_proxy and case insensitive env variables (#9372 ) ### Changes proposed in this PR - Add support for the `no_proxy` and `NO_PROXY` environment variables - Internally rely on urllib's [`proxy_bypass_environment`](`bdb941be42/Lib/urllib/request.py (L2519)`) - Extract env variables using urllib's `getproxies`/[`getproxies_environment`](`bdb941be42/Lib/urllib/request.py (L2488)`) which supports lowercase + uppercase, preferring lowercase, except for `HTTP_PROXY` in a CGI environment This does contain behaviour changes for consumers so making sure these are called out: - `no_proxy`/`NO_PROXY` is now respected - lowercase `https_proxy` is now allowed and taken over `HTTPS_PROXY` Related to #9306 which also uses `ProxyAgent` Signed-off-by: Timothy Leung tim95@hotmail.co.uk	4 years ago
Patrick Cloke	8ec2217103	Reduce the memory usage of previewing media files. (#9421 ) This reduces the memory usage of previewing media files which end up larger than the `max_spider_size` by avoiding buffering content internally in treq. It also checks the `Content-Length` header in additional places instead of streaming the content to check the body length.	4 years ago
Eric Eastwood	0a00b7ff14	Update black, and run auto formatting over the codebase (#9381 ) - Update black version to the latest - Run black auto formatting over the codebase - Run autoformatting according to [`docs/code_style.md `](`80d6dc9783/docs/code_style.md`) - Update `code_style.md` docs around installing black to use the correct version	4 years ago
Richard van der Hoff	0cd2938bc8	Support icons for Identity Providers (#9154 )	4 years ago
Patrick Cloke	74dd906041	Avoid raising the body exceeded error multiple times. (#9108 ) Previously this code generated unreferenced `Deferred` instances which caused "Unhandled Deferreds" errors to appear in error situations.	4 years ago
Marcus	e385c8b473	Don't apply the IP range blacklist to proxy connections (#9084 ) It is expected that the proxy would be on a private IP address so the configured proxy should be connected to regardless of the IP range blacklist.	4 years ago
Patrick Cloke	723b19748a	Handle bad JSON data being returned from the federation API. (#9070 )	4 years ago
Patrick Cloke	ff5c4da128	Add a maximum size for well-known lookups. (#8950 )	4 years ago
Richard van der Hoff	394516ad1b	Remove spurious "SynapseRequest" result from `make_request" This was never used, so let's get rid of it.	4 years ago
Richard van der Hoff	7eebe4b3fc	Replace `request.code` with `channel.code` The two are equivalent, but really we want to check the HTTP result that got returned to the channel, not the code that the Request object intended to return to the channel.	4 years ago
Patrick Cloke	30fba62108	Apply an IP range blacklist to push and key revocation requests. (#8821 ) Replaces the `federation_ip_range_blacklist` configuration setting with an `ip_range_blacklist` setting with wider scope. It now applies to: * Federation * Identity servers * Push notifications * Checking key validitity for third-party invite events The old `federation_ip_range_blacklist` setting is still honored if present, but with reduced scope (it only applies to federation and identity servers).	4 years ago
Richard van der Hoff	be8fa65d0b	Remove redundant calls to `render()`	4 years ago
Richard van der Hoff	cfd895a22e	use global make_request() directly where we have a custom Resource Where we want to render a request against a specific Resource, call the global make_request() function rather than the one in HomeserverTestCase, allowing us to pass in an appropriate `Site`.	4 years ago
Richard van der Hoff	1c262431f9	Fix handling of connection timeouts in outgoing http requests (#8400 ) * Remove `on_timeout_cancel` from `timeout_deferred` The `on_timeout_cancel` param to `timeout_deferred` wasn't always called on a timeout (in particular if the canceller raised an exception), so it was unreliable. It was also only used in one place, and to be honest it's easier to do what it does a different way. * Fix handling of connection timeouts in outgoing http requests Turns out that if we get a timeout during connection, then a different exception is raised, which wasn't always handled correctly. To fix it, catch the exception in SimpleHttpClient and turn it into a RequestTimedOutError (which is already a documented exception). Also add a description to RequestTimedOutError so that we can see which stage it failed at. * Fix incorrect handling of timeouts reading federation responses This was trapping the wrong sort of TimeoutError, so was never being hit. The effect was relatively minor, but we should fix this so that it does the expected thing. * Fix inconsistent handling of `timeout` param between methods `get_json`, `put_json` and `delete_json` were applying a different timeout to the response body to `post_json`; bring them in line and test. Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com> Co-authored-by: Erik Johnston <erik@matrix.org>	4 years ago
Patrick Cloke	b86764662b	Fix the exception that is raised when invalid JSON is encountered. (#8291 )	4 years ago
Dan Callaghan	c312ee3cde	Use TLSv1.2 for fake servers in tests (#8208 ) Some Linux distros have begun disabling TLSv1.0 and TLSv1.1 by default for security reasons, for example in Fedora 33 onwards: https://fedoraproject.org/wiki/Changes/StrongCryptoSettings2 Use TLSv1.2 for the fake TLS servers created in the test suite, to avoid failures due to OpenSSL disallowing TLSv1.0: <twisted.python.failure.Failure OpenSSL.SSL.Error: [('SSL routines', 'ssl_choose_client_version', 'unsupported protocol')]> Signed-off-by: Dan Callaghan <djc@djc.id.au>	4 years ago
Patrick Cloke	c619253db8	Stop sub-classing object (#8249 )	4 years ago
Patrick Cloke	5bf8e5f55b	Convert the well known resolver to async (#8214 )	4 years ago
Andrew Morgan	a466b67972	Reduce run-times of tests by advancing the reactor less (#7757 )	4 years ago
Patrick Cloke	c978f6c451	Convert federation client to async/await. (#7975 )	4 years ago
Patrick Cloke	68cd935826	Convert the federation agent and related code to async/await. (#7874 )	4 years ago
Erik Johnston	5cdca53aa0	Merge different Resource implementation classes (#7732 )	4 years ago
Patrick Cloke	ac51bd581a	Include a user agent in federation requests. (#7677 )	5 years ago
Richard van der Hoff	39230d2171	Clean up some LoggingContext stuff (#7120 ) * Pull Sentinel out of LoggingContext ... and drop a few unnecessary references to it * Factor out LoggingContext.current_context move `current_context` and `set_context` out to top-level functions. Mostly this means that I can more easily trace what's actually referring to LoggingContext, but I think it's generally neater. * move copy-to-parent into `stop` this really just makes `start` and `stop` more symetric. It also means that it behaves correctly if you manually `set_log_context` rather than using the context manager. * Replace `LoggingContext.alive` with `finished` Turn `alive` into `finished` and make it a bit better defined.	5 years ago
Richard van der Hoff	c37db0211e	Share SSL contexts for non-federation requests (#7094 ) Extends #5794 etc to the SimpleHttpClient so that it also applies to non-federation requests. Fixes #7092.	5 years ago
Richard van der Hoff	1cb84c6486	Support for routing outbound HTTP requests via a proxy (#6239 ) The `http_proxy` and `HTTPS_PROXY` env vars can be set to a `host[:port]` value which should point to a proxy. The address of the proxy should be excluded from IP blacklists such as the `url_preview_ip_range_blacklist`. The proxy will then be used for * push * url previews * phone-home stats * recaptcha validation * CAS auth validation It will not be used for: * Application Services * Identity servers * Outbound federation * In worker configurations, connections from workers to masters Fixes #4198.	5 years ago
Erik Johnston	7777d353bf	Remove test debugs	5 years ago
Erik Johnston	1dec31560e	Change jitter to be a factor rather than absolute value	5 years ago
Erik Johnston	f299c5414c	Refactor MatrixFederationAgent to retry SRV. This refactors MatrixFederationAgent to move the SRV lookup into the endpoint code, this has two benefits: 1. Its easier to retry different host/ports in the same way as HostnameEndpoint. 2. We avoid SRV lookups if we have a free connection in the pool	5 years ago
Erik Johnston	e6e136decc	Retry well known on fail. If we have recently seen a valid well-known for a domain we want to retry on (non-final) errors a few times, to handle temporary blips in networking/etc.	5 years ago
Erik Johnston	17e1e80726	Retry well-known lookup before expiry. This gives a bit of a grace period where we can attempt to refetch a remote `well-known`, while still using the cached result if that fails. Hopefully this will make the well-known resolution a bit more torelant of failures, rather than it immediately treating failures as "no result" and caching that for an hour.	5 years ago
Erik Johnston	107ad133fc	Move well known lookup into a separate clas	5 years ago
Erik Johnston	af9f1c0764	Add a lower bound for TTL on well known results. It costs both us and the remote server for us to fetch the well known for every single request we send, so we add a minimum cache period. This is set to 5m so that we still honour the basic premise of "refetch frequently".	5 years ago
Erik Johnston	a9bcae9f50	Share SSL options for well-known requests	5 years ago
Amber Brown	4806651744	Replace returnValue with return (#5736 )	5 years ago
Amber Brown	463b072b12	Move logging utilities out of the side drawer of util/ and into logging/ (#5606 )	5 years ago
Richard van der Hoff	c3c6b00d95	Pass config_dir_path and data_dir_path into Config.read_config. (#5522 ) * Pull config_dir_path and data_dir_path calculation out of read_config_files * Pass config_dir_path and data_dir_path into read_config	6 years ago
Amber Brown	32e7c9e7f2	Run Black. (#5482 )	6 years ago
Richard van der Hoff	db74c4fc6c	fix ci on py2, again	6 years ago
Richard van der Hoff	19780a521e	fix CI on python 2.7	6 years ago
Richard van der Hoff	e016681221	Tests for SSL certs for federation connections Add some tests for bad certificates for federation and .well-known connections	6 years ago
Richard van der Hoff	9fbb20a531	Stop hardcoding trust of old matrix.org key (#5374 ) There are a few changes going on here: * We make checking the signature on a key server response optional: if no verify_keys are specified, we trust to TLS to validate the connection. * We change the default config so that it does not require responses to be signed by the old key. * We replace the old 'perspectives' config with 'trusted_key_servers', which is also formatted slightly differently. * We emit a warning to the logs every time we trust a key server response signed by the old key.	6 years ago

1 2

95 Commits (9752849e2b41968613ca244e86311d805bdd27df)