77 Commits (8c56e18e4786a28bedbafb1a73fd1abc8806933f)

Author	SHA1	Message	Date
Erik Johnston	0aba4a4eaa	Add cache to `get_server_keys_json_for_remote` (#16123)	1 year ago
Patrick Cloke	ad3f43be9a	Run pyupgrade for python 3.7 & 3.8. (#16110)	1 year ago
Patrick Cloke	5e024a0645	Modify StoreKeyFetcher to read from server_keys_json. (#15417) ... Before this change: * `PerspectivesKeyFetcher` and `ServerKeyFetcher` write to `server_keys_json`. * `PerspectivesKeyFetcher` also writes to `server_signature_keys`. * `StoreKeyFetcher` reads from `server_signature_keys`. After this change: * `PerspectivesKeyFetcher` and `ServerKeyFetcher` write to `server_keys_json`. * `PerspectivesKeyFetcher` also writes to `server_signature_keys`. * `StoreKeyFetcher` reads from `server_keys_json`. This results in `StoreKeyFetcher` now using the results from `ServerKeyFetcher` in addition to those from `PerspectivesKeyFetcher`, i.e. keys which are directly fetched from a server will now be pulled from the database instead of refetched. An additional minor change is included to avoid creating a `PerspectivesKeyFetcher` (and checking it) if no `trusted_key_servers` are configured. The overall impact of this should be better usage of cached results: * If a server has no trusted key servers configured then it should reduce how often keys are fetched. * if a server's trusted key server does not have a requested server's keys cached then it should reduce how often keys are directly fetched.	2 years ago
reivilibre	edae20f926	Improve robustness when handling a perspective key response by deduplicating received server keys. (#15423) ... * Change `store_server_verify_keys` to take a `Mapping[(str, str), FKR]` This is because we already can't handle duplicate keys — leads to cardinality violation * Newsfile Signed-off-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org> --------- Signed-off-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org>	2 years ago
Patrick Cloke	42aea0d8af	Add final type hint to tests.unittest. (#15072) ... Adds a return type to HomeServerTestCase.make_homeserver and deal with any variables which are no longer Any.	2 years ago
Patrick Cloke	a4ca770655	Add missing type hints to tests. (#14687) ... Adds type hints to tests.metrics and tests.crypto.	2 years ago
Richard van der Hoff	ecb6fe9d9c	Stop using deprecated `keyIds` param on /key/v2/server (#14525) ... Fixes #14523.	2 years ago
David Robertson	e1b15f25f3	Fix /key/v2/server calls with URL-unsafe key IDs (#14490) ... Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>	2 years ago
Jacek Kuśnierz	88193f2125	Remove direct refeferences to PyNaCl (use signedjson instead). (#12902)	3 years ago
David Robertson	e0bb268134	Fix typechecker problems exposed by signedjson 1.1.2 (#12326)	3 years ago
Patrick Cloke	02d708568b	Replace assertEquals and friends with non-deprecated versions. (#12092)	3 years ago
Richard van der Hoff	e24ff8ebe3	Remove `HomeServer.get_datastore()` (#12031) ... The presence of this method was confusing, and mostly present for backwards compatibility. Let's get rid of it. Part of #11733	3 years ago
David Robertson	1b1aed38e3	Deal with mypy errors w/ type-hinted pynacl 1.5.0 (#11714) ... * Deal with mypy errors w/ type-hinted pynacl 1.5.0 Fixes #11644. I really don't like that we're monkey patching pynacl SignedKey instances with alg and version objects. But I'm too scared to make the changes necessary right now. (Ideally I would replace `signedjson.types.SingingKey` with a runtime class which wraps or inherits from `nacl.signing.SigningKey`.) C.f. https://github.com/matrix-org/python-signedjson/issues/16	3 years ago
David Robertson	c43dd4d01b	Deal with mypy errors w/ type-hinted pynacl 1.5.0 (#11714) ... * Deal with mypy errors w/ type-hinted pynacl 1.5.0 Fixes #11644. I really don't like that we're monkey patching pynacl SignedKey instances with alg and version objects. But I'm too scared to make the changes necessary right now. (Ideally I would replace `signedjson.types.SingingKey` with a runtime class which wraps or inherits from `nacl.signing.SigningKey`.) C.f. https://github.com/matrix-org/python-signedjson/issues/16	3 years ago
Olivier Wilkinson (reivilibre)	4dd9ea8f4f	Revert "Revert accidental fast-forward merge from v1.49.0rc1" ... This reverts commit 158d73ebdd.	3 years ago
Sean Quah	158d73ebdd	Revert accidental fast-forward merge from v1.49.0rc1 ... Revert "Sort internal changes in changelog" Revert "Update CHANGES.md" Revert "1.49.0rc1" Revert "Revert "Move `glob_to_regex` and `re_word_boundary` to `matrix-python-common` (#11505) (#11527)" Revert "Refactors in `_generate_sync_entry_for_rooms` (#11515)" Revert "Correctly register shutdown handler for presence workers (#11518)" Revert "Fix `ModuleApi.looping_background_call` for non-async functions (#11524)" Revert "Fix 'delete room' admin api to work on incomplete rooms (#11523)" Revert "Correctly ignore invites from ignored users (#11511)" Revert "Fix the test breakage introduced by #11435 as a result of concurrent PRs (#11522)" Revert "Stabilise support for MSC2918 refresh tokens as they have now been merged into the Matrix specification. (#11435)" Revert "Save the OIDC session ID (sid) with the device on login (#11482)" Revert "Add admin API to get some information about federation status (#11407)" Revert "Include bundled aggregations in /sync and related fixes (#11478)" Revert "Move `glob_to_regex` and `re_word_boundary` to `matrix-python-common` (#11505)" Revert "Update backward extremity docs to make it clear that it does not indicate whether we have fetched an events' `prev_events` (#11469)" Revert "Support configuring the lifetime of non-refreshable access tokens separately to refreshable access tokens. (#11445)" Revert "Add type hints to `synapse/tests/rest/admin` (#11501)" Revert "Revert accidental commits to develop." Revert "Newsfile" Revert "Give `tests.server.setup_test_homeserver` (nominally!) the same behaviour" Revert "Move `tests.utils.setup_test_homeserver` to `tests.server`" Revert "Convert one of the `setup_test_homeserver`s to `make_test_homeserver_synchronous`" Revert "Disambiguate queries on `state_key` (#11497)" Revert "Comments on the /sync tentacles (#11494)" Revert "Clean up tests.storage.test_appservice (#11492)" Revert "Clean up `tests.storage.test_main` to remove use of legacy code. (#11493)" Revert "Clean up `tests.test_visibility` to remove legacy code. (#11495)" Revert "Minor cleanup on recently ported doc pages (#11466)" Revert "Add most of the missing type hints to `synapse.federation`. (#11483)" Revert "Avoid waiting for zombie processes in `synctl stop` (#11490)" Revert "Fix media repository failing when media store path contains symlinks (#11446)" Revert "Add type annotations to `tests.storage.test_appservice`. (#11488)" Revert "`scripts-dev/sign_json`: support for signing events (#11486)" Revert "Add MSC3030 experimental client and federation API endpoints to get the closest event to a given timestamp (#9445)" Revert "Port wiki pages to documentation website (#11402)" Revert "Add a license header and comment. (#11479)" Revert "Clean-up get_version_string (#11468)" Revert "Link background update controller docs to summary (#11475)" Revert "Additional type hints for config module. (#11465)" Revert "Register the login redirect endpoint for v3. (#11451)" Revert "Update openid.md" Revert "Remove mention of OIDC certification from Dex (#11470)" Revert "Add a note about huge pages to our Postgres doc (#11467)" Revert "Don't start Synapse master process if `worker_app` is set (#11416)" Revert "Expose worker & homeserver as entrypoints in `setup.py` (#11449)" Revert "Bundle relations of relations into the `/relations` result. (#11284)" Revert "Fix `LruCache` corruption bug with a `size_callback` that can return 0 (#11454)" Revert "Eliminate a few `Any`s in `LruCache` type hints (#11453)" Revert "Remove unnecessary `json.dumps` from `tests.rest.admin` (#11461)" Revert "Merge branch 'master' into develop" This reverts commit 26b5d2320f. This reverts commit bce4220f38. This reverts commit 966b5d0fa0. This reverts commit 088d748f2c. This reverts commit 14d593f72d. This reverts commit 2a3ec6facf. This reverts commit eccc49d755. This reverts commit b1ecd19c5d. This reverts commit 9c55dedc8c. This reverts commit 2d42e586a8. This reverts commit 2f053f3f82. This reverts commit a15a893df8. This reverts commit 8b4b153c9e. This reverts commit 494ebd7347. This reverts commit a77c369897. This reverts commit 4eb77965cd. This reverts commit 637df95de6. This reverts commit e5f426cd54. This reverts commit 8cd68b8102. This reverts commit 6cae125e20. This reverts commit 7be88fbf48. This reverts commit b3fd99b74a. This reverts commit f7ec6e7d9e. This reverts commit 5640992d17. This reverts commit d26808dd85. This reverts commit f91624a595. This reverts commit 16d39a5490. This reverts commit 8a4c296987. This reverts commit 49e1356ee3. This reverts commit d2279f471b. This reverts commit b50e39df57. This reverts commit 858d80bf0f. This reverts commit 435f044807. This reverts commit f61462e1be. This reverts commit a6f1a3abec. This reverts commit 84dc50e160. This reverts commit ed635d3285. This reverts commit 7b62791e00. This reverts commit 153194c771. This reverts commit f44d729d4c. This reverts commit a265fbd397. This reverts commit b9fef1a7cd. This reverts commit b0eb64ff7b. This reverts commit f1795463bf. This reverts commit 70cbb1a5e3. This reverts commit 42bf020463. This reverts commit 379f2650cf. This reverts commit 7ff22d6da4. This reverts commit 5a0b652d36. This reverts commit 432a174bc1. This reverts commit b14f8a1baf, reversing changes made to e713855dca.	3 years ago
Richard van der Hoff	9cd13c5f63	Fix perspectives requests for multiple keys for the same server (#11440) ... If we tried to request multiple keys for the same server, we would end up dropping some of those requests.	3 years ago
Richard van der Hoff	a6f7f84570	Fix verification of objects signed with old local keys (#11379) ... Fixes a bug introduced in #11129: objects signed by the local server, but with keys other than the current one, could not be successfully verified. We need to check the key id in the signature, and track down the right key.	3 years ago
Shay	e002faee01	Fetch verify key locally rather than trying to do so over federation if origin and host are the same. (#11129) ... * add tests for fetching key locally * add logic to check if origin server is same as host and fetch verify key locally rather than over federation * add changelog * slight refactor, add docstring, change changelog entry * Make changelog entry one line * remove verify_json_locally and push locality check to process_request, add function process_request_locally * remove leftover code reference * refactor to add common call to 'verify_json and associated handling code * add type hint to process_json * add some docstrings + very slight refactor	3 years ago
Erik Johnston	fc3d2dc269	Rewrite the KeyRing (#10035)	4 years ago
Jonathan de Jong	4b965c862d	Remove redundant "coding: utf-8" lines (#9786) ... Part of #9744 Removes all redundant `# -*- coding: utf-8 -*-` lines from files, as python 3 automatically reads source code as utf-8 now. `Signed-off-by: Jonathan de Jong <jonathan@automatia.nl>`	4 years ago
Patrick Cloke	0b3112123d	Use mock from the stdlib. (#9772)	4 years ago
Patrick Cloke	48d44ab142	Record more information into structured logs. (#9654) ... Records additional request information into the structured logs, e.g. the requester, IP address, etc.	4 years ago
Eric Eastwood	0a00b7ff14	Update black, and run auto formatting over the codebase (#9381) ... - Update black version to the latest - Run black auto formatting over the codebase - Run autoformatting according to [`docs/code_style.md `](80d6dc9783/docs/code_style.md) - Update `code_style.md` docs around installing black to use the correct version	4 years ago
Patrick Cloke	1c9a850562	Add type hints to the crypto module. (#8999)	4 years ago
Patrick Cloke	30fba62108	Apply an IP range blacklist to push and key revocation requests. (#8821) ... Replaces the `federation_ip_range_blacklist` configuration setting with an `ip_range_blacklist` setting with wider scope. It now applies to: * Federation * Identity servers * Push notifications * Checking key validitity for third-party invite events The old `federation_ip_range_blacklist` setting is still honored if present, but with reduced scope (it only applies to federation and identity servers).	4 years ago
Patrick Cloke	c9c0ad5e20	Remove the deprecated Handlers object (#8494) ... All handlers now available via get_*_handler() methods on the HomeServer.	4 years ago
Richard van der Hoff	5e3ca12b15	Create a mechanism for marking tests "logcontext clean" (#8399)	4 years ago
Richard van der Hoff	fec6f9ac17	Fix occasional "Re-starting finished log context" from keyring (#8398) ... * Fix test_verify_json_objects_for_server_awaits_previous_requests It turns out that this wasn't really testing what it thought it was testing (in particular, `check_context` was turning failures into success, which was making the tests pass even though it wasn't clear they should have been. It was also somewhat overcomplex - we can test what it was trying to test without mocking out perspectives servers. * Fix warnings about finished logcontexts in the keyring We need to make sure that we finish the key fetching magic before we run the verifying code, to ensure that we don't mess up our logcontexts.	4 years ago
Patrick Cloke	c619253db8	Stop sub-classing object (#8249)	4 years ago
Patrick Cloke	e00816ad98	Do not yield on awaitables in tests. (#8193)	4 years ago
Patrick Cloke	2a89ce8cd4	Convert the crypto module to async/await. (#8003)	4 years ago
Patrick Cloke	c978f6c451	Convert federation client to async/await. (#7975)	4 years ago
Patrick Cloke	38e1fac886	Fix some spelling mistakes / typos. (#7811)	4 years ago
Richard van der Hoff	39230d2171	Clean up some LoggingContext stuff (#7120) ... * Pull Sentinel out of LoggingContext ... and drop a few unnecessary references to it * Factor out LoggingContext.current_context move `current_context` and `set_context` out to top-level functions. Mostly this means that I can more easily trace what's actually referring to LoggingContext, but I think it's generally neater. * move copy-to-parent into `stop` this really just makes `start` and `stop` more symetric. It also means that it behaves correctly if you manually `set_log_context` rather than using the context manager. * Replace `LoggingContext.alive` with `finished` Turn `alive` into `finished` and make it a bit better defined.	5 years ago
Richard van der Hoff	799001f2c0	Add a `make_event_from_dict` method (#6858) ... ... and use it in places where it's trivial to do so. This will make it easier to pass room versions into the FrozenEvent constructors.	5 years ago
Richard van der Hoff	2a81393a4b	Pass room_version into add_hashes_and_signatures	5 years ago
Richard van der Hoff	4b36b482e0	Fix exception when fetching notary server's old keys (#6625) ... Lift the restriction that *all* the keys used for signing v2 key responses be present in verify_keys. Fixes #6596.	5 years ago
Erik Johnston	a8d16f6c00	Review comments	5 years ago
Amber Brown	4806651744	Replace returnValue with return (#5736)	5 years ago
Richard van der Hoff	c7095be913	Refactor Keyring._start_key_lookups ... There's an awful lot of deferreds and dictionaries flying around here. The whole thing can be made much simpler and achieve the same effect.	5 years ago
Amber Brown	463b072b12	Move logging utilities out of the side drawer of util/ and into logging/ (#5606)	5 years ago
Amber Brown	32e7c9e7f2	Run Black. (#5482)	6 years ago
Erik Johnston	43badd2cd4	Fix key verification when key stored with null valid_until_ms ... Some keys are stored in the synapse database with a null valid_until_ms which caused an exception to be thrown when using that key. We fix this by treating nulls as zeroes, i.e. they keys will match verification requests with a minimum_valid_until_ms of zero (i.e. don't validate ts) but will not match requests with a non-zero minimum_valid_until_ms. Fixes #5391.	6 years ago
Richard van der Hoff	9fbb20a531	Stop hardcoding trust of old matrix.org key (#5374) ... There are a few changes going on here: * We make checking the signature on a key server response optional: if no verify_keys are specified, we trust to TLS to validate the connection. * We change the default config so that it does not require responses to be signed by the old key. * We replace the old 'perspectives' config with 'trusted_key_servers', which is also formatted slightly differently. * We emit a warning to the logs every time we trust a key server response signed by the old key.	6 years ago
Richard van der Hoff	cea9750d11	Associate a request_name with each verify request, for logging ... Also: * rename VerifyKeyRequest->VerifyJsonRequest * calculate key_ids on VerifyJsonRequest construction * refactor things to pass around VerifyJsonRequests instead of 4-tuples	6 years ago
Richard van der Hoff	def5ea4062	Don't bomb out on direct key fetches as soon as one fails	6 years ago
Richard van der Hoff	fec2dcb1a5	Enforce validity period on server_keys for fed requests. (#5321) ... When handling incoming federation requests, make sure that we have an up-to-date copy of the signing key. We do not yet enforce the validity period for event signatures.	6 years ago
Richard van der Hoff	fa1b293da2	Simplification to Keyring.wait_for_previous_lookups. (#5250) ... The list of server names was redundant, since it was equivalent to the keys on the server_to_deferred map. This reduces the number of large lists being passed around, and has the benefit of deduplicating the entries in `wait_on`.	6 years ago
Richard van der Hoff	753b1270da	Require sig from origin server on perspectives responses	6 years ago