13589 Commits (9f111075e8204dd55e141f04d78e06c718892956)

Author	SHA1	Message	Date
Patrick Cloke	3a541a7daa	Improve failover logic for MSC3083 restricted rooms. (#10447) ... If the federation client receives an M_UNABLE_TO_AUTHORISE_JOIN or M_UNABLE_TO_GRANT_JOIN response it will attempt another server before giving up completely.	3 years ago
Erik Johnston	5522a103a9	1.39.0	3 years ago
Richard van der Hoff	858363d0b7	Generics for `ObservableDeferred` (#10491) ... Now that `Deferred` is a generic class, let's update `ObeservableDeferred` to follow suit.	3 years ago
Eric Eastwood	d0b294ad97	Make historical events discoverable from backfill for servers without any scrollback history (MSC2716) (#10245) ... * Make historical messages available to federated servers Part of MSC2716: https://github.com/matrix-org/matrix-doc/pull/2716 Follow-up to https://github.com/matrix-org/synapse/pull/9247 * Debug message not available on federation * Add base starting insertion point when no chunk ID is provided * Fix messages from multiple senders in historical chunk Follow-up to https://github.com/matrix-org/synapse/pull/9247 Part of MSC2716: https://github.com/matrix-org/matrix-doc/pull/2716 --- Previously, Synapse would throw a 403, `Cannot force another user to join.`, because we were trying to use `?user_id` from a single virtual user which did not match with messages from other users in the chunk. * Remove debug lines * Messing with selecting insertion event extremeties * Move db schema change to new version * Add more better comments * Make a fake requester with just what we need See https://github.com/matrix-org/synapse/pull/10276#discussion_r660999080 * Store insertion events in table * Make base insertion event float off on its own See https://github.com/matrix-org/synapse/pull/10250#issuecomment-875711889 Conflicts: synapse/rest/client/v1/room.py * Validate that the app service can actually control the given user See https://github.com/matrix-org/synapse/pull/10276#issuecomment-876316455 Conflicts: synapse/rest/client/v1/room.py * Add some better comments on what we're trying to check for * Continue debugging * Share validation logic * Add inserted historical messages to /backfill response * Remove debug sql queries * Some marker event implemntation trials * Clean up PR * Rename insertion_event_id to just event_id * Add some better sql comments * More accurate description * Add changelog * Make it clear what MSC the change is part of * Add more detail on which insertion event came through * Address review and improve sql queries * Only use event_id as unique constraint * Fix test case where insertion event is already in the normal DAG * Remove debug changes * Switch to chunk events so we can auth via power_levels Previously, we were using `content.chunk_id` to connect one chunk to another. But these events can be from any `sender` and we can't tell who should be able to send historical events. We know we only want the application service to do it but these events have the sender of a real historical message, not the application service user ID as the sender. Other federated homeservers also have no indicator which senders are an application service on the originating homeserver. So we want to auth all of the MSC2716 events via power_levels and have them be sent by the application service with proper PL levels in the room. * Switch to chunk events for federation * Add unstable room version to support new historical PL * Fix federated events being rejected for no state_groups Add fix from https://github.com/matrix-org/synapse/pull/10439 until it merges. * Only connect base insertion event to prev_event_ids Per discussion with @erikjohnston, https://matrix.to/#/!UytJQHLQYfvYWsGrGY:jki.re/$12bTUiObDFdHLAYtT7E-BvYRp3k_xv8w0dUQHibasJk?via=jki.re&via=matrix.org * Make it possible to get the room_version with txn * Allow but ignore historical events in unsupported room version See https://github.com/matrix-org/synapse/pull/10245#discussion_r675592489 We can't reject historical events on unsupported room versions because homeservers without knowledge of MSC2716 or the new room version don't reject historical events either. Since we can't rely on the auth check here to stop historical events on unsupported room versions, I've added some additional checks in the processing/persisting code (`synapse/storage/databases/main/events.py` -> `_handle_insertion_event` and `_handle_chunk_event`). I've had to do some refactoring so there is method to fetch the room version by `txn`. * Move to unique index syntax See https://github.com/matrix-org/synapse/pull/10245#discussion_r675638509 * High-level document how the insertion->chunk lookup works * Remove create_event fallback for room_versions See https://github.com/matrix-org/synapse/pull/10245/files#r677641879 * Use updated method name	3 years ago
Erik Johnston	5146e19880	1.39.0rc3	3 years ago
Richard van der Hoff	d9cb658c78	Fix up type hints for Twisted 21.7 (#10490) ... Mostly this involves decorating a few Deferred declarations with extra type hints. We wrap the types in quotes to avoid runtime errors when running against older versions of Twisted that don't have generics on Deferred.	3 years ago
Richard van der Hoff	9643dfde6a	improve typing annotations in CachedCall (#10450) ... tighten up some of the typing in CachedCall, which is going to be needed when Twisted 21.7 brings better typing on Deferred.	3 years ago
Patrick Cloke	752fe0cd98	Restricted rooms (MSC3083) should not have their allow key redacted. (#10489)	3 years ago
Šimon Brandner	c3b037795a	Support for MSC2285 (hidden read receipts) (#10413) ... Implementation of matrix-org/matrix-doc#2285	3 years ago
sri-vidyut	8e1febc6a1	Support underscores (in addition to hyphens) for charset detection. (#10410)	3 years ago
Erik Johnston	5b22d5ee03	Fix `oldest_pdu_in_federation_staging` (#10455) ... If the staging area was empty we'd report an age of 51 years, which is not true or helpful.	3 years ago
Dirk Klimpel	076deade02	allow specifying https:// proxy (#10411)	3 years ago
Jason Robinson	31c6b30dd4	Fix import of the default SAML mapping provider. (#10477) ... Fix a circular import, which was causing exceptions on boot if SAML was configured.	3 years ago
reivilibre	10dcfae46f	Fix typo that causes R30v2 to actually be old R30 (#10486) ... Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net>	3 years ago
Andrew Morgan	74d09a43d9	Always communicate device OTK counts to clients (#10485) ... Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>	3 years ago
reivilibre	e16eab29d6	Add a PeriodicallyFlushingMemoryHandler to prevent logging silence (#10407) ... Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net>	3 years ago
Denis Kasak	2476d5373c	Mitigate media repo XSSs on IE11. (#10468) ... IE11 doesn't support Content-Security-Policy but it has support for a non-standard X-Content-Security-Policy header, which only supports the sandbox directive. This prevents script execution, so it at least offers some protection against media repo-based attacks. Signed-off-by: Denis Kasak <dkasak@termina.org.uk>	3 years ago
Travis Ralston	b3a757eb3b	Support MSC2033: Device ID on whoami (#9918) ... * Fix no-access-token bug in deactivation tests * Support MSC2033: Device ID on whoami * Test for appservices too MSC: https://github.com/matrix-org/matrix-doc/pull/2033 The MSC has passed FCP, which means stable endpoints can be used.	3 years ago
Patrick Cloke	b7186c6e8d	Add type hints to state handler. (#10482)	3 years ago
Patrick Cloke	228decfce1	Update the MSC3083 support to verify if joins are from an authorized server. (#10254)	3 years ago
Patrick Cloke	4fb92d93ea	Add type hints to synapse.federation.transport.client. (#10408)	3 years ago
Erik Johnston	f76f8c1567	1.39.0rc2	3 years ago
Erik Johnston	283bb5c94e	1.38.1	3 years ago
David Baker	7da24b975d	Always send device_one_time_keys_count (#10457) ... As per comment Fixes https://github.com/matrix-org/synapse/issues/10456 See also https://github.com/vector-im/element-android/issues/3725	3 years ago
Dirk Klimpel	89c4ca81bb	Add `creation_ts` to list users admin API (#10448) ... Signed-off-by: Dirk Klimpel dirk@klimpel.org	3 years ago
Erik Johnston	38b346a504	Replace `or_ignore` in `simple_insert` with `simple_upsert` (#10442) ... Now that we have `simple_upsert` that should be used in preference to trying to insert and looking for an exception. The main benefit is that we ERROR message don't get written to postgres logs. We also have tidy up the return value on `simple_upsert`, rather than having a tri-state of inserted/not-inserted/unknown.	3 years ago
Richard van der Hoff	d8324b8238	Fix a handful of type annotations. (#10446) ... * switch from `types.CoroutineType` to `typing.Coroutine` these should be identical semantically, and since `defer.ensureDeferred` is defined to take a `typing.Coroutine`, will keep mypy happy * Fix some annotations on inlineCallbacks functions * changelog	3 years ago
Patrick Cloke	590cc4e888	Add type hints to additional servlet functions (#10437) ... Improves type hints for: * parse_{boolean,integer} * parse_{boolean,integer}_from_args * parse_json_{value,object}_from_request And fixes any incorrect calls that resulted from unknown types.	3 years ago
Patrick Cloke	d15e72e511	Update the notification email subject when invited to a space. (#10426)	3 years ago
Patrick Cloke	5db118626b	Add a return type to parse_string. (#10438) ... And set the required attribute in a few places which will error if a parameter is not provided.	3 years ago
Eric Eastwood	2d89c66b88	Switch to `chunk` events so we can auth via power_levels (MSC2716) (#10432) ... Previously, we were using `content.chunk_id` to connect one chunk to another. But these events can be from any `sender` and we can't tell who should be able to send historical events. We know we only want the application service to do it but these events have the sender of a real historical message, not the application service user ID as the sender. Other federated homeservers also have no indicator which senders are an application service on the originating homeserver. So we want to auth all of the MSC2716 events via power_levels and have them be sent by the application service with proper PL levels in the room.	3 years ago
Erik Johnston	54389d5697	Fix dropping locks on shut down (#10433)	3 years ago
Michael Telatynski	69226c1ab4	MSC3244 room capabilities implementation (#10283)	3 years ago
Andrew Morgan	12623cf38c	1.39.0rc1	3 years ago
Erik Johnston	794371b1bf	Revert "Fix dropping locks on shut down" ... This reverts commit 83f1ccfcab.	3 years ago
Erik Johnston	83f1ccfcab	Fix dropping locks on shut down	3 years ago
Brendan Abolivier	a743bf4694	Port the ThirdPartyEventRules module interface to the new generic interface (#10386) ... Port the third-party event rules interface to the generic module interface introduced in v1.37.0	3 years ago
Erik Johnston	f3ac9c6750	Fix exception when failing to get remote room list (#10414)	3 years ago
reivilibre	eebfd024e9	Factorise `get_datastore` calls in phone_stats_home. (#10427) ... Follow-up to #10332.	3 years ago
reivilibre	4e340412c0	Add a new version of the R30 phone-home metric, which removes a false impression of retention given by the old R30 metric (#10332) ... Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net>	3 years ago
Jonathan de Jong	95e47b2e78	[pyupgrade] `synapse/` (#10348) ... This PR is tantamount to running ``` pyupgrade --py36-plus --keep-percent-format `find synapse/ -type f -name "*.py"` ``` Part of #9744	3 years ago
Eric Eastwood	7387d6f624	Remove unused `events_by_room` (#10421) ... It looks like it was first used and introduced in 5130d80d79 (diff-8a4a36a7728107b2ccaff2cb405dbab229a1100fe50653a63d1aa9ac10ae45e8R305) but the But the usage was removed in 4c6a31cd6e (diff-8a4a36a7728107b2ccaff2cb405dbab229a1100fe50653a63d1aa9ac10ae45e8)	3 years ago
Jonathan de Jong	323452944e	One last inline type hint (for the whole repo) (#10418)	3 years ago
Jonathan de Jong	98aec1cc9d	Use inline type hints in `handlers/` and `rest/`. (#10382)	3 years ago
Brendan Abolivier	36dc15412d	Add a module type for account validity (#9884) ... This adds an API for third-party plugin modules to implement account validity, so they can provide this feature instead of Synapse. The module implementing the current behaviour for this feature can be found at https://github.com/matrix-org/synapse-email-account-validity. To allow for a smooth transition between the current feature and the new module, hooks have been added to the existing account validity endpoints to allow their behaviours to be overridden by a module.	3 years ago
Patrick Cloke	d427f64724	Do not include signatures/hashes in make_{join,leave,knock} responses. (#10404) ... These signatures would end up invalid since the joining/leaving/knocking server would modify the response before calling send_{join,leave,knock}.	3 years ago
Jonathan de Jong	bdfde6dca1	Use inline type hints in `http/federation/`, `storage/` and `util/` (#10381)	3 years ago
Erik Johnston	3acf85c85f	Reduce likelihood of Postgres table scanning `state_groups_state`. (#10359) ... The postgres statistics collector sometimes massively underestimates the number of distinct state groups are in the `state_groups_state`, which can cause postgres to use table scans for queries for multiple state groups. We fix this by manually setting `n_distinct` on the column.	3 years ago
Erik Johnston	ac5c221208	Stagger send presence to remotes (#10398) ... This is to help with performance, where trying to connect to thousands of hosts at once can consume a lot of CPU (due to TLS etc). Co-authored-by: Brendan Abolivier <babolivier@matrix.org>	3 years ago
Jonathan de Jong	bf72d10dbf	Use inline type hints in various other places (in `synapse/`) (#10380)	3 years ago