272 Commits (87e5df9a6ed772daf1070919845942b39143252e)

Author	SHA1	Message	Date
Andrew Morgan	2e59e97ebd	Move ThirdPartyEventRules into module_api/callbacks (#15535)	2 years ago
Sean Quah	04e79e6a18	Add config option to forget rooms automatically when users leave them (#15224) ... This is largely based off the stats and user directory updater code. Signed-off-by: Sean Quah <seanq@matrix.org>	2 years ago
Quentin Gliech	8b3a502996	Experimental support for MSC3970: per-device transaction IDs (#15318)	2 years ago
Andrew Morgan	aec639e3e3	Move Spam Checker callbacks to a dedicated file (#15453)	2 years ago
DeepBlueV7.X	753d1d9cde	Fix joining rooms you have been unbanned from (#15323) ... * Fix joining rooms you have been unbanned from Since forever synapse did not allow you to join a room after you have been unbanned from it over federation. This was not actually because of the unban event not federating. Synapse simply used outdated state to validate the join transition. This skips the validation if we are not in the room and for that reason won't have the current room state. Fixes #1563 Signed-off-by: Nicolas Werner <nicolas.werner@hotmail.de> * Add changelog Signed-off-by: Nicolas Werner <nicolas.werner@hotmail.de> * Update changelog.d/15323.bugfix --------- Signed-off-by: Nicolas Werner <nicolas.werner@hotmail.de>	2 years ago
Dirk Klimpel	ecbe0ddbe7	Add support for knocking to workers. (#15133)	2 years ago
Shay	1c95ddd09b	Batch up storing state groups when creating new room (#14918)	2 years ago
Mathieu Velten	6cddf24e36	Faster joins: don't stall when a user joins during a fast join (#14606) ... Fixes #12801. Complement tests are at https://github.com/matrix-org/complement/pull/567. Avoid blocking on full state when handling a subsequent join into a partial state room. Also always perform a remote join into partial state rooms, since we do not know whether the joining user has been banned and want to avoid leaking history to banned users. Signed-off-by: Mathieu Velten <mathieuv@matrix.org> Co-authored-by: Sean Quah <seanq@matrix.org> Co-authored-by: David Robertson <davidr@element.io>	2 years ago
Patrick Cloke	cf5233b783	Avoid fetching unused account data in sync. (#14973) ... The per-room account data is no longer unconditionally fetched, even if all rooms will be filtered out. Global account data will not be fetched if it will all be filtered out.	2 years ago
Mathieu Velten	54c012c5a8	Make `handle_new_client_event` throws `PartialStateConflictError` (#14665) ... Then adapts calling code to retry when needed so it doesn't 500 to clients. Signed-off-by: Mathieu Velten <mathieuv@matrix.org> Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com>	2 years ago
David Robertson	b5b5f66084	Move `StateFilter` to `synapse.types` (#14668) ... * Move `StateFilter` to `synapse.types` * Changelog	2 years ago
lukasdenk	719488dda8	Add query parameter `ts` to allow appservices set the `origin_server_ts` for state events. (#11866) ... MSC3316 declares that both /rooms/{roomId}/send and /rooms/{roomId}/state should accept a ts parameter for appservices. This change expands support to /state and adds tests.	2 years ago
Eric Eastwood	a52c40e2a6	Fix `get_users_in_room` mis-use in `transfer_room_state_on_room_upgrade` (#13960) ... Spawning from looking into `get_users_in_room` while investigating https://github.com/matrix-org/synapse/issues/13942#issuecomment-1262787050. See https://github.com/matrix-org/synapse/pull/13575#discussion_r953023755 for the original exploration around finding `get_users_in_room` mis-uses. Related to the following PRs where we also cleaned up some `get_users_in_room` mis-uses: - https://github.com/matrix-org/synapse/pull/13605 - https://github.com/matrix-org/synapse/pull/13608 - https://github.com/matrix-org/synapse/pull/13606 - https://github.com/matrix-org/synapse/pull/13958	2 years ago
Shay	8ab16a92ed	Persist CreateRoom events to DB in a batch (#13800)	2 years ago
reivilibre	6302753012	Deduplicate `is_server_notices_room`. (#13780)	2 years ago
Jacek Kuśnierz	84ddcd7bbf	Drop support for calling `/_matrix/client/v3/rooms/{roomId}/invite` without an `id_access_token` (#13241) ... Fixes #13206 Signed-off-by: Jacek Kusnierz jacek.kusnierz@tum.de	2 years ago
Dirk Klimpel	682dfcfc0d	Fix that user cannot `/forget` rooms after the last member has left (#13546)	2 years ago
Eric Eastwood	d58615c82c	Directly lookup local membership instead of getting all members in a room first (`get_users_in_room` mis-use) (#13608) ... See https://github.com/matrix-org/synapse/pull/13575#discussion_r953023755	2 years ago
Dirk Klimpel	37f329c9ad	Fix that sending server notices fail if avatar is `None` (#13566) ... Indroduced in #11846.	2 years ago
Quentin Gliech	3dd175b628	`synapse.api.auth.Auth` cleanup: make permission-related methods use `Requester` instead of the `UserID` (#13024) ... Part of #13019 This changes all the permission-related methods to rely on the Requester instead of the UserID. This is a first step towards enabling scoped access tokens at some point, since I expect the Requester to have scope-related informations in it. It also changes methods which figure out the user/device/appservice out of the access token to return a Requester instead of something else. This avoids having store-related objects in the methods signatures.	2 years ago
Shay	a648a06d52	Add some tracing spans to give insight into local joins (#13439)	2 years ago
reivilibre	39be5bc550	Make minor clarifications to the error messages given when we fail to join a room via any server. (#13160)	2 years ago
David Robertson	b977867358	Rate limit joins per-room (#13276)	2 years ago
David Robertson	7281591f4c	Use state before join to determine if we `_should_perform_remote_join` (#13270) ... Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>	2 years ago
Nick Mills-Barrett	982fe29655	Optimise room creation event lookups part 2 (#13224)	2 years ago
David Teller	11f811470f	Uniformize spam-checker API, part 5: expand other spam-checker callbacks to return `Tuple[Codes, dict]` (#13044) ... Signed-off-by: David Teller <davidt@element.io> Co-authored-by: Brendan Abolivier <babolivier@matrix.org>	2 years ago
Eric Eastwood	2c2a42cc10	Fix application service not being able to join remote federated room without a profile set (#13131) ... Fix https://github.com/matrix-org/synapse/issues/4778 Complement tests: https://github.com/matrix-org/complement/pull/399	2 years ago
David Teller	80c7a06777	Rate limiting invites per issuer (#13125) ... Co-authored-by: reivilibre <oliverw@matrix.org>	2 years ago
reivilibre	7552615247	Reduce the duplication of code that invokes the rate limiter. (#13070)	2 years ago
David Teller	a164a46038	Uniformize spam-checker API, part 4: port other spam-checker callbacks to return `Union[Allow, Codes]`. (#12857) ... Co-authored-by: Brendan Abolivier <babolivier@matrix.org>	2 years ago
Erik Johnston	e3163e2e11	Reduce the amount of state we pull from the DB (#12811)	3 years ago
Erik Johnston	888a29f412	Wait for lazy join to complete when getting current state (#12872)	3 years ago
Patrick Cloke	a8db8c6eba	Remove user-visible groups/communities code (#12553) ... Makes it so that groups/communities no longer exist from a user-POV. E.g. we remove: * All API endpoints (including Client-Server, Server-Server, and admin). * Documented configuration options (and the experimental flag, which is now unused). * Special handling during room upgrades. * The `groups` section of the `/sync` response.	3 years ago
Shay	71e8afe34d	Update EventContext `get_current_event_ids` and `get_prev_event_ids` to accept state filters and update calls where possible (#12791)	3 years ago
Sean Quah	800ba87cc8	Refactor and convert `Linearizer` to async (#12357) ... Refactor and convert `Linearizer` to async. This makes a `Linearizer` cancellation bug easier to fix. Also refactor to use an async context manager, which eliminates an unlikely footgun where code that doesn't immediately use the context manager could forget to release the lock. Signed-off-by: Sean Quah <seanq@element.io>	3 years ago
Eric Eastwood	6f2943714b	Remove unused `auth_event_ids` argument plumbing (#12304) ... Follow-up to https://github.com/matrix-org/synapse/pull/12083 Since we are now using the new `state_event_ids` parameter to do all of the heavy lifting. We can remove any spots where we plumbed `auth_event_ids` just for MSC2716 things in https://github.com/matrix-org/synapse/pull/9247/files. Removing `auth_event_ids` from following functions: - `create_and_send_nonmember_event` - `_local_membership_update` - `update_membership` - `update_membership_locked`	3 years ago
Eric Eastwood	14662d3c18	Refactor `create_new_client_event` to use a new parameter, `state_event_ids`, which accurately describes the usage with MSC2716 instead of abusing `auth_event_ids` (#12083) ... Spawned from https://github.com/matrix-org/synapse/pull/10975#discussion_r813183430 Part of [MSC2716](https://github.com/matrix-org/matrix-spec-proposals/pull/2716)	3 years ago
Richard van der Hoff	8533c8b03d	Avoid generating state groups for local out-of-band leaves (#12154) ... If we locally generate a rejection for an invite received over federation, it is stored as an outlier (because we probably don't have the state for the room). However, currently we still generate a state group for it (even though the state in that state group will be nonsense). By setting the `outlier` param on `create_event`, we avoid the nonsensical state.	3 years ago
Richard van der Hoff	e24ff8ebe3	Remove `HomeServer.get_datastore()` (#12031) ... The presence of this method was confusing, and mostly present for backwards compatibility. Let's get rid of it. Part of #11733	3 years ago
Brendan Abolivier	130fd45393	Limit concurrent AS joins (#11996) ... Initially introduced in matrix-org-hotfixes by e5537cf (and tweaked by later commits). Fixes #11995 See also #4826	3 years ago
Eric Eastwood	fef2e792be	Fix historical messages backfilling in random order on remote homeservers (MSC2716) (#11114) ... Fix https://github.com/matrix-org/synapse/issues/11091 Fix https://github.com/matrix-org/synapse/issues/10764 (side-stepping the issue because we no longer have to deal with `fake_prev_event_id`) 1. Made the `/backfill` response return messages in `(depth, stream_ordering)` order (previously only sorted by `depth`) - Technically, it shouldn't really matter how `/backfill` returns things but I'm just trying to make the `stream_ordering` a little more consistent from the origin to the remote homeservers in order to get the order of messages from `/messages` consistent ([sorted by `(topological_ordering, stream_ordering)`](https://github.com/matrix-org/synapse/blob/develop/docs/development/room-dag-concepts.md#depth-and-stream-ordering)). - Even now that we return backfilled messages in order, it still doesn't guarantee the same `stream_ordering` (and more importantly the [`/messages` order](https://github.com/matrix-org/synapse/blob/develop/docs/development/room-dag-concepts.md#depth-and-stream-ordering)) on the other server. For example, if a room has a bunch of history imported and someone visits a permalink to a historical message back in time, their homeserver will skip over the historical messages in between and insert the permalink as the next message in the `stream_order` and totally throw off the sort. - This will be even more the case when we add the [MSC3030 jump to date API endpoint](https://github.com/matrix-org/matrix-doc/pull/3030) so the static archives can navigate and jump to a certain date. - We're solving this in the future by switching to [online topological ordering](https://github.com/matrix-org/gomatrixserverlib/issues/187) and [chunking](https://github.com/matrix-org/synapse/issues/3785) which by its nature will apply retroactively to fix any inconsistencies introduced by people permalinking 2. As we're navigating `prev_events` to return in `/backfill`, we order by `depth` first (newest -> oldest) and now also tie-break based on the `stream_ordering` (newest -> oldest). This is technically important because MSC2716 inserts a bunch of historical messages at the same `depth` so it's best to be prescriptive about which ones we should process first. In reality, I think the code already looped over the historical messages as expected because the database is already in order. 3. Making the historical state chain and historical event chain float on their own by having no `prev_events` instead of a fake `prev_event` which caused backfill to get clogged with an unresolvable event. Fixes https://github.com/matrix-org/synapse/issues/11091 and https://github.com/matrix-org/synapse/issues/10764 4. We no longer find connected insertion events by finding a potential `prev_event` connection to the current event we're iterating over. We now solely rely on marker events which when processed, add the insertion event as an extremity and the federating homeserver can ask about it when time calls. - Related discussion, https://github.com/matrix-org/synapse/pull/11114#discussion_r741514793 Before | After --- | ---  |  #### Why aren't we sorting topologically when receiving backfill events? > The main reason we're going to opt to not sort topologically when receiving backfill events is because it's probably best to do whatever is easiest to make it just work. People will probably have opinions once they look at [MSC2716](https://github.com/matrix-org/matrix-doc/pull/2716) which could change whatever implementation anyway. > > As mentioned, ideally we would do this but code necessary to make the fake edges but it gets confusing and gives an impression of “just whyyyy” (feels icky). This problem also dissolves with online topological ordering. > > -- https://github.com/matrix-org/synapse/pull/11114#discussion_r741517138 See https://github.com/matrix-org/synapse/pull/11114#discussion_r739610091 for the technical difficulties	3 years ago
Brendan Abolivier	d80d39b035	Add a ratelimiter for 3pid invite (#11892)	3 years ago
Brendan Abolivier	bf60da1a60	Configurable limits on avatars (#11846) ... Only allow files which file size and content types match configured limits to be set as avatar. Most of the inspiration from the non-test code comes from matrix-org/synapse-dinsic#19	3 years ago
Sean Quah	c7fe32edb4	Add type hints to `synapse/storage/databases/main/room.py` (#11575)	3 years ago
Eric Eastwood	aa8708ebed	Allow events to be created with no `prev_events` (MSC2716) (#11243) ... The event still needs to have `auth_events` defined to be valid. Split out from https://github.com/matrix-org/synapse/pull/11114	3 years ago
Eric Eastwood	7ffddd819c	Prevent historical state from being pushed to an application service via `/transactions` (MSC2716) (#11265) ... Mark historical state from the MSC2716 `/batch_send` endpoint as `historical` which makes it `backfilled` and have a negative `stream_ordering` so it doesn't get queried by `/transactions`. Fix https://github.com/matrix-org/synapse/issues/11241 Complement tests: https://github.com/matrix-org/complement/pull/221	3 years ago
Patrick Cloke	c01bc5f43d	Add remaining type hints to `synapse.events`. (#11098)	3 years ago
Patrick Cloke	eb9ddc8c2e	Remove the deprecated BaseHandler. (#11005) ... The shared ratelimit function was replaced with a dedicated RequestRatelimiter class (accessible from the HomeServer object). Other properties were copied to each sub-class that inherited from BaseHandler.	3 years ago
Brendan Abolivier	4e51621064	Add a spamchecker method to allow or deny 3pid invites (#10894) ... This is in the context of creating new module callbacks that modules in https://github.com/matrix-org/synapse-dinsic can use, in an effort to reconcile the spam checker API in synapse-dinsic with the one in mainline. Note that a module callback already exists for 3pid invites (https://matrix-org.github.io/synapse/develop/modules/third_party_rules_callbacks.html#check_threepid_can_be_invited) but it doesn't check whether the sender of the invite is allowed to send it.	3 years ago
Patrick Cloke	f4b1a9a527	Require direct references to configuration variables. (#10985) ... This removes the magic allowing accessing configurable variables directly from the config object. It is now required that a specific configuration class is used (e.g. `config.foo` must be replaced with `config.server.foo`).	3 years ago