watcha-synapse

Commit Graph

Author	SHA1	Message	Date
reivilibre	5172c8c403	Faster remote room joins (worker mode): do not populate external hosts-in-room cache when sending events as this requires blocking for full state. (#14749 ) Signed-off-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org> Co-authored-by: Sean Quah <seanq@matrix.org>	2 years ago
Mathieu Velten	54c012c5a8	Make `handle_new_client_event` throws `PartialStateConflictError` (#14665 ) Then adapts calling code to retry when needed so it doesn't 500 to clients. Signed-off-by: Mathieu Velten <mathieuv@matrix.org> Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com>	2 years ago
David Robertson	e2a1adbf5d	Allow selecting "prejoin" events by state keys (#14642 ) * Declare new config * Parse new config * Read new config * Don't use trial/our TestCase where it's not needed Before: ``` $ time trial tests/events/test_utils.py > /dev/null real 0m2.277s user 0m2.186s sys 0m0.083s ``` After: ``` $ time trial tests/events/test_utils.py > /dev/null real 0m0.566s user 0m0.508s sys 0m0.056s ``` * Helper to upsert to event fields without exceeding size limits. * Use helper when adding invite/knock state Now that we allow admins to include events in prejoin room state with arbitrary state keys, be a good Matrix citizen and ensure they don't accidentally create an oversized event. * Changelog * Move StateFilter tests should have done this in #14668 * Add extra methods to StateFilter * Use StateFilter * Ensure test file enforces typed defs; alphabetise * Workaround surprising get_current_state_ids * Whoops, fix mypy	2 years ago
David Robertson	b5b5f66084	Move `StateFilter` to `synapse.types` (#14668 ) * Move `StateFilter` to `synapse.types` * Changelog	2 years ago
Shay	72f3e38137	Fix possible variable shadow in `create_new_client_event` (#14575 )	2 years ago
Brendan Abolivier	86c5a710d8	Implement MSC3912: Relation-based redactions (#14260 ) Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com>	2 years ago
Shay	b7a7ff6ee3	Add initial power level event to batch of bulk persisted events when creating a new room. (#14228 )	2 years ago
Shay	847e2393f3	Prepatory work for adding power level event to batched events (#14214 )	2 years ago
Shay	b6baa46db0	Fix a bug where the joined hosts for a given event were not being properly cached (#14125 )	2 years ago
Shay	7b7478e8b6	Batch up notifications after event persistence (#14033 )	2 years ago
Shay	8ab16a92ed	Persist CreateRoom events to DB in a batch (#13800 )	2 years ago
Shay	a2cf66a94d	Prepatory work for batching events to send (#13487 ) This PR begins work on batching up events during the creation of a room. The PR splits out the creation and sending/persisting of the events. The first three events in the creation of the room-creating the room, joining the creator to the room, and the power levels event are sent sequentially, while the subsequent events are created and collected to be sent at the end of the function. This is currently done by appending them to a list and then iterating over the list to send, the next step (after this PR) would be to send and persist the collected events as a batch.	2 years ago
reivilibre	6302753012	Deduplicate `is_server_notices_room`. (#13780 )	2 years ago
Eric Eastwood	d58615c82c	Directly lookup local membership instead of getting all members in a room first (`get_users_in_room` mis-use) (#13608 ) See https://github.com/matrix-org/synapse/pull/13575#discussion_r953023755	2 years ago
Quentin Gliech	3dd175b628	`synapse.api.auth.Auth` cleanup: make permission-related methods use `Requester` instead of the `UserID` (#13024 ) Part of #13019 This changes all the permission-related methods to rely on the Requester instead of the UserID. This is a first step towards enabling scoped access tokens at some point, since I expect the Requester to have scope-related informations in it. It also changes methods which figure out the user/device/appservice out of the access token to return a Requester instead of something else. This avoids having store-related objects in the methods signatures.	2 years ago
reivilibre	c3516e9dec	Faster room joins: make `/joined_members` block whilst the room is partial stated. (#13514 )	2 years ago
Shay	a648a06d52	Add some tracing spans to give insight into local joins (#13439 )	2 years ago
andrew do	78a3111c41	Return 404 or member list when getting joined_members after leaving (#13374 ) Signed-off-by: Andrew Doh <andrewddo@gmail.com> Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com> Co-authored-by: Andrew Morgan <andrewm@element.io> Co-authored-by: Brendan Abolivier <babolivier@matrix.org>	2 years ago
Will Hunt	502f075e96	Implement MSC3848: Introduce errcodes for specific event sending failures (#13343 ) Implements MSC3848	2 years ago
Sean Quah	335ebb21cc	Faster room joins: avoid blocking when pulling events with missing prevs (#13355 ) Avoid blocking on full state in `_resolve_state_at_missing_prevs` and return a new flag indicating whether the resolved state is partial. Thread that flag around so that it makes it into the event context. Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>	2 years ago
David Robertson	b977867358	Rate limit joins per-room (#13276 )	2 years ago
Erik Johnston	cf5fa5063d	Don't pull out full state when sending dummy events (#13310 )	2 years ago
Erik Johnston	0ca4172b5d	Don't pull out state in `compute_event_context` for unconflicted state (#13267 )	2 years ago
Sean Quah	68db233f0c	Handle race between persisting an event and un-partial stating a room (#13100 ) Whenever we want to persist an event, we first compute an event context, which includes the state at the event and a flag indicating whether the state is partial. After a lot of processing, we finally try to store the event in the database, which can fail for partial state events when the containing room has been un-partial stated in the meantime. We detect the race as a foreign key constraint failure in the data store layer and turn it into a special `PartialStateConflictError` exception, which makes its way up to the method in which we computed the event context. To make things difficult, the exception needs to cross a replication request: `/fed_send_events` for events coming over federation and `/send_event` for events from clients. We transport the `PartialStateConflictError` as a `409 Conflict` over replication and turn `409`s back into `PartialStateConflictError`s on the worker making the request. All client events go through `EventCreationHandler.handle_new_client_event`, which is called in a lot of places. Instead of trying to update all the code which creates client events, we turn the `PartialStateConflictError` into a `429 Too Many Requests` in `EventCreationHandler.handle_new_client_event` and hope that clients take it as a hint to retry their request. On the federation event side, there are 7 places which compute event contexts. 4 of them use outlier event contexts: `FederationEventHandler._auth_and_persist_outliers_inner`, `FederationHandler.do_knock`, `FederationHandler.on_invite_request` and `FederationHandler.do_remotely_reject_invite`. These events won't have the partial state flag, so we do not need to do anything for then. The remaining 3 paths which create events are `FederationEventHandler.process_remote_join`, `FederationEventHandler.on_send_membership_event` and `FederationEventHandler._process_received_pdu`. We can't experience the race in `process_remote_join`, unless we're handling an additional join into a partial state room, which currently blocks, so we make no attempt to handle it correctly. `on_send_membership_event` is only called by `FederationServer._on_send_membership_event`, so we catch the `PartialStateConflictError` there and retry just once. `_process_received_pdu` is called by `on_receive_pdu` for incoming events and `_process_pulled_event` for backfill. The latter should never try to persist partial state events, so we ignore it. We catch the `PartialStateConflictError` in `on_receive_pdu` and retry just once. Refering to the graph of code paths in https://github.com/matrix-org/synapse/issues/12988#issuecomment-1156857648 may make the above make more sense. Signed-off-by: Sean Quah <seanq@matrix.org>	2 years ago
Shay	046a6513bc	Don't process /send requests for users who have hit their ratelimit (#13134 )	2 years ago
Quentin Gliech	92103cb2c8	Decouple `synapse.api.auth_blocking.AuthBlocking` from `synapse.api.auth.Auth`. (#13021 )	2 years ago
David Teller	a164a46038	Uniformize spam-checker API, part 4: port other spam-checker callbacks to return `Union[Allow, Codes]`. (#12857 ) Co-authored-by: Brendan Abolivier <babolivier@matrix.org>	2 years ago
Richard van der Hoff	c1b28b8842	Remove redundant `room_version` param from `check_auth_rules_from_context` It's now implied by the room_version property on the event.	3 years ago
Richard van der Hoff	68be42f6b6	Remove `room_version` param from `validate_event_for_room_version` Instead, use the `room_version` property of the event we're validating. The `room_version` was originally added as a parameter somewhere around #4482, but really it's been redundant since #6875 added a `room_version` field to `EventBase`.	3 years ago
Richard van der Hoff	7c6b2204d1	Faster joins: add issue links to the TODOs (#13004 ) ... to help us keep track of these things	3 years ago
Erik Johnston	e3163e2e11	Reduce the amount of state we pull from the DB (#12811 )	3 years ago
Erik Johnston	888a29f412	Wait for lazy join to complete when getting current state (#12872 )	3 years ago
Richard van der Hoff	79dadf7216	Fix 404 on `/sync` when the last event is a redaction of an unknown/purged event (#12905 ) Currently, we try to pull the event corresponding to a sync token from the database. However, when we fetch redaction events, we check the target of that redaction (because we aren't allowed to send redactions to clients without validating them). So, if the sync token points to a redaction of an event that we don't have, we have a problem. It turns out we don't really need that event, and can just work with its ID and metadata, which sidesteps the whole problem.	3 years ago
Erik Johnston	1e453053cb	Rename storage classes (#12913 )	3 years ago
Brendan Abolivier	8fd87739bf	Fix import in module_api module and docs on the new check_event_for_spam signature (#12918 ) Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>	3 years ago
David Teller	af7db19e1e	Uniformize spam-checker API, part 3: Expand check_event_for_spam with the ability to return additional fields (#12846 ) Signed-off-by: David Teller <davidt@element.io>	3 years ago
Erik Johnston	b83bc5fab5	Pull out less state when handling gaps mk2 (#12852 )	3 years ago
Erik Johnston	4660d9fdcf	Fix up `state_store` naming (#12871 )	3 years ago
David Teller	28199e9357	Uniformize spam-checker API, part 2: check_event_for_spam (#12808 ) Signed-off-by: David Teller <davidt@element.io>	3 years ago
Shay	71e8afe34d	Update EventContext `get_current_event_ids` and `get_prev_event_ids` to accept state filters and update calls where possible (#12791 )	3 years ago
Patrick Cloke	86a515ccbf	Consolidate logic for parsing relations. (#12693 ) Parse the `m.relates_to` event content field (which describes relations) in a single place, this is used during: * Event persistence. * Validation of the Client-Server API. * Fetching bundled aggregations. * Processing of push rules. Each of these separately implement the logic and each made slightly different assumptions about what was valid. Some had minor / potential bugs.	3 years ago
Patrick Cloke	a4c75918b3	Remove unneeded `ActionGenerator` class. (#12691 ) It simply passes through to `BulkPushRuleEvaluator`, which can be called directly instead.	3 years ago
Erik Johnston	c72d26c1e1	Refactor `EventContext` (#12689 ) Refactor how the `EventContext` class works, with the intention of reducing the amount of state we fetch from the DB during event processing. The idea here is to get rid of the cached `current_state_ids` and `prev_state_ids` that live in the `EventContext`, and instead defer straight to the database (and its caching). One change that may have a noticeable effect is that we now no longer prefill the `get_current_state_ids` cache on a state change. However, that query is relatively light, since its just a case of reading a table from the DB (unlike fetching state at an event which is more heavyweight). For deployments with workers this cache isn't even used. Part of #12684	3 years ago
David Robertson	2607b3e181	Update mypy to 0.950 and fix complaints (#12650 )	3 years ago
andrew do	01e625513a	remove constantly lib use and switch to enums. (#12624 )	3 years ago
Nick Mills-Barrett	e3a49f4784	Fix missing sync events during historical batch imports (#12319 ) Discovered after much in-depth investigation in #12281. Closes: #12281 Closes: #3305 Signed off by: Nick Mills-Barrett nick@beeper.com	3 years ago
Patrick Cloke	86cf6a3a17	Remove references to unstable identifiers from MSC3440. (#12382 ) Removes references to unstable thread relation, unstable identifiers for filtering parameters, and the experimental config flag.	3 years ago
Sean Quah	800ba87cc8	Refactor and convert `Linearizer` to async (#12357 ) Refactor and convert `Linearizer` to async. This makes a `Linearizer` cancellation bug easier to fix. Also refactor to use an async context manager, which eliminates an unlikely footgun where code that doesn't immediately use the context manager could forget to release the lock. Signed-off-by: Sean Quah <seanq@element.io>	3 years ago
Eric Eastwood	6f2943714b	Remove unused `auth_event_ids` argument plumbing (#12304 ) Follow-up to https://github.com/matrix-org/synapse/pull/12083 Since we are now using the new `state_event_ids` parameter to do all of the heavy lifting. We can remove any spots where we plumbed `auth_event_ids` just for MSC2716 things in https://github.com/matrix-org/synapse/pull/9247/files. Removing `auth_event_ids` from following functions: - `create_and_send_nonmember_event` - `_local_membership_update` - `update_membership` - `update_membership_locked`	3 years ago
Eric Eastwood	14662d3c18	Refactor `create_new_client_event` to use a new parameter, `state_event_ids`, which accurately describes the usage with MSC2716 instead of abusing `auth_event_ids` (#12083 ) Spawned from https://github.com/matrix-org/synapse/pull/10975#discussion_r813183430 Part of [MSC2716](https://github.com/matrix-org/matrix-spec-proposals/pull/2716)	3 years ago

1 2 3 4 5 ...

492 Commits (510d4b06e7d346b4f94cb5598da90c9f668b62bb)