72 Commits (e7c77a8750094616419720379afa02506e716c7d)

Author	SHA1	Message	Date
Dirk Klimpel	50ae4eafe1	Add some type hints to `event_federation` datastore (#12753) ... Co-authored-by: David Robertson <david.m.robertson1@gmail.com>	3 years ago
David Robertson	a7293ef16f	Bump `black` and `click` versions (#12320)	3 years ago
Richard van der Hoff	f0b03186d9	Add type hints for `tests/unittest.py`. (#12347) ... In particular, add type hints for get_success and friends, which are then helpful in a bunch of places.	3 years ago
Richard van der Hoff	319a805cd3	Raise an exception when getting state at an outlier (#12191) ... It seems like calling `_get_state_group_for_events` for an event where the state is unknown is an error. Accordingly, let's raise an exception rather than silently returning an empty result.	3 years ago
David Robertson	a2b00a4486	Bump `black` and `click` versions (#12320)	3 years ago
Dirk Klimpel	9e06e22064	Add type hints to more tests files. (#12240)	3 years ago
Richard van der Hoff	e24ff8ebe3	Remove `HomeServer.get_datastore()` (#12031) ... The presence of this method was confusing, and mostly present for backwards compatibility. Let's get rid of it. Part of #11733	3 years ago
Richard van der Hoff	0fb3dd0830	Refactor the way we set `outlier` (#11634) ... * `_auth_and_persist_outliers`: mark persisted events as outliers Mark any events that get persisted via `_auth_and_persist_outliers` as, well, outliers. Currently this will be a no-op as everything will already be flagged as an outlier, but I'm going to change that. * `process_remote_join`: stop flagging as outlier The events are now flagged as outliers later on, by `_auth_and_persist_outliers`. * `send_join`: remove `outlier=True` The events created here are returned in the result of `send_join` to `FederationHandler.do_invite_join`. From there they are passed into `FederationEventHandler.process_remote_join`, which passes them to `_auth_and_persist_outliers`... which sets the `outlier` flag. * `get_event_auth`: remove `outlier=True` stop flagging the events returned by `get_event_auth` as outliers. This method is only called by `_get_remote_auth_chain_for_event`, which passes the results into `_auth_and_persist_outliers`, which will flag them as outliers. * `_get_remote_auth_chain_for_event`: remove `outlier=True` we pass all the events into `_auth_and_persist_outliers`, which will now flag the events as outliers. * `_check_sigs_and_hash_and_fetch`: remove unused `outlier` parameter This param is now never set to True, so we can remove it. * `_check_sigs_and_hash_and_fetch_one`: remove unused `outlier` param This is no longer set anywhere, so we can remove it. * `get_pdu`: remove unused `outlier` parameter ... and chase it down into `get_pdu_from_destination_raw`. * `event_from_pdu_json`: remove redundant `outlier` param This is never set to `True`, so can be removed. * changelog * update docstring	3 years ago
Eric Eastwood	daf498e099	Fix 500 error on `/messages` when we accumulate more than 5 backward extremities (#11027) ... Found while working on the Gitter backfill script and noticed it only happened after we sent 7 batches, https://gitlab.com/gitterHQ/webapp/-/merge_requests/2229#note_665906390 When there are more than 5 backward extremities for a given depth, backfill will throw an error because we sliced the extremity list to 5 but then try to iterate over the full list. This causes us to look for state that we never fetched and we get a `KeyError`. Before when calling `/messages` when there are more than 5 backward extremities: ``` Traceback (most recent call last): File "/usr/local/lib/python3.8/site-packages/synapse/http/server.py", line 258, in _async_render_wrapper callback_return = await self._async_render(request) File "/usr/local/lib/python3.8/site-packages/synapse/http/server.py", line 446, in _async_render callback_return = await raw_callback_return File "/usr/local/lib/python3.8/site-packages/synapse/rest/client/room.py", line 580, in on_GET msgs = await self.pagination_handler.get_messages( File "/usr/local/lib/python3.8/site-packages/synapse/handlers/pagination.py", line 396, in get_messages await self.hs.get_federation_handler().maybe_backfill( File "/usr/local/lib/python3.8/site-packages/synapse/handlers/federation.py", line 133, in maybe_backfill return await self._maybe_backfill_inner(room_id, current_depth, limit) File "/usr/local/lib/python3.8/site-packages/synapse/handlers/federation.py", line 386, in _maybe_backfill_inner likely_extremeties_domains = get_domains_from_state(states[e_id]) KeyError: '$zpFflMEBtZdgcMQWTakaVItTLMjLFdKcRWUPHbbSZJl' ```	3 years ago
Richard van der Hoff	85551b7a85	Factor out common code for persisting fetched auth events (#10896) ... * Factor more stuff out of `_get_events_and_persist` It turns out that the event-sorting algorithm in `_get_events_and_persist` is also useful in other circumstances. Here we move the current `_auth_and_persist_fetched_events` to `_auth_and_persist_fetched_events_inner`, and then factor the sorting part out to `_auth_and_persist_fetched_events`. * `_get_remote_auth_chain_for_event`: remove redundant `outlier` assignment `get_event_auth` returns events with the outlier flag already set, so this is redundant (though we need to update a test where `get_event_auth` is mocked). * `_get_remote_auth_chain_for_event`: move existing-event tests earlier Move a couple of tests outside the loop. This is a bit inefficient for now, but a future commit will make it better. It should be functionally identical. * `_get_remote_auth_chain_for_event`: use `_auth_and_persist_fetched_events` We can use the same codepath for persisting the events fetched as part of an auth chain as for those fetched individually by `_get_events_and_persist` for building the state at a backwards extremity. * `_get_remote_auth_chain_for_event`: use a dict for efficiency `_auth_and_persist_fetched_events` sorts the events itself, so we no longer need to care about maintaining the ordering from `get_event_auth` (and no longer need to sort by depth in `get_event_auth`). That means that we can use a map, making it easier to filter out events we already have, etc. * changelog * `_auth_and_persist_fetched_events`: improve docstring	3 years ago
Richard van der Hoff	1800aabfc2	Split `FederationHandler` in half (#10692) ... The idea here is to take anything to do with incoming events and move it out to a separate handler, as a way of making FederationHandler smaller.	3 years ago
reivilibre	642a42edde	Flatten the synapse.rest.client package (#10600)	3 years ago
Eric Eastwood	db6e7f15ea	Fix backfilled events being rejected for no `state_groups` (#10439) ... Reproducible on a federated homeserver when there is a membership auth event as a floating outlier. Then when we try to backfill one of that persons messages, it has missing membership auth to fetch which caused us to mistakenly replace the `context` for the message with that of the floating membership `outlier` event. Since `outliers` have no `state` or `state_group`, the error bubbles up when we continue down the persisting route: `sqlite3.IntegrityError: NOT NULL constraint failed: event_to_state_groups.state_group` Call stack: ``` backfill _auth_and_persist_event _check_event_auth _update_auth_events_and_context_for_auth ```	3 years ago
Richard van der Hoff	6e8fb42be7	Improve validation for `send_{join,leave,knock}` (#10225) ... The idea here is to stop people sending things that aren't joins/leaves/knocks through these endpoints: previously you could send anything you liked through them. I wasn't able to find any security holes from doing so, but it doesn't sound like a good thing.	3 years ago
Jonathan de Jong	495b214f4f	Fix (final) Bugbear violations (#9838)	4 years ago
Jonathan de Jong	4b965c862d	Remove redundant "coding: utf-8" lines (#9786) ... Part of #9744 Removes all redundant `# -*- coding: utf-8 -*-` lines from files, as python 3 automatically reads source code as utf-8 now. `Signed-off-by: Jonathan de Jong <jonathan@automatia.nl>`	4 years ago
Eric Eastwood	0a00b7ff14	Update black, and run auto formatting over the codebase (#9381) ... - Update black version to the latest - Run black auto formatting over the codebase - Run autoformatting according to [`docs/code_style.md `](80d6dc9783/docs/code_style.md) - Update `code_style.md` docs around installing black to use the correct version	4 years ago
Erik Johnston	ff55300b91	Honour ratelimit flag for application services for invite ratelimiting (#9302)	4 years ago
Erik Johnston	f2c1560eca	Ratelimit invites by room and target user (#9258)	4 years ago
Patrick Cloke	1619802228	Various clean-ups to the logging context code (#8935)	4 years ago
Richard van der Hoff	3ce2f303f1	Consistently use room_id from federation request body (#8776) ... * Consistently use room_id from federation request body Some federation APIs have a redundant `room_id` path param (see https://github.com/matrix-org/matrix-doc/issues/2330). We should make sure we consistently use either the path param or the body param, and the body param is easier. * Kill off some references to "context" Once upon a time, "rooms" were known as "contexts". I think this kills of the last references to "contexts".	4 years ago
Patrick Cloke	30fba62108	Apply an IP range blacklist to push and key revocation requests. (#8821) ... Replaces the `federation_ip_range_blacklist` configuration setting with an `ip_range_blacklist` setting with wider scope. It now applies to: * Federation * Identity servers * Push notifications * Checking key validitity for third-party invite events The old `federation_ip_range_blacklist` setting is still honored if present, but with reduced scope (it only applies to federation and identity servers).	4 years ago
Richard van der Hoff	950bb0305f	Consistently use room_id from federation request body (#8776) ... * Consistently use room_id from federation request body Some federation APIs have a redundant `room_id` path param (see https://github.com/matrix-org/matrix-doc/issues/2330). We should make sure we consistently use either the path param or the body param, and the body param is easier. * Kill off some references to "context" Once upon a time, "rooms" were known as "contexts". I think this kills of the last references to "contexts".	4 years ago
Patrick Cloke	c9c0ad5e20	Remove the deprecated Handlers object (#8494) ... All handlers now available via get_*_handler() methods on the HomeServer.	4 years ago
Patrick Cloke	08bc80ef09	Implement room version 6 (MSC2240). (#7506)	5 years ago
Patrick Cloke	56b66db78a	Strictly enforce canonicaljson requirements in a new room version (#7381)	5 years ago
Richard van der Hoff	928edef979	Pass room_version into `event_from_pdu_json` ... It's called from all over the shop, so this one's a bit messy.	5 years ago
Richard van der Hoff	807ec3bd99	Fix bug which caused rejected events to be stored with the wrong room state (#6320) ... Fixes a bug where rejected events were persisted with the wrong state group. Also fixes an occasional internal-server-error when receiving events over federation which are rejected and (possibly because they are backwards-extremities) have no prev_group. Fixes #6289.	5 years ago
Brendan Abolivier	8a5e8e829b	Lint (again)	5 years ago
Brendan Abolivier	ebcb6a30d7	Lint	5 years ago
Brendan Abolivier	6527fa18c1	Add test case	5 years ago
Erik Johnston	2c176e02ae	Make unit tests work	9 years ago
Matthew Hodgson	6c28ac260c	copyrights	9 years ago
David Baker	0e48f7f245	fix tests	9 years ago
David Baker	92a1e74b20	fix tests	9 years ago
David Baker	d79e90f078	Add mocks to make tests work again	9 years ago
Erik Johnston	1c3d844e73	Don't needlessly compute context	10 years ago
Mark Haines	63878c0379	Don't bother checking for updates if the stream token hasn't advanced for a user	10 years ago
Erik Johnston	9236136f3a	Make work in both Maria and SQLite. Fix tests	10 years ago
Mark Haines	896253e085	Factor out some of the common homeserver setup code into a ... setup_test_homeserver function in utils.	10 years ago
Erik Johnston	e7ca813dd4	Try to ensure we don't persist an event we have already persisted. In persist_event check if we already have the event, if so then update instead of replacing so that we don't cause a bump of the stream_ordering.	10 years ago
Erik Johnston	78015948a7	Initial implementation of auth conflict resolution	10 years ago
Mark Haines	c3eae8a88c	Construct the EventContext in the state handler rather than constructing one and then immediately calling state_handler.annotate_context_with_state	10 years ago
Erik Johnston	3c77d13aa5	Kill off synapse.api.events.*	10 years ago
Erik Johnston	d3eb12c7b8	Fix federation test	10 years ago
Matthew Hodgson	8ada2d2018	fix UTs by telling all the mock stores about the new methods for tracking retries	10 years ago
Erik Johnston	027542e2e5	Fix bugs when joining a remote room that has dodgy event graphs. This should also fix the number of times a HS will trigger a GET /event/	10 years ago
Erik Johnston	822b15ea43	Fix tests.	10 years ago
Erik Johnston	64fc859dac	Fix bugs in invite/join dances. ... We now do more implement more of the auth on the events so that we don't reject valid events.	10 years ago
Mark Haines	c5eabe3143	replace user_id with sender	10 years ago