watcha-synapse

Commit Graph

Author	SHA1	Message	Date
Eric Eastwood	70a4317692	Track when the pulled event signature fails (#13815 ) Because we're doing the recording in `_check_sigs_and_hash_for_pulled_events_and_fetch` (previously named `_check_sigs_and_hash_and_fetch`), this means we will track signature failures for `backfill`, `get_room_state`, `get_event_auth`, and `get_missing_events` (all pulled event scenarios). And we also record signature failures from `get_pdu`. Part of https://github.com/matrix-org/synapse/issues/13700 Part of https://github.com/matrix-org/synapse/issues/13676 and https://github.com/matrix-org/synapse/issues/13356 This PR will be especially important for https://github.com/matrix-org/synapse/pull/13816 so we can avoid the costly `_get_state_ids_after_missing_prev_event` down the line when `/messages` calls backfill.	2 years ago
Patrick Cloke	b706111b78	Do not return unspecced original_event field when using the stable /relations endpoint. (#14025 ) Keep the old behavior (of including the original_event field) for any requests to the /unstable version of the endpoint, but do not include the field when the /v1 version is used. This should avoid new clients from depending on this field, but will not help with current dependencies.	2 years ago
lukasdenk	719488dda8	Add query parameter `ts` to allow appservices set the `origin_server_ts` for state events. (#11866 ) MSC3316 declares that both /rooms/{roomId}/send and /rooms/{roomId}/state should accept a ts parameter for appservices. This change expands support to /state and adds tests.	2 years ago
Erik Johnston	535f8c8f7d	Skip filtering during push if there are no push actions (#13992 )	2 years ago
David Robertson	285d72556b	Update mypy and mypy-zope, attempt 3 (#13993 ) Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>	2 years ago
David Robertson	8e52cb0bce	Revert "Update mypy and mypy-zope (#13925 )" This reverts commit `6d543d6d9f`.	2 years ago
David Robertson	6d543d6d9f	Update mypy and mypy-zope (#13925 ) * Update mypy and mypy-zope * Unignore assigning to LogRecord attributes Presumably https://github.com/python/typeshed/pull/8064 makes this ok Cherry-picked from #13521 * Remove unused ignores due to mypy ParamSpec fixes https://github.com/python/mypy/pull/12668 Cherry-picked from #13521 * Remove additional unused ignores * Fix new mypy complaints related to `assertGreater` Presumably due to https://github.com/python/typeshed/pull/8077 * Changelog * Reword changelog Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com> Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>	2 years ago
Erik Johnston	285b9e9b6c	Speed up calculating push actions in large rooms (#13973 ) We move the expensive check of visibility to after calculating push actions, avoiding the expensive check for users who won't get pushed anyway. I think this should have a big impact on rooms with large numbers of local users that have pushed disabled.	2 years ago
David Robertson	e8f30a76ca	Fix overflows in /messages backfill calculation (#13936 ) * Reproduce bug * Compute `least_function` first * Substitute `least_function` with an f-string * Bugfix: avoid overflow Co-authored-by: Eric Eastwood <erice@element.io>	2 years ago
David Robertson	6f0c3e669d	Don't require `setuptools_rust` at runtime (#13952 )	2 years ago
Erik Johnston	ebd9e2dac6	Implement push rule evaluation in Rust. (#13838 )	2 years ago
Nick Mills-Barrett	a466164647	Optimise get_rooms_for_user (drop with_stream_ordering) (#13787 )	2 years ago
Brendan Abolivier	be76cd8200	Allow admins to require a manual approval process before new accounts can be used (using MSC3866) (#13556 )	2 years ago
Hugh Nimmo-Smith	e5fdf16d46	Expose MSC3882 only be under an unstable endpoint. (#13868 )	2 years ago
Patrick Cloke	568016929f	Clarify that a method returns only unthreaded receipts. (#13937 ) By renaming it and updating the docstring. Additionally, refactors a method which is used only by tests.	2 years ago
Eric Eastwood	df8b91ed2b	Limit and filter the number of backfill points to get from the database (#13879 ) There is no need to grab thousands of backfill points when we only need 5 to make the `/backfill` request with. We need to grab a few extra in case the first few aren't visible in the history. Previously, we grabbed thousands of backfill points from the database, then sorted and filtered them in the app. Fetching the 4.6k backfill points for `#matrix:matrix.org` from the database takes ~50ms - ~570ms so it's not like this saves a lot of time 🤷. But it might save us more time now that `get_backfill_points_in_room`/`get_insertion_event_backward_extremities_in_room` are more complicated after https://github.com/matrix-org/synapse/pull/13635 This PR moves the filtering and limiting to the SQL query so we just have less data to work with in the first place. Part of https://github.com/matrix-org/synapse/issues/13356	2 years ago
Shay	8ab16a92ed	Persist CreateRoom events to DB in a batch (#13800 )	2 years ago
Shay	a2cf66a94d	Prepatory work for batching events to send (#13487 ) This PR begins work on batching up events during the creation of a room. The PR splits out the creation and sending/persisting of the events. The first three events in the creation of the room-creating the room, joining the creator to the room, and the power levels event are sent sequentially, while the subsequent events are created and collected to be sent at the end of the function. This is currently done by appending them to a list and then iterating over the list to send, the next step (after this PR) would be to send and persist the collected events as a batch.	2 years ago
Eric Eastwood	29269d9d3f	Fix `have_seen_event` cache not being invalidated (#13863 ) Fix https://github.com/matrix-org/synapse/issues/13856 Fix https://github.com/matrix-org/synapse/issues/13865 > Discovered while trying to make Synapse fast enough for [this MSC2716 test for importing many batches](https://github.com/matrix-org/complement/pull/214#discussion_r741678240). As an example, disabling the `have_seen_event` cache saves 10 seconds for each `/messages` request in that MSC2716 Complement test because we're not making as many federation requests for `/state` (speeding up `have_seen_event` itself is related to https://github.com/matrix-org/synapse/issues/13625) > > But this will also make `/messages` faster in general so we can include it in the [faster `/messages` milestone](https://github.com/matrix-org/synapse/milestone/11). > > -- https://github.com/matrix-org/synapse/issues/13856 ### The problem `_invalidate_caches_for_event` doesn't run in monolith mode which means we never even tried to clear the `have_seen_event` and other caches. And even in worker mode, it only runs on the workers, not the master (AFAICT). Additionally there was bug with the key being wrong so `_invalidate_caches_for_event` never invalidates the `have_seen_event` cache even when it does run. Because we were using the `@cachedList` wrong, it was putting items in the cache under keys like `((room_id, event_id),)` with a `set` in a `set` (ex. `(('!TnCIJPKzdQdUlIyXdQ:test', '$Iu0eqEBN7qcyF1S9B3oNB3I91v2o5YOgRNPwi_78s-k'),)`) and we we're trying to invalidate with just `(room_id, event_id)` which did nothing.	2 years ago
Patrick Cloke	87fe9db467	Support the stable dir parameter for /relations. (#13920 ) Since MSC3715 has passed FCP, the stable parameter can be used. This currently falls back to the unstable parameter if the stable parameter is not provided (and MSC3715 support is enabled in the configuration).	2 years ago
Patrick Cloke	2fae1a3f78	Improve tests for get_unread_push_actions_for_user_in_range_. (#13893 ) Adds a docstring. * Reduces a small amount of duplicated code. * Improves tests.	2 years ago
Mathieu Velten	41461fd4d6	typing: check origin server of typing event against room's servers (#13830 ) This is also using the partial state approximation if needed so we do not block here during a fast join. Signed-off-by: Mathieu Velten <mathieuv@matrix.org> Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com>	2 years ago
Eric Eastwood	ac1a31740b	Only try to backfill event if we haven't tried before recently (#13635 ) Only try to backfill event if we haven't tried before recently (exponential backoff). No need to keep trying the same backfill point that fails over and over. Fix https://github.com/matrix-org/synapse/issues/13622 Fix https://github.com/matrix-org/synapse/issues/8451 Follow-up to https://github.com/matrix-org/synapse/pull/13589 Part of https://github.com/matrix-org/synapse/issues/13356	2 years ago
Patrick Cloke	efd108b45d	Accept & store thread IDs for receipts (implement MSC3771). (#13782 ) Updates the `/receipts` endpoint and receipt EDU handler to parse a `thread_id` from the body and insert it in the database.	2 years ago
reivilibre	c06b2b7142	Faster Remote Room Joins: tell remote homeservers that we are unable to authorise them if they query a room which has partial state on our server. (#13823 )	2 years ago
Patrick Cloke	b7272b73aa	Properly paginate forward in the /relations API. (#13840 ) This fixes a bug where the `/relations` API with `dir=f` would skip the first item of each page (except the first page), causing incomplete data to be returned to the client.	2 years ago
Brendan Abolivier	ccca14140a	Track device IDs for pushers (#13831 ) Second half of the MSC3881 implementation	2 years ago
Hugh Nimmo-Smith	0fd2f2d460	Implementation of MSC3882 login token request (#13722 )	2 years ago
Brendan Abolivier	8ae42ab8fa	Support enabling/disabling pushers (from MSC3881) (#13799 ) Partial implementation of MSC3881	2 years ago
Mathieu Velten	6bd8763804	Add cache invalidation across workers to module API (#13667 ) Signed-off-by: Mathieu Velten <mathieuv@matrix.org>	2 years ago
Quentin Gliech	e0804ef898	Improve the `synapse.api.auth.Auth` mock used in unit tests. (#13809 ) To return the proper type (`Requester`) instead of a `dict`.	2 years ago
Erik Johnston	42d261c32f	Port the push rule classes to Rust. (#13768 )	2 years ago
Quentin Gliech	74f60cec92	Add an admin API endpoint to find a user based on its external ID in an auth provider. (#13810 )	2 years ago
Sean Quah	b73cbb8215	Avoid putting rejected events in room state (#13723 ) Signed-off-by: Sean Quah <seanq@matrix.org>	2 years ago
David Robertson	742f9f9d78	A third batch of Pydantic validation for rest/client/account.py (#13736 )	2 years ago
Andrew Morgan	918c74bfb5	Add a `MXCUri` class to make working with mxc uri's easier. (#13162 )	2 years ago
Eric Eastwood	957e3d74fc	Keep track when we try and fail to process a pulled event (#13589 ) We can follow-up this PR with: 1. Only try to backfill from an event if we haven't tried recently -> https://github.com/matrix-org/synapse/issues/13622 1. When we decide to backfill that event again, process it in the background so it doesn't block and make `/messages` slow when we know it will probably fail again -> https://github.com/matrix-org/synapse/issues/13623 1. Generally track failures everywhere we try and fail to pull an event over federation -> https://github.com/matrix-org/synapse/issues/13700 Fix https://github.com/matrix-org/synapse/issues/13621 Part of https://github.com/matrix-org/synapse/issues/13356 Mentioned in [internal doc](https://docs.google.com/document/d/1lvUoVfYUiy6UaHB6Rb4HicjaJAU40-APue9Q4vzuW3c/edit#bookmark=id.qv7cj51sv9i5)	2 years ago
Patrick Cloke	666ae87729	Update event push action and receipt tables to support threads. (#13753 ) Adds a `thread_id` column to the `event_push_actions`, `event_push_actions_staging`, and `event_push_summary` tables. This will notifications to be segmented by the thread in a future pull request. The `thread_id` column stores the root event ID or the special value `"main"`. The `thread_id` column for `event_push_actions` and `event_push_summary` is backfilled with `"main"` for all existing rows. New entries into `event_push_actions` and `event_push_actions_staging` will get the proper thread ID. `receipts_linearized` and `receipts_graph` also gain a `thread_id` column, which is similar, except `NULL` is a special value meaning the receipt is "unthreaded". See MSC3771 and MSC3773 for where this data will be useful.	2 years ago
Sean Quah	c73774467e	Fix bug in device list caching when remote users leave rooms (#13749 ) When a remote user leaves the last room shared with the homeserver, we have to mark their device list as unsubscribed, otherwise we would hold on to a stale device list in our cache. Crucially, the device list would remain cached even after the remote user rejoined the room, which could lead to E2EE failures until the next change to the remote user's device list. Fixes #13651. Signed-off-by: Sean Quah <seanq@matrix.org>	2 years ago
David Robertson	f2d2481e56	Require SQLite >= 3.27.0 (#13760 )	2 years ago
Dirk Klimpel	f799eac7ea	Add timestamp to user's consent (#13741 ) Co-authored-by: reivilibre <olivier@librepush.net>	2 years ago
Sean Quah	8ef0c8ff14	Fix error in `is_mine_id` when encountering a malformed ID (#13746 ) Previously, `is_mine_id` would raise an exception when passed an ID with no colons. Return `False` instead. Fixes #13040. Signed-off-by: Sean Quah <seanq@matrix.org>	2 years ago
reivilibre	d3d9ca156e	Cancel the processing of key query requests when they time out. (#13680 )	2 years ago
reivilibre	c2fe48a6ff	Rename the `EventFormatVersions` enum values so that they line up with room version numbers. (#13706 )	2 years ago
Connor Davis	bb5b47b62a	Add Admin API to Fetch Messages Within a Particular Window (#13672 ) This adds two new admin APIs that allow us to fetch messages from a room within a particular time.	2 years ago
Erik Johnston	c9b7e97355	Add a stub Rust crate (#12595 )	2 years ago
Patrick Cloke	32fc3b7ba4	Remove configuration options for direct TCP replication. (#13647 ) Removes the ability to configure legacy direct TCP replication. Workers now require Redis to run.	2 years ago
Brendan Abolivier	898fef2789	Share some metrics between the Prometheus exporter and the phone home stats (#13671 )	2 years ago
Patrick Cloke	390b7ce946	Disable calculating unread counts unless the config flag is enabled. (#13694 ) This avoids doing work that will never be used (since the resulting unread counts will never be sent in a /sync response). The negative of doing this is that unread counts will be incorrect when the feature is initially enabled.	2 years ago
Šimon Brandner	0e99f07952	Remove support for unstable private read receipts (#13653 ) Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>	2 years ago

1 2 3 4 5 ...

3098 Commits (720b12c2098912bcf7568dd071ea213937b6de1e)