411 Commits (36ae8611fe98977153387308311f7d38b67e39c8)

Author	SHA1	Message	Date
dependabot[bot]	001fc7bd19	Bump ruff from 0.0.277 to 0.0.286 (#16198) ... Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	1 year ago
Shay	81a6f8c9ae	Drop backwards compat hack for event serialization (#16069)	1 year ago
Patrick Cloke	d98a43d922	Stabilize support for MSC3970: updated transaction semantics (scope to `device_id`) (#15629) ... For now this maintains compatible with old Synapses by falling back to using transaction semantics on a per-access token. A future version of Synapse will drop support for this.	1 year ago
Patrick Cloke	4f5bccbbba	Add forward-compatibility for the redacts property (MSC2174). (#16013) ... The location of the redacts field changes in room version 11. Ensure it is copied to the *new* location for *old* room versions for forwards-compatibility with clients. Note that copying it to the *old* location for the *new* room version was previously handled.	1 year ago
Shay	5c7364fea5	Properly handle redactions of creation events (#15973)	1 year ago
Patrick Cloke	6d81aec09f	Support room version 11 (#15912) ... And fix a bug in the implementation of the updated redaction format (MSC2174) where the top-level redacts field was not properly added for backwards-compatibility.	1 year ago
Eric Eastwood	0f02f0b4da	Remove experimental MSC2716 implementation to incrementally import history into existing rooms (#15748) ... Context for why we're removing the implementation: - https://github.com/matrix-org/matrix-spec-proposals/pull/2716#issuecomment-1487441010 - https://github.com/matrix-org/matrix-spec-proposals/pull/2716#issuecomment-1504262734 Anyone wanting to continue MSC2716, should also address these leftover tasks: https://github.com/matrix-org/synapse/issues/10737 Closes https://github.com/matrix-org/synapse/issues/10737 in the fact that it is not longer necessary to track those things.	1 year ago
Shay	553f2f53e7	Replace `EventContext` fields `prev_group` and `delta_ids` with field `state_group_deltas` (#15233)	1 year ago
Patrick Cloke	f880e64b11	Stabilize support for MSC3952: Intentional mentions. (#15520)	1 year ago
Patrick Cloke	c5d1e6d414	Properly parse event_fields in filters (#15607) ... The event_fields property in filters should use the proper escape rules, namely backslashes can be escaped with an additional backslash. This adds tests (adapted from matrix-js-sdk) and implements the logic to properly split the event_fields strings.	2 years ago
Eric Eastwood	703a8f9c67	Instrument `state` and `state_group` storage related things (tracing) (#15610) ... Instrument `state` and `state_group` storage related things (tracing) so it's a little more clear where these database transactions are coming from as there is a lot of wires crossing in these functions. Part of `/messages` performance investigation: https://github.com/matrix-org/synapse/issues/13356	2 years ago
Patrick Cloke	f2905d827f	Implement MSC3821 to update redaction rules (`third_party_invite.signed`) (#15563) ... Updates the redaction rules to protect enough information that the event can still be properly verified.	2 years ago
Patrick Cloke	ba6b21c81e	Implement MSC3389 to protect relations from redaction. (#15565) ... MSC3389 proposes protecting the relation type & parent event ID from redaction. This keeps the relation information intact after redaction which helps with some UX flaws (e.g. deleting an event causes it to no longer be in a thread, which is confusing).	2 years ago
Andrew Morgan	2e59e97ebd	Move ThirdPartyEventRules into module_api/callbacks (#15535)	2 years ago
Quentin Gliech	8b3a502996	Experimental support for MSC3970: per-device transaction IDs (#15318)	2 years ago
Andrew Morgan	aec639e3e3	Move Spam Checker callbacks to a dedicated file (#15453)	2 years ago
Patrick Cloke	2503126d52	Implement MSC2174: move redacts to a content property. (#15395) ... This moves `redacts` from being a top-level property to a `content` property in a new room version. MSC2176 (which was previously implemented) states to not `redact` this property.	2 years ago
Patrick Cloke	83649b891d	Implement MSC3989 to redact the origin field. (#15393) ... This will be done in a future room version, for now an unstable room version is added which redacts the origin field.	2 years ago
dependabot[bot]	bd4d958aaf	Bump ruff from 0.0.252 to 0.0.259 (#15328) ... * Bump ruff from 0.0.252 to 0.0.259 Bumps [ruff](https://github.com/charliermarsh/ruff) from 0.0.252 to 0.0.259. - [Release notes](https://github.com/charliermarsh/ruff/releases) - [Changelog](https://github.com/charliermarsh/ruff/blob/main/BREAKING_CHANGES.md) - [Commits](https://github.com/charliermarsh/ruff/compare/v0.0.252...v0.0.259) --- updated-dependencies: - dependency-name: ruff dependency-type: direct:development update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * Fix new warnings * Mypy * Newsfile --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Erik Johnston <erik@matrix.org>	2 years ago
Patrick Cloke	68a6717312	Reject mentions on the C-S API which are invalid. (#15311) ... Invalid mentions data received over the Client-Server API should be rejected with a 400 error. This will hopefully stop clients from sending invalid data, although does not help with data received over federation.	2 years ago
David Robertson	3b0083c92a	Use immutabledict instead of frozendict (#15113) ... Additionally: * Consistently use `freeze()` in test --------- Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com> Co-authored-by: 6543 <6543@obermui.de>	2 years ago
Andrew Morgan	e157c63f68	Fix missing conditional for registering `on_remove_user_third_party_identifier` module api callbacks (#15227	2 years ago
Shay	a368d30c1c	More speedups/fixes to creating batched events (#15195)	2 years ago
Quentin Gliech	41f127e068	Pass the requester during event serialization. (#15174) ... This allows Synapse to properly include the transaction ID in the unsigned data of events.	2 years ago
Patrick Cloke	05e0a4089a	Stop applying edits to event contents (MSC3925). (#15193) ... Enables MSC3925 support by default, which: * Includes the full edit event in the bundled aggregations of an edited event. * Stops modifying the original event's content to return the new content from the edit event. This is a backwards-incompatible change that is considered to be "correct" by the spec.	2 years ago
Richard van der Hoff	2b78981736	Remove support for aggregating reactions (#15172) ... It turns out that no clients rely on server-side aggregation of `m.annotation` relationships: it's just not very useful as currently implemented. It's also non-trivial to calculate. I want to remove it from MSC2677, so to keep the implementation in line, let's remove it here.	2 years ago
Andrew Morgan	b40657314e	Add module API callbacks for adding and deleting local 3PID associations (#15044	2 years ago
Patrick Cloke	4fc8875876	Refactor media modules. (#15146) ... * Removes the `v1` directory from `test.rest.media.v1`. * Moves the non-REST code from `synapse.rest.media.v1` to `synapse.media`. * Flatten the `v1` directory from `synapse.rest.media`, but leave compatiblity with 3rd party media repositories and spam checkers.	2 years ago
Shay	1c95ddd09b	Batch up storing state groups when creating new room (#14918)	2 years ago
dependabot[bot]	9bb2eac719	Bump black from 22.12.0 to 23.1.0 (#15103)	2 years ago
Harishankar Kumar	db2b105d69	Change collection[str] to StrCollection in event_auth code (#14929) ... Signed-off-by: Harishankar Kumar <hari01584@gmail.com>	2 years ago
Sean Quah	d0c713cc85	Return read-only collections from `@cached` methods (#13755) ... It's important that collections returned from `@cached` methods are not modified, otherwise future retrievals from the cache will return the modified collection. This applies to the return values from `@cached` methods and the values inside the dictionaries returned by `@cachedList` methods. It's not necessary for the dictionaries returned by `@cachedList` methods themselves to be read-only. Signed-off-by: Sean Quah <seanq@matrix.org> Co-authored-by: David Robertson <davidr@element.io>	2 years ago
Shay	03bccd542b	Add a class UnpersistedEventContext to allow for the batching up of storing state groups (#14675) ... * add class UnpersistedEventContext * modify create new client event to create unpersistedeventcontexts * persist event contexts after creation * fix tests to persist unpersisted event contexts * cleanup * misc lints + cleanup * changelog + fix comments * lints * fix batch insertion? * reduce redundant calculation * add unpersisted event classes * rework compute_event_context, split into function that returns unpersisted event context and then persists it * use calculate_context_info to create unpersisted event contexts * update typing * $%#^&* * black * fix comments and consolidate classes, use attr.s for class * requested changes * lint * requested changes * requested changes * refactor to be stupidly explicit * clearer renaming and flow * make partial state non-optional * update docstrings --------- Co-authored-by: Erik Johnston <erik@matrix.org>	2 years ago
David Robertson	a134e626e4	Reject boolean power levels (#14944) ... * Better test for bad values in power levels events The previous test only checked that Synapse didn't raise an exception, but didn't check that we had correctly interpreted the value of the dodgy power level. It also conflated two things: bad room notification levels, and bad user levels. There _is_ logic for converting the latter to integers, but we should test it separately. * Check we ignore types that don't convert to int * Handle `None` values in `notifications.room` * Changelog * Also test that bad values are rejected by event auth * Docstring * linter scripttttttttt * Test boolean values in PL content * Reject boolean power levels * Changelog	2 years ago
David Robertson	796a4b7482	Prefer `type(x) is int` to `isinstance(x, int)` (#14945) ... * Perfer `type(x) is int` to `isinstance(x, int)` This covered all additional instances I could see where `x` was user-controlled. The remaining cases are ``` $ rg -s 'isinstance.*[^_]int' tests/replication/_base.py 576: if isinstance(obj, int): synapse/util/caches/stream_change_cache.py 136: assert isinstance(stream_pos, int) 214: assert isinstance(stream_pos, int) 246: assert isinstance(stream_pos, int) 267: assert isinstance(stream_pos, int) synapse/replication/tcp/external_cache.py 133: if isinstance(result, int): synapse/metrics/__init__.py 100: if isinstance(calls, (int, float)): synapse/handlers/appservice.py 262: assert isinstance(new_token, int) synapse/config/_util.py 62: if isinstance(p, int): ``` which cover metrics, logic related to `jsonschema`, and replication and data streams. AFAICS these are all internal to Synapse * Changelog	2 years ago
Patrick Cloke	3c3ba31507	Add missing type hints for tests.events. (#14904)	2 years ago
Richard van der Hoff	06ab64f201	Implement MSC3925: changes to bundling of edits (#14811) ... Two parts to this: * Bundle the whole of the replacement with any edited events. This is backwards-compatible so I haven't put it behind a flag. * Optionally, inhibit server-side replacement of edited events. This has scope to break things, so it is currently disabled by default.	2 years ago
David Robertson	e2a1adbf5d	Allow selecting "prejoin" events by state keys (#14642) ... * Declare new config * Parse new config * Read new config * Don't use trial/our TestCase where it's not needed Before: ``` $ time trial tests/events/test_utils.py > /dev/null real 0m2.277s user 0m2.186s sys 0m0.083s ``` After: ``` $ time trial tests/events/test_utils.py > /dev/null real 0m0.566s user 0m0.508s sys 0m0.056s ``` * Helper to upsert to event fields without exceeding size limits. * Use helper when adding invite/knock state Now that we allow admins to include events in prejoin room state with arbitrary state keys, be a good Matrix citizen and ensure they don't accidentally create an oversized event. * Changelog * Move StateFilter tests should have done this in #14668 * Add extra methods to StateFilter * Use StateFilter * Ensure test file enforces typed defs; alphabetise * Workaround surprising get_current_state_ids * Whoops, fix mypy	2 years ago
David Robertson	b5b5f66084	Move `StateFilter` to `synapse.types` (#14668) ... * Move `StateFilter` to `synapse.types` * Changelog	2 years ago
Mathieu Velten	75888c2b1f	Faster joins: do not wait for full state when creating events to send (#14403) ... Signed-off-by: Mathieu Velten <mathieuv@matrix.org>	2 years ago
Patrick Cloke	d8cc86eff4	Remove redundant types from comments. (#14412) ... Remove type hints from comments which have been added as Python type hints. This helps avoid drift between comments and reality, as well as removing redundant information. Also adds some missing type hints which were simple to fill in.	2 years ago
Shay	1c777ef1e8	Fix docstring in EventContext (#14145)	2 years ago
Kateřina Churanová	6caa303083	fix: Push notifications for invite over federation (#13719)	2 years ago
Denis	c802ef1411	Don't include redundant prev_state in new events (#13791)	2 years ago
reivilibre	c2fe48a6ff	Rename the `EventFormatVersions` enum values so that they line up with room version numbers. (#13706)	2 years ago
Eric Eastwood	7af07f9716	Instrument `_check_sigs_and_hash_and_fetch` to trace time spent in child concurrent calls (#13588) ... Instrument `_check_sigs_and_hash_and_fetch` to trace time spent in child concurrent calls because I've see `_check_sigs_and_hash_and_fetch` take [10.41s to process 100 events](https://github.com/matrix-org/synapse/issues/13587) Fix https://github.com/matrix-org/synapse/issues/13587 Part of https://github.com/matrix-org/synapse/issues/13356	2 years ago
Eric Eastwood	06df5d4250	MSC2716v4 room version - remove namespace from MSC2716 event content fields (#13551) ... Complement PR: https://github.com/matrix-org/complement/pull/450 As suggested in https://github.com/matrix-org/matrix-spec-proposals/pull/2716#discussion_r941444525	2 years ago
Richard van der Hoff	96d92156d0	Update type of `EventContext.rejected` (#13460)	2 years ago
Erik Johnston	c6a05063ff	Don't pull out the full state when creating an event (#13281)	2 years ago
David Robertson	7281591f4c	Use state before join to determine if we `_should_perform_remote_join` (#13270) ... Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>	2 years ago