watcha-synapse

Commit Graph

Author	SHA1	Message	Date
dlamarcheteamnet	eff5ada386	merge	8 months ago
Erik Johnston	23740eaa3d	Correctly mention previous copyright (#16820 ) During the migration the automated script to update the copyright headers accidentally got rid of some of the existing copyright lines. Reinstate them.	10 months ago
Erik Johnston	5d3850b038	Port `EventInternalMetadata` class to Rust (#16782 ) There are a couple of things we need to be careful of here: 1. The current python code does no validation when loading from the DB, so we need to be careful to ignore such errors (at least on jki.re there are some old events with internal metadata fields of the wrong type). 2. We want to be memory efficient, as we often have many hundreds of thousands of events in the cache at a time. --------- Co-authored-by: Quentin Gliech <quenting@element.io>	11 months ago
David Robertson	44377f5ac0	Revert postgres logical replication deltaas This reverts two commits: `0bb8e418a4` "Fix postgres schema after dropping old tables (#16730)" and `51e4e35653` "Add a Postgres `REPLICA IDENTITY` to tables that do not have an implicit one. This should allow use of Postgres logical replication. (take 2, now with no added deadlocks!) (#16658)" and also amends the changelog.	12 months ago
reivilibre	51e4e35653	Add a Postgres `REPLICA IDENTITY` to tables that do not have an implicit one. This should allow use of Postgres logical replication. (take 2, now with no added deadlocks!) (#16658 ) * Add `ALTER TABLE ... REPLICA IDENTITY ...` for individual tables We can't combine them into one file as it makes it likely to hit a deadlock if Synapse is running, as it only takes one other transaction to access two tables in a different order to the schema delta. * Add notes * Newsfile Signed-off-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org> * Re-introduce REPLICA IDENTITY test --------- Signed-off-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org>	12 months ago
Patrick Cloke	8e1e62c9e0	Update license headers	1 year ago
Erik Johnston	ef5329a9f9	Revert "Add a Postgres `REPLICA IDENTITY` to tables that do not have an implicit one. This should allow use of Postgres logical replication. (#16456 )" (#16651 ) This reverts commit `69afe3f7a0`.	1 year ago
reivilibre	830988ae72	Fix test not detecting tables with missing primary keys and missing replica identities, then add more replica identities. (#16647 ) * Fix the CI query that did not detect all cases of missing primary keys * Add more missing REPLICA IDENTITY entries * Newsfile Signed-off-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org> --------- Signed-off-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org>	1 year ago
David Robertson	43d1aa75e8	Add an Admin API to temporarily grant the ability to update an existing cross-signing key without UIA (#16634 )	1 year ago
Patrick Cloke	f2f2c7c1f0	Use full GitHub links instead of bare issue numbers. (#16637 )	1 year ago
reivilibre	69afe3f7a0	Add a Postgres `REPLICA IDENTITY` to tables that do not have an implicit one. This should allow use of Postgres logical replication. (#16456 ) * Add Postgres replica identities to tables that don't have an implicit one Fixes #16224 * Newsfile Signed-off-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org> * Move the delta to version 83 as we missed the boat for 82 * Add a test that all tables have a REPLICA IDENTITY * Extend the test to include when indices are deleted * isort * black * Fully qualify `oid` as it is a 'hidden attribute' in Postgres 11 * Update tests/storage/test_database.py Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com> * Add missed tables --------- Signed-off-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org> Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>	1 year ago
Patrick Cloke	ab3f1b3b53	Convert simple_select_one_txn and simple_select_one to return tuples. (#16612 )	1 year ago
David Robertson	91587d4cf9	Bulk-invalidate e2e cached queries after claiming keys (#16613 ) Co-authored-by: Patrick Cloke <patrickc@matrix.org>	1 year ago
Patrick Cloke	455ef04187	Avoid updating the same rows multiple times with simple_update_many_txn. (#16609 ) simple_update_many_txn had a bug in it which would cause each update to be applied twice.	1 year ago
Patrick Cloke	9738b1c497	Avoid executing no-op queries. (#16583 ) If simple_{insert,upsert,update}_many_txn is called without any data to modify then return instead of executing the query. This matches the behavior of simple_{select,delete}_many_txn.	1 year ago
Patrick Cloke	ec9ff389f4	More tests for the simple_* methods. (#16596 ) Expand tests for the simple_* database methods, additionally test against both PostgreSQL and SQLite variants.	1 year ago
Patrick Cloke	cfb6d38c47	Remove remaining usage of cursor_to_dict. (#16564 )	1 year ago
Patrick Cloke	679c691f6f	Remove more usages of cursor_to_dict. (#16551 ) Mostly to improve type safety.	1 year ago
Patrick Cloke	9407d5ba78	Convert simple_select_list and simple_select_list_txn to return lists of tuples (#16505 ) This should use fewer allocations and improves type hints.	1 year ago
Erik Johnston	8f35f8148e	Fix bug where a new writer advances their token too quickly (#16473 ) * Fix bug where a new writer advances their token too quickly When starting a new writer (for e.g. persisting events), the `MultiWriterIdGenerator` doesn't have a minimum token for it as there are no rows matching that new writer in the DB. This results in the the first stream ID it acquired being announced as persisted before it actually finishes persisting, if another writer gets and persists a subsequent stream ID. This is due to the logic of setting the minimum persisted position to the minimum known position of across all writers, and the new writer starts off not being considered. * Fix sending out POSITIONs when our token advances without update Broke in #14820 * For replication HTTP requests, only wait for minimal position	1 year ago
Patrick Cloke	6ad1f9eac2	Convert DeviceLastConnectionInfo to attrs. (#16507 ) To improve type safety & memory usage.	1 year ago
Patrick Cloke	a4904dcb04	Convert simple_select_many_batch, simple_select_many_txn to tuples. (#16444 )	1 year ago
Patrick Cloke	85bfd4735e	Return an immutable value from get_latest_event_ids_in_room. (#16326 )	1 year ago
Erik Johnston	954921736b	Refactor `get_user_by_id` (#16316 )	1 year ago
Erik Johnston	2b35626b6b	Refactor storing of server keys (#16261 )	1 year ago
Patrick Cloke	aa483cb4c9	Update ruff config (#16283 ) Enable additional checks & clean-up unneeded configuration.	1 year ago
Mathieu Velten	dcb2778341	Add last_seen_ts to the admin users API (#16218 )	1 year ago
David Robertson	6525fd65ee	Log the details of background update failures (#16212 )	1 year ago
Erik Johnston	a2e0d4cd60	Fix rare bug that broke looping calls (#16210 ) * Fix rare bug that broke looping calls We can't interact with the reactor from the main thread via looping call. Introduced in v1.90.0 / #15791. * Newsfile	1 year ago
Patrick Cloke	9ec3da06da	Bump mypy-zope & mypy. (#16188 )	1 year ago
V02460	84f441f88f	Prepare unit tests for Python 3.12 (#16099 )	1 year ago
Patrick Cloke	a8a46b1336	Replace simple_async_mock with AsyncMock (#16180 ) Python 3.8 has a native AsyncMock, use it instead of a custom implementation.	1 year ago
Patrick Cloke	daf11e26ef	Replace make_awaitable with AsyncMock (#16179 ) Python 3.8 provides a native AsyncMock, we can replace the homegrown version we have.	1 year ago
Neil Johnson	ec662bbe41	Filter out unwanted user_agents from udv. (#16124 )	1 year ago
Erik Johnston	bd558a6dc3	Speed up state res in rare case we don't have all events (#16116 ) If we don't have all the auth events in a room then not all state events will have a chain cover index. Even so, we can still use the chain cover index on the events that do have it, rather than bailing and using the slower functions. This situation should not arise for newly persisted rooms, as we check we have the full auth chain for each event, but can happen for existing rooms. c.f. #15245	1 year ago
Erik Johnston	eb0dbab15b	Fix database performance of read/write worker locks (#16061 ) We were seeing serialization errors when taking out multiple read locks. The transactions were retried, so isn't causing any failures. Introduced in #15782.	1 year ago
Patrick Cloke	ad3f43be9a	Run pyupgrade for python 3.7 & 3.8. (#16110 )	1 year ago
Mathieu Velten	dac97642e4	Implements admin API to lock an user (MSC3939) (#15870 )	1 year ago
Mathieu Velten	f0a860908b	Allow config of the backoff algorithm for the federation client. (#15754 ) Adds three new configuration variables: * destination_min_retry_interval is identical to before (10mn). * destination_retry_multiplier is now 2 instead of 5, the maximum value will be reached slower. * destination_max_retry_interval is one day instead of (essentially) infinity. Capping this will cause destinations to continue to be retried sometimes instead of being lost forever. The previous value was 2 ^ 62 milliseconds.	1 year ago
Erik Johnston	ae55cc1e6b	Add ability to wait for locks and add locks to purge history / room deletion (#15791 ) c.f. #13476	1 year ago
Shay	e625c3dca0	Revert "Stop writing to column `user_id` of tables `profiles` and `user_filters`. (#15953 ) * Revert "Stop writing to column `user_id` of tables `profiles` and `user_filters` (#15787)" This reverts commit `f25b0f8808`. * newsfragement	1 year ago
Eric Eastwood	1c802de626	Re-introduce the outbound federation proxy (#15913 ) Allow configuring the set of workers to proxy outbound federation traffic through (`outbound_federation_restricted_to`). This is useful when you have a worker setup with `federation_sender` instances responsible for sending outbound federation requests and want to make sure all outbound federation traffic goes through those instances. Before this change, the generic workers would still contact federation themselves for things like profile lookups, backfill, etc. This PR allows you to set more strict access controls/firewall for all workers and only allow the `federation_sender`'s to contact the outside world.	1 year ago
Eric Eastwood	c9bf644fa0	Revert "Federation outbound proxy" (#15910 ) Revert "Federation outbound proxy (#15773)" This reverts commit `b07b14b494`.	1 year ago
Erik Johnston	e55a9b3e41	Fix downgrading to previous version of Synapse (#15907 ) We do this by marking the constraint as deferrable.	1 year ago
Shay	f25b0f8808	Stop writing to column `user_id` of tables `profiles` and `user_filters` (#15787 )	1 year ago
Eric Eastwood	b07b14b494	Federation outbound proxy (#15773 ) Allow configuring the set of workers to proxy outbound federation traffic through (`outbound_federation_restricted_to`). This is useful when you have a worker setup with `federation_sender` instances responsible for sending outbound federation requests and want to make sure all outbound federation traffic goes through those instances. Before this change, the generic workers would still contact federation themselves for things like profile lookups, backfill, etc. This PR allows you to set more strict access controls/firewall for all workers and only allow the `federation_sender`'s to contact the outside world. The original code is from @erikjohnston's branches which I've gotten in-shape to merge.	1 year ago
Erik Johnston	39d131b016	Add basic read/write lock (#15782 )	1 year ago
Erik Johnston	95a96b21eb	Add foreign key constraint to `event_forward_extremities`. (#15751 )	1 year ago
Eric Eastwood	0f02f0b4da	Remove experimental MSC2716 implementation to incrementally import history into existing rooms (#15748 ) Context for why we're removing the implementation: - https://github.com/matrix-org/matrix-spec-proposals/pull/2716#issuecomment-1487441010 - https://github.com/matrix-org/matrix-spec-proposals/pull/2716#issuecomment-1504262734 Anyone wanting to continue MSC2716, should also address these leftover tasks: https://github.com/matrix-org/synapse/issues/10737 Closes https://github.com/matrix-org/synapse/issues/10737 in the fact that it is not longer necessary to track those things.	1 year ago
Jason Little	21fea6b749	Prefill events after invalidate not before when persisting events (#15758 ) Fixes #15757	1 year ago

1 2 3 4 5 ...

754 Commits (ec63f5a8bda1bcdbca85c6f601b8dcfd0d1e9b66)