72 Commits (watcha-develop)

Author	SHA1	Message	Date
Erik Johnston	23740eaa3d	Correctly mention previous copyright (#16820) ... During the migration the automated script to update the copyright headers accidentally got rid of some of the existing copyright lines. Reinstate them.	2 years ago
elara-leitstellentechnik	10ada2ff6d	Write signing keys with file mode 0640 (#16740) ... Co-authored-by: Fabian Klemp <fabian.klemp@frequentis.com>	2 years ago
Patrick Cloke	8e1e62c9e0	Update license headers	2 years ago
Patrick Cloke	ab3f1b3b53	Convert simple_select_one_txn and simple_select_one to return tuples. (#16612)	2 years ago
David Robertson	747416e94c	Portdb: don't copy a table that gets rebuilt (#16563)	2 years ago
Denis Kasak	3a0aa6fe76	Force TLS certificate verification in registration script. (#16530) ... If using the script remotely, there's no particularly convincing reason to disable certificate verification, as this makes the connection interceptible. If on the other hand, the script is used locally (the most common use case), you can simply target the HTTP listener and avoid TLS altogether. This is what the script already attempts to do if passed a homeserver configuration YAML file.	2 years ago
Patrick Cloke	aa483cb4c9	Update ruff config (#16283) ... Enable additional checks & clean-up unneeded configuration.	2 years ago
Patrick Cloke	9ec3da06da	Bump mypy-zope & mypy. (#16188)	2 years ago
Patrick Cloke	ad3f43be9a	Run pyupgrade for python 3.7 & 3.8. (#16110)	2 years ago
Mathieu Velten	dac97642e4	Implements admin API to lock an user (MSC3939) (#15870)	2 years ago
Patrick Cloke	90ad836ed8	Properly setup the additional sequences in the portdb script. (#16043) ... The un_partial_stated_event_stream_sequence and application_services_txn_id_seq were never properly configured in the portdb script, resulting in an error on start-up.	2 years ago
Erik Johnston	39d131b016	Add basic read/write lock (#15782)	3 years ago
Erik Johnston	95a96b21eb	Add foreign key constraint to `event_forward_extremities`. (#15751)	3 years ago
Erik Johnston	289ce3b8d9	Fix harmless exception in port DB script (#15814) ... The port DB script would try and run database background tasks, which could fail if the data they acted on was in the process of being ported. These exceptions were non fatal. Fixes #15789	3 years ago
Shay	89f6fb0d5a	Add an admin API endpoint to support per-user feature flags (#15344)	3 years ago
Shay	301b4156d5	Add column `full_user_id` to tables `profiles` and `user_filters`. (#15458)	3 years ago
Shay	6b23d74ad1	Delete server-side backup keys when deactivating an account. (#15181)	3 years ago
Quentin Gliech	5b70f240cf	Make cleaning up pushers depend on the device_id instead of the token_id (#15280) ... This makes it so that we rely on the `device_id` to delete pushers on logout, instead of relying on the `access_token_id`. This ensures we're not removing pushers on token refresh, and prepares for a world without access token IDs (also known as the OIDC). This actually runs the `set_device_id_for_pushers` background update, which was forgotten in #13831. Note that for backwards compatibility it still deletes pushers based on the `access_token` until the background update finishes.	3 years ago
reivilibre	98fd558382	Add a primitive helper script for listing worker endpoints. (#15243) ... Co-authored-by: Patrick Cloke <patrickc@matrix.org>	3 years ago
Shay	7f02fafa28	Add a check to SQLite port DB script to ensure that the sqlite database passed to the script exists before trying to port from it (#15306)	3 years ago
Shay	96bcc5d902	Revert "check sqlite database file exists before porting/#14692" (#15301)	3 years ago
Patrick Cloke	4fc8875876	Refactor media modules. (#15146) ... * Removes the `v1` directory from `test.rest.media.v1`. * Moves the non-REST code from `synapse.rest.media.v1` to `synapse.media`. * Flatten the `v1` directory from `synapse.rest.media`, but leave compatiblity with 3rd party media repositories and spam checkers.	3 years ago
dependabot[bot]	9bb2eac719	Bump black from 22.12.0 to 23.1.0 (#15103)	3 years ago
David Robertson	e26d7d5ae7	Teach portdb about `un_partial_stated_event_stream` (#15108) ... * Sort BOOLEAN_COLUMNS and APPEND_ONLY_TABLES So I can see if a given table is present in logarithmic time, rather than linear. * Teach portdb about `un_partial_stated_event_streams` * Comments comments comments * Changelog	3 years ago
Erik Johnston	65d0386693	Always notify replication when a stream advances (#14877) ... This ensures that all other workers are told about stream updates in a timely manner, without having to remember to manually poke replication.	3 years ago
Jeyachandran Rathnam	5c9be9c760	Check sqlite database file exists before porting. (#14692) ... To avoid creating an empty SQLite file if the given path is incorrect.	3 years ago
reivilibre	96251af50d	Fix a bug introduced in v1.67.0 where not specifying a config file or a server URL would lead to the `register_new_matrix_user` script failing. (#14637)	3 years ago
Finn	fe50738e59	let update_synapse_database run on a multi-database configurations (#13422) ... * Allow sharded database in db migrate script Signed-off-by: Finn Herzfeld <finn@beeper.com> * Update changelog.d/13422.bugfix Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com> * Remove check entirely * remove unused import Signed-off-by: Finn Herzfeld <finn@beeper.com> Co-authored-by: finn <finn@beeper.com> Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>	3 years ago
Patrick Cloke	3bbe532abb	Add an API for listing threads in a room. (#13394) ... Implement the /threads endpoint from MSC3856. This is currently unstable and behind an experimental configuration flag. It includes a background update to backfill data, results from the /threads endpoint will be partial until that finishes.	3 years ago
Brendan Abolivier	be76cd8200	Allow admins to require a manual approval process before new accounts can be used (using MSC3866) (#13556)	3 years ago
Brendan Abolivier	8ae42ab8fa	Support enabling/disabling pushers (from MSC3881) (#13799) ... Partial implementation of MSC3881	3 years ago
David Robertson	fff9b955fa	Generate separate snapshots for logical databases (#13792) ... * Generate separate snapshots for sqlite, postgres and common * Cleanup postgres dbs in the TRAP * Say which logical DB we're applying updates to * Run background updates on the state DB * Add new option for accepting a SCHEMA_NUMBER	3 years ago
Nick Mills-Barrett	cdbb641232	Add receipts event stream ordering (#13703)	3 years ago
Richard van der Hoff	d092e6f32a	Support `registration_shared_secret` in a file (#13614) ... A new `registration_shared_secret_path` option. This is kinda handy for k8s deployments and things.	3 years ago
Richard van der Hoff	a2ce614447	register_new_matrix_user: read server url from config (#13616) ... Fixes https://github.com/matrix-org/synapse/issues/3672: `https://localhost:8448` is virtually never right.	3 years ago
Andrew Morgan	6faaf76a32	Remove 'anonymised' from the phone home stats documentation (#13321)	3 years ago
Patrick Cloke	4db7862e0f	Drop unused tables from groups/communities. (#12967) ... These tables have been unused since Synapse v1.61.0, although schema version 72 was added in Synapse v1.62.0.	4 years ago
David Robertson	28d96cb2b4	Ensure portdb selects _all_ rows with negative rowids (#13226)	4 years ago
reivilibre	fb7d24ab6d	Check that `auto_vacuum` is disabled when porting a SQLite database to Postgres, as `VACUUM`s must not be performed between runs of the script. (#13195)	4 years ago
Erik Johnston	a3a05c812d	Add index to help delete old push actions (#13141)	4 years ago
Erik Johnston	92a0c18ef0	Improve performance of getting unread counts in rooms (#13119)	4 years ago
Erik Johnston	0d1d3e0708	Speed up `get_unread_event_push_actions_by_room` (#13005) ... Fixes #11887 hopefully. The core change here is that `event_push_summary` now holds a summary of counts up until a much more recent point, meaning that the range of rows we need to count in `event_push_actions` is much smaller. This needs two major changes: 1. When we get a receipt we need to recalculate `event_push_summary` rather than just delete it 2. The logic for deleting `event_push_actions` is now divorced from calculating `event_push_summary`. In future it would be good to calculate `event_push_summary` while we persist a new event (it should just be a case of adding one to the relevant rows in `event_push_summary`), as that will further simplify the get counts logic and remove the need for us to periodically update `event_push_summary` in a background job.	4 years ago
Patrick Cloke	53b77b203a	Replace noop background updates with DELETE. (#12954) ... Removes the `register_noop_background_update` and deletes the background updates directly in a delta file.	4 years ago
David Robertson	f30bcbd84a	Fix Synapse git info missing in version strings (#12973)	4 years ago
Patrick Cloke	f7baffd8ec	Remove remaining pieces of groups code. (#12966) ... * Remove an unused stream ID generator. * Remove the now unused remote profile cache.	4 years ago
Patrick Cloke	6b46c3eb3d	Remove groups code from synapse_port_db. (#12899)	4 years ago
Patrick Cloke	2e8763ec96	Remove most groups datastore code. (#12895) ... The remaining piece is a background update that is needed for backwards compatibility.	4 years ago
Jae Lo Presti	f16ec055cc	hash_password: raise an error if no config file is specified (#12789)	4 years ago
Patrick Cloke	4586119f0b	Add missing type hints to config classes. (#12402)	4 years ago
David Robertson	961ee75a9b	Disallow untyped defs in synapse._scripts (#12422) ... Of note: * No untyped defs in `register_new_matrix_user` This one might be contraversial. `request_registration` has three dependency-injection arguments used for testing. I'm removing the injection of the `requests` module and using `unitest.mock.patch` in the test cases instead. Doing `reveal_type(requests)` and `reveal_type(requests.get)` before the change: ``` synapse/_scripts/register_new_matrix_user.py:45: note: Revealed type is "Any" synapse/_scripts/register_new_matrix_user.py:46: note: Revealed type is "Any" ``` And after: ``` synapse/_scripts/register_new_matrix_user.py:44: note: Revealed type is "types.ModuleType" synapse/_scripts/register_new_matrix_user.py:45: note: Revealed type is "def (url: Union[builtins.str, builtins.bytes], params: Union[Union[_typeshed.SupportsItems[Union[builtins.str, builtins.bytes, builtins.int, builtins.float], Union[builtins.str, builtins.bytes, builtins.int, builtins.float, typing.Iterable[Union[builtins.str, builtins.bytes, builtins.int, builtins.float]], None]], Tuple[Union[builtins.str, builtins.bytes, builtins.int, builtins.float], Union[builtins.str, builtins.bytes, builtins.int, builtins.float, typing.Iterable[Union[builtins.str, builtins.bytes, builtins.int, builtins.float]], None]], typing.Iterable[Tuple[Union[builtins.str, builtins.bytes, builtins.int, builtins.float], Union[builtins.str, builtins.bytes, builtins.int, builtins.float, typing.Iterable[Union[builtins.str, builtins.bytes, builtins.int, builtins.float]], None]]], builtins.str, builtins.bytes], None] =, data: Union[Any, None] =, headers: Union[Any, None] =, cookies: Union[Any, None] =, files: Union[Any, None] =, auth: Union[Any, None] =, timeout: Union[Any, None] =, allow_redirects: builtins.bool =, proxies: Union[Any, None] =, hooks: Union[Any, None] =, stream: Union[Any, None] =, verify: Union[Any, None] =, cert: Union[Any, None] =, json: Union[Any, None] =) -> requests.models.Response" ``` * Drive-by comment in `synapse.storage.types` * No untyped defs in `synapse_port_db` This was by far the most painful. I'm happy to break this up into smaller pieces for review if it's not managable as-is.	4 years ago