50 Commits (4953cd71dfbb1925dc4a477efd2ed48c2dfd70d6)

Author	SHA1	Message	Date
Patrick Cloke	4fc8875876	Refactor media modules. (#15146) ... * Removes the `v1` directory from `test.rest.media.v1`. * Moves the non-REST code from `synapse.rest.media.v1` to `synapse.media`. * Flatten the `v1` directory from `synapse.rest.media`, but leave compatiblity with 3rd party media repositories and spam checkers.	3 years ago
dependabot[bot]	9bb2eac719	Bump black from 22.12.0 to 23.1.0 (#15103)	3 years ago
David Robertson	e26d7d5ae7	Teach portdb about `un_partial_stated_event_stream` (#15108) ... * Sort BOOLEAN_COLUMNS and APPEND_ONLY_TABLES So I can see if a given table is present in logarithmic time, rather than linear. * Teach portdb about `un_partial_stated_event_streams` * Comments comments comments * Changelog	3 years ago
Erik Johnston	65d0386693	Always notify replication when a stream advances (#14877) ... This ensures that all other workers are told about stream updates in a timely manner, without having to remember to manually poke replication.	3 years ago
Jeyachandran Rathnam	5c9be9c760	Check sqlite database file exists before porting. (#14692) ... To avoid creating an empty SQLite file if the given path is incorrect.	3 years ago
reivilibre	96251af50d	Fix a bug introduced in v1.67.0 where not specifying a config file or a server URL would lead to the `register_new_matrix_user` script failing. (#14637)	3 years ago
Finn	fe50738e59	let update_synapse_database run on a multi-database configurations (#13422) ... * Allow sharded database in db migrate script Signed-off-by: Finn Herzfeld <finn@beeper.com> * Update changelog.d/13422.bugfix Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com> * Remove check entirely * remove unused import Signed-off-by: Finn Herzfeld <finn@beeper.com> Co-authored-by: finn <finn@beeper.com> Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>	3 years ago
Patrick Cloke	3bbe532abb	Add an API for listing threads in a room. (#13394) ... Implement the /threads endpoint from MSC3856. This is currently unstable and behind an experimental configuration flag. It includes a background update to backfill data, results from the /threads endpoint will be partial until that finishes.	3 years ago
Brendan Abolivier	be76cd8200	Allow admins to require a manual approval process before new accounts can be used (using MSC3866) (#13556)	3 years ago
Brendan Abolivier	8ae42ab8fa	Support enabling/disabling pushers (from MSC3881) (#13799) ... Partial implementation of MSC3881	3 years ago
David Robertson	fff9b955fa	Generate separate snapshots for logical databases (#13792) ... * Generate separate snapshots for sqlite, postgres and common * Cleanup postgres dbs in the TRAP * Say which logical DB we're applying updates to * Run background updates on the state DB * Add new option for accepting a SCHEMA_NUMBER	3 years ago
Nick Mills-Barrett	cdbb641232	Add receipts event stream ordering (#13703)	3 years ago
Richard van der Hoff	d092e6f32a	Support `registration_shared_secret` in a file (#13614) ... A new `registration_shared_secret_path` option. This is kinda handy for k8s deployments and things.	3 years ago
Richard van der Hoff	a2ce614447	register_new_matrix_user: read server url from config (#13616) ... Fixes https://github.com/matrix-org/synapse/issues/3672: `https://localhost:8448` is virtually never right.	3 years ago
Andrew Morgan	6faaf76a32	Remove 'anonymised' from the phone home stats documentation (#13321)	4 years ago
Patrick Cloke	4db7862e0f	Drop unused tables from groups/communities. (#12967) ... These tables have been unused since Synapse v1.61.0, although schema version 72 was added in Synapse v1.62.0.	4 years ago
David Robertson	28d96cb2b4	Ensure portdb selects _all_ rows with negative rowids (#13226)	4 years ago
reivilibre	fb7d24ab6d	Check that `auto_vacuum` is disabled when porting a SQLite database to Postgres, as `VACUUM`s must not be performed between runs of the script. (#13195)	4 years ago
Erik Johnston	a3a05c812d	Add index to help delete old push actions (#13141)	4 years ago
Erik Johnston	92a0c18ef0	Improve performance of getting unread counts in rooms (#13119)	4 years ago
Erik Johnston	0d1d3e0708	Speed up `get_unread_event_push_actions_by_room` (#13005) ... Fixes #11887 hopefully. The core change here is that `event_push_summary` now holds a summary of counts up until a much more recent point, meaning that the range of rows we need to count in `event_push_actions` is much smaller. This needs two major changes: 1. When we get a receipt we need to recalculate `event_push_summary` rather than just delete it 2. The logic for deleting `event_push_actions` is now divorced from calculating `event_push_summary`. In future it would be good to calculate `event_push_summary` while we persist a new event (it should just be a case of adding one to the relevant rows in `event_push_summary`), as that will further simplify the get counts logic and remove the need for us to periodically update `event_push_summary` in a background job.	4 years ago
Patrick Cloke	53b77b203a	Replace noop background updates with DELETE. (#12954) ... Removes the `register_noop_background_update` and deletes the background updates directly in a delta file.	4 years ago
David Robertson	f30bcbd84a	Fix Synapse git info missing in version strings (#12973)	4 years ago
Patrick Cloke	f7baffd8ec	Remove remaining pieces of groups code. (#12966) ... * Remove an unused stream ID generator. * Remove the now unused remote profile cache.	4 years ago
Patrick Cloke	6b46c3eb3d	Remove groups code from synapse_port_db. (#12899)	4 years ago
Patrick Cloke	2e8763ec96	Remove most groups datastore code. (#12895) ... The remaining piece is a background update that is needed for backwards compatibility.	4 years ago
Jae Lo Presti	f16ec055cc	hash_password: raise an error if no config file is specified (#12789)	4 years ago
Patrick Cloke	4586119f0b	Add missing type hints to config classes. (#12402)	4 years ago
David Robertson	961ee75a9b	Disallow untyped defs in synapse._scripts (#12422) ... Of note: * No untyped defs in `register_new_matrix_user` This one might be contraversial. `request_registration` has three dependency-injection arguments used for testing. I'm removing the injection of the `requests` module and using `unitest.mock.patch` in the test cases instead. Doing `reveal_type(requests)` and `reveal_type(requests.get)` before the change: ``` synapse/_scripts/register_new_matrix_user.py:45: note: Revealed type is "Any" synapse/_scripts/register_new_matrix_user.py:46: note: Revealed type is "Any" ``` And after: ``` synapse/_scripts/register_new_matrix_user.py:44: note: Revealed type is "types.ModuleType" synapse/_scripts/register_new_matrix_user.py:45: note: Revealed type is "def (url: Union[builtins.str, builtins.bytes], params: Union[Union[_typeshed.SupportsItems[Union[builtins.str, builtins.bytes, builtins.int, builtins.float], Union[builtins.str, builtins.bytes, builtins.int, builtins.float, typing.Iterable[Union[builtins.str, builtins.bytes, builtins.int, builtins.float]], None]], Tuple[Union[builtins.str, builtins.bytes, builtins.int, builtins.float], Union[builtins.str, builtins.bytes, builtins.int, builtins.float, typing.Iterable[Union[builtins.str, builtins.bytes, builtins.int, builtins.float]], None]], typing.Iterable[Tuple[Union[builtins.str, builtins.bytes, builtins.int, builtins.float], Union[builtins.str, builtins.bytes, builtins.int, builtins.float, typing.Iterable[Union[builtins.str, builtins.bytes, builtins.int, builtins.float]], None]]], builtins.str, builtins.bytes], None] =, data: Union[Any, None] =, headers: Union[Any, None] =, cookies: Union[Any, None] =, files: Union[Any, None] =, auth: Union[Any, None] =, timeout: Union[Any, None] =, allow_redirects: builtins.bool =, proxies: Union[Any, None] =, hooks: Union[Any, None] =, stream: Union[Any, None] =, verify: Union[Any, None] =, cert: Union[Any, None] =, json: Union[Any, None] =) -> requests.models.Response" ``` * Drive-by comment in `synapse.storage.types` * No untyped defs in `synapse_port_db` This was by far the most painful. I'm happy to break this up into smaller pieces for review if it's not managable as-is.	4 years ago
David Robertson	0cd182f296	Make `synapse._scripts` pass typechecks (#12421)	4 years ago
Erik Johnston	5c9e39e619	Track device list updates per room. (#12321) ... This is a first step in dealing with #7721. The idea is basically that rather than calculating the full set of users a device list update needs to be sent to up front, we instead simply record the rooms the user was in at the time of the change. This will allow a few things: 1. we can defer calculating the set of remote servers that need to be poked about the change; and 2. during `/sync` and `/keys/changes` we can avoid also avoid calculating users who share rooms with other users, and instead just look at the rooms that have changed. However, care needs to be taken to correctly handle server downgrades. As such this PR writes to both `device_lists_changes_in_room` and the `device_lists_outbound_pokes` table synchronously. In a future release we can then bump the database schema compat version to `69` and then we can assume that the new `device_lists_changes_in_room` exists and is handled. There is a temporary option to disable writing to `device_lists_outbound_pokes` synchronously, allowing us to test the new code path does work (and by implication upgrading to a future release and downgrading to this one will work correctly). Note: Ideally we'd do the calculation of room to servers on a worker (e.g. the background worker), but currently only master can write to the `device_list_outbound_pokes` table.	4 years ago
David Robertson	4aeb00ca20	Move synctl into `synapse._scripts` and expose as an entrypoint (#12140)	4 years ago
David Robertson	f3f0ab10fe	Move scripts directory inside synapse, exposing as setuptools entry_points (#12118) ... * Two scripts are basically entry_points already * Move and rename scripts/* to synapse/_scripts/*.py * Delete sync_room_to_group.pl * Expose entry points in setup.py * Update linter script and config * Fixup scripts & docs mentioning scripts that moved Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>	4 years ago
lukasdenk	d8be9924ef	Add a flag to the `synapse_review_recent_signups` script to ignore and filter appservice users. (#11675)	4 years ago
Patrick Cloke	66c4b774fd	Add type hints to synapse._scripts (#11297)	4 years ago
Samuel Philipp	4e393af52f	Fixed config parse bug in review_recent_signups (#11191)	4 years ago
V02460	b7f7ca24b1	Remove shebang line from module files (#10415) ... Signed-off-by: Kai A. Hiller <V02460@gmail.com>	5 years ago
Erik Johnston	6655ea5587	Add script for getting info about recently registered users (#10290)	5 years ago
Jonathan de Jong	4b965c862d	Remove redundant "coding: utf-8" lines (#9786) ... Part of #9744 Removes all redundant `# -*- coding: utf-8 -*-` lines from files, as python 3 automatically reads source code as utf-8 now. `Signed-off-by: Jonathan de Jong <jonathan@automatia.nl>`	5 years ago
Dmitry Borodaenko	7c43447477	Strip trailing / from server_url in register_new_matrix_user (#8823) ... When server URL provided to register_new_matrix_user includes path component (e.g. "http://localhost:8008/"), the command fails with "ERROR! Received 400 Bad Request". Stripping trailing slash from the server_url command argument makes sure combined endpoint URL remains valid. Signed-off-by: Dmitry Borodaenko angdraug@debian.org	5 years ago
Dirk Klimpel	3f0ff53158	Remove deprecated `/_matrix/client/*/admin` endpoints (#8785) ... These are now only available via `/_synapse/admin/v1`.	5 years ago
Jonathan de Jong	837293c314	Remove obsolete __future__ imports (#8337)	5 years ago
Dagfinn Ilmari Mannsåker	a3f11567d9	Replace all remaining six usage with native Python 3 equivalents (#7704)	6 years ago
Andrew Morgan	bc29a19731	Replace instance variations of homeserver with correct case/spacing	6 years ago
Amber Brown	32e7c9e7f2	Run Black. (#5482)	7 years ago
Amber Brown	3f189c902e	Fix flake8 (#4519)	7 years ago
Neil Johnson	d2f7c4e6b1	create support user (#4141) ... Allow for the creation of a support user. A support user can access the server, join rooms, interact with other users, but does not appear in the user directory nor does it contribute to monthly active user limits.	7 years ago
Amber Brown	77d70a7646	Port register_new_matrix_user to Python 3 and add tests (#4085)	7 years ago