watcha-synapse

Commit Graph

Author	SHA1	Message	Date
Eric Eastwood	1c802de626	Re-introduce the outbound federation proxy (#15913 ) Allow configuring the set of workers to proxy outbound federation traffic through (`outbound_federation_restricted_to`). This is useful when you have a worker setup with `federation_sender` instances responsible for sending outbound federation requests and want to make sure all outbound federation traffic goes through those instances. Before this change, the generic workers would still contact federation themselves for things like profile lookups, backfill, etc. This PR allows you to set more strict access controls/firewall for all workers and only allow the `federation_sender`'s to contact the outside world.	1 year ago
Eric Eastwood	0371a354cf	Better clarify how to run a worker instance (pass both configs) (#15921 ) Previously, if you just followed the instructions per the docs, you just ran into an error: ```sh $ poetry run synapse_worker --config-path homeserver_generic_worker1.yaml Missing mandatory `server_name` config option. ```	1 year ago
Jason Little	224ef0b669	Unix Sockets for HTTP Replication (#15708 ) Unix socket support for `federation` and `client` Listeners has existed now for a little while(since [1.81.0](https://github.com/matrix-org/synapse/pull/15353)), but there was one last hold out before it could be complete: HTTP Replication communication. This should finish it up. The Listeners would have always worked, but would have had no way to be talked to/at. --------- Co-authored-by: Eric Eastwood <madlittlemods@gmail.com> Co-authored-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org> Co-authored-by: Eric Eastwood <erice@element.io>	1 year ago
Eric Eastwood	c9bf644fa0	Revert "Federation outbound proxy" (#15910 ) Revert "Federation outbound proxy (#15773)" This reverts commit `b07b14b494`.	1 year ago
Jason Little	2481b7dfa4	Remove `worker_replication_*` deprecated settings, with helpful errors on startup (#15860 ) Co-authored-by: reivilibre <oliverw@matrix.org>	1 year ago
Eric Eastwood	b07b14b494	Federation outbound proxy (#15773 ) Allow configuring the set of workers to proxy outbound federation traffic through (`outbound_federation_restricted_to`). This is useful when you have a worker setup with `federation_sender` instances responsible for sending outbound federation requests and want to make sure all outbound federation traffic goes through those instances. Before this change, the generic workers would still contact federation themselves for things like profile lookups, backfill, etc. This PR allows you to set more strict access controls/firewall for all workers and only allow the `federation_sender`'s to contact the outside world. The original code is from @erikjohnston's branches which I've gotten in-shape to merge.	1 year ago
Eric Eastwood	0f02f0b4da	Remove experimental MSC2716 implementation to incrementally import history into existing rooms (#15748 ) Context for why we're removing the implementation: - https://github.com/matrix-org/matrix-spec-proposals/pull/2716#issuecomment-1487441010 - https://github.com/matrix-org/matrix-spec-proposals/pull/2716#issuecomment-1504262734 Anyone wanting to continue MSC2716, should also address these leftover tasks: https://github.com/matrix-org/synapse/issues/10737 Closes https://github.com/matrix-org/synapse/issues/10737 in the fact that it is not longer necessary to track those things.	1 year ago
Jason Little	e4f545c452	Remove `worker_replication_` settings (#15491 ) Add master to the instance_map as part of Complement, have ReplicationEndpoint look at instance_map for master. * Fix typo in drive by. * Remove unnecessary worker_replication_* bits from unit tests and add master to instance_map(hopefully in the right place) * Several updates: 1. Switch from master to main for naming the main process in the instance_map. Add useful constants for easier adjustment of names in the future. 2. Add backwards compatibility for worker_replication_* to allow time to transition to new style. Make sure to prioritize declaring main directly on the instance_map. 3. Clean up old comments/commented out code. 4. Adjust unit tests to match with new code. 5. Adjust Complement setup infrastructure to only add main to the instance_map if workers are used and remove now unused options from the worker.yaml template. * Initial Docs upload * Changelog * Missed some commented out code that can go now * Remove TODO comment that no longer holds true. * Fix links in docs * More docs * Remove debug logging * Apply suggestions from code review Co-authored-by: reivilibre <olivier@librepush.net> * Apply suggestions from code review Co-authored-by: reivilibre <olivier@librepush.net> * Update version to latest, include completeish before/after examples in upgrade notes. * Fix up and docs too --------- Co-authored-by: reivilibre <olivier@librepush.net>	2 years ago
Tatu Wikman	3e95c19911	Docs: Add Nginx loadbalancing example with sticky mxid for workers (#15411 ) * Docs: Add Nginx loadbalancing example with sticky mxid for workers Add example nginx configuration snippet that * does load balancing for workers * respects mxid part of the token * from both url parameter and auth header * and handles since parameter Thanks to @olmari for pushing me to write this and testing the configs Signed-off-by: Tatu Wikman <tatu.wikman@gmail.com> * Add changelog entry Signed-off-by: Tatu Wikman <tatu.wikman@gmail.com> * Update codeblock formatter Co-authored-by: Dirk Klimpel <5740567+dklimpel@users.noreply.github.com> * Remove indirectly related nginx-config Signed-off-by: Sami Olmari <sami@olmari.fi> * Proper definition of action how to target username for worker Signed-off-by: Sami Olmari <sami@olmari.fi> * Change "nginx" to general "reverse proxy" as it's concept now. Signed-off-by: Sami Olmari <sami@olmari.fi> * Wording in better English Co-authored-by: Tatu Wikman <tatu.wikman@gmail.com> * rename changelog entry to have correct extension --------- Signed-off-by: Tatu Wikman <tatu.wikman@gmail.com> Signed-off-by: Sami Olmari <sami@olmari.fi> Co-authored-by: Dirk Klimpel <5740567+dklimpel@users.noreply.github.com> Co-authored-by: Sami Olmari <sami@olmari.fi> Co-authored-by: Sami Olmari <sami+github@olmari.fi>	2 years ago
Dirk Klimpel	e4a25d022c	Load `/capabilities` endpoint on workers (#15436 )	2 years ago
Dirk Klimpel	4af0aec54d	Load `/directory/room/{roomAlias}` endpoint on workers (#15333 ) * Enable `directory` * move to worker store * newsfile * disable `ClientDirectoryListServer` and `ClientAppserviceDirectoryListServer` for workers	2 years ago
Dirk Klimpel	4fc85e5a92	Load `/password_policy` endpoint on workers. (#15331 )	2 years ago
Jason Little	3d70cc393f	Load `/register/available` endpoint on workers (#15268 )	2 years ago
Patrick Cloke	f4fc83ac75	Add a missing endpoint to the workers documentation. (#15223 )	2 years ago
Dirk Klimpel	ecbe0ddbe7	Add support for knocking to workers. (#15133 )	2 years ago
reivilibre	682d31c702	Allow use of the `/filter` Client-Server APIs on workers. (#15134 )	2 years ago
ZAID BIN TARIQ	ad1f3fa8e1	Document how to start Synapse with Poetry (#14892 ) * Add Start Synapse with Poetry * Create 14892.doc * Apply suggestions from code review Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com> * Update docs/workers.md --------- Co-authored-by: David Robertson <david.m.robertson1@gmail.com> Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com>	2 years ago
Dirk Klimpel	3479599387	Add missing worker settings to shared configuration (#14748 ) * Add missing worker settings to shared configuration * newsfile * update docs after review * more update for doc * This -> These Co-authored-by: David Robertson <david.m.robertson1@gmail.com>	2 years ago
reivilibre	4eb2f4e02b	Fix broken links in the Synapse documentation. (#14744 ) * Fix stale external links * Fix some internal links * Fix URLs without trailing / where needed * Fix more links * Newsfile Signed-off-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org> * Reapply docs/openid.md fix after conflict Signed-off-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org>	2 years ago
realtyem	6acb6d772a	Update worker docs to update preferred settings for pusher and federation_sender (#14493 ) * Fix one typo on line 3700(and apparently do something to other lines, no idea) * Update config_documentation.md with more information about how federation_senders and pushers settings can be handled. Specifically, that the instance map style of config does not require the special other variables that enable and disable functionality and that a single worker CAN be added to the map not only just two or more. * Extra line here for consistency and appearance. * Add link to sygnal repo. * Add deprecation notice to workers.md and point to the newer alternative method of defining this functionality. * Changelog * Correct version number of Synapse the deprecation is happening in. * Update quiet deprecation with simple notice and suggestion.	2 years ago
Eric Eastwood	8f10c8b054	Move MSC3030 `/timestamp_to_event` endpoint to stable v1 location (#14471 ) Fix https://github.com/matrix-org/synapse/issues/14390 - Client API: `/_matrix/client/unstable/org.matrix.msc3030/rooms/<roomID>/timestamp_to_event?ts=<timestamp>&dir=<direction>` -> `/_matrix/client/v1/rooms/<roomID>/timestamp_to_event?ts=<timestamp>&dir=<direction>` - Federation API: `/_matrix/federation/unstable/org.matrix.msc3030/timestamp_to_event/<roomID>?ts=<timestamp>&dir=<direction>` -> `/_matrix/federation/v1/timestamp_to_event/<roomID>?ts=<timestamp>&dir=<direction>` Complement test changes: https://github.com/matrix-org/complement/pull/559	2 years ago
David Robertson	01a0527892	Fix version that `worker_main_http_uri` is redundant from (#14476 ) * Fix version that `worker_main_http_uri` is redundant from * Changelog	2 years ago
realtyem	c15e9a0edb	Remove need for `worker_main_http_uri` setting to use /keys/upload. (#14400 )	2 years ago
aceArt-GmbH	42f9d414c2	Add example on how to load balance /sync requests (#14297 ) Signed-off-by: lukas <lukas.walter@aceart.de> Signed-off-by: lukas <lukas.walter@aceart.de>	2 years ago
Dirk Klimpel	1357ae869f	Add workers settings to configuration manual (#14086 ) * Add workers settings to configuration manual * Update `pusher_instances` * update url to python logger * update headlines * update links after headline change * remove link from `daemon process` There is no docs in Synapse for this * extend example for `federation_sender_instances` and `pusher_instances` * more infos about stream writers * add link to DAG * update `pusher_instances` * update `worker_listeners` * update `stream_writers` * Update `worker_name` Co-authored-by: David Robertson <davidr@element.io>	2 years ago
Patrick Cloke	c3e4edb4d6	Stabilize the threads API. (#14175 ) Stabilize the threads API (MSC3856) by supporting (only) the v1 path for the endpoint. This also marks the API as safe for workers since it is a read-only API.	2 years ago
Patrick Cloke	c604d2c218	Mark /relations endpoint as usable on workers. (#14028 ) Co-authored-by: Eric Eastwood <erice@element.io>	2 years ago
Dirk Klimpel	0adeccafc6	Add sample worker files for `pusher` and `federation_sender` (#14077 ) Signed-off-by: Dirk Klimpel <dirk@klimpel.org>	2 years ago
Dirk Klimpel	4490697b98	Add some cross references to worker documentation (#13974 ) Signed-off-by: Dirk Klimpel <dirk@klimpel.org>	2 years ago
Patrick Cloke	32fc3b7ba4	Remove configuration options for direct TCP replication. (#13647 ) Removes the ability to configure legacy direct TCP replication. Workers now require Redis to run.	2 years ago
Richard van der Hoff	372136d3a8	Remove documentation of legacy `frontend_proxy` worker app (#13645 ) This has been the same as a generic_worker since #6964, so let's get rid of it. Fixes #3717	2 years ago
Patrick Cloke	e761e8b475	Clarify documentation about replication traffic. (#13656 ) It can be authenticated with the worker_replication_secret setting, but is always unencrypted.	2 years ago
Patrick Cloke	7bc08f3201	Remove remaining bits of groups code. (#12936 ) * Update worker docs to remove group endpoints. * Removes an unused parameter to `ApplicationService`. * Break dependency between media repo and groups. * Avoid copying `m.room.related_groups` state events during room upgrades.	3 years ago
Patrick Cloke	1885ee0113	Remove unstable APIs for /hierarchy. (#12851 ) Removes the unstable endpoint as well as a duplicated field which was modified during stabilization.	3 years ago
Nick Mills-Barrett	1f9013ce60	Add the `batch_send` endpoint to generic workers (#12868 )	3 years ago
Nick Mills-Barrett	2e5f88b5e6	Add the `/account/whoami` endpoint to generic workers (#12866 )	3 years ago
Nick Mills-Barrett	b4fab0b14f	Fix incorrect worker-allowed path in documentation (#12867 )	3 years ago
Dirk Klimpel	298911555c	Fix typos in documentation (#12863 )	3 years ago
Sami Olmari	47619017f9	Add missing user directory search endpoint to the generic worker documentation (#12773 ) Signed-off-by: Sami Olmari <sami@olmari.fi>	3 years ago
reivilibre	699192fc1a	Add the `update_user_directory_from_worker` configuration option (superseding `update_user_directory`) to allow a generic worker to be designated as the worker to update the user directory. (#12654 ) Co-authored-by: Shay <hillerys@element.io>	3 years ago
reivilibre	c2d50e9f6c	Add the `notify_appservices_from_worker` configuration option (superseding `notify_appservices`) to allow a generic worker to be designated as the worker to send traffic to Application Services. (#12452 )	3 years ago
Dirk Klimpel	a36a38b1ca	Add some example configurations for worker (#12492 ) Signed-off-by: Dirk Klimpel <dirk@klimpel.org>	3 years ago
reivilibre	7efddbebef	Update documentation to reflect that `run_background_tasks_on` is no longer experimental. (#12451 ) * Background workers aren't experimental anymore * Newsfile Signed-off-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org> * Stream writers aren't experimental either	3 years ago
Dirk Klimpel	a743f7d33e	Replace `federation_reader` with `generic_worker` in docs (#12457 )	3 years ago
Shay	f608e6c8cf	Update broken link in `workers.md` (#12369 )	3 years ago
Jonathan de Jong	c5776780f0	Remove mutual_rooms `update_user_directory` check, and add extra documentation (#12038 ) Resolves #10339	3 years ago
Patrick Cloke	2177e356bc	Sync more worker regexes in the documentation. (#12243 )	3 years ago
~creme	7577894bec	Document that most streams can only have a single writer. (#12196 ) This includes the `typing`, `to_device`, `account_data`, `receipts`, and `presence` streams (really anything except the `events` stream).	3 years ago
Patrick Cloke	7754af24ab	Remove the unstable `/spaces` endpoint. (#12073 ) ...and various code supporting it. The /spaces endpoint was from an old version of MSC2946 and included both a Client-Server and Server-Server API. Note that the unstable /hierarchy endpoint (from the final version of MSC2946) is not yet removed.	3 years ago
Patrick Cloke	c56bfb08bc	Add documentation for missing worker types. (#11599 ) And clean-up the endpoints which should be routed to workers.	3 years ago

1 2 3

107 Commits (36ae8611fe98977153387308311f7d38b67e39c8)