20 Commits (285b9e9b6c3558718e7d4f513062e277948ac35d)

Author	SHA1	Message	Date
reivilibre	65e675504f	Add the ability to set the log level using the `SYNAPSE_TEST_LOG_LEVEL` environment when using `complement.sh`. (#13152)	2 years ago
reivilibre	9667bad55d	Improve startup times in Complement test runs against workers, particularly in CPU-constrained environments. (#13127) ... Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>	2 years ago
reivilibre	3c5549e74a	Refactor the Dockerfile-workers configuration script to use Jinja2 templates in Synapse workers' Supervisord blocks. (#13054) ... Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>	2 years ago
reivilibre	212be2edc1	Use updated `update_user_directory_from_worker` options in Complement tests. (#13069)	2 years ago
reivilibre	67f51c84f8	Merge the Complement testing Docker images into a single, multi-purpose image. (#12881) ... Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>	3 years ago
Nick Mills-Barrett	1f9013ce60	Add the `batch_send` endpoint to generic workers (#12868)	3 years ago
reivilibre	67aae05ece	Support registering Application Services when running with workers under Complement. (#12826) ... Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>	3 years ago
reivilibre	4fef76ca34	Remove Caddy from the Synapse workers image used in Complement. (#12818)	3 years ago
reivilibre	c2d50e9f6c	Add the `notify_appservices_from_worker` configuration option (superseding `notify_appservices`) to allow a generic worker to be designated as the worker to send traffic to Application Services. (#12452)	3 years ago
reivilibre	d743b25c8f	Use supervisord to supervise Postgres and Caddy in the Complement image. (#12480) ... Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>	3 years ago
David Robertson	8bac3e0435	disallow-untyped-defs in `docker` and `stubs` directories (#12528)	3 years ago
Richard van der Hoff	aaaff98202	Dockerfile-workers: reduce the amount we install (#12464) ... This is an attempt to reduce the rebuild time. In short, we reduce the amount of stuff that the dockerfile installs, so as to give a faster startup.	3 years ago
Richard van der Hoff	8af8a9bce5	Dockerfile-workers: give the master its own log config (#12466) ... When we run a worker-mode synapse under docker, everything gets logged to stdout. Currently, output from the workers is tacked with a worker name, for example: ``` 2022-04-13 15:27:56,810 - worker:frontend_proxy1 - synapse.util.caches.lrucache - 154 - INFO - LruCache._expire_old_entries-0 - Dropped 0 items from caches ``` - note `worker:frontend_proxy1`. No such tag is applied to log lines from the master, which makes somewhat confusing reading. To fix this, we generate a dedicated log config file for the master in the same way that we do for the workers, and use that.	3 years ago
Richard van der Hoff	3cdf5a1386	Fix up healthcheck generation for workers docker image (#12405) ... This wasn't quite generating the right thing.	3 years ago
Jorge Florian	78e4d96a4d	Add missing type definitions for scripts in docker folder (#12280) ... Signed-off-by: Jorge Florian <jafn28@gmail.com>	3 years ago
Olivier Wilkinson (reivilibre)	4dd9ea8f4f	Revert "Revert accidental fast-forward merge from v1.49.0rc1" ... This reverts commit 158d73ebdd.	3 years ago
Sean Quah	158d73ebdd	Revert accidental fast-forward merge from v1.49.0rc1 ... Revert "Sort internal changes in changelog" Revert "Update CHANGES.md" Revert "1.49.0rc1" Revert "Revert "Move `glob_to_regex` and `re_word_boundary` to `matrix-python-common` (#11505) (#11527)" Revert "Refactors in `_generate_sync_entry_for_rooms` (#11515)" Revert "Correctly register shutdown handler for presence workers (#11518)" Revert "Fix `ModuleApi.looping_background_call` for non-async functions (#11524)" Revert "Fix 'delete room' admin api to work on incomplete rooms (#11523)" Revert "Correctly ignore invites from ignored users (#11511)" Revert "Fix the test breakage introduced by #11435 as a result of concurrent PRs (#11522)" Revert "Stabilise support for MSC2918 refresh tokens as they have now been merged into the Matrix specification. (#11435)" Revert "Save the OIDC session ID (sid) with the device on login (#11482)" Revert "Add admin API to get some information about federation status (#11407)" Revert "Include bundled aggregations in /sync and related fixes (#11478)" Revert "Move `glob_to_regex` and `re_word_boundary` to `matrix-python-common` (#11505)" Revert "Update backward extremity docs to make it clear that it does not indicate whether we have fetched an events' `prev_events` (#11469)" Revert "Support configuring the lifetime of non-refreshable access tokens separately to refreshable access tokens. (#11445)" Revert "Add type hints to `synapse/tests/rest/admin` (#11501)" Revert "Revert accidental commits to develop." Revert "Newsfile" Revert "Give `tests.server.setup_test_homeserver` (nominally!) the same behaviour" Revert "Move `tests.utils.setup_test_homeserver` to `tests.server`" Revert "Convert one of the `setup_test_homeserver`s to `make_test_homeserver_synchronous`" Revert "Disambiguate queries on `state_key` (#11497)" Revert "Comments on the /sync tentacles (#11494)" Revert "Clean up tests.storage.test_appservice (#11492)" Revert "Clean up `tests.storage.test_main` to remove use of legacy code. (#11493)" Revert "Clean up `tests.test_visibility` to remove legacy code. (#11495)" Revert "Minor cleanup on recently ported doc pages (#11466)" Revert "Add most of the missing type hints to `synapse.federation`. (#11483)" Revert "Avoid waiting for zombie processes in `synctl stop` (#11490)" Revert "Fix media repository failing when media store path contains symlinks (#11446)" Revert "Add type annotations to `tests.storage.test_appservice`. (#11488)" Revert "`scripts-dev/sign_json`: support for signing events (#11486)" Revert "Add MSC3030 experimental client and federation API endpoints to get the closest event to a given timestamp (#9445)" Revert "Port wiki pages to documentation website (#11402)" Revert "Add a license header and comment. (#11479)" Revert "Clean-up get_version_string (#11468)" Revert "Link background update controller docs to summary (#11475)" Revert "Additional type hints for config module. (#11465)" Revert "Register the login redirect endpoint for v3. (#11451)" Revert "Update openid.md" Revert "Remove mention of OIDC certification from Dex (#11470)" Revert "Add a note about huge pages to our Postgres doc (#11467)" Revert "Don't start Synapse master process if `worker_app` is set (#11416)" Revert "Expose worker & homeserver as entrypoints in `setup.py` (#11449)" Revert "Bundle relations of relations into the `/relations` result. (#11284)" Revert "Fix `LruCache` corruption bug with a `size_callback` that can return 0 (#11454)" Revert "Eliminate a few `Any`s in `LruCache` type hints (#11453)" Revert "Remove unnecessary `json.dumps` from `tests.rest.admin` (#11461)" Revert "Merge branch 'master' into develop" This reverts commit 26b5d2320f. This reverts commit bce4220f38. This reverts commit 966b5d0fa0. This reverts commit 088d748f2c. This reverts commit 14d593f72d. This reverts commit 2a3ec6facf. This reverts commit eccc49d755. This reverts commit b1ecd19c5d. This reverts commit 9c55dedc8c. This reverts commit 2d42e586a8. This reverts commit 2f053f3f82. This reverts commit a15a893df8. This reverts commit 8b4b153c9e. This reverts commit 494ebd7347. This reverts commit a77c369897. This reverts commit 4eb77965cd. This reverts commit 637df95de6. This reverts commit e5f426cd54. This reverts commit 8cd68b8102. This reverts commit 6cae125e20. This reverts commit 7be88fbf48. This reverts commit b3fd99b74a. This reverts commit f7ec6e7d9e. This reverts commit 5640992d17. This reverts commit d26808dd85. This reverts commit f91624a595. This reverts commit 16d39a5490. This reverts commit 8a4c296987. This reverts commit 49e1356ee3. This reverts commit d2279f471b. This reverts commit b50e39df57. This reverts commit 858d80bf0f. This reverts commit 435f044807. This reverts commit f61462e1be. This reverts commit a6f1a3abec. This reverts commit 84dc50e160. This reverts commit ed635d3285. This reverts commit 7b62791e00. This reverts commit 153194c771. This reverts commit f44d729d4c. This reverts commit a265fbd397. This reverts commit b9fef1a7cd. This reverts commit b0eb64ff7b. This reverts commit f1795463bf. This reverts commit 70cbb1a5e3. This reverts commit 42bf020463. This reverts commit 379f2650cf. This reverts commit 7ff22d6da4. This reverts commit 5a0b652d36. This reverts commit 432a174bc1. This reverts commit b14f8a1baf, reversing changes made to e713855dca.	3 years ago
Michael Kaye	e2c300e7e4	Create healthcheck script for synapse-workers container (#11429) ... The intent is to iterate through all the worker ports and only report healthy when all are healthy, starting with the main process.	3 years ago
Aaron R	d993c3bb1e	Add support for `/_matrix/media/v3` APIs (#11371) ... * Add support for `/_matrix/media/v3` APIs Signed-off-by: Aaron Raimist <aaron@raim.ist> * Update `workers.md` to use v3 client and media APIs Signed-off-by: Aaron Raimist <aaron@raim.ist> * Add changelog Signed-off-by: Aaron Raimist <aaron@raim.ist>	3 years ago
kegsay	01df612e1e	Fix frontend_proxy jinja script in docker workers (#10783)	3 years ago
Patrick Cloke	7d90d6ce9b	Run complement with Synapse workers manually. (#10039) ... Adds an option to complement.sh to run Synapse in worker mode (instead of the default monolith mode).	4 years ago
Andrew Morgan	7e460ec2a5	Add a dockerfile for running a set of Synapse worker processes (#9162) ... This PR adds a Dockerfile and some supporting files to the `docker/` directory. The Dockerfile's intention is to spin up a container with: * A Synapse main process. * Any desired worker processes, defined by a `SYNAPSE_WORKERS` environment variable supplied at runtime. * A redis for worker communication. * A nginx for routing traffic. * A supervisord to start all worker processes and monitor them if any go down. Note that **this is not currently intended to be used in production**. If you'd like to use Synapse workers with Docker, instead make use of the official image, with one worker per container. The purpose of this dockerfile is currently to allow testing Synapse in worker mode with the [Complement](https://github.com/matrix-org/complement/) test suite. `configure_workers_and_start.py` is where most of the magic happens in this PR. It reads from environment variables (documented in the file) and creates all necessary config files for the processes. It is the entrypoint of the Dockerfile, and thus is run any time the docker container is spun up, recreating all config files in case you want to use a different set of workers. One can specify which workers they'd like to use by setting the `SYNAPSE_WORKERS` environment variable (as a comma-separated list of arbitrary worker names) or by setting it to `*` for all worker processes. We will be using the latter in CI. Huge thanks to @MatMaul for helping get this all working 🎉 This PR is paired with its equivalent on the Complement side: https://github.com/matrix-org/complement/pull/62. Note, for the purpose of testing this PR before it's merged: You'll need to (re)build the base Synapse docker image for everything to work (`matrixdotorg/synapse:latest`). Then build the worker-based docker image on top (`matrixdotorg/synapse:workers`).	4 years ago