watcha-synapse

Commit Graph

Author	SHA1	Message	Date
Richard van der Hoff	f61462e1be	`scripts-dev/sign_json`: support for signing events (#11486 )	3 years ago
Patrick Cloke	a4521ce0a8	Support the stable /hierarchy endpoint from MSC2946 (#11329 ) This also makes additional updates where the implementation had drifted from the approved MSC. Unstable endpoints will be removed at a later data.	3 years ago
Patrick Cloke	9d1971a5c4	Return the stable `event` field from `/send_join` per MSC3083. (#11413 ) This does not remove the unstable field and still parses both. Handling of the unstable field will need to be removed in the future.	3 years ago
Patrick Cloke	5505da2109	Remove msc2716 from the list of tests for complement. (#11389 ) As the tests are currently failing and not run in CI.	3 years ago
Patrick Cloke	60ecb6b4d4	Fix running complement.sh script. (#11368 ) By reverting changes from #11166 in this script. Specifically commit `13f084eb58`.	3 years ago
Dan Callahan	d7141e0b8b	Fix Shellcheck SC2006: Use $(...) notation Use $(...) notation instead of legacy backticked `...`. https://github.com/koalaman/shellcheck/wiki/SC2006 Signed-off-by: Dan Callahan <danc@element.io>	3 years ago
Dan Callahan	13f084eb58	Fix Shellcheck SC2086: Quote to prevent splitting Double quote to prevent globbing and word splitting. https://github.com/koalaman/shellcheck/wiki/SC2086 Signed-off-by: Dan Callahan <danc@element.io>	3 years ago
Dan Callahan	6a9d84a676	Fix Shellcheck SC2166: test -a is not well defined Prefer [ p ] && [ q ] as [ p -a q ] is not well defined. https://github.com/koalaman/shellcheck/wiki/SC2166 Signed-off-by: Dan Callahan <danc@element.io>	3 years ago
Dan Callahan	64adbb7b54	Fix Shellcheck SC2046: Quote to prevent word split Quote this to prevent word splitting https://www.shellcheck.net/wiki/SC2046 Signed-off-by: Dan Callahan <danc@element.io>	3 years ago
Dan Callahan	12d79ff1b6	Fix Shellcheck SC2164: exit in case cd fails. Use `cd ... \|\| exit` in case cd fails. https://github.com/koalaman/shellcheck/wiki/SC2164 Signed-off-by: Dan Callahan <danc@element.io>	3 years ago
Richard van der Hoff	57501d9194	Update `sign_json` to support inline key config (#11139 ) It's been possible to configure a key inline in the homeserver.yaml since `13bc1e0746`. Update `sign_json` to work with this.	3 years ago
Hillery Shay	5dcacdf6d1	Add support for ubuntu 21.10 "Impish Indri" (#11024 ) * support ubuntu 21.10 indri * add changelog * update to correct codename Co-authored-by: Brendan Abolivier <github@brendanabolivier.com> Co-authored-by: Brendan Abolivier <github@brendanabolivier.com>	3 years ago
reivilibre	b742cb2e4a	Release script improvements (#10966 ) Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>	3 years ago
Nick Barrett	c80878d22a	Add `--run-background-updates` option to `update_database` script. (#10954 ) Signed-off-by: Nick Barrett <nick@beeper.com>	3 years ago
Erik Johnston	d37841787a	Sign the git tag in release script (#10925 )	3 years ago
David Robertson	ea01d4c2de	Update postgresql testing script (#10906 ) - Use sytest:bionic. Sytest:latest is two years old (do we want CI to push out latest at all?) and comes with Python 3.5, which we explictly no longer support. The script now runs under PostgreSQL 10 as a result. - Advertise script in the docs - Move pg testing script to scripts-dev directory - Write to host as the script's exector, not root A few changes to make it speedier to re-run the tests: - Create blank DB in the container, not the script, so we don't have to `initdb` each time - Use a named volume to persist the tox environment, so we don't have to fetch and install a bunch of packages from PyPI each time Co-authored-by: reivilibre <olivier@librepush.net>	3 years ago
Brendan Abolivier	5154afc00d	Add a script to update the debian changelog for non-Debian systems (#10778 ) When releasing 1.42.0 with @Azrenbeth and talking with @clokep yesterday I realised doing the dch incantations related to releasing Synapse wasn't trivial on eg a macOS system, so this is a script to run in a Debian container to make things a bit easier.	3 years ago
Patrick Cloke	882539e423	Ensure the base Docker image is rebuilt when running complement with workers. (#10686 ) We now always rebuild the matrixdotorg/synapse image, then build the matrixdotorg/synapse-workers image on top of it.	3 years ago
Richard van der Hoff	c8132f4a31	Build debs for bookworm (#10612 )	3 years ago
David Robertson	d2ad397d3c	Stop building a debian package for Groovy Gorilla (#10588 )	3 years ago
David Robertson	314a739160	Also rename in lint.sh	3 years ago
David Robertson	c0ebdfc77e	Kill off the .buildkite dir completely	3 years ago
David Robertson	03fb99a5c8	check-newsfragment: pass pr number explicitly use PULL_REQUEST_NUMBER instead of BUILDKITE_PULL_REQUEST remove the other user of BUILDKITE_PULL_REQUEST, namely merge_base_branch.sh	3 years ago
Eric Eastwood	684d19a11c	Add support for MSC2716 marker events (#10498 ) * Make historical messages available to federated servers Part of MSC2716: https://github.com/matrix-org/matrix-doc/pull/2716 Follow-up to https://github.com/matrix-org/synapse/pull/9247 * Debug message not available on federation * Add base starting insertion point when no chunk ID is provided * Fix messages from multiple senders in historical chunk Follow-up to https://github.com/matrix-org/synapse/pull/9247 Part of MSC2716: https://github.com/matrix-org/matrix-doc/pull/2716 --- Previously, Synapse would throw a 403, `Cannot force another user to join.`, because we were trying to use `?user_id` from a single virtual user which did not match with messages from other users in the chunk. * Remove debug lines * Messing with selecting insertion event extremeties * Move db schema change to new version * Add more better comments * Make a fake requester with just what we need See https://github.com/matrix-org/synapse/pull/10276#discussion_r660999080 * Store insertion events in table * Make base insertion event float off on its own See https://github.com/matrix-org/synapse/pull/10250#issuecomment-875711889 Conflicts: synapse/rest/client/v1/room.py * Validate that the app service can actually control the given user See https://github.com/matrix-org/synapse/pull/10276#issuecomment-876316455 Conflicts: synapse/rest/client/v1/room.py * Add some better comments on what we're trying to check for * Continue debugging * Share validation logic * Add inserted historical messages to /backfill response * Remove debug sql queries * Some marker event implemntation trials * Clean up PR * Rename insertion_event_id to just event_id * Add some better sql comments * More accurate description * Add changelog * Make it clear what MSC the change is part of * Add more detail on which insertion event came through * Address review and improve sql queries * Only use event_id as unique constraint * Fix test case where insertion event is already in the normal DAG * Remove debug changes * Add support for MSC2716 marker events * Process markers when we receive it over federation * WIP: make hs2 backfill historical messages after marker event * hs2 to better ask for insertion event extremity But running into the `sqlite3.IntegrityError: NOT NULL constraint failed: event_to_state_groups.state_group` error * Add insertion_event_extremities table * Switch to chunk events so we can auth via power_levels Previously, we were using `content.chunk_id` to connect one chunk to another. But these events can be from any `sender` and we can't tell who should be able to send historical events. We know we only want the application service to do it but these events have the sender of a real historical message, not the application service user ID as the sender. Other federated homeservers also have no indicator which senders are an application service on the originating homeserver. So we want to auth all of the MSC2716 events via power_levels and have them be sent by the application service with proper PL levels in the room. * Switch to chunk events for federation * Add unstable room version to support new historical PL * Messy: Fix undefined state_group for federated historical events ``` 2021-07-13 02:27:57,810 - synapse.handlers.federation - 1248 - ERROR - GET-4 - Failed to backfill from hs1 because NOT NULL constraint failed: event_to_state_groups.state_group Traceback (most recent call last): File "/usr/local/lib/python3.8/site-packages/synapse/handlers/federation.py", line 1216, in try_backfill await self.backfill( File "/usr/local/lib/python3.8/site-packages/synapse/handlers/federation.py", line 1035, in backfill await self._auth_and_persist_event(dest, event, context, backfilled=True) File "/usr/local/lib/python3.8/site-packages/synapse/handlers/federation.py", line 2222, in _auth_and_persist_event await self._run_push_actions_and_persist_event(event, context, backfilled) File "/usr/local/lib/python3.8/site-packages/synapse/handlers/federation.py", line 2244, in _run_push_actions_and_persist_event await self.persist_events_and_notify( File "/usr/local/lib/python3.8/site-packages/synapse/handlers/federation.py", line 3290, in persist_events_and_notify events, max_stream_token = await self.storage.persistence.persist_events( File "/usr/local/lib/python3.8/site-packages/synapse/logging/opentracing.py", line 774, in _trace_inner return await func(args, kwargs) File "/usr/local/lib/python3.8/site-packages/synapse/storage/persist_events.py", line 320, in persist_events ret_vals = await yieldable_gather_results(enqueue, partitioned.items()) File "/usr/local/lib/python3.8/site-packages/synapse/storage/persist_events.py", line 237, in handle_queue_loop ret = await self._per_item_callback( File "/usr/local/lib/python3.8/site-packages/synapse/storage/persist_events.py", line 577, in _persist_event_batch await self.persist_events_store._persist_events_and_state_updates( File "/usr/local/lib/python3.8/site-packages/synapse/storage/databases/main/events.py", line 176, in _persist_events_and_state_updates await self.db_pool.runInteraction( File "/usr/local/lib/python3.8/site-packages/synapse/storage/database.py", line 681, in runInteraction result = await self.runWithConnection( File "/usr/local/lib/python3.8/site-packages/synapse/storage/database.py", line 770, in runWithConnection return await make_deferred_yieldable( File "/usr/local/lib/python3.8/site-packages/twisted/python/threadpool.py", line 238, in inContext result = inContext.theWork() # type: ignore[attr-defined] File "/usr/local/lib/python3.8/site-packages/twisted/python/threadpool.py", line 254, in <lambda> inContext.theWork = lambda: context.call( # type: ignore[attr-defined] File "/usr/local/lib/python3.8/site-packages/twisted/python/context.py", line 118, in callWithContext return self.currentContext().callWithContext(ctx, func, args, *kw) File "/usr/local/lib/python3.8/site-packages/twisted/python/context.py", line 83, in callWithContext return func(args, *kw) File "/usr/local/lib/python3.8/site-packages/twisted/enterprise/adbapi.py", line 293, in _runWithConnection compat.reraise(excValue, excTraceback) File "/usr/local/lib/python3.8/site-packages/twisted/python/deprecate.py", line 298, in deprecatedFunction return function(args, *kwargs) File "/usr/local/lib/python3.8/site-packages/twisted/python/compat.py", line 403, in reraise raise exception.with_traceback(traceback) File "/usr/local/lib/python3.8/site-packages/twisted/enterprise/adbapi.py", line 284, in _runWithConnection result = func(conn, args, *kw) File "/usr/local/lib/python3.8/site-packages/synapse/storage/database.py", line 765, in inner_func return func(db_conn, args, *kwargs) File "/usr/local/lib/python3.8/site-packages/synapse/storage/database.py", line 549, in new_transaction r = func(cursor, args, *kwargs) File "/usr/local/lib/python3.8/site-packages/synapse/logging/utils.py", line 69, in wrapped return f(args, *kwargs) File "/usr/local/lib/python3.8/site-packages/synapse/storage/databases/main/events.py", line 385, in _persist_events_txn self._store_event_state_mappings_txn(txn, events_and_contexts) File "/usr/local/lib/python3.8/site-packages/synapse/storage/databases/main/events.py", line 2065, in _store_event_state_mappings_txn self.db_pool.simple_insert_many_txn( File "/usr/local/lib/python3.8/site-packages/synapse/storage/database.py", line 923, in simple_insert_many_txn txn.execute_batch(sql, vals) File "/usr/local/lib/python3.8/site-packages/synapse/storage/database.py", line 280, in execute_batch self.executemany(sql, args) File "/usr/local/lib/python3.8/site-packages/synapse/storage/database.py", line 300, in executemany self._do_execute(self.txn.executemany, sql, args) File "/usr/local/lib/python3.8/site-packages/synapse/storage/database.py", line 330, in _do_execute return func(sql, args) sqlite3.IntegrityError: NOT NULL constraint failed: event_to_state_groups.state_group ``` Revert "Messy: Fix undefined state_group for federated historical events" This reverts commit 187ab28611546321e02770944c86f30ee2bc742a. * Fix federated events being rejected for no state_groups Add fix from https://github.com/matrix-org/synapse/pull/10439 until it merges. * Adapting to experimental room version * Some log cleanup * Add better comments around extremity fetching code and why * Rename to be more accurate to what the function returns * Add changelog * Ignore rejected events * Use simplified upsert * Add Erik's explanation of extra event checks See https://github.com/matrix-org/synapse/pull/10498#discussion_r680880332 * Clarify that the depth is not directly correlated to the backwards extremity that we return See https://github.com/matrix-org/synapse/pull/10498#discussion_r681725404 * lock only matters for sqlite See https://github.com/matrix-org/synapse/pull/10498#discussion_r681728061 * Move new SQL changes to its own delta file * Clean up upsert docstring * Bump database schema version (62)	3 years ago
Erik Johnston	6878e10653	Fix release script URL (#10516 )	3 years ago
Erik Johnston	a7bacccd85	Extend the release script to tag and create the releases. (#10496 )	3 years ago
Patrick Cloke	13944678c3	Use new go test running syntax for complement. (#10488 ) Updates CI and the helper script t ensures all tests are run (in parallel).	3 years ago
Erik Johnston	92a882254b	Change release script to update debian changelog for RCs (#10465 )	3 years ago
Richard van der Hoff	f22252d4f9	Enable docker image caching for the deb build (#10431 )	3 years ago
Eric Eastwood	cd5fcd2731	Disable msc2716 until Complement update is merged (#10463 )	3 years ago
Richard van der Hoff	5f2848f379	build debs in GHA (#10247 ) GHA workflow to build the debs	3 years ago
Brendan Abolivier	d731ed70d9	Fixes to the release script (#10239 ) * rename major/minor into the right semver terminology minor/patch (since this was something that got me very confused the first couple of times I've used the script) * name the release branch based on the new version, not the previous one	3 years ago
Richard van der Hoff	8beead66ae	Send out invite rejections and knocks over federation (#10223 ) ensure that events sent via `send_leave` and `send_knock` are sent on to the rest of the federation.	3 years ago
Eric Eastwood	96f6293de5	Add endpoints for backfilling history (MSC2716) (#9247 ) Work on https://github.com/matrix-org/matrix-doc/pull/2716	3 years ago
Dan Callahan	7dc14730d9	Name release branches just after major.minor (#10013 ) With the prior format, 1.33.0 / 1.33.1 / 1.33.2 got separate branches: release-v1.33.0 release-v1.33.1 release-v1.33.2 Under the new model, all three would share a common branch: release-v1.33 As before, RCs and actual releases exist as tags on these branches. This better reflects our support model, e.g., that the "1.33" series had a formal release followed by two patches / updates. Signed-off-by: Dan Callahan <danc@element.io>	4 years ago
Richard van der Hoff	5447a76332	Remove redundant, unmaintained `convert_server_keys` script. (#10055 )	4 years ago
Patrick Cloke	7d90d6ce9b	Run complement with Synapse workers manually. (#10039 ) Adds an option to complement.sh to run Synapse in worker mode (instead of the default monolith mode).	4 years ago
Jerin J Titus	057ce7b754	Remove tls_fingerprints option (#9280 ) Signed-off-by: Jerin J Titus <72017981+jerinjtitus@users.noreply.github.com>	4 years ago
Richard van der Hoff	6482075c95	Run `black` on the scripts (#9981 ) Turns out these scripts weren't getting linted.	4 years ago
Richard van der Hoff	7562d887e1	Change the format of access tokens away from macaroons (#5588 )	4 years ago
Richard van der Hoff	765473567c	Fix make_full_schema to create the db with the right options and user (#9931 )	4 years ago
Dan Callahan	56c4b47df3	Build Debian packages for Ubuntu 21.04 Hirsute (#9909 ) Signed-off-by: Dan Callahan <danc@element.io>	4 years ago
Jonathan de Jong	495b214f4f	Fix (final) Bugbear violations (#9838 )	4 years ago
Jonathan de Jong	4b965c862d	Remove redundant "coding: utf-8" lines (#9786 ) Part of #9744 Removes all redundant `# -- coding: utf-8 --` lines from files, as python 3 automatically reads source code as utf-8 now. `Signed-off-by: Jonathan de Jong <jonathan@automatia.nl>`	4 years ago
Erik Johnston	c1dbe84c3d	Add release helper script (#9713 ) Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com> Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>	4 years ago
Dan Callahan	3efde8b69a	Add option to skip unit tests when building debs (#9793 ) Signed-off-by: Dan Callahan <danc@element.io>	4 years ago
Patrick Cloke	abc814dcbf	Enable complement tests for MSC2946. (#9771 ) By providing the additional build tag for `msc2946`.	4 years ago
Richard van der Hoff	9278eb701e	drop support for stretch and xenial	4 years ago
Patrick Cloke	452991527a	MSC3083: Check for space membership during a local join of restricted rooms. (#9735 ) When joining a room with join rules set to 'restricted', check if the user is a member of the spaces defined in the 'allow' key of the join rules. This only applies to an experimental room version, as defined in MSC3083.	4 years ago
Patrick Cloke	5fe38e07e7	Revert "Use 'dmypy run' in lint.sh instead of 'mypy' (#9701 )" (#9720 )	4 years ago

1 2 3 4

162 Commits (b1ecd19c5d19815b69e425d80f442bf2877cab76)