11794 Commits (8cced497640525ff1bed7011509b88306c7e7ea3)

Author	SHA1	Message	Date
Andrew Morgan	20ffaa7209	1.13.0rc1	5 years ago
Patrick Cloke	0ad6d28b0d	Rework UI Auth session validation for registration (#7455) ... Be less strict about validation of UI authentication sessions during registration to match client expecations.	5 years ago
Richard van der Hoff	aa5aa6f96a	Fix errors from malformed log line (#7454)	5 years ago
Richard van der Hoff	da9b2db3af	Drop support for redis.dbid (#7450) ... Since we only use pubsub, the dbid is irrelevant.	5 years ago
Brendan Abolivier	d9b8d27494	Add a configuration setting for the dummy event threshold (#7422) ... Add dummy_events_threshold which allows configuring the number of forward extremities a room needs for Synapse to send forward extremities in it.	5 years ago
Patrick Cloke	d7c2df2fa3	Improve per-block CPU and DB usage metrics (#7426)	5 years ago
Andrew Morgan	4162c39dcf	Port group attestation renewal slow down from matrix-org-hotfixes (#7442)	5 years ago
Richard van der Hoff	e053c86a96	Make redis go faster with hiredis (#7439) ... For the record, the reason we need this is as follows: each RDATA command comes down the redis pipe as a subscription message. txredisapi as written needs at least three reactor ticks to read each subscription message from the tcp buffer. Hence, once the process gets loaded, it starts getting behind, and eventually redis knifes the connection. it then takes ages for the master to work its way through the backlog, before it reconnects again, during which any commands from any workers are dropped.	5 years ago
Andrew Morgan	aee9130a83	Stop Auth methods from polling the config on every req. (#7420)	5 years ago
Richard van der Hoff	7f7eedbebb	Wait for a POSITION on the right connection before accepting RDATA ... ... otherwise we can believe we're up to date when we're not.	5 years ago
Brendan Abolivier	5b8023dc7f	Move logs about discarded RDATA to debug (#7421)	5 years ago
Richard van der Hoff	d78265af0c	Wait to subscribe before sending REPLICATE	5 years ago
Erik Johnston	f9073893af	Speed up fetching device lists changes in sync. ... Currently we copy `users_who_share_room` needlessly about three times, which is expensive when the set is large (which it can easily be).	5 years ago
Patrick Cloke	fe69fb6263	Add backwards compatibility codepath to LoggingContext. (#7408)	5 years ago
Erik Johnston	7941a70fa8	Fix bug in EventContext.deserialize. (#7393) ... This caused `prev_state_ids` to be incorrect if the state event was not replacing an existing state entry.	5 years ago
Erik Johnston	350421e058	Fix redis password support. (#7401) ... We forgot to set the password on the subscriber connection, as well as not calling super methods for overridden connectionMade/connectionLost functions.	5 years ago
Patrick Cloke	032e5a2aca	Convert synapse.server_notices to async/await. (#7394)	5 years ago
Patrick Cloke	b0cbc57375	Convert the synapse.notifier module to async/await. (#7395)	5 years ago
Richard van der Hoff	97ef1471c6	Wait for current_state_events_membership before populate_stats_process_rooms (#7387) ... populate_stats_process_rooms was added in #5971 / v1.4.0; current_state_events_membership was added in #5706 / v1.3.0. Fixes #7380.	5 years ago
Erik Johnston	0e719f2398	Thread through instance name to replication client. (#7369) ... For in memory streams when fetching updates on workers we need to query the source of the stream, which currently is hard coded to be master. This PR threads through the source instance we received via `POSITION` through to the update function in each stream, which can then be passed to the replication client for in memory streams.	5 years ago
Erik Johnston	3085cde577	Use `stream.current_token()` and remove `stream_positions()` (#7172) ... We move the processing of typing and federation replication traffic into their handlers so that `Stream.current_token()` points to a valid token. This allows us to remove `get_streams_to_replicate()` and `stream_positions()`.	5 years ago
Andrew Morgan	6b22921b19	async/await is_server_admin (#7363)	5 years ago
Andrew Morgan	2e8955f4a6	Further improvements to requesting the public rooms list on a homeserver which has it set to private (#7368)	5 years ago
Richard van der Hoff	b2dba06079	Workaround for assertion errors from db_query_to_update_function (#7378) ... Hopefully this is no worse than what we have on master...	5 years ago
Patrick Cloke	627b0f5f27	Persist user interactive authentication sessions (#7302) ... By persisting the user interactive authentication sessions to the database, this fixes situations where a user hits different works throughout their auth session and also allows sessions to persist through restarts of Synapse.	5 years ago
Andrew Morgan	9d8ecc9e6c	Apply federation check for /publicRooms with filter list (#7367)	5 years ago
Erik Johnston	37f6823f5b	Add instance name to RDATA/POSITION commands (#7364) ... This is primarily for allowing us to send those commands from workers, but for now simply allows us to ignore echoed RDATA/POSITION commands that we sent (we get echoes of sent commands when using redis). Currently we log a WARNING on the master process every time we receive an echoed RDATA.	5 years ago
Erik Johnston	3eab76ad43	Don't relay REMOTE_SERVER_UP cmds to same conn. (#7352) ... For direct TCP connections we need the master to relay REMOTE_SERVER_UP commands to the other connections so that all instances get notified about it. The old implementation just relayed to all connections, assuming that sending back to the original sender of the command was safe. This is not true for redis, where commands sent get echoed back to the sender, which was causing master to effectively infinite loop sending and then re-receiving REMOTE_SERVER_UP commands that it sent. The fix is to ensure that we only relay to *other* connections and not to the connection we received the notification from. Fixes #7334.	5 years ago
Richard van der Hoff	c2e1a2110f	Fix limit logic for EventsStream (#7358) ... * Factor out functions for injecting events into database I want to add some more flexibility to the tools for injecting events into the database, and I don't want to clutter up HomeserverTestCase with them, so let's factor them out to a new file. * Rework TestReplicationDataHandler This wasn't very easy to work with: the mock wrapping was largely superfluous, and it's useful to be able to inspect the received rows, and clear out the received list. * Fix AssertionErrors being thrown by EventsStream Part of the problem was that there was an off-by-one error in the assertion, but also the limit logic was too simple. Fix it all up and add some tests.	5 years ago
Andrew Morgan	eeef9633af	Fix fallback value for account_threepid_delegates.email (#7316)	5 years ago
Manuel Stahl	04dd7d182d	Return total number of users and profile attributes in admin users endpoint (#6881) ... Signed-off-by: Manuel Stahl <manuel.stahl@awesome-technologies.de>	5 years ago
Andrew Morgan	ce207aa0ff	Fix typo 'datbases' in ConfigError	5 years ago
Erik Johnston	38919b521e	Run replication streamers on workers (#7146) ... Currently we never write to streams from workers, but that will change soon	5 years ago
Richard van der Hoff	07337fe30b	Fix incorrect metrics reporting for renew_attestations (#7344) ... We need to wait for the renewals to finish, so that the metrics are correctly reported.	5 years ago
Brendan Abolivier	cc9eceb00d	Don't crash when one of the configuration files is empty (#7341) ... If the admin adds a `.yaml` file that's either empty or doesn't parse into a dict to a config directory (e.g. `conf.d` for debs installs), stuff like https://github.com/matrix-org/synapse/issues/7322 would happen. This PR checks that the file is correctly parsed into a dict, or ignores it with a warning if it parses into any other type (including `None` for empty files). Fixes https://github.com/matrix-org/synapse/issues/7322	5 years ago
Patrick Cloke	7bfe0902ce	Add documentation to the sample config about the templates for SSO. (#7343)	5 years ago
Patrick Cloke	33bceb7f70	Convert some of the federation handler methods to async/await. (#7338)	5 years ago
Richard van der Hoff	ce428a1abe	Fix EventsStream raising assertions when it falls behind ... Figuring out how to correctly limit updates from this stream without dropping entries is far more complicated than just counting the number of rows being returned. We need to consider each query separately and, if any one query hits the limit, truncate the results from the others. I think this also fixes some potentially long-standing bugs where events or state changes could get missed if we hit the limit on either query.	5 years ago
Patrick Cloke	ce9b62e13f	1.12.4	5 years ago
Richard van der Hoff	9cbdfb3a2f	Make it clear that the limit for an update_function is a target	5 years ago
Richard van der Hoff	23b28266ac	Remove 'limit' param from `get_repl_stream_updates` API ... there doesn't seem to be much point in passing this limit all around, since both sides agree it's meant to be 100.	5 years ago
Brendan Abolivier	2e3b9a0fcb	Revert "Revert "Merge pull request #7315 from matrix-org/babolivier/request_token"" ... This reverts commit 1adf6a5587.	5 years ago
Brendan Abolivier	1adf6a5587	Revert "Merge pull request #7315 from matrix-org/babolivier/request_token" ... This reverts commit 6f4319368b, reversing changes made to 0d775fcc2d.	5 years ago
Richard van der Hoff	71a1abb8a1	Stop the master relaying USER_SYNC for other workers (#7318) ... Long story short: if we're handling presence on the current worker, we shouldn't be sending USER_SYNC commands over replication. In an attempt to figure out what is going on here, I ended up refactoring some bits of the presencehandler code, so the first 4 commits here are non-functional refactors to move this code slightly closer to sanity. (There's still plenty to do here :/). Suggest reviewing individual commits. Fixes (I hope) #7257.	5 years ago
Brendan Abolivier	69ad7cc13b	Config option to inhibit 3PID errors on /requestToken ... Adds a request_token_inhibit_errors configuration flag (disabled by default) which, if enabled, change the behaviour of all /requestToken endpoints so that they return a 200 and a fake sid if the 3PID was/was not found associated with an account (depending on the endpoint), instead of an error. Co-Authored-By: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>	5 years ago
Erik Johnston	841c581c40	Fix replication metrics when using redis (#7325)	5 years ago
Richard van der Hoff	82d8b1dd1f	Another go at fixing one-word commands (#7326) ... I messed this up last time I tried (#7239 / e13c6c7).	5 years ago
Richard van der Hoff	83af1079d6	1.12.4rc1	5 years ago
Richard van der Hoff	13683a3a22	Extend StreamChangeCache to support multiple entities per stream ID (#7303) ... First some background: StreamChangeCache is used to keep track of what "entities" have changed since a given stream ID. So for example, we might use it to keep track of when the last to-device message for a given user was received [1], and hence whether we need to pull any to-device messages from the database on a sync [2]. Now, it turns out that StreamChangeCache didn't support more than one thing being changed at a given stream_id (this was part of the problem with #7206). However, it's entirely valid to send to-device messages to more than one user at a time. As it turns out, this did in fact work, because *some* methods of StreamChangeCache coped ok with having multiple things changing on the same stream ID, and it seems we never actually use the methods which don't work on the stream change caches where we allow multiple changes at the same stream ID. But that feels horribly fragile, hence: let's update StreamChangeCache to properly support this, and add some typing and some more tests while we're at it. [1]: https://github.com/matrix-org/synapse/blob/release-v1.12.3/synapse/storage/data_stores/main/deviceinbox.py#L301 [2]: https://github.com/matrix-org/synapse/blob/release-v1.12.3/synapse/storage/data_stores/main/deviceinbox.py#L47-L51	5 years ago
Dirk Klimpel	6b6685db9f	Extend room admin api with additional attributes (#7225)	5 years ago