90 Commits (eb609c65d0794dd49efcd924bdc8743fd4253a93)

Author	SHA1	Message	Date
Richard van der Hoff	b38bdae3a2	Fix AssertionErrors after purging events (#11642) ... * Fix AssertionErrors after purging events If you purged a bunch of events from your database, and then restarted synapse without receiving more events, then you would get a bunch of AssertionErrors on restart. This fixes the situation by rewinding the stream processors. * `check-newsfragment`: ignore deleted newsfiles	3 years ago
David Robertson	2d91b6256e	Fix adding excluded users to the private room sharing tables when joining a room (#11143) ... * We only need to fetch users in private rooms * Filter out `user_id` at the top * Discard excluded users in the top loop We weren't doing this in the "First, if they're our user" branch so this is a bugfix. * The caller must check that `user_id` is included This is in the docstring. There are two call sites: - one in `_handle_room_publicity_change`, which explicitly checks before calling; - and another in `_handle_room_membership_event`, which returns early if the user is excluded. So this change is safe. * Test joining a private room with an excluded user * Tweak an existing test * Changelog * test docstring * lint	3 years ago
David Robertson	37b845dabc	Don't remove local users from dir when the leave their last room (#11103)	3 years ago
David Robertson	e09be0c87a	Correctly exclude users when making a room public or private (#11075) ... Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>	3 years ago
David Robertson	317e9e415c	Rearrange the user_directory's `_handle_deltas` function (#11035) ... * Pull out `_handle_room_membership_event` * Discard excluded users early * Rearrange logic so the change is membership is effectively switched over. See PR for rationale.	3 years ago
David Robertson	670a8d9a1e	Fix overwriting profile when making room public (#11003) ... This splits apart `handle_new_user` into a function which adds an entry to the `user_directory` and a function which updates the room sharing tables. I plan to continue doing more of this kind of refactoring to clarify the implementation.	3 years ago
David Robertson	e0bf34dada	Don't alter directory entries for local users when setting a per-room nickname (#11002) ... Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>	3 years ago
David Robertson	370bca32e6	Don't drop user dir deltas when server leaves room (#10982) ... Fix a long-standing bug where a batch of user directory changes would be silently dropped if the server left a room early in the batch. * Pull out `wait_for_background_update` in tests Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com> Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>	3 years ago
David Robertson	f7b034a24b	Consistently exclude from user_directory (#10960) ... * Introduce `should_include_local_users_in_dir` We exclude three kinds of local users from the user_directory tables. At present we don't consistently exclude all three in the same places. This commit introduces a new function to gather those exclusion conditions together. Because we have to handle local and remote users in different ways, I've made that function only consider the case of remote users. It's the caller's responsibility to make the local versus remote distinction clear and correct. A test fixup is required. The test now hits a path which makes db queries against the users table. The expected rows were missing, because we were using a dummy user that hadn't actually been registered. We also add new test cases to covert the exclusion logic. ---- By my reading this makes these changes: * When an app service user registers or changes their profile, they will _not_ be added to the user directory. (Previously only support and deactivated users were excluded). This is consistent with the logic that rebuilds the user directory. See also [the discussion here](https://github.com/matrix-org/synapse/pull/10914#discussion_r716859548). * When rebuilding the directory, exclude support and disabled users from room sharing tables. Previously only appservice users were excluded. * Exclude all three categories of local users when rebuilding the directory. Previously `_populate_user_directory_process_users` didn't do any exclusion. Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>	3 years ago
Patrick Cloke	94b620a5ed	Use direct references for configuration variables (part 6). (#10916)	3 years ago
Patrick Cloke	bb7fdd821b	Use direct references for configuration variables (part 5). (#10897)	3 years ago
Patrick Cloke	b3590614da	Require type hints in the handlers module. (#10831) ... Adds missing type hints to methods in the synapse.handlers module and requires all methods to have type hints there. This also removes the unused construct_auth_difference method from the FederationHandler.	3 years ago
David Robertson	318162f5de	Easy refactors of the user directory (#10789) ... No functional changes here. This came out as I was working to tackle #5677	3 years ago
Jonathan de Jong	98aec1cc9d	Use inline type hints in `handlers/` and `rest/`. (#10382)	3 years ago
Andrew Morgan	c571736c6c	User directory: use calculated room membership state instead (#9821) ... Fixes: #9797. Should help reduce CPU usage on the user directory, especially when memberships change in rooms with lots of state history.	4 years ago
Jonathan de Jong	4b965c862d	Remove redundant "coding: utf-8" lines (#9786) ... Part of #9744 Removes all redundant `# -*- coding: utf-8 -*-` lines from files, as python 3 automatically reads source code as utf-8 now. `Signed-off-by: Jonathan de Jong <jonathan@automatia.nl>`	4 years ago
Patrick Cloke	b7748d3c00	Import HomeServer from the proper module. (#9665)	4 years ago
Patrick Cloke	43f1c82457	Add back the guard against the user directory stream position not existing. (#9428) ... As the comment says, this guard was there for when the initial user directory update has yet to happen.	4 years ago
Eric Eastwood	0a00b7ff14	Update black, and run auto formatting over the codebase (#9381) ... - Update black version to the latest - Run black auto formatting over the codebase - Run autoformatting according to [`docs/code_style.md `](80d6dc9783/docs/code_style.md) - Update `code_style.md` docs around installing black to use the correct version	4 years ago
Patrick Cloke	1baab20352	Add type hints to various handlers. (#9223) ... With this change all handlers except the e2e_* ones have type hints enabled.	4 years ago
Patrick Cloke	31b1905e13	Add type hints to the receipts and user directory handlers. (#8976)	4 years ago
Dirk Klimpel	c07022303e	Fix a bug that deactivated users appear in the directory (#8933) ... Fixes a bug that deactivated users appear in the directory when their profile information was updated. To change profile information of deactivated users is neccesary for example you will remove displayname or avatar. But they should not appear in directory. They are deactivated. Co-authored-by: Erik Johnston <erikj@jki.re>	4 years ago
Patrick Cloke	be2db93b3c	Do not assume that the contents dictionary includes history_visibility. (#8945)	4 years ago
David Teller	f14428b25c	Allow spam-checker modules to be provide async methods. (#8890) ... Spam checker modules can now provide async methods. This is implemented in a backwards-compatible manner.	4 years ago
Patrick Cloke	34a5696f93	Fix typos and spelling errors. (#8639)	4 years ago
Patrick Cloke	8a4a4186de	Simplify super() calls to Python 3 syntax. (#8344) ... This converts calls like super(Foo, self) -> super(). Generated with: sed -i "" -Ee 's/super\([^\(]+\)/super()/g' **/*.py	4 years ago
Patrick Cloke	b939251c37	Fix errors when updating the user directory with invalid data (#8223)	4 years ago
Patrick Cloke	bd6dc17221	Replace iteritems/itervalues/iterkeys with native versions. (#7692)	5 years ago
Patrick Cloke	737b4a936e	Convert user directory handler and related classes to async/await. (#7640)	5 years ago
Patrick Cloke	49f877d32e	Filter the results of user directory searching via the spam checker (#6888) ... Add a method to the spam checker to filter the user directory results.	5 years ago
Erik Johnston	ed630ea17c	Reduce amount of logging at INFO level. (#6862) ... A lot of the things we log at INFO are now a bit superfluous, so lets make them DEBUG logs to reduce the amount we log by default. Co-Authored-By: Brendan Abolivier <babolivier@matrix.org> Co-authored-by: Brendan Abolivier <github@brendanabolivier.com>	5 years ago
Richard van der Hoff	a139420a3c	Fix races in room stats (and other) updates. (#6187) ... Hopefully this will fix the occasional failures we were seeing in the room directory. The problem was that events are not necessarily persisted (and `current_state_delta_stream` updated) in the same order as their stream_id. So for instance current_state_delta 9 might be persisted *before* current_state_delta 8. Then, when the room stats saw stream_id 9, it assumed it had done everything up to 9, and never came back to do stream_id 8. We can solve this easily by only processing up to the stream_id where we know all events have been persisted.	5 years ago
Amber Brown	4806651744	Replace returnValue with return (#5736)	5 years ago
Andrew Morgan	4a4d5c4fd6	Fix grammar and document get_current_users_in_room (#4998)	6 years ago
Amber Brown	ac396a0d32	Refactor out state delta handling into its own class (#4917)	6 years ago
Amber Brown	282c97327f	Migrate the user directory initial population to a background task (#4864)	6 years ago
Amber Brown	c0332d095f	fixup	6 years ago
Amber Brown	10480c4348	fixup	6 years ago
Amber Brown	5ba8ceab4c	fixes	6 years ago
Amber Brown	4abc988c6a	initial	6 years ago
Amber Brown	f6135d06cf	Rewrite userdir to be faster (#4537)	6 years ago
Amber Brown	bb4fd8f927	Run `black` on user directory code (#4635)	6 years ago
Andrew Morgan	82a92ba535	Add metric for user dir current event stream position	6 years ago
Neil Johnson	d2f7c4e6b1	create support user (#4141) ... Allow for the creation of a support user. A support user can access the server, join rooms, interact with other users, but does not appear in the user directory nor does it contribute to monthly active user limits.	6 years ago
Richard van der Hoff	b3f6dddad2	Give some more things logcontexts (#4077)	6 years ago
Erik Johnston	ab822a2d1f	Add some fixmes	6 years ago
Amber Brown	49af402019	run isort	6 years ago
Amber Brown	77ac14b960	Pass around the reactor explicitly (#3385)	7 years ago
David Baker	7a1af504d7	Remove users from user directory on deactivate	7 years ago
Adrian Tschira	933bf2dd35	replace some iteritems with six ... Signed-off-by: Adrian Tschira <nota@notafile.com>	7 years ago