578 Commits (6e8fb42be7657f9d4958c02d87cff865225714d2)

Author	SHA1	Message	Date
Brendan Abolivier	41ac128fd3	Split multiplart email sending into a dedicated handler (#9977) ... Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>	4 years ago
Richard van der Hoff	03318a766c	Merge pull request from GHSA-x345-32rc-8h85 ... * tests for push rule pattern matching * tests for acl pattern matching * factor out common `re.escape` * Factor out common re.compile * Factor out common anchoring code * add word_boundary support to `glob_to_regex` * Use `glob_to_regex` in push rule evaluator NB that this drops support for character classes. I don't think anyone ever used them. * Improve efficiency of globs with multiple wildcards The idea here is that we compress multiple `*` globs into a single `.*`. We also need to consider `?`, since `*?*` is as hard to implement efficiently as `**`. * add assertion on regex pattern * Fix mypy * Simplify glob_to_regex * Inline the glob_to_regex helper function Signed-off-by: Dan Callahan <danc@element.io> * Moar comments Signed-off-by: Dan Callahan <danc@element.io> Co-authored-by: Dan Callahan <danc@element.io>	4 years ago
Erik Johnston	3853a7edfc	Only store data in caches, not "smart" objects (#9845)	4 years ago
Erik Johnston	177dae2704	Limit length of accepted email addresses (#9855)	4 years ago
Andrew Morgan	71f0623de9	Port "Allow users to click account renewal links multiple times without hitting an 'Invalid Token' page #74" from synapse-dinsic (#9832) ... This attempts to be a direct port of https://github.com/matrix-org/synapse-dinsic/pull/74 to mainline. There was some fiddling required to deal with the changes that have been made to mainline since (mainly dealing with the split of `RegistrationWorkerStore` from `RegistrationStore`, and the changes made to `self.make_request` in test code).	4 years ago
Jonathan de Jong	4b965c862d	Remove redundant "coding: utf-8" lines (#9786) ... Part of #9744 Removes all redundant `# -*- coding: utf-8 -*-` lines from files, as python 3 automatically reads source code as utf-8 now. `Signed-off-by: Jonathan de Jong <jonathan@automatia.nl>`	4 years ago
Patrick Cloke	b7748d3c00	Import HomeServer from the proper module. (#9665)	4 years ago
Ankit Dobhal	d66f9070cd	Fixed code misc. quality issues (#9649) ... - Merge 'isinstance' calls. - Remove unnecessary dict call outside of comprehension. - Use 'sys.exit()' calls.	4 years ago
Patrick Cloke	55da8df078	Fix additional type hints from Twisted 21.2.0. (#9591)	4 years ago
Patrick Cloke	33a02f0f52	Fix additional type hints from Twisted upgrade. (#9518)	4 years ago
Erik Johnston	2566dc57ce	Test that we require validated email for email pushers (#9496)	4 years ago
Erik Johnston	2927921942	Clean up `ShardedWorkerHandlingConfig` (#9466) ... * Split ShardedWorkerHandlingConfig This is so that we have a type level understanding of when it is safe to call `get_instance(..)` (as opposed to `should_handle(..)`). * Remove special cases in ShardedWorkerHandlingConfig. `ShardedWorkerHandlingConfig` tried to handle the various different ways it was possible to configure federation senders and pushers. This led to special cases that weren't hit during testing. To fix this the handling of the different cases is moved from there and `generic_worker` into the worker config class. This allows us to have the logic in one place and allows the rest of the code to ignore the different cases.	4 years ago
Erik Johnston	66f4949e7f	Fix deleting pushers when using sharded pushers. (#9465)	4 years ago
Eric Eastwood	0a00b7ff14	Update black, and run auto formatting over the codebase (#9381) ... - Update black version to the latest - Run black auto formatting over the codebase - Run autoformatting according to [`docs/code_style.md `](80d6dc9783/docs/code_style.md) - Update `code_style.md` docs around installing black to use the correct version	4 years ago
Patrick Cloke	7950aa8a27	Fix some typos.	4 years ago
Patrick Cloke	792263c97c	Handle empty rooms when generating email notifications. (#9257) ... Fixes some exceptions if the room state isn't quite as expected. If the expected state events aren't found, try to find them in the historical room state. If they still aren't found, fallback to a reasonable, although ugly, value.	4 years ago
Patrick Cloke	5d38a3c97f	Refactor email summary generation. (#9260) ... * Fixes a case where no summary text was returned. * The use of messages_from_person vs. messages_from_person_and_others was tweaked to depend on whether there was 1 sender or multiple senders, not based on if there was 1 room or multiple rooms.	4 years ago
Patrick Cloke	e54746bdf7	Clean-up the template loading code. (#9200) ... * Enables autoescape by default for HTML files. * Adds a new read_template method for reading a single template. * Some logic clean-up.	4 years ago
Patrick Cloke	6f7417c3db	Handle missing content keys when calculating presentable names. (#9165) ... Treat the content as untrusted and do not assume it is of the proper form.	4 years ago
Patrick Cloke	23d701864f	Improve the performance of calculating ignored users in large rooms (#9024) ... This allows for efficiently finding which users ignore a particular user. Co-authored-by: Erik Johnston <erik@matrix.org>	4 years ago
Patrick Cloke	37eaf9c272	Fix-up assertions about last stream token in push (#9020) ... The last stream token is always known and we do not need to handle none.	4 years ago
Patrick Cloke	4136255d3c	Ensure that a URL exists in the content during push. (#8965) ... This fixes an KeyError exception, after this PR the content is just considered unknown.	4 years ago
Patrick Cloke	bd30cfe86a	Convert internal pusher dicts to attrs classes. (#8940) ... This improves type hinting and should use less memory.	4 years ago
Patrick Cloke	b3a4b53587	Fix handling of stream tokens for push. (#8943) ... Removes faulty assertions and fixes the logic to ensure the max stream token is always set.	4 years ago
Patrick Cloke	5d34f40d49	Add type hints to the push module. (#8901)	4 years ago
Patrick Cloke	92d87c6882	Add type hints for HTTP and email pushers. (#8880)	4 years ago
Patrick Cloke	02e588856a	Add type hints to the push mailer module. (#8882)	4 years ago
Patrick Cloke	b774c555d8	Add additional validation to pusher URLs. (#8865) ... Pusher URLs now must end in `/_matrix/push/v1/notify` per the specification.	4 years ago
Patrick Cloke	30fba62108	Apply an IP range blacklist to push and key revocation requests. (#8821) ... Replaces the `federation_ip_range_blacklist` configuration setting with an `ip_range_blacklist` setting with wider scope. It now applies to: * Federation * Identity servers * Push notifications * Checking key validitity for third-party invite events The old `federation_ip_range_blacklist` setting is still honored if present, but with reduced scope (it only applies to federation and identity servers).	4 years ago
Andrew Morgan	17fa58bdd1	Add a config option to change whether unread push notification counts are per-message or per-room (#8820) ... This PR adds a new config option to the `push` section of the homeserver config, `group_unread_count_by_room`. By default Synapse will group push notifications by room (so if you have 1000 unread messages, if they lie in 55 rooms, you'll see an unread count on your phone of 55). However, it is also useful to be able to send out the true count of unread messages if desired. If `group_unread_count_by_room` is set to `false`, then with the above example, one would see an unread count of 1000 (email anyone?).	4 years ago
David Baker	59cc2472b3	Add base pushrule to notify for jitsi conferences (#8286) ... This could be customised to trigger a different kind of notification in the future, but for now it's a normal non-highlight one.	4 years ago
Richard van der Hoff	cbc82aa09f	Implement and use an @lru_cache decorator (#8595) ... We don't always need the full power of a DeferredCache.	4 years ago
Erik Johnston	2b7c180879	Start fewer opentracing spans (#8640) ... #8567 started a span for every background process. This is good as it means all Synapse code that gets run should be in a span (unless in the sentinel logging context), but it means we generate about 15x the number of spans as we did previously. This PR attempts to reduce that number by a) not starting one for send commands to Redis, and b) deferring starting background processes until after we're sure they're necessary. I don't really know how much this will help.	4 years ago
Patrick Cloke	34a5696f93	Fix typos and spelling errors. (#8639)	4 years ago
Erik Johnston	db9ef792f0	Fix email notifications for invites without local state. (#8627) ... This can happen if e.g. the room invited into is no longer on the server (or if all users left the room).	4 years ago
Richard van der Hoff	903d11c43a	Add `DeferredCache.get_immediate` method (#8568) ... * Add `DeferredCache.get_immediate` method A bunch of things that are currently calling `DeferredCache.get` are only really interested in the result if it's completed. We can optimise and simplify this case. * Remove unused 'default' parameter to DeferredCache.get() * another get_immediate instance	4 years ago
Patrick Cloke	c356b4bf42	Include a simple message in email notifications that include encrypted content (#8545)	4 years ago
Richard van der Hoff	995cc615a0	Apply suggestions from code review ... Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>	4 years ago
Richard van der Hoff	0ec0bc3886	type annotations for LruCache	4 years ago
Richard van der Hoff	3ee17585cd	Make LruCache register its own metrics (#8561) ... rather than have everything that instantiates an LruCache manage metrics separately, have LruCache do it itself.	4 years ago
Erik Johnston	921a3f8a59	Fix not sending events over federation when using sharded event persisters (#8536) ... * Fix outbound federaion with multiple event persisters. We incorrectly notified federation senders that the minimum persisted stream position had advanced when we got an `RDATA` from an event persister. Notifying of federation senders already correctly happens in the notifier, so we just delete the offending line. * Change some interfaces to use RoomStreamToken. By enforcing use of `RoomStreamTokens` we make it less likely that people pass in random ints that they got from somewhere random.	4 years ago
Patrick Cloke	4ff0201e62	Enable mypy checking for unreachable code and fix instances. (#8432)	4 years ago
Mathieu Velten	916bb9d0d1	Don't push if an user account has expired (#8353)	4 years ago
Patrick Cloke	8a4a4186de	Simplify super() calls to Python 3 syntax. (#8344) ... This converts calls like super(Foo, self) -> super(). Generated with: sed -i "" -Ee 's/super\([^\(]+\)/super()/g' **/*.py	4 years ago
Erik Johnston	5d3e306d9f	Clean up `Notifier.on_new_room_event` code path (#8288) ... The idea here is that we pass the `max_stream_id` to everything, and only use the stream ID of the particular event to figure out *when* the max stream position has caught up to the event and we can notify people about it. This is to maintain the distinction between the position of an item in the stream (i.e. event A has stream ID 513) and a token that can be used to partition the stream (i.e. give me all events after stream ID 352). This distinction becomes important when the tokens are more complicated than a single number, which they will be once we start tracking the position of multiple writers in the tokens. The valid operations here are: 1. Is a position before or after a token 2. Fetching all events between two tokens 3. Merging multiple tokens to get the "max", i.e. `C = max(A, B)` means that for all positions P where P is before A *or* before B, then P is before C. Future PR will change the token type to a dedicated type.	4 years ago
Andrew Morgan	a3a90ee031	Show a confirmation page during user password reset (#8004) ... This PR adds a confirmation step to resetting your user password between clicking the link in your email and your password actually being reset. This is to better align our password reset flow with the industry standard of requiring a confirmation from the user after email validation.	4 years ago
Erik Johnston	c9dbee50ae	Fixup pusher pool notifications (#8287) ... `pusher_pool.on_new_notifications` expected a min and max stream ID, however that was not what we were passing in. Instead, let's just pass it the current max stream ID and have it track the last stream ID it got passed. I believe that it mostly worked as we called the function for every event. However, it would break for events that got persisted out of order, i.e, that were persisted but the max stream ID wasn't incremented as not all preceding events had finished persisting, and push for that event would be delayed until another event got pushed to the effected users.	4 years ago
Erik Johnston	dc9dcdbd59	Revert "Fixup pusher pool notifications" ... This reverts commit e7fd336a53.	4 years ago
Erik Johnston	e7fd336a53	Fixup pusher pool notifications	4 years ago
Brendan Abolivier	d8762cc116	Only add rows to the push actions table if the event notifies or should be marked unread (#8274)	4 years ago