655 Commits (8a4b3738f3dcf49f70c08204ad457559dcef4112)

Author	SHA1	Message	Date
Richard van der Hoff	7eb6e39a8f	Record the SSO Auth Provider in the login token (#9510) ... This great big stack of commits is a a whole load of hoop-jumping to make it easier to store additional values in login tokens, and then to actually store the SSO Identity Provider in the login token. (Making use of that data will follow in a subsequent PR.)	4 years ago
Patrick Cloke	aee10768d8	Revert "Fix #8518 (sync requests being cached wrongly on timeout) (#9358)" ... This reverts commit f5c93fc993. This is being backed out due to a regression (#9507) and additional review feedback being provided.	4 years ago
Jonathan de Jong	f5c93fc993	Fix #8518 (sync requests being cached wrongly on timeout) (#9358) ... This fixes #8518 by adding a conditional check on `SyncResult` in a function when `prev_stream_token == current_stream_token`, as a sanity check. In `CachedResponse.set.<remove>()`, the result is immediately popped from the cache if the conditional function returns "false". This prevents the caching of a timed-out `SyncResult` (that has `next_key` as the stream key that produced that `SyncResult`). The cache is prevented from returning a `SyncResult` that makes the client request the same stream key over and over again, effectively making it stuck in a loop of requesting and getting a response immediately for as long as the cache keeps those values. Signed-off-by: Jonathan de Jong <jonathan@automatia.nl>	4 years ago
Eric Eastwood	0a00b7ff14	Update black, and run auto formatting over the codebase (#9381) ... - Update black version to the latest - Run black auto formatting over the codebase - Run autoformatting according to [`docs/code_style.md `](80d6dc9783/docs/code_style.md) - Update `code_style.md` docs around installing black to use the correct version	4 years ago
Richard van der Hoff	3b754aea27	Clean up caching/locking of OIDC metadata load (#9362) ... Ensure that we lock correctly to prevent multiple concurrent metadata load requests, and generally clean up the way we construct the metadata cache.	4 years ago
Patrick Cloke	7950aa8a27	Fix some typos.	4 years ago
Patrick Cloke	e40d88cff3	Backout changes for automatically calculating the public baseurl. (#9313) ... This breaks some people's configurations (if their Client-Server API is not accessed via port 443).	4 years ago
Patrick Cloke	3f58fc848d	Type hints and validation improvements. (#9321) ... * Adds type hints to the groups servlet and stringutils code. * Assert the maximum length of some input values for spec compliance.	4 years ago
Richard van der Hoff	4167494c90	Replace username picker with a template (#9275) ... There's some prelimiary work here to pull out the construction of a jinja environment to a separate function. I wanted to load the template at display time rather than load time, so that it's easy to update on the fly. Honestly, I think we should do this with all our templates: the risk of ending up with malformed templates is far outweighed by the improved turnaround time for an admin trying to update them.	4 years ago
Andrew Morgan	a64c29926e	Pass a dict, instead of None, to modules if a None config is specified in the homeserver config (#9229) ... If a Synapse module's config block were empty in YAML, thus being translated to a `Nonetype` in Python, then some modules could fail as that None ends up getting passed to their `parse_config` method. Modules are expected to accept a `dict` instead. This PR ensures that if the user does end up specifying an empty config block (such as what [the default oidc config in the sample config](5310808d3b/docs/sample_config.yaml (L1816-L1845)) states) then `None` is not passed to the module. An empty dict is passed instead. This code assumes that no existing modules are relying on receiving a `None` config block, but I'd really hope that they aren't.	4 years ago
Erik Johnston	056327457f	Fix chain cover update to handle events with duplicate auth events (#9210)	4 years ago
Richard van der Hoff	0cd2938bc8	Support icons for Identity Providers (#9154)	4 years ago
Richard van der Hoff	9ffac2bef1	Remote dependency on distutils (#9125) ... `distutils` is pretty much deprecated these days, and replaced with `setuptools`. It's also annoying because it's you can't `pip install` it, and it's hard to figure out which debian package we should depend on to make sure it's there. Since we only use it for a tiny function anyway, let's just vendor said function into our codebase.	4 years ago
Erik Johnston	1a08e0cdab	Fix event chain bg update. (#9118) ... We passed in a graph to `sorted_topologically` which didn't have an entry for each node (as we dropped nodes with no edges).	4 years ago
Erik Johnston	1315a2e8be	Use a chain cover index to efficiently calculate auth chain difference (#8868)	4 years ago
Erik Johnston	a03d71dc9d	Fix "Starting metrics collection from sentinel context" errors (#9053)	4 years ago
Patrick Cloke	1b4d5d6acf	Empty iterables should count towards cache usage. (#9028)	4 years ago
Patrick Cloke	06fefe0bb1	Add type hints to the logging context code. (#8939)	4 years ago
David Teller	f14428b25c	Allow spam-checker modules to be provide async methods. (#8890) ... Spam checker modules can now provide async methods. This is implemented in a backwards-compatible manner.	4 years ago
Richard van der Hoff	ab7a24cc6b	Better formatting for config errors from modules (#8874) ... The idea is that the parse_config method of extension modules can raise either a ConfigError or a JsonValidationError, and it will be magically turned into a legible error message. There's a few components to it: * Separating the "path" and the "message" parts of a ConfigError, so that we can fiddle with the path bit to turn it into an absolute path. * Generally improving the way ConfigErrors get printed. * Passing in the config path to load_module so that it can wrap any exceptions that get caught appropriately.	4 years ago
Richard van der Hoff	90c900a8ff	Merge pull request #8678 from matrix-org/rav/fix_frozen_events ... Fix serialisation errors when using third-party event rules.	4 years ago
Richard van der Hoff	cbc82aa09f	Implement and use an @lru_cache decorator (#8595) ... We don't always need the full power of a DeferredCache.	4 years ago
Richard van der Hoff	b6ca69e4f1	Remove frozendict_json_encoder and support frozendicts everywhere ... Not being able to serialise `frozendicts` is fragile, and it's annoying to have to think about which serialiser you want. There's no real downside to supporting frozendicts, so let's just have one json encoder.	4 years ago
Dan Callahan	aff1eb7c67	Tell Black to format code for Python 3.5 (#8664) ... This allows trailing commas in multi-line arg lists. Minor, but we might as well keep our formatting current with regard to our minimum supported Python version. Signed-off-by: Dan Callahan <danc@element.io>	4 years ago
Richard van der Hoff	b28aaeb3a5	Optimise CacheDescriptor (#8594) ... don't bother constricting a CacheContext unless we need one.	4 years ago
Richard van der Hoff	c13820bcee	fix failure case	4 years ago
Richard van der Hoff	2b3af01791	optimise DeferredCache.set	4 years ago
Richard van der Hoff	1f4269700c	Push some deferred wrangling down into DeferredCache	4 years ago
Richard van der Hoff	96e7d3c4a0	Fix 'LruCache' object has no attribute '_on_resize' (#8591) ... We need to make sure we are readu for the `set_cache_factor` callback.	4 years ago
Richard van der Hoff	903d11c43a	Add `DeferredCache.get_immediate` method (#8568) ... * Add `DeferredCache.get_immediate` method A bunch of things that are currently calling `DeferredCache.get` are only really interested in the result if it's completed. We can optimise and simplify this case. * Remove unused 'default' parameter to DeferredCache.get() * another get_immediate instance	4 years ago
Richard van der Hoff	97647b33c2	Replace DeferredCache with LruCache where possible (#8563) ... Most of these uses don't need a full-blown DeferredCache; LruCache is lighter and more appropriate.	4 years ago
Richard van der Hoff	6d7b22041d	review comments	4 years ago
Richard van der Hoff	995cc615a0	Apply suggestions from code review ... Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>	4 years ago
Richard van der Hoff	0ec0bc3886	type annotations for LruCache	4 years ago
Richard van der Hoff	3ee17585cd	Make LruCache register its own metrics (#8561) ... rather than have everything that instantiates an LruCache manage metrics separately, have LruCache do it itself.	4 years ago
Richard van der Hoff	8075504a60	Enable mypy for synapse.util.caches (#8547) ... This seemed to entail dragging in a type stub for SortedList.	4 years ago
Richard van der Hoff	4182bb812f	move DeferredCache into its own module	4 years ago
Richard van der Hoff	9f87da0a84	Rename Cache->DeferredCache	4 years ago
Richard van der Hoff	7eff59ec91	Add some more type annotations to Cache	4 years ago
Patrick Cloke	1781bbe319	Add type hints to response cache. (#8507)	4 years ago
Richard van der Hoff	b28bfd905d	Clarify error message when plugin config parsers raise an error (#8492) ... This turns: Failed to parse config for 'myplugin': Exception('error message') into: Failed to parse config for 'myplugin': error message.	4 years ago
Richard van der Hoff	ba700074c6	Expose a `get_resource_usage` method in `Measure`	4 years ago
Richard van der Hoff	1c262431f9	Fix handling of connection timeouts in outgoing http requests (#8400) ... * Remove `on_timeout_cancel` from `timeout_deferred` The `on_timeout_cancel` param to `timeout_deferred` wasn't always called on a timeout (in particular if the canceller raised an exception), so it was unreliable. It was also only used in one place, and to be honest it's easier to do what it does a different way. * Fix handling of connection timeouts in outgoing http requests Turns out that if we get a timeout during connection, then a different exception is raised, which wasn't always handled correctly. To fix it, catch the exception in SimpleHttpClient and turn it into a RequestTimedOutError (which is already a documented exception). Also add a description to RequestTimedOutError so that we can see which stage it failed at. * Fix incorrect handling of timeouts reading federation responses This was trapping the wrong sort of TimeoutError, so was never being hit. The effect was relatively minor, but we should fix this so that it does the expected thing. * Fix inconsistent handling of `timeout` param between methods `get_json`, `put_json` and `delete_json` were applying a different timeout to the response body to `post_json`; bring them in line and test. Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com> Co-authored-by: Erik Johnston <erik@matrix.org>	4 years ago
Patrick Cloke	8a4a4186de	Simplify super() calls to Python 3 syntax. (#8344) ... This converts calls like super(Foo, self) -> super(). Generated with: sed -i "" -Ee 's/super\([^\(]+\)/super()/g' **/*.py	4 years ago
Jonathan de Jong	837293c314	Remove obsolete __future__ imports (#8337)	4 years ago
Patrick Cloke	aec294ee0d	Use slots in attrs classes where possible (#8296) ... slots use less memory (and attribute access is faster) while slightly limiting the flexibility of the class attributes. This focuses on objects which are instantiated "often" and for short periods of time.	4 years ago
Patrick Cloke	b86764662b	Fix the exception that is raised when invalid JSON is encountered. (#8291)	4 years ago
Patrick Cloke	2ea1c68249	Remove some unused distributor signals (#8216) ... Removes the `user_joined_room` and stops calling it since there are no observers. Also cleans-up some other unused signals and related code.	4 years ago
Patrick Cloke	e45b834119	Add types to async_helpers (#8260)	4 years ago
Patrick Cloke	72bec36d50	Directly import json from the standard library. (#8259) ... By importing from canonicaljson the simplejson module was still being used in some situations. After this change the std lib json is consistenty used throughout Synapse.	4 years ago