27 Commits (87b111f96a244cd212e8fff7f1a0f661fdd8073e)

Author	SHA1	Message	Date
Richard van der Hoff	667fba68f3	Run things as background processes ... This fixes #3518, and ensures that we get useful logs and metrics for lots of things that happen in the background. (There are certainly more things that happen in the background; these are just the common ones I've found running a single-process synapse locally).	6 years ago
Amber Brown	49af402019	run isort	6 years ago
Amber Brown	6350bf925e	Attempt to be more performant on PyPy (#3462)	7 years ago
Amber Brown	77ac14b960	Pass around the reactor explicitly (#3385)	7 years ago
Erik Johnston	926ba76e23	Replace ujson with simplejson	7 years ago
Richard van der Hoff	6b02fc80d1	Reinstate event_search_postgres_gist handler ... People may have queued updates for this, so we can't just delete it.	7 years ago
Richard van der Hoff	2908f955d1	Check database in has_completed_background_updates ... so that the right thing happens on workers.	7 years ago
Richard van der Hoff	7098b65cb8	Fix error on sqlite 3.7 ... Create the url_cache index on local_media_repository as a background update, so that we can detect whether we are on sqlite or not and create a partial or complete index accordingly. To avoid running the cleanup job before we have built the index, add a bailout which will defer the cleanup if the bg updates are still running. Fixes https://github.com/matrix-org/synapse/issues/2572.	7 years ago
Richard van der Hoff	6cfee09be9	Make __init__ consitstent across Store heirarchy ... Add db_conn parameters to the `__init__` methods of the *Store classes, so that they are all consistent, which makes the multiple inheritance work correctly (and so that we can later extract mixins which can be used in the slavedstores)	7 years ago
Richard van der Hoff	eaaabc6c4f	replace 'except:' with 'except Exception:' ... what could possibly go wrong	7 years ago
Richard van der Hoff	34194aaff7	Don't create event_search index on sqlite ... ... because the table is virtual	8 years ago
Richard van der Hoff	baafb85ba4	Add an index to event_search ... - to make the purge API quicker	8 years ago
Richard van der Hoff	c84770b877	Fix bgupdate error if index already exists (#2167) ... When creating a new table index in the background, guard against it existing already. Fixes https://github.com/matrix-org/synapse/issues/2135. Also, make sure we restore the autocommit flag when we're done, otherwise we get more failures from other operations later on. Fixes https://github.com/matrix-org/synapse/issues/1890 (hopefully).	8 years ago
Richard van der Hoff	994d7ae7c5	Remove broken use of clock.call_later ... background_updates was using `call_later` in a way that leaked the logcontext into the reactor. We could have rewritten it to do it properly, but given that we weren't using the fancier facilities provided by `call_later`, we might as well just use `async.sleep`, which does the logcontext stuff properly.	8 years ago
Erik Johnston	c94de0ab60	Add WHERE clause support to index creation	8 years ago
Erik Johnston	5beda10bbd	Reindex state_groups_state after pruning	8 years ago
Mark Haines	955ef1f06c	fix: defer.returnValue takes one argument	8 years ago
Richard van der Hoff	2ee4c9ee02	background updates: fix assert again	8 years ago
Richard van der Hoff	9dbd903f41	background updates: Fix assertion to do something	8 years ago
Richard van der Hoff	465117d7ca	Fix background_update tests ... A bit of a cleanup for background_updates, and make sure that the real background updates have run before we start the unit tests, so that they don't interfere with the tests.	8 years ago
Richard van der Hoff	ec5717caf5	Create index on user_ips in the background ... user_ips is kinda big, so really we want to add the index in the background once we're running. Replace the schema delta with one which will do that. I've done this in a way that's reasonably easy to reuse as there a few other indexes I need, and I don't suppose they will be the last.	8 years ago
Erik Johnston	183cacac90	Simplify query and handle finishing correctly	9 years ago
Matthew Hodgson	6c28ac260c	copyrights	9 years ago
Mark Haines	940a161192	Fix the background update	9 years ago
Mark Haines	a412b9a465	Run the background updates when starting synapse.	9 years ago
Mark Haines	c6a01f2ed0	Add storage module for tracking background updates. ... The progress for each background update is stored as a JSON blob in the database. Each background update is broken up into separate batches. The batch size is automatically tuned to try avoid blocking single threaded databases for too long.	9 years ago