57 Commits (1553adc83122ac245f523524ae1583cd556ed121)

Author	SHA1	Message	Date
Patrick Cloke	c619253db8	Stop sub-classing object (#8249)	4 years ago
Patrick Cloke	37db6252b7	Convert additional databases to async/await part 3 (#8201)	4 years ago
Patrick Cloke	30426c7063	Convert additional database methods to async (select list, search, insert_many, delete_*) (#8168)	4 years ago
Patrick Cloke	dbc630a628	Use the JSON encoder without whitespace in more places. (#8124)	4 years ago
Patrick Cloke	894dae74fe	Convert misc database code to async (#8087)	4 years ago
Erik Johnston	a7bdf98d01	Rename database classes to make some sense (#8033)	4 years ago
Patrick Cloke	f460da6031	Consistently use `db_to_json` to convert from database values to JSON objects. (#7849)	4 years ago
Richard van der Hoff	fcc2de7a0c	Update docstring per review comments	5 years ago
Richard van der Hoff	af47264b78	review comment	5 years ago
Richard van der Hoff	7b608cf468	Only run one background update at a time	5 years ago
Richard van der Hoff	b4c2234232	Make do_next_background_update return a bool ... returning a None or an int that we don't use is confusing.	5 years ago
Richard van der Hoff	26d17b9bdc	Make `has_completed_background_updates` async ... (Almost) everywhere that uses it is happy with an awaitable.	5 years ago
Richard van der Hoff	cfe8c8ab8e	Remove unused `start_background_update` ... This was only used in a unit test, so let's just inline it in the test.	5 years ago
Patrick Cloke	509e381afa	Clarify list/set/dict/tuple comprehensions and enforce via flake8 (#6957) ... Ensure good comprehension hygiene using flake8-comprehensions.	5 years ago
Richard van der Hoff	85db7f73be	Add a background update to clear tombstoned rooms from the directory (#6648) ... * Add a background update to clear tombstoned rooms from the directory * use the ABC metaclass	5 years ago
Richard van der Hoff	9824a39d80	Async/await for background updates (#6647) ... so that bg update routines can be async	5 years ago
Erik Johnston	d537be1ebd	Pass Database into the data store	5 years ago
Erik Johnston	4a33a6dd19	Move background update handling out of store	5 years ago
Erik Johnston	756d4942f5	Move DB pool and helper functions into dedicated Database class	5 years ago
Erik Johnston	ee86abb2d6	Remove underscore from SQLBaseStore functions	5 years ago
Brendan Abolivier	14504ad573	Add CI for synapse_port_db (#6140) ... This adds: * a test sqlite database * a configuration file for the sqlite database * a configuration file for a postgresql database (using the credentials in `.buildkite/docker-compose.pyXX.pgXX.yaml`) as well as a new script named `.buildkite/scripts/test_synapse_port_db.sh` that: 1. installs Synapse 2. updates the test sqlite database to the latest schema and runs background updates on it 3. creates an empty postgresql database 4. run the `synapse_port_db` script to migrate the test sqlite database to the empty postgresql database (with coverage) Step `2` is done via a new script located at `scripts-dev/update_database`. The test sqlite database is extracted from a SyTest run, so that it can be considered as an actual homeserver's database with actual data in it.	5 years ago
Erik Johnston	39b50ad42a	Review comments	5 years ago
Erik Johnston	2135c198d1	Add has_completed_background_update ... This allows checking if a specific background update has completed.	5 years ago
Brendan Abolivier	f02f14e09a	Fix logging	5 years ago
Amber Brown	4806651744	Replace returnValue with return (#5736)	5 years ago
Amber Brown	32e7c9e7f2	Run Black. (#5482)	6 years ago
Amber Brown	7efd1d87c2	Run black on the rest of the storage module (#4996)	6 years ago
Amber Brown	282c97327f	Migrate the user directory initial population to a background task (#4864)	6 years ago
Andrew Morgan	25d64a846a	Fix typos	6 years ago
Richard van der Hoff	667fba68f3	Run things as background processes ... This fixes #3518, and ensures that we get useful logs and metrics for lots of things that happen in the background. (There are certainly more things that happen in the background; these are just the common ones I've found running a single-process synapse locally).	6 years ago
Amber Brown	49af402019	run isort	6 years ago
Amber Brown	6350bf925e	Attempt to be more performant on PyPy (#3462)	7 years ago
Amber Brown	77ac14b960	Pass around the reactor explicitly (#3385)	7 years ago
Erik Johnston	926ba76e23	Replace ujson with simplejson	7 years ago
Richard van der Hoff	6b02fc80d1	Reinstate event_search_postgres_gist handler ... People may have queued updates for this, so we can't just delete it.	7 years ago
Richard van der Hoff	2908f955d1	Check database in has_completed_background_updates ... so that the right thing happens on workers.	7 years ago
Richard van der Hoff	7098b65cb8	Fix error on sqlite 3.7 ... Create the url_cache index on local_media_repository as a background update, so that we can detect whether we are on sqlite or not and create a partial or complete index accordingly. To avoid running the cleanup job before we have built the index, add a bailout which will defer the cleanup if the bg updates are still running. Fixes https://github.com/matrix-org/synapse/issues/2572.	7 years ago
Richard van der Hoff	6cfee09be9	Make __init__ consitstent across Store heirarchy ... Add db_conn parameters to the `__init__` methods of the *Store classes, so that they are all consistent, which makes the multiple inheritance work correctly (and so that we can later extract mixins which can be used in the slavedstores)	7 years ago
Richard van der Hoff	eaaabc6c4f	replace 'except:' with 'except Exception:' ... what could possibly go wrong	7 years ago
Richard van der Hoff	34194aaff7	Don't create event_search index on sqlite ... ... because the table is virtual	8 years ago
Richard van der Hoff	baafb85ba4	Add an index to event_search ... - to make the purge API quicker	8 years ago
Richard van der Hoff	c84770b877	Fix bgupdate error if index already exists (#2167) ... When creating a new table index in the background, guard against it existing already. Fixes https://github.com/matrix-org/synapse/issues/2135. Also, make sure we restore the autocommit flag when we're done, otherwise we get more failures from other operations later on. Fixes https://github.com/matrix-org/synapse/issues/1890 (hopefully).	8 years ago
Richard van der Hoff	994d7ae7c5	Remove broken use of clock.call_later ... background_updates was using `call_later` in a way that leaked the logcontext into the reactor. We could have rewritten it to do it properly, but given that we weren't using the fancier facilities provided by `call_later`, we might as well just use `async.sleep`, which does the logcontext stuff properly.	8 years ago
Erik Johnston	c94de0ab60	Add WHERE clause support to index creation	8 years ago
Erik Johnston	5beda10bbd	Reindex state_groups_state after pruning	8 years ago
Mark Haines	955ef1f06c	fix: defer.returnValue takes one argument	8 years ago
Richard van der Hoff	2ee4c9ee02	background updates: fix assert again	8 years ago
Richard van der Hoff	9dbd903f41	background updates: Fix assertion to do something	8 years ago
Richard van der Hoff	465117d7ca	Fix background_update tests ... A bit of a cleanup for background_updates, and make sure that the real background updates have run before we start the unit tests, so that they don't interfere with the tests.	8 years ago
Richard van der Hoff	ec5717caf5	Create index on user_ips in the background ... user_ips is kinda big, so really we want to add the index in the background once we're running. Replace the schema delta with one which will do that. I've done this in a way that's reasonably easy to reuse as there a few other indexes I need, and I don't suppose they will be the last.	8 years ago