196 Commits (bcaec2915ac74937171e27d507b8f9c0e39d3677)

Author	SHA1	Message	Date
Erik Johnston	d72667fcce	Speed up get_current_state_ids ... Using _simple_select_list is fairly expensive for functions that return a lot of rows and/or get called a lot. (This is because it carefully constructs a list of dicts). get_current_state_ids gets called a lot on startup and e.g. when the IRC bridge decided to send tonnes of joins/leaves (as it invalidates the cache). We therefore replace it with a custon txn function that builds up the final result dict without building up and intermediate representation.	8 years ago
Erik Johnston	3ce8d59176	Increase cache size for _get_state_group_for_event	8 years ago
Erik Johnston	d58b1ffe94	Replace some calls to cursor_to_dict ... cursor_to_dict can be surprisinglh expensive for large result sets, so lets only call it when we need to.	8 years ago
Erik Johnston	e71940aa64	Use iter(items|values)	8 years ago
Erik Johnston	00957d1aa4	User Cursor.__iter__ instead of fetchall ... This prevents unnecessary construction of lists	8 years ago
Richard van der Hoff	0c01f829ae	Avoid resetting state on rejected events ... When we get a rejected event, give it the same state_group as its prev_event, rather than no state_group at all. This should fix https://github.com/matrix-org/synapse/issues/1935.	8 years ago
Erik Johnston	8ffbe43ba1	Get current state by using current_state_events table	8 years ago
Richard van der Hoff	fc2f29c1d0	Fix bugs in the /keys/changes api ... * `get_forward_extremeties_for_room` takes a numeric `stream_ordering`. We were passing a `RoomStreamToken`, which meant that it returned the *current* extremities, rather than those corresponding to the `from_token`. However: * `get_state_ids_for_events` required a second ('types') parameter; this meant that a `TypeError` was thrown and we ended up acting as though there was *no* prev state. * `get_state_ids_for_events` actually returns a map from event_id to state dictionary - just looking up the state keys in it again meant that we acted as though there was no prev state. We now check if each member's state has changed since *any* of the extremities. Also add/fix some comments.	8 years ago
Erik Johnston	21b7375778	Add an index to make membership queries faster	8 years ago
Erik Johnston	a55fa2047f	Insert delta of current_state_events to be more efficient	8 years ago
Erik Johnston	897f8752da	Up cache max entries for state	8 years ago
Erik Johnston	01521299c7	Increase cache size limit	8 years ago
Erik Johnston	2fae34bd2c	Optionally measure size of cache by sum of length of values	8 years ago
Matthew Hodgson	8cfc0165e9	fix annoying typos	8 years ago
Erik Johnston	587d8ac60f	Correctly intern keys in state cache	8 years ago
Erik Johnston	4974147aa3	Remove duplication	8 years ago
Erik Johnston	13122e5e24	Remove unused variable	8 years ago
Erik Johnston	cf3e1cc200	Fix perf of fetching state in SQLite	8 years ago
Erik Johnston	00f51493f5	Fix reindex	8 years ago
Erik Johnston	d5ae1f1291	Ensure we don't mutate state cache entries	8 years ago
Erik Johnston	03a98aff3c	Create new index concurrently	8 years ago
Erik Johnston	897d57bc58	Change state fetch query for postgres to be faster ... It turns out that postgres doesn't like doing a list of OR's and is about 1000x slower, so we just issue a query for each specific type seperately.	8 years ago
Erik Johnston	5beda10bbd	Reindex state_groups_state after pruning	8 years ago
Erik Johnston	b568ca309c	Temporarily disable sequential scans for state fetching	8 years ago
Erik Johnston	513188aa56	Comment	8 years ago
Erik Johnston	fadb01551a	Add appopriate framing clause	8 years ago
Erik Johnston	d25c20ccbe	Use windowing function to make use of index	8 years ago
Erik Johnston	0595413c0f	Scale the batch size so that we're not bitten by the minimum	8 years ago
Erik Johnston	a7032abb2e	Correctly handle reindexing state groups that already have an edge	8 years ago
Erik Johnston	70332a12dd	Take value in a better way	8 years ago
Erik Johnston	373654c635	Comment about sqlite and WITH RECURSIVE	8 years ago
Erik Johnston	628e65721b	Add comments	8 years ago
Erik Johnston	a99e933550	Add upgrade script that will slowly prune state_groups_state entries	8 years ago
Erik Johnston	598317927c	Limit the length of state chains	8 years ago
Erik Johnston	9e25443db8	Move to storing state_groups_state as deltas	8 years ago
Erik Johnston	0cfd6c3161	Use state_groups table to test existence	8 years ago
Erik Johnston	c10cb581c6	Correctly handle the difference between prev and current state	8 years ago
Erik Johnston	1bb8ec296d	Generate state group ids in state layer	8 years ago
Erik Johnston	5dc2a702cf	Make _state_groups_id_gen a normal IdGenerator	8 years ago
Erik Johnston	778fa85f47	Make sync not pull out full state	8 years ago
Erik Johnston	a3dc1e9cbe	Replace context.current_state with context.current_state_ids	8 years ago
Erik Johnston	17f4f14df7	Pull out event ids rather than full events for state	8 years ago
Erik Johnston	ba214a5e32	Remove lru option	8 years ago
Mark Haines	1b3c3e6d68	Only update the events and event_json tables for rejected events	8 years ago
Mark Haines	2623cec874	Don't add rejections to the state_group, persist all rejections	8 years ago
Erik Johnston	61c7edfd34	Add cache to _get_state_groups_from_groups	9 years ago
Mark Haines	87f2dec8d4	Make the cache objects be per instance rather than being global	9 years ago
Mark Haines	2a37467fa1	Use google style doc strings. ... pycharm supports them so there is no need to use the other format. Might as well convert the existing strings to reduce the risk of people accidentally cargo culting the wrong doc string format.	9 years ago
Mark Haines	e36bfbab38	Use a stream id generator for backfilled ids	9 years ago
Mark Haines	31a9eceda5	Add a replication stream for state groups	9 years ago