11 Commits (595f28529caa9a5a785666dde2f9dc35ff075ece)

Author	SHA1	Message	Date
Erik Johnston	66f4949e7f	Fix deleting pushers when using sharded pushers. (#9465)	4 years ago
Erik Johnston	8de3703d21	Make event persisters periodically announce position over replication. (#8499) ... Currently background proccesses stream the events stream use the "minimum persisted position" (i.e. `get_current_token()`) rather than the vector clock style tokens. This is broadly fine as it doesn't matter if the background processes lag a small amount. However, in extreme cases (i.e. SyTests) where we only write to one event persister the background processes will never make progress. This PR changes it so that the `MultiWriterIDGenerator` keeps the current position of a given instance as up to date as possible (i.e using the latest token it sees if its not in the process of persisting anything), and then periodically announces that over replication. This then allows the "minimum persisted position" to advance, albeit with a small lag.	4 years ago
Erik Johnston	d7983b63a6	Support any process writing to cache invalidation stream. (#7436)	5 years ago
Erik Johnston	37f6823f5b	Add instance name to RDATA/POSITION commands (#7364) ... This is primarily for allowing us to send those commands from workers, but for now simply allows us to ignore echoed RDATA/POSITION commands that we sent (we get echoes of sent commands when using redis). Currently we log a WARNING on the master process every time we receive an echoed RDATA.	5 years ago
Richard van der Hoff	71a1abb8a1	Stop the master relaying USER_SYNC for other workers (#7318) ... Long story short: if we're handling presence on the current worker, we shouldn't be sending USER_SYNC commands over replication. In an attempt to figure out what is going on here, I ended up refactoring some bits of the presencehandler code, so the first 4 commits here are non-functional refactors to move this code slightly closer to sanity. (There's still plenty to do here :/). Suggest reviewing individual commits. Fixes (I hope) #7257.	5 years ago
Erik Johnston	4f21c33be3	Remove usage of "conn_id" for presence. (#7128) ... * Remove `conn_id` usage for UserSyncCommand. Each tcp replication connection is assigned a "conn_id", which is used to give an ID to a remotely connected worker. In a redis world, there will no longer be a one to one mapping between connection and instance, so instead we need to replace such usages with an ID generated by the remote instances and included in the replicaiton commands. This really only effects UserSyncCommand. * Add CLEAR_USER_SYNCS command that is sent on shutdown. This should help with the case where a synchrotron gets restarted gracefully, rather than rely on 5 minute timeout.	5 years ago
Erik Johnston	4cff617df1	Move catchup of replication streams to worker. (#7024) ... This changes the replication protocol so that the server does not send down `RDATA` for rows that happened before the client connected. Instead, the server will send a `POSITION` and clients then query the database (or master out of band) to get up to date.	5 years ago
Erik Johnston	5d7a6ad223	Allow streaming cache invalidate all to workers. (#6749)	5 years ago
Erik Johnston	a8a50f5b57	Wake up transaction queue when remote server comes back online (#6706) ... This will be used to retry outbound transactions to a remote server if we think it might have come back up.	5 years ago
Richard van der Hoff	cc6243b4c0	document the REPLICATE command a bit better (#6305) ... since I found myself wonder how it works	5 years ago
dstipp	379d2a8c39	(#5849) Convert rst to markdown (#6040) ... Converting some of the rst documentation to markdown. Attempted to preserve whitespace and line breaks to minimize cosmetic change.	5 years ago