4 Commits (07fe8e49cddfb20394dd50b1330a0336e30ef8b7)

Author	SHA1	Message	Date
Andrew Pogrebnoi	07fe8e49cd	Fix decryption of large tuples/TOASTs (#73) ... Encryption/decryption of the same data should be exact as long as the offset is the same. But as we encode in 16-byte blocks, the size of `encKey` always is a multiple of 16. We start from `aes_block_no`-th index of the encKey[] so N-th will be crypted with the same encKey byte despite what start_offset `pg_tde_crypt()` was called with. For example `start_offset = 10; MAX_AES_ENC_BATCH_KEY_SIZE = 6`: ``` data: [10 11 12 13 14 15 16] encKeys: [...][0 1 2 3 4 5][0 1 2 3 4 5] ``` so the 10th data byte is encoded with the 4th byte of the 2nd encKey etc. We need this shift so each byte will be coded the same despite the initial offset. Let's see the same data but sent to the func starting from the offset 0: ``` data: [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16] encKeys: [0 1 2 3 4 5][0 1 2 3 4 5][ 0 1 2 3 4 5] ``` again, the 10th data byte is encoded with the 4th byte of the 2nd `encKey` etc. The issue in `pg_tde_crypt()` was that along with shifting encKeys in the first batch, we started skipping `aes_block_no` bytes in `data` with every batch after the first one. That lead to: 1. Some bytes remained unencrypted. 2. If data was encrypted and decrypted in different batches these skipped bytes would differ hence decrypted data would be wrong. TOASTed data, for example, is encrypted as one chunk but decrypted in `TOAST_MAX_CHUNK_SIZE` chunks. The issue with pg_tde_move_encrypted_data() was that encryption and decryption were happening in the same loop but `encKeys` may have had different shifting as in and out data may have different start_offset. It wasn't an issue if the data was less than `DATA_BYTES_PER_AES_BATCH`. However, it resulted in data corruption while moving the larger tuples. Plus, it makes sense to reuse `pg_tde_crypt()` instead of copying its tricky code. Also, encKey had a maximum size of 1632 bytes but only 1600 bytes maximum could have been used. We don't need that extra buffer 32 bytes buffer anymore. The same with the `dataLen` in `Aes128EncryptedZeroBlocks2()` - I don't see why do we need that extra block. Fixes https://github.com/Percona-Lab/postgres-tde-ext/issues/72	2 years ago
Zsolt Parragi	2b7ecbe042	Fix update crash after code cleanup (#67) ... Issue: the code cleanup introduced in PR #52 modified the original tuple in the update method instead of decrypting the tuple data into a copy. This caused data curruption crashes in some test. Fix: reintroduce the missing palloc to the update method. Fixes #61 Fixes #62 Fixes #64	2 years ago
Andrew Pogrebnoi	a4711f682b	XLog: replicate fork file and encrypt tuples (#59) ... 1. Inserts and Updates now are encrypted in WAL. We encrypt new tuples directly in Buffer after they were insrerted there. To pass it to XLog we could memcpy Buffer data into into the tuple. But later tuple has to be unencrypted for index instertions etc. So we pass directly data from the Buffer into XLog. 2. Log into WAL and replicate *.tde forks creation. 3. Added docker-compose for the streaming replication test setup. (not perfect - needs two `up -d` in a row to start the secondary) 4. Added tests for multi inserts. Need tests for replications though.	2 years ago
Zsolt Parragi	4c95782aa3	Encrypt fork files with master key and improve tests ... Fork file encryption was missing from the previous pull requests, resulting in the server not initializing the keyring on normal executions, and because of this, missing if the keyring configuration wasn't specified at all. Fixes #46	2 years ago