open-dsi
/
postgres
mirror of https://github.com/postgres/postgres
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
104 Commits
38 Branches
662 Tags
735 MiB
Tag: Branch: Tree: 87d132977e
Branches Tags
${ item.name }
Create tag ${ searchTerm }
Create branch ${ searchTerm }
from '87d132977e'
${ noResults }
postgres/sql/move_large_tuples.sql

34 lines
811 B
Raw Normal View History Unescape

Fix decryption of large tuples/TOASTs (#73) Encryption/decryption of the same data should be exact as long as the offset is the same. But as we encode in 16-byte blocks, the size of `encKey` always is a multiple of 16. We start from `aes_block_no`-th index of the encKey[] so N-th will be crypted with the same encKey byte despite what start_offset `pg_tde_crypt()` was called with. For example `start_offset = 10; MAX_AES_ENC_BATCH_KEY_SIZE = 6`: ``` data: [10 11 12 13 14 15 16] encKeys: [...][0 1 2 3 4 5][0 1 2 3 4 5] ``` so the 10th data byte is encoded with the 4th byte of the 2nd encKey etc. We need this shift so each byte will be coded the same despite the initial offset. Let's see the same data but sent to the func starting from the offset 0: ``` data: [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16] encKeys: [0 1 2 3 4 5][0 1 2 3 4 5][ 0 1 2 3 4 5] ``` again, the 10th data byte is encoded with the 4th byte of the 2nd `encKey` etc. The issue in `pg_tde_crypt()` was that along with shifting encKeys in the first batch, we started skipping `aes_block_no` bytes in `data` with every batch after the first one. That lead to: 1. Some bytes remained unencrypted. 2. If data was encrypted and decrypted in different batches these skipped bytes would differ hence decrypted data would be wrong. TOASTed data, for example, is encrypted as one chunk but decrypted in `TOAST_MAX_CHUNK_SIZE` chunks. The issue with pg_tde_move_encrypted_data() was that encryption and decryption were happening in the same loop but `encKeys` may have had different shifting as in and out data may have different start_offset. It wasn't an issue if the data was less than `DATA_BYTES_PER_AES_BATCH`. However, it resulted in data corruption while moving the larger tuples. Plus, it makes sense to reuse `pg_tde_crypt()` instead of copying its tricky code. Also, encKey had a maximum size of 1632 bytes but only 1600 bytes maximum could have been used. We don't need that extra buffer 32 bytes buffer anymore. The same with the `dataLen` in `Aes128EncryptedZeroBlocks2()` - I don't see why do we need that extra block. Fixes https://github.com/Percona-Lab/postgres-tde-ext/issues/72
2 years ago
-- test pg_tde_move_encrypted_data()
CREATE EXTENSION pg_tde;
CREATE TABLE sbtest2(
id SERIAL,
k TEXT STORAGE PLAIN,
PRIMARY KEY (id)
) USING pg_tde;
INSERT INTO sbtest2(k) VALUES(repeat('a', 2500));
INSERT INTO sbtest2(k) VALUES(repeat('b', 2500));
INSERT INTO sbtest2(k) VALUES(repeat('c', 2500));
INSERT INTO sbtest2(k) VALUES(repeat('d', 2500));
INSERT INTO sbtest2(k) VALUES(repeat('e', 2500));
DELETE FROM sbtest2 WHERE id IN (2,3,4);
VACUUM sbtest2;
SELECT * FROM sbtest2;
INSERT INTO sbtest2(k) VALUES(repeat('b', 2500));
INSERT INTO sbtest2(k) VALUES(repeat('c', 2500));
INSERT INTO sbtest2(k) VALUES(repeat('d', 2500));
DELETE FROM sbtest2 WHERE id IN (7);
VACUUM sbtest2;
SELECT * FROM sbtest2;
VACUUM FULL sbtest2;
SELECT * FROM sbtest2;
DROP TABLE sbtest2;
DROP EXTENSION pg_tde;