Instead of using ugly hacks like extern and defining kmip_ereoprt() in
a separate file we just move all code which requires the KMIP headers to
a separate file which does not use any PostgreSQL features.
Since we only ever returned false in one odd internal error, i.e.
when something is wrong with the file system, we may as well remove
the returned boolean flag and jsut always raise an error when the
rotation fails.
To protect us against people decrypting relation keys using the wrong
principal key we change to encrypting intenral keys using AES-128-GCM
which verifies that the data was encrypted with the same key as we
decrypt it with. Additionally we make sure to add some of the plain
test data when claculating the AEAD tag.
To imrpove security plus prepare for implementing AES-GCM we save a
unique initialization vector per entry in the key map. This required
refactoring the API a bit but that is a nice thing for GCM too.
Since partitioned tables, and indexes, lack storage the "is encrypted"
property is not relevant to them because encryption is done at the SMGR
level. Therefore we should either throw an error or return NULL, and
here we choose to return NULL to make the function easier to work with.
Since only WAL encryption use the cached context it should not be part
of every internal key. Instead store it in the WAL decryption key cache
and the backend global state for WAL encryption.
It is unclear to me why we stopped encrypting the FSM and visibility
map forks in commit e514ac5cc1 which
sadly does not explain why we stopped doing so. Both of them leak
metadata that we do not have to and the write load to these tables
are not heavy enough to excuse it for perfomrance reasons.
So instead of leaking data and having to jsutify that we jsut encrypt
all forks.
We never took advantage of that the relation keys were stored in two
different files, one for the RelFileNumber and flags and one for the
actual keys so this just complicated the code for almost no gain.
Bumps the version of the file format.
Updating of principal keys has not been used since commit
06885e3559 and the code is broken anyway
due to the byte offset being wrong after writing a header to the map
file. That broken code will be remove in a future commit.
The WAL record has a field for which principal key was used to encrypt
the internal key but it was never initialized. The reaosn it worked is
because if we were lucky enough to have the data on stack be zero we
would treat it as no principal key info was passed and that code path
worked well since the parameter is optional.
Alternatively we could drop the field from the WAL record and pass NULL
instead pg_tde_write_map_entry() since that seems to so far have worked.
smgr_create is called for all forks. It is possible that additional
forks for existing tables are created during a tde creation event.
In practice this happens quite often with CREATE INDEX CONCURRENTLY.
Without this fix, pg_tde created encryption keys for these existing
tables, and later writes and reads tried to use these keys with
all these issues. In practice some of the file got encrypted, some
didn't, and earlier records that weren't encrypted became unreadable.
Andreas Karlsson