Now that we have removed tde_heap_basic we may as well remove all tests
specific to its properties. These tests give us a false sense of test
coverage since they do not actually test things related to SMGR-level
encryption.
Copy the check from pg_checksums which adds some sanity checking by
refusing to start if the cluster is already running. It takes no
lock or anything like that so the cluster could be start while we
run this tool but since it is quick the risk is pretty smaller,
certainly smaller than for pg_checksums.
By using RESET ROLE we can test permissions even without using TAP.
We skip porting the part which tests if we can read tables since
there is nothing in our code which could even affect that so testing
it makes little sense.
Previusly the tool was named pg_tde_alter_key_provider, but in the
help message it referred to itself as pg_tde_modify_key_provider.
Instead of any of that, the name should align with the online
function, which i pg_tde_change_key_provider.
This commit renames the tool and addresses the help message.
Also, the help message is now more detailed to make the parameters
cleaner.
This tool should be considered part of our contrib module similar to
how pglogical considers pglogical_create_subscriber. It is a bit
confusing that we have to use SCRIPTS_built when building with make
but it works.
Since we always call the get functions with throw_error = false and
always call the set functions with throw_error = true the code can be
simplified by hardcoding the logic instead of having it configurable.
This commit changes the approach to WAL encryption. Instead of
encrypting each WAL page and keeping its header unencrypted with a special
encrypted flag, this change moves encrypted/unencrypted bookkeeping to
WAL internal keys. Now every [WAL] internal key has an additional field
`start_lsn`. The field indicates the first WAL record that was encrypted
with this key. This means everything starting from that LSN is encrypted
with the key until the next or the end of the WAL. In order to have
unencrypted WAL (when the user sets `pg_tde.wal_encrypt = off`), we insert
a special key with the flag `TDE_KEY_TYPE_WAL_UNENCRYPTED`.
The user can turn WAL encryption on and off, which will generate a new WAL
key with the respective state (`TDE_KEY_TYPE_WAL_ENCRYPTED` or
`TDE_KEY_TYPE_WAL_UNENCRYPTED `). If GUC pg_tde.wal_encrypt was changed,
the server will generate a new WAL key with `start_lsn` set to
`InvalidXLogRecPtr` on start. WAL writer, in turn, will update `start_lsn`
with the actual LSN on the first write since the key creation.
We use the current key _map and _dat files infrastructure along with the
Internal key cache but with some special cases. There might be multiple
internal keys for WAL but only one for the SMGR (relations, indexes etc).
Creating a new WAL key, we write it the same as the SMGR key, so key
rotation, for example, doesn't require any changes. But reads and start_lsn
happen directly from/in _dat file (omitting _map). This needs revision and
refactoring (along with _map, _dat files in general). As well as WAL keys
cache, which is currently a simple linked list referencing the actual
internal key case. That allows WAL key changing.
Changed the get started to global provider setting
Moved multi-tenant setup to a How to doc
Moved WAL encryption to a separate doc and Get started step
Updated functions description
modified: documentation/docs/apt.md
modified: documentation/docs/functions.md
modified: documentation/docs/install.md
new file: documentation/docs/multi-tenant-setup.md
modified: documentation/docs/setup.md
modified: documentation/docs/switch.md
new file: documentation/docs/wal-encryption.md
modified: documentation/docs/yum.md
modified: documentation/mkdocs.yml
new file: documentation/snippets/kms-considerations.md
The collision of variable and field names vs typedefs caused various
weird indentation issues so we rename the two typedefs to avoid
these collisions. This also reduces the diff between our stuff and
the upstream.
Instead of checking for the oid of the AM we check for if the
access method of the relation uses the routine struct as the
heap does. This allows the tools to be support all access
methods which are simple copies of heap.
I am uncertain if this is something which the project would be
interested in but it decreases the size of our diff against
upstream.
We check if a file is encrypted by looking at if it has a storage
manager encryption key for the relation file and if it has a key
we skip it.
These lookups require use to retrieve the tablespace and database
oids from the path to to the relation file.
- Do not assume that people always will need xlog and smgr
- Use an inline function instead of a macro
- Make the initialization of the path more explicit
Sequences are not updated after ALTER TABLE ... SET ACCESS METHOD.
This commit adds a workaround by executing a dummy ALTER SEQUENCE
on all sequences owned by the table in the post event trigger,
correcting their encryption status.
While not allowing the strings to be NULL simplified validation slightly
it made other parts of the code more complex and introduced a security
issue in the form of the strncpy() uses. And while we could have fixed
the security issue this code is cleaner.
Andreas Karlsson