Since partitioned tables do not have any sotrage and only control the
default access method of their children we should not try to change the
encryption status of anything when changing the AM of a partitioned
table.
Since we only looked at the parent table and not on the whole tree when
setting the status of the encrypted indexes we could easily accidentally
create a plain text index on an encrypted table.
This patch also makes sure to disallow adding indexes to an inheritance
tree where the tables are a mix of encrypted and unecrypted tables.
We removed the ability for the various set_key functions to actually
create new keys and instead have a separate function for that as the API
around "ensure_new_key" was confusing at best.
We no longer run these tests when pg_tde is turned on globally.
There is no reason for us to CREATE IF NOT EXISTS in tests as we should
_know_ what state the database is in when running them.
Add new functions pg_tde_create_key_using_database/global_key_provider()
to create keys instead of key creation being a side effect of setting
the key.
Also remove support for "create if not exists" semantics as any user
should know what keys their key provider contains.
To ensure the tests are always run from the same state we remove any key
provider files so that pg_tde_add_database/global_key_provider_file()
always creates a new file.
updated this topic with:
- added warning to not use it in PROD as it is in beta
- Updated the SELECT parameters to be easier to read
- Removed topic header as it is not necessary and updated the paragraph
introducing the steps
The modified files allow the creation of the pdf for pg_tde
modified:
contrib/pg_tde/documentation/_resource/overrides/partials/banner.html
new file: contrib/pg_tde/documentation/_resourcepdf/overrides/404.html
new file: contrib/pg_tde/documentation/_resourcepdf/overrides/main.html
new file:
contrib/pg_tde/documentation/_resourcepdf/overrides/partials/banner.html
new file:
contrib/pg_tde/documentation/_resourcepdf/overrides/partials/copyright.html
new file:
contrib/pg_tde/documentation/_resourcepdf/overrides/partials/header.html
new file: contrib/pg_tde/documentation/docs/templates/pdf_cover_page.tpl
deleted: contrib/pg_tde/documentation/mkdocs-pdf.yml
modified: contrib/pg_tde/documentation/mkdocs.yml
modified: contrib/pg_tde/documentation/requirements.txt
Co-authored-by: Dragos Andriciuc <dragos.andriciuc@percona.com>
In the output from the key info functions the key_ prefix for the
provider name and id columns doesn't add any value so let's harmonize
them with how we name arguments to the key creation functions.
There was no reason for name and type to be prefixed in the add/change
key provider functions while options was not.
The output from the list_providers functions had them for name and type,
but not id nor options. So remove the prefixes there aswell.
In other functions the prefixes serve to show what parameters is about
the provider and which are about something else.
We're long past the years where someone is gonna read a print-out of
these files and thus need a reminder of what the file name is. And even
if someone wanted to do that it's easy to have your printer program
automatically add the file name to the header or footer of each page
now-a-days.
All different provider types except Vault treated finding no key as
SUCCESS but with NULL as the key. Let's do this for Vault too which
slightly simplifies the callers which used to have to understand both
ways to handle a key not existing.
None of these headers were necessary to include, some because they are
already included by postgres.h and others because we have removed the
functions which we used to use from them.
Add SQL functions that allow user to remove principal key.
* Database level principal key can be removed if there are no encrypted tables or if there is default key. For the first case we just drop key map file completely, for the second we perform key rotation.
* Default principal key can be removed if there are no databases that use it.
Readded the DELETE key function to docs based on 1257 in Architecture chapter where we also update from DROP to DELETE.
Since the only thing our key provider code does with shared memory is
look up a LWLock tranche it is quite a waste of lines of codes to use
our own layer on top of the PostgreSQL shared memory.
As far as I understand you need at least one element to write a literal
which initializes a struct to all zeros. In these particular cases I do
not think it matters since static variables are always zero initialized
but I prefer being clear.
Inspired by how PostgreSQL does it we support our includes in the
following order:
1. postgres.h
2. System headers
3. PostgreSQL headers
4. pg_tde headers
5. Frontend/backend specific headers
Within every section we sort includes in alphabetical order where
possible, which right now was everywhere except in keyring_kmip_impl.c.
There was no point in having the function for attaching the the shared
memory hash table be inline if the non-inlined code had to be called
every time anyway. We still maybe should just stop using an explicit
inline since this is unlikely to be that performace critical code but at
least if we use it we should use it correctly.
There are crash scenarios where keys are left behind in the key file
even though the OID for the table goes unused. This meant that we could
have keys laying around for newly created plaintext relations after OID
wraparound.
Simply removing any existing keys when relations are created seems
appriopriate.
Also move creation of pg_tde data dir to library init. This directory is
used by the SMgr which is loaded regardless of whether any database are
yet to create extension or not.
Only create keys when MAIN fork is created, and trust
tde_smgr_should_encrypt() to know when to encrypt.
Also trust that the key has already been created if we're in recovery or
replication.
Since we can use Fortanix, I added it to the config KMIP section, it
tells the user where to look for more details on setting it up and
provides a bit of examples with pg_tde_add_global_key_provider_kmip
By using OpenTransientFile() we do not have to close file descriptors on
error plus PostgreSQL will check if we have forgot to close any files on
commit.
This change made us find one instance where we had forgot to close a
file which is also fixed in this commit.
Instead of returning NULL when we failed the generate a key we throw an
error like all other places where random fails. Additionally we stop
hardcoding 32 in the assertion.
These comments are repeated many times but add little to no value in
practice since any such issues should be releaved by making sure we test
all these functions.
Since the dev mode of Vault supports generating HTTPs ceritficates we
should just use that since in production everyone will use HTTPS we
should run our tests with HTTPS too.
Andreas Karlsson