|
|
|
|
/*
|
|
|
|
|
* check.c
|
|
|
|
|
*
|
|
|
|
|
* server checks and output routines
|
|
|
|
|
*
|
|
|
|
|
* Copyright (c) 2010-2024, PostgreSQL Global Development Group
|
|
|
|
|
* src/bin/pg_upgrade/check.c
|
|
|
|
|
*/
|
|
|
|
|
|
Create libpgcommon, and move pg_malloc et al to it
libpgcommon is a new static library to allow sharing code among the
various frontend programs and backend; this lets us eliminate duplicate
implementations of common routines. We avoid libpgport, because that's
intended as a place for porting issues; per discussion, it seems better
to keep them separate.
The first use case, and the only implemented by this patch, is pg_malloc
and friends, which many frontend programs were already using.
At the same time, we can use this to provide palloc emulation functions
for the frontend; this way, some palloc-using files in the backend can
also be used by the frontend cleanly. To do this, we change palloc() in
the backend to be a function instead of a macro on top of
MemoryContextAlloc(). This was previously believed to cause loss of
performance, but this implementation has been tweaked by Tom and Andres
so that on modern compilers it provides a slight improvement over the
previous one.
This lets us clean up some places that were already with
localized hacks.
Most of the pg_malloc/palloc changes in this patch were authored by
Andres Freund. Zoltán Böszörményi also independently provided a form of
that. libpgcommon infrastructure was authored by Álvaro.
13 years ago
|
|
|
#include "postgres_fe.h"
|
|
|
|
|
|
|
|
|
|
#include "catalog/pg_authid_d.h"
|
|
|
|
|
#include "catalog/pg_class_d.h"
|
|
|
|
|
#include "catalog/pg_collation.h"
|
|
|
|
|
#include "fe_utils/string_utils.h"
|
Be forgiving of variant spellings of locale names in pg_upgrade.
Even though the server tries to canonicalize stored locale names, the
platform often doesn't cooperate, so it's entirely possible that one DB
thinks its locale is, say, "en_US.UTF-8" while the other has "en_US.utf8".
Rather than failing, we should try to allow this where it's clearly OK.
There is already pretty robust encoding lookup in encnames.c, so make
use of that to compare the encoding parts of the names. The locale
identifier parts are just compared case-insensitively, which we were
already doing. The major problem known to exist in the field is variant
encoding-name spellings, so hopefully this will be Good Enough. If not,
we can try being even laxer.
Pavel Raiskup, reviewed by Rushabh Lathia
12 years ago
|
|
|
#include "mb/pg_wchar.h"
|
|
|
|
|
#include "pg_upgrade.h"
|
|
|
|
|
|
|
|
|
|
static void check_new_cluster_is_empty(void);
|
|
|
|
|
static void check_is_install_user(ClusterInfo *cluster);
|
|
|
|
|
static void check_proper_datallowconn(ClusterInfo *cluster);
|
|
|
|
|
static void check_for_prepared_transactions(ClusterInfo *cluster);
|
|
|
|
|
static void check_for_isn_and_int8_passing_mismatch(ClusterInfo *cluster);
|
Remove support for postfix (right-unary) operators.
This feature has been a thorn in our sides for a long time, causing
many grammatical ambiguity problems. It doesn't seem worth the
pain to continue to support it, so remove it.
There are some follow-on improvements we can make in the grammar,
but this commit only removes the bare minimum number of productions,
plus assorted backend support code.
Note that pg_dump and psql continue to have full support, since
they may be used against older servers. However, pg_dump warns
about postfix operators. There is also a check in pg_upgrade.
Documentation-wise, I (tgl) largely removed the "left unary"
terminology in favor of saying "prefix operator", which is
a more standard and IMO less confusing term.
I included a catversion bump, although no initial catalog data
changes here, to mark the boundary at which oprkind = 'r'
stopped being valid in pg_operator.
Mark Dilger, based on work by myself and Robert Haas;
review by John Naylor
Discussion: https://postgr.es/m/38ca86db-42ab-9b48-2902-337a0d6b8311@2ndquadrant.com
5 years ago
|
|
|
static void check_for_user_defined_postfix_ops(ClusterInfo *cluster);
|
|
|
|
|
static void check_for_incompatible_polymorphics(ClusterInfo *cluster);
|
Remove WITH OIDS support, change oid catalog column visibility.
Previously tables declared WITH OIDS, including a significant fraction
of the catalog tables, stored the oid column not as a normal column,
but as part of the tuple header.
This special column was not shown by default, which was somewhat odd,
as it's often (consider e.g. pg_class.oid) one of the more important
parts of a row. Neither pg_dump nor COPY included the contents of the
oid column by default.
The fact that the oid column was not an ordinary column necessitated a
significant amount of special case code to support oid columns. That
already was painful for the existing, but upcoming work aiming to make
table storage pluggable, would have required expanding and duplicating
that "specialness" significantly.
WITH OIDS has been deprecated since 2005 (commit ff02d0a05280e0).
Remove it.
Removing includes:
- CREATE TABLE and ALTER TABLE syntax for declaring the table to be
WITH OIDS has been removed (WITH (oids[ = true]) will error out)
- pg_dump does not support dumping tables declared WITH OIDS and will
issue a warning when dumping one (and ignore the oid column).
- restoring an pg_dump archive with pg_restore will warn when
restoring a table with oid contents (and ignore the oid column)
- COPY will refuse to load binary dump that includes oids.
- pg_upgrade will error out when encountering tables declared WITH
OIDS, they have to be altered to remove the oid column first.
- Functionality to access the oid of the last inserted row (like
plpgsql's RESULT_OID, spi's SPI_lastoid, ...) has been removed.
The syntax for declaring a table WITHOUT OIDS (or WITH (oids = false)
for CREATE TABLE) is still supported. While that requires a bit of
support code, it seems unnecessary to break applications / dumps that
do not use oids, and are explicit about not using them.
The biggest user of WITH OID columns was postgres' catalog. This
commit changes all 'magic' oid columns to be columns that are normally
declared and stored. To reduce unnecessary query breakage all the
newly added columns are still named 'oid', even if a table's column
naming scheme would indicate 'reloid' or such. This obviously
requires adapting a lot code, mostly replacing oid access via
HeapTupleGetOid() with access to the underlying Form_pg_*->oid column.
The bootstrap process now assigns oids for all oid columns in
genbki.pl that do not have an explicit value (starting at the largest
oid previously used), only oids assigned later by oids will be above
FirstBootstrapObjectId. As the oid column now is a normal column the
special bootstrap syntax for oids has been removed.
Oids are not automatically assigned during insertion anymore, all
backend code explicitly assigns oids with GetNewOidWithIndex(). For
the rare case that insertions into the catalog via SQL are called for
the new pg_nextoid() function can be used (which only works on catalog
tables).
The fact that oid columns on system tables are now normal columns
means that they will be included in the set of columns expanded
by * (i.e. SELECT * FROM pg_class will now include the table's oid,
previously it did not). It'd not technically be hard to hide oid
column by default, but that'd mean confusing behavior would either
have to be carried forward forever, or it'd cause breakage down the
line.
While it's not unlikely that further adjustments are needed, the
scope/invasiveness of the patch makes it worthwhile to get merge this
now. It's painful to maintain externally, too complicated to commit
after the code code freeze, and a dependency of a number of other
patches.
Catversion bump, for obvious reasons.
Author: Andres Freund, with contributions by John Naylor
Discussion: https://postgr.es/m/20180930034810.ywp2c7awz7opzcfr@alap3.anarazel.de
7 years ago
|
|
|
static void check_for_tables_with_oids(ClusterInfo *cluster);
|
|
|
|
|
static void check_for_pg_role_prefix(ClusterInfo *cluster);
|
|
|
|
|
static void check_for_new_tablespace_dir(void);
|
|
|
|
|
static void check_for_user_defined_encoding_conversions(ClusterInfo *cluster);
|
Migrate logical slots to the new node during an upgrade.
While reading information from the old cluster, a list of logical
slots is fetched. At the later part of upgrading, pg_upgrade revisits the
list and restores slots by executing pg_create_logical_replication_slot()
on the new cluster. Migration of logical replication slots is only
supported when the old cluster is version 17.0 or later.
If the old node has invalid slots or slots with unconsumed WAL records,
the pg_upgrade fails. These checks are needed to prevent data loss.
The significant advantage of this commit is that it makes it easy to
continue logical replication even after upgrading the publisher node.
Previously, pg_upgrade allowed copying publications to a new node. With
this patch, adjusting the connection string to the new publisher will
cause the apply worker on the subscriber to connect to the new publisher
automatically. This enables seamless continuation of logical replication,
even after an upgrade.
Author: Hayato Kuroda, Hou Zhijie
Reviewed-by: Peter Smith, Bharath Rupireddy, Dilip Kumar, Vignesh C, Shlok Kyal
Discussion: http://postgr.es/m/TYAPR01MB58664C81887B3AF2EB6B16E3F5939@TYAPR01MB5866.jpnprd01.prod.outlook.com
Discussion: http://postgr.es/m/CAA4eK1+t7xYcfa0rEQw839=b2MzsfvYDPz3xbD+ZqOdP3zpKYg@mail.gmail.com
2 years ago
|
|
|
static void check_new_cluster_logical_replication_slots(void);
|
Allow upgrades to preserve the full subscription's state.
This feature will allow us to replicate the changes on subscriber nodes
after the upgrade.
Previously, only the subscription metadata information was preserved.
Without the list of relations and their state, it's not possible to
re-enable the subscriptions without missing some records as the list of
relations can only be refreshed after enabling the subscription (and
therefore starting the apply worker). Even if we added a way to refresh
the subscription while enabling a publication, we still wouldn't know
which relations are new on the publication side, and therefore should be
fully synced, and which shouldn't.
To preserve the subscription relations, this patch teaches pg_dump to
restore the content of pg_subscription_rel from the old cluster by using
binary_upgrade_add_sub_rel_state SQL function. This is supported only
in binary upgrade mode.
The subscription's replication origin is needed to ensure that we don't
replicate anything twice.
To preserve the replication origins, this patch teaches pg_dump to update
the replication origin along with creating a subscription by using
binary_upgrade_replorigin_advance SQL function to restore the
underlying replication origin remote LSN. This is supported only in
binary upgrade mode.
pg_upgrade will check that all the subscription relations are in 'i'
(init) or in 'r' (ready) state and will error out if that's not the case,
logging the reason for the failure. This helps to avoid the risk of any
dangling slot or origin after the upgrade.
Author: Vignesh C, Julien Rouhaud, Shlok Kyal
Reviewed-by: Peter Smith, Masahiko Sawada, Michael Paquier, Amit Kapila, Hayato Kuroda
Discussion: https://postgr.es/m/20230217075433.u5mjly4d5cr4hcfe@jrouhaud
2 years ago
|
|
|
static void check_new_cluster_subscription_configuration(void);
|
|
|
|
|
static void check_old_cluster_for_valid_slots(void);
|
Allow upgrades to preserve the full subscription's state.
This feature will allow us to replicate the changes on subscriber nodes
after the upgrade.
Previously, only the subscription metadata information was preserved.
Without the list of relations and their state, it's not possible to
re-enable the subscriptions without missing some records as the list of
relations can only be refreshed after enabling the subscription (and
therefore starting the apply worker). Even if we added a way to refresh
the subscription while enabling a publication, we still wouldn't know
which relations are new on the publication side, and therefore should be
fully synced, and which shouldn't.
To preserve the subscription relations, this patch teaches pg_dump to
restore the content of pg_subscription_rel from the old cluster by using
binary_upgrade_add_sub_rel_state SQL function. This is supported only
in binary upgrade mode.
The subscription's replication origin is needed to ensure that we don't
replicate anything twice.
To preserve the replication origins, this patch teaches pg_dump to update
the replication origin along with creating a subscription by using
binary_upgrade_replorigin_advance SQL function to restore the
underlying replication origin remote LSN. This is supported only in
binary upgrade mode.
pg_upgrade will check that all the subscription relations are in 'i'
(init) or in 'r' (ready) state and will error out if that's not the case,
logging the reason for the failure. This helps to avoid the risk of any
dangling slot or origin after the upgrade.
Author: Vignesh C, Julien Rouhaud, Shlok Kyal
Reviewed-by: Peter Smith, Masahiko Sawada, Michael Paquier, Amit Kapila, Hayato Kuroda
Discussion: https://postgr.es/m/20230217075433.u5mjly4d5cr4hcfe@jrouhaud
2 years ago
|
|
|
static void check_old_cluster_subscription_state(void);
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* DataTypesUsageChecks - definitions of data type checks for the old cluster
|
|
|
|
|
* in order to determine if an upgrade can be performed. See the comment on
|
|
|
|
|
* data_types_usage_checks below for a more detailed description.
|
|
|
|
|
*/
|
|
|
|
|
typedef struct
|
|
|
|
|
{
|
|
|
|
|
/* Status line to print to the user */
|
|
|
|
|
const char *status;
|
|
|
|
|
/* Filename to store report to */
|
|
|
|
|
const char *report_filename;
|
|
|
|
|
/* Query to extract the oid of the datatype */
|
|
|
|
|
const char *base_query;
|
|
|
|
|
/* Text to store to report in case of error */
|
|
|
|
|
const char *report_text;
|
|
|
|
|
/* The latest version where the check applies */
|
|
|
|
|
int threshold_version;
|
|
|
|
|
/* A function pointer for determining if the check applies */
|
|
|
|
|
DataTypesUsageVersionCheck version_hook;
|
|
|
|
|
} DataTypesUsageChecks;
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Special values for threshold_version for indicating that a check applies to
|
|
|
|
|
* all versions, or that a custom function needs to be invoked to determine
|
|
|
|
|
* if the check applies.
|
|
|
|
|
*/
|
|
|
|
|
#define MANUAL_CHECK 1
|
|
|
|
|
#define ALL_VERSIONS -1
|
|
|
|
|
|
|
|
|
|
/*--
|
|
|
|
|
* Data type usage checks. Each check for problematic data type usage is
|
|
|
|
|
* defined in this array with metadata, SQL query for finding the data type
|
|
|
|
|
* and functionality for deciding if the check is applicable to the version
|
|
|
|
|
* of the old cluster. The struct members are described in detail below:
|
|
|
|
|
*
|
|
|
|
|
* status A oneline string which can be printed to the user to
|
|
|
|
|
* inform about progress. Should not end with newline.
|
|
|
|
|
* report_filename The filename in which the list of problems detected by
|
|
|
|
|
* the check will be printed.
|
|
|
|
|
* base_query A query which extracts the Oid of the datatype checked
|
|
|
|
|
* for.
|
|
|
|
|
* report_text The text which will be printed to the user to explain
|
|
|
|
|
* what the check did, and why it failed. The text should
|
|
|
|
|
* end with a newline, and does not need to refer to the
|
|
|
|
|
* report_filename as that is automatically appended to
|
|
|
|
|
* the report with the path to the log folder.
|
|
|
|
|
* threshold_version The major version of PostgreSQL for which to run the
|
|
|
|
|
* check. Iff the old cluster is less than, or equal to,
|
|
|
|
|
* the threshold version then the check will be executed.
|
|
|
|
|
* If the old version is greater than the threshold then
|
|
|
|
|
* the check is skipped. If the threshold_version is set
|
|
|
|
|
* to ALL_VERSIONS then it will be run unconditionally,
|
|
|
|
|
* if set to MANUAL_CHECK then the version_hook function
|
|
|
|
|
* will be executed in order to determine whether or not
|
|
|
|
|
* to run.
|
|
|
|
|
* version_hook A function pointer to a version check function of type
|
|
|
|
|
* DataTypesUsageVersionCheck which is used to determine
|
|
|
|
|
* if the check is applicable to the old cluster. If the
|
|
|
|
|
* version_hook returns true then the check will be run,
|
|
|
|
|
* else it will be skipped. The function will only be
|
|
|
|
|
* executed iff threshold_version is set to MANUAL_CHECK.
|
|
|
|
|
*/
|
|
|
|
|
static DataTypesUsageChecks data_types_usage_checks[] =
|
|
|
|
|
{
|
|
|
|
|
/*
|
|
|
|
|
* Look for composite types that were made during initdb *or* belong to
|
|
|
|
|
* information_schema; that's important in case information_schema was
|
|
|
|
|
* dropped and reloaded.
|
|
|
|
|
*
|
|
|
|
|
* The cutoff OID here should match the source cluster's value of
|
|
|
|
|
* FirstNormalObjectId. We hardcode it rather than using that C #define
|
|
|
|
|
* because, if that #define is ever changed, our own version's value is
|
|
|
|
|
* NOT what to use. Eventually we may need a test on the source cluster's
|
|
|
|
|
* version to select the correct value.
|
|
|
|
|
*/
|
|
|
|
|
{
|
|
|
|
|
.status = gettext_noop("Checking for system-defined composite types in user tables"),
|
|
|
|
|
.report_filename = "tables_using_composite.txt",
|
|
|
|
|
.base_query =
|
|
|
|
|
"SELECT t.oid FROM pg_catalog.pg_type t "
|
|
|
|
|
"LEFT JOIN pg_catalog.pg_namespace n ON t.typnamespace = n.oid "
|
|
|
|
|
" WHERE typtype = 'c' AND (t.oid < 16384 OR nspname = 'information_schema')",
|
|
|
|
|
.report_text =
|
|
|
|
|
gettext_noop("Your installation contains system-defined composite types in user tables.\n"
|
|
|
|
|
"These type OIDs are not stable across PostgreSQL versions,\n"
|
|
|
|
|
"so this cluster cannot currently be upgraded. You can drop the\n"
|
|
|
|
|
"problem columns and restart the upgrade.\n"),
|
|
|
|
|
.threshold_version = ALL_VERSIONS
|
|
|
|
|
},
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* 9.3 -> 9.4 Fully implement the 'line' data type in 9.4, which
|
|
|
|
|
* previously returned "not enabled" by default and was only functionally
|
|
|
|
|
* enabled with a compile-time switch; as of 9.4 "line" has a different
|
|
|
|
|
* on-disk representation format.
|
|
|
|
|
*/
|
|
|
|
|
{
|
|
|
|
|
.status = gettext_noop("Checking for incompatible \"line\" data type"),
|
|
|
|
|
.report_filename = "tables_using_line.txt",
|
|
|
|
|
.base_query =
|
|
|
|
|
"SELECT 'pg_catalog.line'::pg_catalog.regtype AS oid",
|
|
|
|
|
.report_text =
|
|
|
|
|
gettext_noop("Your installation contains the \"line\" data type in user tables.\n"
|
|
|
|
|
"This data type changed its internal and input/output format\n"
|
|
|
|
|
"between your old and new versions so this\n"
|
|
|
|
|
"cluster cannot currently be upgraded. You can\n"
|
|
|
|
|
"drop the problem columns and restart the upgrade.\n"),
|
|
|
|
|
.threshold_version = 903
|
|
|
|
|
},
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* pg_upgrade only preserves these system values: pg_class.oid pg_type.oid
|
|
|
|
|
* pg_enum.oid
|
|
|
|
|
*
|
|
|
|
|
* Many of the reg* data types reference system catalog info that is not
|
|
|
|
|
* preserved, and hence these data types cannot be used in user tables
|
|
|
|
|
* upgraded by pg_upgrade.
|
|
|
|
|
*/
|
|
|
|
|
{
|
|
|
|
|
.status = gettext_noop("Checking for reg* data types in user tables"),
|
|
|
|
|
.report_filename = "tables_using_reg.txt",
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Note: older servers will not have all of these reg* types, so we
|
|
|
|
|
* have to write the query like this rather than depending on casts to
|
|
|
|
|
* regtype.
|
|
|
|
|
*/
|
|
|
|
|
.base_query =
|
|
|
|
|
"SELECT oid FROM pg_catalog.pg_type t "
|
|
|
|
|
"WHERE t.typnamespace = "
|
|
|
|
|
" (SELECT oid FROM pg_catalog.pg_namespace "
|
|
|
|
|
" WHERE nspname = 'pg_catalog') "
|
|
|
|
|
" AND t.typname IN ( "
|
|
|
|
|
/* pg_class.oid is preserved, so 'regclass' is OK */
|
|
|
|
|
" 'regcollation', "
|
|
|
|
|
" 'regconfig', "
|
|
|
|
|
" 'regdictionary', "
|
|
|
|
|
" 'regnamespace', "
|
|
|
|
|
" 'regoper', "
|
|
|
|
|
" 'regoperator', "
|
|
|
|
|
" 'regproc', "
|
|
|
|
|
" 'regprocedure' "
|
|
|
|
|
/* pg_authid.oid is preserved, so 'regrole' is OK */
|
|
|
|
|
/* pg_type.oid is (mostly) preserved, so 'regtype' is OK */
|
|
|
|
|
" )",
|
|
|
|
|
.report_text =
|
|
|
|
|
gettext_noop("Your installation contains one of the reg* data types in user tables.\n"
|
|
|
|
|
"These data types reference system OIDs that are not preserved by\n"
|
|
|
|
|
"pg_upgrade, so this cluster cannot currently be upgraded. You can\n"
|
|
|
|
|
"drop the problem columns and restart the upgrade.\n"),
|
|
|
|
|
.threshold_version = ALL_VERSIONS
|
|
|
|
|
},
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* PG 16 increased the size of the 'aclitem' type, which breaks the
|
|
|
|
|
* on-disk format for existing data.
|
|
|
|
|
*/
|
|
|
|
|
{
|
|
|
|
|
.status = gettext_noop("Checking for incompatible \"aclitem\" data type"),
|
|
|
|
|
.report_filename = "tables_using_aclitem.txt",
|
|
|
|
|
.base_query =
|
|
|
|
|
"SELECT 'pg_catalog.aclitem'::pg_catalog.regtype AS oid",
|
|
|
|
|
.report_text =
|
|
|
|
|
gettext_noop("Your installation contains the \"aclitem\" data type in user tables.\n"
|
|
|
|
|
"The internal format of \"aclitem\" changed in PostgreSQL version 16\n"
|
|
|
|
|
"so this cluster cannot currently be upgraded. You can drop the\n"
|
|
|
|
|
"problem columns and restart the upgrade.\n"),
|
|
|
|
|
.threshold_version = 1500
|
|
|
|
|
},
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* It's no longer allowed to create tables or views with "unknown"-type
|
|
|
|
|
* columns. We do not complain about views with such columns, because
|
|
|
|
|
* they should get silently converted to "text" columns during the DDL
|
|
|
|
|
* dump and reload; it seems unlikely to be worth making users do that by
|
|
|
|
|
* hand. However, if there's a table with such a column, the DDL reload
|
|
|
|
|
* will fail, so we should pre-detect that rather than failing
|
|
|
|
|
* mid-upgrade. Worse, if there's a matview with such a column, the DDL
|
|
|
|
|
* reload will silently change it to "text" which won't match the on-disk
|
|
|
|
|
* storage (which is like "cstring"). So we *must* reject that.
|
|
|
|
|
*/
|
|
|
|
|
{
|
|
|
|
|
.status = gettext_noop("Checking for invalid \"unknown\" user columns"),
|
|
|
|
|
.report_filename = "tables_using_unknown.txt",
|
|
|
|
|
.base_query =
|
|
|
|
|
"SELECT 'pg_catalog.unknown'::pg_catalog.regtype AS oid",
|
|
|
|
|
.report_text =
|
|
|
|
|
gettext_noop("Your installation contains the \"unknown\" data type in user tables.\n"
|
|
|
|
|
"This data type is no longer allowed in tables, so this cluster\n"
|
|
|
|
|
"cannot currently be upgraded. You can drop the problem columns\n"
|
|
|
|
|
"and restart the upgrade.\n"),
|
|
|
|
|
.threshold_version = 906
|
|
|
|
|
},
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* PG 12 changed the 'sql_identifier' type storage to be based on name,
|
|
|
|
|
* not varchar, which breaks on-disk format for existing data. So we need
|
|
|
|
|
* to prevent upgrade when used in user objects (tables, indexes, ...). In
|
|
|
|
|
* 12, the sql_identifier data type was switched from name to varchar,
|
|
|
|
|
* which does affect the storage (name is by-ref, but not varlena). This
|
|
|
|
|
* means user tables using sql_identifier for columns are broken because
|
|
|
|
|
* the on-disk format is different.
|
|
|
|
|
*/
|
|
|
|
|
{
|
|
|
|
|
.status = gettext_noop("Checking for invalid \"sql_identifier\" user columns"),
|
|
|
|
|
.report_filename = "tables_using_sql_identifier.txt",
|
|
|
|
|
.base_query =
|
|
|
|
|
"SELECT 'information_schema.sql_identifier'::pg_catalog.regtype AS oid",
|
|
|
|
|
.report_text =
|
|
|
|
|
gettext_noop("Your installation contains the \"sql_identifier\" data type in user tables.\n"
|
|
|
|
|
"The on-disk format for this data type has changed, so this\n"
|
|
|
|
|
"cluster cannot currently be upgraded. You can drop the problem\n"
|
|
|
|
|
"columns and restart the upgrade.\n"),
|
|
|
|
|
.threshold_version = 1100
|
|
|
|
|
},
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* JSONB changed its storage format during 9.4 beta, so check for it.
|
|
|
|
|
*/
|
|
|
|
|
{
|
|
|
|
|
.status = gettext_noop("Checking for incompatible \"jsonb\" data type in user tables"),
|
|
|
|
|
.report_filename = "tables_using_jsonb.txt",
|
|
|
|
|
.base_query =
|
|
|
|
|
"SELECT 'pg_catalog.jsonb'::pg_catalog.regtype AS oid",
|
|
|
|
|
.report_text =
|
|
|
|
|
gettext_noop("Your installation contains the \"jsonb\" data type in user tables.\n"
|
|
|
|
|
"The internal format of \"jsonb\" changed during 9.4 beta so this\n"
|
|
|
|
|
"cluster cannot currently be upgraded. You can drop the problem \n"
|
|
|
|
|
"columns and restart the upgrade.\n"),
|
|
|
|
|
.threshold_version = MANUAL_CHECK,
|
|
|
|
|
.version_hook = jsonb_9_4_check_applicable
|
|
|
|
|
},
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* PG 12 removed types abstime, reltime, tinterval.
|
|
|
|
|
*/
|
|
|
|
|
{
|
|
|
|
|
.status = gettext_noop("Checking for removed \"abstime\" data type in user tables"),
|
|
|
|
|
.report_filename = "tables_using_abstime.txt",
|
|
|
|
|
.base_query =
|
|
|
|
|
"SELECT 'pg_catalog.abstime'::pg_catalog.regtype AS oid",
|
|
|
|
|
.report_text =
|
|
|
|
|
gettext_noop("Your installation contains the \"abstime\" data type in user tables.\n"
|
|
|
|
|
"The \"abstime\" type has been removed in PostgreSQL version 12,\n"
|
|
|
|
|
"so this cluster cannot currently be upgraded. You can drop the\n"
|
|
|
|
|
"problem columns, or change them to another data type, and restart\n"
|
|
|
|
|
"the upgrade.\n"),
|
|
|
|
|
.threshold_version = 1100
|
|
|
|
|
},
|
|
|
|
|
{
|
|
|
|
|
.status = gettext_noop("Checking for removed \"reltime\" data type in user tables"),
|
|
|
|
|
.report_filename = "tables_using_reltime.txt",
|
|
|
|
|
.base_query =
|
|
|
|
|
"SELECT 'pg_catalog.reltime'::pg_catalog.regtype AS oid",
|
|
|
|
|
.report_text =
|
|
|
|
|
gettext_noop("Your installation contains the \"reltime\" data type in user tables.\n"
|
|
|
|
|
"The \"reltime\" type has been removed in PostgreSQL version 12,\n"
|
|
|
|
|
"so this cluster cannot currently be upgraded. You can drop the\n"
|
|
|
|
|
"problem columns, or change them to another data type, and restart\n"
|
|
|
|
|
"the upgrade.\n"),
|
|
|
|
|
.threshold_version = 1100
|
|
|
|
|
},
|
|
|
|
|
{
|
|
|
|
|
.status = gettext_noop("Checking for removed \"tinterval\" data type in user tables"),
|
|
|
|
|
.report_filename = "tables_using_tinterval.txt",
|
|
|
|
|
.base_query =
|
|
|
|
|
"SELECT 'pg_catalog.tinterval'::pg_catalog.regtype AS oid",
|
|
|
|
|
.report_text =
|
|
|
|
|
gettext_noop("Your installation contains the \"tinterval\" data type in user tables.\n"
|
|
|
|
|
"The \"tinterval\" type has been removed in PostgreSQL version 12,\n"
|
|
|
|
|
"so this cluster cannot currently be upgraded. You can drop the\n"
|
|
|
|
|
"problem columns, or change them to another data type, and restart\n"
|
|
|
|
|
"the upgrade.\n"),
|
|
|
|
|
.threshold_version = 1100
|
|
|
|
|
},
|
|
|
|
|
|
|
|
|
|
/* End of checks marker, must remain last */
|
|
|
|
|
{
|
|
|
|
|
NULL, NULL, NULL, NULL, 0, NULL
|
|
|
|
|
}
|
|
|
|
|
};
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Private state for check_for_data_types_usage()'s UpgradeTask.
|
|
|
|
|
*/
|
|
|
|
|
struct data_type_check_state
|
|
|
|
|
{
|
|
|
|
|
DataTypesUsageChecks *check; /* the check for this step */
|
|
|
|
|
bool *result; /* true if check failed for any database */
|
|
|
|
|
PQExpBuffer *report; /* buffer for report on failed checks */
|
|
|
|
|
};
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Returns a palloc'd query string for the data type check, for use by
|
|
|
|
|
* check_for_data_types_usage()'s UpgradeTask.
|
|
|
|
|
*/
|
|
|
|
|
static char *
|
|
|
|
|
data_type_check_query(int checknum)
|
|
|
|
|
{
|
|
|
|
|
DataTypesUsageChecks *check = &data_types_usage_checks[checknum];
|
|
|
|
|
|
|
|
|
|
return psprintf("WITH RECURSIVE oids AS ( "
|
|
|
|
|
/* start with the type(s) returned by base_query */
|
|
|
|
|
" %s "
|
|
|
|
|
" UNION ALL "
|
|
|
|
|
" SELECT * FROM ( "
|
|
|
|
|
/* inner WITH because we can only reference the CTE once */
|
|
|
|
|
" WITH x AS (SELECT oid FROM oids) "
|
|
|
|
|
/* domains on any type selected so far */
|
|
|
|
|
" SELECT t.oid FROM pg_catalog.pg_type t, x WHERE typbasetype = x.oid AND typtype = 'd' "
|
|
|
|
|
" UNION ALL "
|
|
|
|
|
/* arrays over any type selected so far */
|
|
|
|
|
" SELECT t.oid FROM pg_catalog.pg_type t, x WHERE typelem = x.oid AND typtype = 'b' "
|
|
|
|
|
" UNION ALL "
|
|
|
|
|
/* composite types containing any type selected so far */
|
|
|
|
|
" SELECT t.oid FROM pg_catalog.pg_type t, pg_catalog.pg_class c, pg_catalog.pg_attribute a, x "
|
|
|
|
|
" WHERE t.typtype = 'c' AND "
|
|
|
|
|
" t.oid = c.reltype AND "
|
|
|
|
|
" c.oid = a.attrelid AND "
|
|
|
|
|
" NOT a.attisdropped AND "
|
|
|
|
|
" a.atttypid = x.oid "
|
|
|
|
|
" UNION ALL "
|
|
|
|
|
/* ranges containing any type selected so far */
|
|
|
|
|
" SELECT t.oid FROM pg_catalog.pg_type t, pg_catalog.pg_range r, x "
|
|
|
|
|
" WHERE t.typtype = 'r' AND r.rngtypid = t.oid AND r.rngsubtype = x.oid"
|
|
|
|
|
" ) foo "
|
|
|
|
|
") "
|
|
|
|
|
/* now look for stored columns of any such type */
|
|
|
|
|
"SELECT n.nspname, c.relname, a.attname "
|
|
|
|
|
"FROM pg_catalog.pg_class c, "
|
|
|
|
|
" pg_catalog.pg_namespace n, "
|
|
|
|
|
" pg_catalog.pg_attribute a "
|
|
|
|
|
"WHERE c.oid = a.attrelid AND "
|
|
|
|
|
" NOT a.attisdropped AND "
|
|
|
|
|
" a.atttypid IN (SELECT oid FROM oids) AND "
|
|
|
|
|
" c.relkind IN ("
|
|
|
|
|
CppAsString2(RELKIND_RELATION) ", "
|
|
|
|
|
CppAsString2(RELKIND_MATVIEW) ", "
|
|
|
|
|
CppAsString2(RELKIND_INDEX) ") AND "
|
|
|
|
|
" c.relnamespace = n.oid AND "
|
|
|
|
|
/* exclude possible orphaned temp tables */
|
|
|
|
|
" n.nspname !~ '^pg_temp_' AND "
|
|
|
|
|
" n.nspname !~ '^pg_toast_temp_' AND "
|
|
|
|
|
/* exclude system catalogs, too */
|
|
|
|
|
" n.nspname NOT IN ('pg_catalog', 'information_schema')",
|
|
|
|
|
check->base_query);
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Callback function for processing results of queries for
|
|
|
|
|
* check_for_data_types_usage()'s UpgradeTask. If the query returned any rows
|
|
|
|
|
* (i.e., the check failed), write the details to the report file.
|
|
|
|
|
*/
|
|
|
|
|
static void
|
|
|
|
|
process_data_type_check(DbInfo *dbinfo, PGresult *res, void *arg)
|
|
|
|
|
{
|
|
|
|
|
struct data_type_check_state *state = (struct data_type_check_state *) arg;
|
|
|
|
|
int ntups = PQntuples(res);
|
|
|
|
|
|
|
|
|
|
AssertVariableIsOfType(&process_data_type_check, UpgradeTaskProcessCB);
|
|
|
|
|
|
|
|
|
|
if (ntups)
|
|
|
|
|
{
|
|
|
|
|
char output_path[MAXPGPATH];
|
|
|
|
|
int i_nspname;
|
|
|
|
|
int i_relname;
|
|
|
|
|
int i_attname;
|
|
|
|
|
FILE *script = NULL;
|
|
|
|
|
bool db_used = false;
|
|
|
|
|
|
|
|
|
|
snprintf(output_path, sizeof(output_path), "%s/%s",
|
|
|
|
|
log_opts.basedir,
|
|
|
|
|
state->check->report_filename);
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Make sure we have a buffer to save reports to now that we found a
|
|
|
|
|
* first failing check.
|
|
|
|
|
*/
|
|
|
|
|
if (*state->report == NULL)
|
|
|
|
|
*state->report = createPQExpBuffer();
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* If this is the first time we see an error for the check in question
|
|
|
|
|
* then print a status message of the failure.
|
|
|
|
|
*/
|
|
|
|
|
if (!(*state->result))
|
|
|
|
|
{
|
|
|
|
|
pg_log(PG_REPORT, " failed check: %s", _(state->check->status));
|
|
|
|
|
appendPQExpBuffer(*state->report, "\n%s\n%s %s\n",
|
|
|
|
|
_(state->check->report_text),
|
|
|
|
|
_("A list of the problem columns is in the file:"),
|
|
|
|
|
output_path);
|
|
|
|
|
}
|
|
|
|
|
*state->result = true;
|
|
|
|
|
|
|
|
|
|
i_nspname = PQfnumber(res, "nspname");
|
|
|
|
|
i_relname = PQfnumber(res, "relname");
|
|
|
|
|
i_attname = PQfnumber(res, "attname");
|
|
|
|
|
|
|
|
|
|
for (int rowno = 0; rowno < ntups; rowno++)
|
|
|
|
|
{
|
|
|
|
|
if (script == NULL && (script = fopen_priv(output_path, "a")) == NULL)
|
|
|
|
|
pg_fatal("could not open file \"%s\": %m", output_path);
|
|
|
|
|
|
|
|
|
|
if (!db_used)
|
|
|
|
|
{
|
|
|
|
|
fprintf(script, "In database: %s\n", dbinfo->db_name);
|
|
|
|
|
db_used = true;
|
|
|
|
|
}
|
|
|
|
|
fprintf(script, " %s.%s.%s\n",
|
|
|
|
|
PQgetvalue(res, rowno, i_nspname),
|
|
|
|
|
PQgetvalue(res, rowno, i_relname),
|
|
|
|
|
PQgetvalue(res, rowno, i_attname));
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
if (script)
|
|
|
|
|
{
|
|
|
|
|
fclose(script);
|
|
|
|
|
script = NULL;
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* check_for_data_types_usage()
|
|
|
|
|
* Detect whether there are any stored columns depending on given type(s)
|
|
|
|
|
*
|
|
|
|
|
* If so, write a report to the given file name and signal a failure to the
|
|
|
|
|
* user.
|
|
|
|
|
*
|
|
|
|
|
* The checks to run are defined in a DataTypesUsageChecks structure where
|
|
|
|
|
* each check has a metadata for explaining errors to the user, a base_query,
|
|
|
|
|
* a report filename and a function pointer hook for validating if the check
|
|
|
|
|
* should be executed given the cluster at hand.
|
|
|
|
|
*
|
|
|
|
|
* base_query should be a SELECT yielding a single column named "oid",
|
|
|
|
|
* containing the pg_type OIDs of one or more types that are known to have
|
|
|
|
|
* inconsistent on-disk representations across server versions.
|
|
|
|
|
*
|
|
|
|
|
* We check for the type(s) in tables, matviews, and indexes, but not views;
|
|
|
|
|
* there's no storage involved in a view.
|
|
|
|
|
*/
|
|
|
|
|
static void
|
|
|
|
|
check_for_data_types_usage(ClusterInfo *cluster)
|
|
|
|
|
{
|
|
|
|
|
bool *results;
|
|
|
|
|
PQExpBuffer report = NULL;
|
|
|
|
|
DataTypesUsageChecks *tmp = data_types_usage_checks;
|
|
|
|
|
int n_data_types_usage_checks = 0;
|
|
|
|
|
UpgradeTask *task = upgrade_task_create();
|
|
|
|
|
char **queries = NULL;
|
|
|
|
|
struct data_type_check_state *states;
|
|
|
|
|
|
|
|
|
|
prep_status("Checking data type usage");
|
|
|
|
|
|
|
|
|
|
/* Gather number of checks to perform */
|
|
|
|
|
while (tmp->status != NULL)
|
|
|
|
|
{
|
|
|
|
|
n_data_types_usage_checks++;
|
|
|
|
|
tmp++;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/* Prepare an array to store the results of checks in */
|
|
|
|
|
results = pg_malloc0(sizeof(bool) * n_data_types_usage_checks);
|
|
|
|
|
queries = pg_malloc0(sizeof(char *) * n_data_types_usage_checks);
|
|
|
|
|
states = pg_malloc0(sizeof(struct data_type_check_state) * n_data_types_usage_checks);
|
|
|
|
|
|
|
|
|
|
for (int i = 0; i < n_data_types_usage_checks; i++)
|
|
|
|
|
{
|
|
|
|
|
DataTypesUsageChecks *check = &data_types_usage_checks[i];
|
|
|
|
|
|
|
|
|
|
if (check->threshold_version == MANUAL_CHECK)
|
|
|
|
|
{
|
|
|
|
|
Assert(check->version_hook);
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Make sure that the check applies to the current cluster version
|
|
|
|
|
* and skip it if not.
|
|
|
|
|
*/
|
|
|
|
|
if (!check->version_hook(cluster))
|
|
|
|
|
continue;
|
|
|
|
|
}
|
|
|
|
|
else if (check->threshold_version != ALL_VERSIONS)
|
|
|
|
|
{
|
|
|
|
|
if (GET_MAJOR_VERSION(cluster->major_version) > check->threshold_version)
|
|
|
|
|
continue;
|
|
|
|
|
}
|
|
|
|
|
else
|
|
|
|
|
Assert(check->threshold_version == ALL_VERSIONS);
|
|
|
|
|
|
|
|
|
|
queries[i] = data_type_check_query(i);
|
|
|
|
|
|
|
|
|
|
states[i].check = check;
|
|
|
|
|
states[i].result = &results[i];
|
|
|
|
|
states[i].report = &report;
|
|
|
|
|
|
|
|
|
|
upgrade_task_add_step(task, queries[i], process_data_type_check,
|
|
|
|
|
true, &states[i]);
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Connect to each database in the cluster and run all defined checks
|
|
|
|
|
* against that database before trying the next one.
|
|
|
|
|
*/
|
|
|
|
|
upgrade_task_run(task, cluster);
|
|
|
|
|
upgrade_task_free(task);
|
|
|
|
|
|
|
|
|
|
if (report)
|
|
|
|
|
{
|
|
|
|
|
pg_fatal("Data type checks failed: %s", report->data);
|
|
|
|
|
destroyPQExpBuffer(report);
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
pg_free(results);
|
|
|
|
|
for (int i = 0; i < n_data_types_usage_checks; i++)
|
|
|
|
|
{
|
|
|
|
|
if (queries[i])
|
|
|
|
|
pg_free(queries[i]);
|
|
|
|
|
}
|
|
|
|
|
pg_free(queries);
|
|
|
|
|
pg_free(states);
|
|
|
|
|
|
|
|
|
|
check_ok();
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* fix_path_separator
|
|
|
|
|
* For non-Windows, just return the argument.
|
|
|
|
|
* For Windows convert any forward slash to a backslash
|
|
|
|
|
* such as is suitable for arguments to builtin commands
|
|
|
|
|
* like RMDIR and DEL.
|
|
|
|
|
*/
|
|
|
|
|
static char *
|
|
|
|
|
fix_path_separator(char *path)
|
|
|
|
|
{
|
|
|
|
|
#ifdef WIN32
|
|
|
|
|
|
|
|
|
|
char *result;
|
|
|
|
|
char *c;
|
|
|
|
|
|
|
|
|
|
result = pg_strdup(path);
|
|
|
|
|
|
|
|
|
|
for (c = result; *c != '\0'; c++)
|
|
|
|
|
if (*c == '/')
|
|
|
|
|
*c = '\\';
|
|
|
|
|
|
|
|
|
|
return result;
|
|
|
|
|
#else
|
|
|
|
|
|
|
|
|
|
return path;
|
|
|
|
|
#endif
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
void
|
|
|
|
|
output_check_banner(void)
|
|
|
|
|
{
|
|
|
|
|
if (user_opts.live_check)
|
|
|
|
|
{
|
|
|
|
|
pg_log(PG_REPORT,
|
|
|
|
|
"Performing Consistency Checks on Old Live Server\n"
|
Remove trailing newlines in pg_upgrade's message strings.
pg_upgrade does not use common/logging.c, which is unfortunate
but changing it to do so seems like more work than is justified.
However, we really need to make it work more like common/logging.c
in one respect: the latter expects supplied message strings to not
end with a newline, instead adding one internally. As it stands,
pg_upgrade's logging facilities expect a caller-supplied newline
in some cases and not others, which is already an invitation to bugs,
but the inconsistency with our other frontend code makes it worse.
There are already several places with missing or extra newlines,
and it's inevitable that there won't be more if we let this stand.
Hence, run around and get rid of all trailing newlines in message
strings, and add an Assert that there's not one, similar to the
existing Assert in common/logging.c. Adjust the logging functions
to supply a newline at the right places.
(Some of these strings also have a *leading* newline, which would
be a good thing to get rid of too; but this patch doesn't attempt
that.)
There are some consequent minor changes in output. The ones that
aren't outright bug fixes are generally removal of extra blank
lines that the original coding intentionally inserted. It didn't
seem worth being bug-compatible with that.
Patch by me, reviewed by Kyotaro Horiguchi and Peter Eisentraut
Discussion: https://postgr.es/m/113191.1655233060@sss.pgh.pa.us
3 years ago
|
|
|
"------------------------------------------------");
|
|
|
|
|
}
|
|
|
|
|
else
|
|
|
|
|
{
|
|
|
|
|
pg_log(PG_REPORT,
|
|
|
|
|
"Performing Consistency Checks\n"
|
Remove trailing newlines in pg_upgrade's message strings.
pg_upgrade does not use common/logging.c, which is unfortunate
but changing it to do so seems like more work than is justified.
However, we really need to make it work more like common/logging.c
in one respect: the latter expects supplied message strings to not
end with a newline, instead adding one internally. As it stands,
pg_upgrade's logging facilities expect a caller-supplied newline
in some cases and not others, which is already an invitation to bugs,
but the inconsistency with our other frontend code makes it worse.
There are already several places with missing or extra newlines,
and it's inevitable that there won't be more if we let this stand.
Hence, run around and get rid of all trailing newlines in message
strings, and add an Assert that there's not one, similar to the
existing Assert in common/logging.c. Adjust the logging functions
to supply a newline at the right places.
(Some of these strings also have a *leading* newline, which would
be a good thing to get rid of too; but this patch doesn't attempt
that.)
There are some consequent minor changes in output. The ones that
aren't outright bug fixes are generally removal of extra blank
lines that the original coding intentionally inserted. It didn't
seem worth being bug-compatible with that.
Patch by me, reviewed by Kyotaro Horiguchi and Peter Eisentraut
Discussion: https://postgr.es/m/113191.1655233060@sss.pgh.pa.us
3 years ago
|
|
|
"-----------------------------");
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
void
|
|
|
|
|
check_and_dump_old_cluster(void)
|
|
|
|
|
{
|
|
|
|
|
/* -- OLD -- */
|
|
|
|
|
|
|
|
|
|
if (!user_opts.live_check)
|
|
|
|
|
start_postmaster(&old_cluster, true);
|
|
|
|
|
|
Migrate logical slots to the new node during an upgrade.
While reading information from the old cluster, a list of logical
slots is fetched. At the later part of upgrading, pg_upgrade revisits the
list and restores slots by executing pg_create_logical_replication_slot()
on the new cluster. Migration of logical replication slots is only
supported when the old cluster is version 17.0 or later.
If the old node has invalid slots or slots with unconsumed WAL records,
the pg_upgrade fails. These checks are needed to prevent data loss.
The significant advantage of this commit is that it makes it easy to
continue logical replication even after upgrading the publisher node.
Previously, pg_upgrade allowed copying publications to a new node. With
this patch, adjusting the connection string to the new publisher will
cause the apply worker on the subscriber to connect to the new publisher
automatically. This enables seamless continuation of logical replication,
even after an upgrade.
Author: Hayato Kuroda, Hou Zhijie
Reviewed-by: Peter Smith, Bharath Rupireddy, Dilip Kumar, Vignesh C, Shlok Kyal
Discussion: http://postgr.es/m/TYAPR01MB58664C81887B3AF2EB6B16E3F5939@TYAPR01MB5866.jpnprd01.prod.outlook.com
Discussion: http://postgr.es/m/CAA4eK1+t7xYcfa0rEQw839=b2MzsfvYDPz3xbD+ZqOdP3zpKYg@mail.gmail.com
2 years ago
|
|
|
/*
|
|
|
|
|
* Extract a list of databases, tables, and logical replication slots from
|
|
|
|
|
* the old cluster.
|
|
|
|
|
*/
|
|
|
|
|
get_db_rel_and_slot_infos(&old_cluster);
|
|
|
|
|
|
|
|
|
|
init_tablespaces();
|
|
|
|
|
|
|
|
|
|
get_loadable_libraries();
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Check for various failure cases
|
|
|
|
|
*/
|
|
|
|
|
check_is_install_user(&old_cluster);
|
|
|
|
|
check_proper_datallowconn(&old_cluster);
|
|
|
|
|
check_for_prepared_transactions(&old_cluster);
|
|
|
|
|
check_for_isn_and_int8_passing_mismatch(&old_cluster);
|
|
|
|
|
|
Migrate logical slots to the new node during an upgrade.
While reading information from the old cluster, a list of logical
slots is fetched. At the later part of upgrading, pg_upgrade revisits the
list and restores slots by executing pg_create_logical_replication_slot()
on the new cluster. Migration of logical replication slots is only
supported when the old cluster is version 17.0 or later.
If the old node has invalid slots or slots with unconsumed WAL records,
the pg_upgrade fails. These checks are needed to prevent data loss.
The significant advantage of this commit is that it makes it easy to
continue logical replication even after upgrading the publisher node.
Previously, pg_upgrade allowed copying publications to a new node. With
this patch, adjusting the connection string to the new publisher will
cause the apply worker on the subscriber to connect to the new publisher
automatically. This enables seamless continuation of logical replication,
even after an upgrade.
Author: Hayato Kuroda, Hou Zhijie
Reviewed-by: Peter Smith, Bharath Rupireddy, Dilip Kumar, Vignesh C, Shlok Kyal
Discussion: http://postgr.es/m/TYAPR01MB58664C81887B3AF2EB6B16E3F5939@TYAPR01MB5866.jpnprd01.prod.outlook.com
Discussion: http://postgr.es/m/CAA4eK1+t7xYcfa0rEQw839=b2MzsfvYDPz3xbD+ZqOdP3zpKYg@mail.gmail.com
2 years ago
|
|
|
if (GET_MAJOR_VERSION(old_cluster.major_version) >= 1700)
|
Allow upgrades to preserve the full subscription's state.
This feature will allow us to replicate the changes on subscriber nodes
after the upgrade.
Previously, only the subscription metadata information was preserved.
Without the list of relations and their state, it's not possible to
re-enable the subscriptions without missing some records as the list of
relations can only be refreshed after enabling the subscription (and
therefore starting the apply worker). Even if we added a way to refresh
the subscription while enabling a publication, we still wouldn't know
which relations are new on the publication side, and therefore should be
fully synced, and which shouldn't.
To preserve the subscription relations, this patch teaches pg_dump to
restore the content of pg_subscription_rel from the old cluster by using
binary_upgrade_add_sub_rel_state SQL function. This is supported only
in binary upgrade mode.
The subscription's replication origin is needed to ensure that we don't
replicate anything twice.
To preserve the replication origins, this patch teaches pg_dump to update
the replication origin along with creating a subscription by using
binary_upgrade_replorigin_advance SQL function to restore the
underlying replication origin remote LSN. This is supported only in
binary upgrade mode.
pg_upgrade will check that all the subscription relations are in 'i'
(init) or in 'r' (ready) state and will error out if that's not the case,
logging the reason for the failure. This helps to avoid the risk of any
dangling slot or origin after the upgrade.
Author: Vignesh C, Julien Rouhaud, Shlok Kyal
Reviewed-by: Peter Smith, Masahiko Sawada, Michael Paquier, Amit Kapila, Hayato Kuroda
Discussion: https://postgr.es/m/20230217075433.u5mjly4d5cr4hcfe@jrouhaud
2 years ago
|
|
|
{
|
|
|
|
|
/*
|
|
|
|
|
* Logical replication slots can be migrated since PG17. See comments
|
|
|
|
|
* atop get_old_cluster_logical_slot_infos().
|
|
|
|
|
*/
|
|
|
|
|
check_old_cluster_for_valid_slots();
|
Migrate logical slots to the new node during an upgrade.
While reading information from the old cluster, a list of logical
slots is fetched. At the later part of upgrading, pg_upgrade revisits the
list and restores slots by executing pg_create_logical_replication_slot()
on the new cluster. Migration of logical replication slots is only
supported when the old cluster is version 17.0 or later.
If the old node has invalid slots or slots with unconsumed WAL records,
the pg_upgrade fails. These checks are needed to prevent data loss.
The significant advantage of this commit is that it makes it easy to
continue logical replication even after upgrading the publisher node.
Previously, pg_upgrade allowed copying publications to a new node. With
this patch, adjusting the connection string to the new publisher will
cause the apply worker on the subscriber to connect to the new publisher
automatically. This enables seamless continuation of logical replication,
even after an upgrade.
Author: Hayato Kuroda, Hou Zhijie
Reviewed-by: Peter Smith, Bharath Rupireddy, Dilip Kumar, Vignesh C, Shlok Kyal
Discussion: http://postgr.es/m/TYAPR01MB58664C81887B3AF2EB6B16E3F5939@TYAPR01MB5866.jpnprd01.prod.outlook.com
Discussion: http://postgr.es/m/CAA4eK1+t7xYcfa0rEQw839=b2MzsfvYDPz3xbD+ZqOdP3zpKYg@mail.gmail.com
2 years ago
|
|
|
|
Allow upgrades to preserve the full subscription's state.
This feature will allow us to replicate the changes on subscriber nodes
after the upgrade.
Previously, only the subscription metadata information was preserved.
Without the list of relations and their state, it's not possible to
re-enable the subscriptions without missing some records as the list of
relations can only be refreshed after enabling the subscription (and
therefore starting the apply worker). Even if we added a way to refresh
the subscription while enabling a publication, we still wouldn't know
which relations are new on the publication side, and therefore should be
fully synced, and which shouldn't.
To preserve the subscription relations, this patch teaches pg_dump to
restore the content of pg_subscription_rel from the old cluster by using
binary_upgrade_add_sub_rel_state SQL function. This is supported only
in binary upgrade mode.
The subscription's replication origin is needed to ensure that we don't
replicate anything twice.
To preserve the replication origins, this patch teaches pg_dump to update
the replication origin along with creating a subscription by using
binary_upgrade_replorigin_advance SQL function to restore the
underlying replication origin remote LSN. This is supported only in
binary upgrade mode.
pg_upgrade will check that all the subscription relations are in 'i'
(init) or in 'r' (ready) state and will error out if that's not the case,
logging the reason for the failure. This helps to avoid the risk of any
dangling slot or origin after the upgrade.
Author: Vignesh C, Julien Rouhaud, Shlok Kyal
Reviewed-by: Peter Smith, Masahiko Sawada, Michael Paquier, Amit Kapila, Hayato Kuroda
Discussion: https://postgr.es/m/20230217075433.u5mjly4d5cr4hcfe@jrouhaud
2 years ago
|
|
|
/*
|
|
|
|
|
* Subscriptions and their dependencies can be migrated since PG17.
|
pg_upgrade: Retrieve subscription count more efficiently.
Presently, pg_upgrade obtains the number of subscriptions in the
to-be-upgraded cluster by first querying pg_subscription in every
database for the number of subscriptions in only that database.
Then, in count_old_cluster_subscriptions(), it adds all the values
collected in the first step. This is expensive, especially when
there are many databases.
Fortunately, there is a better way to retrieve the subscription
count. Since pg_subscription is a shared catalog, we only need to
connect to a single database and query it once. This commit
modifies pg_upgrade to use that approach, which also allows us to
trim several lines of code. In passing, move the call to
get_db_subscription_count(), which has been renamed to
get_subscription_count(), from get_db_rel_and_slot_infos() to the
dedicated >= v17 section in check_and_dump_old_cluster().
We may be able to make similar improvements to
get_old_cluster_logical_slot_infos(), but that is left as a future
exercise.
Reviewed-by: Michael Paquier, Amit Kapila
Discussion: https://postgr.es/m/ZprQJv_TxccN3tkr%40nathan
Backpatch-through: 17
1 year ago
|
|
|
* Before that the logical slots are not upgraded, so we will not be
|
|
|
|
|
* able to upgrade the logical replication clusters completely.
|
Allow upgrades to preserve the full subscription's state.
This feature will allow us to replicate the changes on subscriber nodes
after the upgrade.
Previously, only the subscription metadata information was preserved.
Without the list of relations and their state, it's not possible to
re-enable the subscriptions without missing some records as the list of
relations can only be refreshed after enabling the subscription (and
therefore starting the apply worker). Even if we added a way to refresh
the subscription while enabling a publication, we still wouldn't know
which relations are new on the publication side, and therefore should be
fully synced, and which shouldn't.
To preserve the subscription relations, this patch teaches pg_dump to
restore the content of pg_subscription_rel from the old cluster by using
binary_upgrade_add_sub_rel_state SQL function. This is supported only
in binary upgrade mode.
The subscription's replication origin is needed to ensure that we don't
replicate anything twice.
To preserve the replication origins, this patch teaches pg_dump to update
the replication origin along with creating a subscription by using
binary_upgrade_replorigin_advance SQL function to restore the
underlying replication origin remote LSN. This is supported only in
binary upgrade mode.
pg_upgrade will check that all the subscription relations are in 'i'
(init) or in 'r' (ready) state and will error out if that's not the case,
logging the reason for the failure. This helps to avoid the risk of any
dangling slot or origin after the upgrade.
Author: Vignesh C, Julien Rouhaud, Shlok Kyal
Reviewed-by: Peter Smith, Masahiko Sawada, Michael Paquier, Amit Kapila, Hayato Kuroda
Discussion: https://postgr.es/m/20230217075433.u5mjly4d5cr4hcfe@jrouhaud
2 years ago
|
|
|
*/
|
pg_upgrade: Retrieve subscription count more efficiently.
Presently, pg_upgrade obtains the number of subscriptions in the
to-be-upgraded cluster by first querying pg_subscription in every
database for the number of subscriptions in only that database.
Then, in count_old_cluster_subscriptions(), it adds all the values
collected in the first step. This is expensive, especially when
there are many databases.
Fortunately, there is a better way to retrieve the subscription
count. Since pg_subscription is a shared catalog, we only need to
connect to a single database and query it once. This commit
modifies pg_upgrade to use that approach, which also allows us to
trim several lines of code. In passing, move the call to
get_db_subscription_count(), which has been renamed to
get_subscription_count(), from get_db_rel_and_slot_infos() to the
dedicated >= v17 section in check_and_dump_old_cluster().
We may be able to make similar improvements to
get_old_cluster_logical_slot_infos(), but that is left as a future
exercise.
Reviewed-by: Michael Paquier, Amit Kapila
Discussion: https://postgr.es/m/ZprQJv_TxccN3tkr%40nathan
Backpatch-through: 17
1 year ago
|
|
|
get_subscription_count(&old_cluster);
|
Allow upgrades to preserve the full subscription's state.
This feature will allow us to replicate the changes on subscriber nodes
after the upgrade.
Previously, only the subscription metadata information was preserved.
Without the list of relations and their state, it's not possible to
re-enable the subscriptions without missing some records as the list of
relations can only be refreshed after enabling the subscription (and
therefore starting the apply worker). Even if we added a way to refresh
the subscription while enabling a publication, we still wouldn't know
which relations are new on the publication side, and therefore should be
fully synced, and which shouldn't.
To preserve the subscription relations, this patch teaches pg_dump to
restore the content of pg_subscription_rel from the old cluster by using
binary_upgrade_add_sub_rel_state SQL function. This is supported only
in binary upgrade mode.
The subscription's replication origin is needed to ensure that we don't
replicate anything twice.
To preserve the replication origins, this patch teaches pg_dump to update
the replication origin along with creating a subscription by using
binary_upgrade_replorigin_advance SQL function to restore the
underlying replication origin remote LSN. This is supported only in
binary upgrade mode.
pg_upgrade will check that all the subscription relations are in 'i'
(init) or in 'r' (ready) state and will error out if that's not the case,
logging the reason for the failure. This helps to avoid the risk of any
dangling slot or origin after the upgrade.
Author: Vignesh C, Julien Rouhaud, Shlok Kyal
Reviewed-by: Peter Smith, Masahiko Sawada, Michael Paquier, Amit Kapila, Hayato Kuroda
Discussion: https://postgr.es/m/20230217075433.u5mjly4d5cr4hcfe@jrouhaud
2 years ago
|
|
|
check_old_cluster_subscription_state();
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
check_for_data_types_usage(&old_cluster);
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* PG 14 changed the function signature of encoding conversion functions.
|
|
|
|
|
* Conversions from older versions cannot be upgraded automatically
|
|
|
|
|
* because the user-defined functions used by the encoding conversions
|
|
|
|
|
* need to be changed to match the new signature.
|
|
|
|
|
*/
|
|
|
|
|
if (GET_MAJOR_VERSION(old_cluster.major_version) <= 1300)
|
|
|
|
|
check_for_user_defined_encoding_conversions(&old_cluster);
|
|
|
|
|
|
Remove support for postfix (right-unary) operators.
This feature has been a thorn in our sides for a long time, causing
many grammatical ambiguity problems. It doesn't seem worth the
pain to continue to support it, so remove it.
There are some follow-on improvements we can make in the grammar,
but this commit only removes the bare minimum number of productions,
plus assorted backend support code.
Note that pg_dump and psql continue to have full support, since
they may be used against older servers. However, pg_dump warns
about postfix operators. There is also a check in pg_upgrade.
Documentation-wise, I (tgl) largely removed the "left unary"
terminology in favor of saying "prefix operator", which is
a more standard and IMO less confusing term.
I included a catversion bump, although no initial catalog data
changes here, to mark the boundary at which oprkind = 'r'
stopped being valid in pg_operator.
Mark Dilger, based on work by myself and Robert Haas;
review by John Naylor
Discussion: https://postgr.es/m/38ca86db-42ab-9b48-2902-337a0d6b8311@2ndquadrant.com
5 years ago
|
|
|
/*
|
|
|
|
|
* Pre-PG 14 allowed user defined postfix operators, which are not
|
|
|
|
|
* supported anymore. Verify there are none, iff applicable.
|
|
|
|
|
*/
|
|
|
|
|
if (GET_MAJOR_VERSION(old_cluster.major_version) <= 1300)
|
|
|
|
|
check_for_user_defined_postfix_ops(&old_cluster);
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* PG 14 changed polymorphic functions from anyarray to
|
|
|
|
|
* anycompatiblearray.
|
|
|
|
|
*/
|
|
|
|
|
if (GET_MAJOR_VERSION(old_cluster.major_version) <= 1300)
|
|
|
|
|
check_for_incompatible_polymorphics(&old_cluster);
|
|
|
|
|
|
Remove WITH OIDS support, change oid catalog column visibility.
Previously tables declared WITH OIDS, including a significant fraction
of the catalog tables, stored the oid column not as a normal column,
but as part of the tuple header.
This special column was not shown by default, which was somewhat odd,
as it's often (consider e.g. pg_class.oid) one of the more important
parts of a row. Neither pg_dump nor COPY included the contents of the
oid column by default.
The fact that the oid column was not an ordinary column necessitated a
significant amount of special case code to support oid columns. That
already was painful for the existing, but upcoming work aiming to make
table storage pluggable, would have required expanding and duplicating
that "specialness" significantly.
WITH OIDS has been deprecated since 2005 (commit ff02d0a05280e0).
Remove it.
Removing includes:
- CREATE TABLE and ALTER TABLE syntax for declaring the table to be
WITH OIDS has been removed (WITH (oids[ = true]) will error out)
- pg_dump does not support dumping tables declared WITH OIDS and will
issue a warning when dumping one (and ignore the oid column).
- restoring an pg_dump archive with pg_restore will warn when
restoring a table with oid contents (and ignore the oid column)
- COPY will refuse to load binary dump that includes oids.
- pg_upgrade will error out when encountering tables declared WITH
OIDS, they have to be altered to remove the oid column first.
- Functionality to access the oid of the last inserted row (like
plpgsql's RESULT_OID, spi's SPI_lastoid, ...) has been removed.
The syntax for declaring a table WITHOUT OIDS (or WITH (oids = false)
for CREATE TABLE) is still supported. While that requires a bit of
support code, it seems unnecessary to break applications / dumps that
do not use oids, and are explicit about not using them.
The biggest user of WITH OID columns was postgres' catalog. This
commit changes all 'magic' oid columns to be columns that are normally
declared and stored. To reduce unnecessary query breakage all the
newly added columns are still named 'oid', even if a table's column
naming scheme would indicate 'reloid' or such. This obviously
requires adapting a lot code, mostly replacing oid access via
HeapTupleGetOid() with access to the underlying Form_pg_*->oid column.
The bootstrap process now assigns oids for all oid columns in
genbki.pl that do not have an explicit value (starting at the largest
oid previously used), only oids assigned later by oids will be above
FirstBootstrapObjectId. As the oid column now is a normal column the
special bootstrap syntax for oids has been removed.
Oids are not automatically assigned during insertion anymore, all
backend code explicitly assigns oids with GetNewOidWithIndex(). For
the rare case that insertions into the catalog via SQL are called for
the new pg_nextoid() function can be used (which only works on catalog
tables).
The fact that oid columns on system tables are now normal columns
means that they will be included in the set of columns expanded
by * (i.e. SELECT * FROM pg_class will now include the table's oid,
previously it did not). It'd not technically be hard to hide oid
column by default, but that'd mean confusing behavior would either
have to be carried forward forever, or it'd cause breakage down the
line.
While it's not unlikely that further adjustments are needed, the
scope/invasiveness of the patch makes it worthwhile to get merge this
now. It's painful to maintain externally, too complicated to commit
after the code code freeze, and a dependency of a number of other
patches.
Catversion bump, for obvious reasons.
Author: Andres Freund, with contributions by John Naylor
Discussion: https://postgr.es/m/20180930034810.ywp2c7awz7opzcfr@alap3.anarazel.de
7 years ago
|
|
|
/*
|
|
|
|
|
* Pre-PG 12 allowed tables to be declared WITH OIDS, which is not
|
|
|
|
|
* supported anymore. Verify there are none, iff applicable.
|
|
|
|
|
*/
|
|
|
|
|
if (GET_MAJOR_VERSION(old_cluster.major_version) <= 1100)
|
|
|
|
|
check_for_tables_with_oids(&old_cluster);
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Pre-PG 10 allowed tables with 'unknown' type columns and non WAL logged
|
|
|
|
|
* hash indexes
|
|
|
|
|
*/
|
Change unknown-type literals to type text in SELECT and RETURNING lists.
Previously, we left such literals alone if the query or subquery had
no properties forcing a type decision to be made (such as an ORDER BY or
DISTINCT clause using that output column). This meant that "unknown" could
be an exposed output column type, which has never been a great idea because
it could result in strange failures later on. For example, an outer query
that tried to do any operations on an unknown-type subquery output would
generally fail with some weird error like "failed to find conversion
function from unknown to text" or "could not determine which collation to
use for string comparison". Also, if the case occurred in a CREATE VIEW's
query then the view would have an unknown-type column, causing similar
failures in queries trying to use the view.
To fix, at the tail end of parse analysis of a query, forcibly convert any
remaining "unknown" literals in its SELECT or RETURNING list to type text.
However, provide a switch to suppress that, and use it in the cases of
SELECT inside a set operation or INSERT command. In those cases we already
had type resolution rules that make use of context information from outside
the subquery proper, and we don't want to change that behavior.
Also, change creation of an unknown-type column in a relation from a
warning to a hard error. The error should be unreachable now in CREATE
VIEW or CREATE MATVIEW, but it's still possible to explicitly say "unknown"
in CREATE TABLE or CREATE (composite) TYPE. We want to forbid that because
it's nothing but a foot-gun.
This change creates a pg_upgrade failure case: a matview that contains an
unknown-type column can't be pg_upgraded, because reparsing the matview's
defining query will now decide that the column is of type text, which
doesn't match the cstring-like storage that the old materialized column
would actually have. Add a checking pass to detect that. While at it,
we can detect tables or composite types that would fail, essentially
for free. Those would fail safely anyway later on, but we might as
well fail earlier.
This patch is by me, but it owes something to previous investigations
by Rahila Syed. Also thanks to Ashutosh Bapat and Michael Paquier for
review.
Discussion: https://postgr.es/m/CAH2L28uwwbL9HUM-WR=hromW1Cvamkn7O-g8fPY2m=_7muJ0oA@mail.gmail.com
9 years ago
|
|
|
if (GET_MAJOR_VERSION(old_cluster.major_version) <= 906)
|
|
|
|
|
{
|
|
|
|
|
if (user_opts.check)
|
|
|
|
|
old_9_6_invalidate_hash_indexes(&old_cluster, true);
|
|
|
|
|
}
|
Change unknown-type literals to type text in SELECT and RETURNING lists.
Previously, we left such literals alone if the query or subquery had
no properties forcing a type decision to be made (such as an ORDER BY or
DISTINCT clause using that output column). This meant that "unknown" could
be an exposed output column type, which has never been a great idea because
it could result in strange failures later on. For example, an outer query
that tried to do any operations on an unknown-type subquery output would
generally fail with some weird error like "failed to find conversion
function from unknown to text" or "could not determine which collation to
use for string comparison". Also, if the case occurred in a CREATE VIEW's
query then the view would have an unknown-type column, causing similar
failures in queries trying to use the view.
To fix, at the tail end of parse analysis of a query, forcibly convert any
remaining "unknown" literals in its SELECT or RETURNING list to type text.
However, provide a switch to suppress that, and use it in the cases of
SELECT inside a set operation or INSERT command. In those cases we already
had type resolution rules that make use of context information from outside
the subquery proper, and we don't want to change that behavior.
Also, change creation of an unknown-type column in a relation from a
warning to a hard error. The error should be unreachable now in CREATE
VIEW or CREATE MATVIEW, but it's still possible to explicitly say "unknown"
in CREATE TABLE or CREATE (composite) TYPE. We want to forbid that because
it's nothing but a foot-gun.
This change creates a pg_upgrade failure case: a matview that contains an
unknown-type column can't be pg_upgraded, because reparsing the matview's
defining query will now decide that the column is of type text, which
doesn't match the cstring-like storage that the old materialized column
would actually have. Add a checking pass to detect that. While at it,
we can detect tables or composite types that would fail, essentially
for free. Those would fail safely anyway later on, but we might as
well fail earlier.
This patch is by me, but it owes something to previous investigations
by Rahila Syed. Also thanks to Ashutosh Bapat and Michael Paquier for
review.
Discussion: https://postgr.es/m/CAH2L28uwwbL9HUM-WR=hromW1Cvamkn7O-g8fPY2m=_7muJ0oA@mail.gmail.com
9 years ago
|
|
|
|
|
|
|
|
/* 9.5 and below should not have roles starting with pg_ */
|
|
|
|
|
if (GET_MAJOR_VERSION(old_cluster.major_version) <= 905)
|
|
|
|
|
check_for_pg_role_prefix(&old_cluster);
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* While not a check option, we do this now because this is the only time
|
|
|
|
|
* the old server is running.
|
|
|
|
|
*/
|
|
|
|
|
if (!user_opts.check)
|
|
|
|
|
generate_old_dump();
|
|
|
|
|
|
|
|
|
|
if (!user_opts.live_check)
|
|
|
|
|
stop_postmaster(false);
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
void
|
|
|
|
|
check_new_cluster(void)
|
|
|
|
|
{
|
|
|
|
|
get_db_rel_and_slot_infos(&new_cluster);
|
|
|
|
|
|
|
|
|
|
check_new_cluster_is_empty();
|
|
|
|
|
|
|
|
|
|
check_loadable_libraries();
|
|
|
|
|
|
|
|
|
|
switch (user_opts.transfer_mode)
|
|
|
|
|
{
|
|
|
|
|
case TRANSFER_MODE_CLONE:
|
|
|
|
|
check_file_clone();
|
|
|
|
|
break;
|
|
|
|
|
case TRANSFER_MODE_COPY:
|
|
|
|
|
break;
|
|
|
|
|
case TRANSFER_MODE_COPY_FILE_RANGE:
|
|
|
|
|
check_copy_file_range();
|
|
|
|
|
break;
|
|
|
|
|
case TRANSFER_MODE_LINK:
|
|
|
|
|
check_hard_link();
|
|
|
|
|
break;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
check_is_install_user(&new_cluster);
|
|
|
|
|
|
|
|
|
|
check_for_prepared_transactions(&new_cluster);
|
|
|
|
|
|
|
|
|
|
check_for_new_tablespace_dir();
|
Migrate logical slots to the new node during an upgrade.
While reading information from the old cluster, a list of logical
slots is fetched. At the later part of upgrading, pg_upgrade revisits the
list and restores slots by executing pg_create_logical_replication_slot()
on the new cluster. Migration of logical replication slots is only
supported when the old cluster is version 17.0 or later.
If the old node has invalid slots or slots with unconsumed WAL records,
the pg_upgrade fails. These checks are needed to prevent data loss.
The significant advantage of this commit is that it makes it easy to
continue logical replication even after upgrading the publisher node.
Previously, pg_upgrade allowed copying publications to a new node. With
this patch, adjusting the connection string to the new publisher will
cause the apply worker on the subscriber to connect to the new publisher
automatically. This enables seamless continuation of logical replication,
even after an upgrade.
Author: Hayato Kuroda, Hou Zhijie
Reviewed-by: Peter Smith, Bharath Rupireddy, Dilip Kumar, Vignesh C, Shlok Kyal
Discussion: http://postgr.es/m/TYAPR01MB58664C81887B3AF2EB6B16E3F5939@TYAPR01MB5866.jpnprd01.prod.outlook.com
Discussion: http://postgr.es/m/CAA4eK1+t7xYcfa0rEQw839=b2MzsfvYDPz3xbD+ZqOdP3zpKYg@mail.gmail.com
2 years ago
|
|
|
|
|
|
|
|
check_new_cluster_logical_replication_slots();
|
Allow upgrades to preserve the full subscription's state.
This feature will allow us to replicate the changes on subscriber nodes
after the upgrade.
Previously, only the subscription metadata information was preserved.
Without the list of relations and their state, it's not possible to
re-enable the subscriptions without missing some records as the list of
relations can only be refreshed after enabling the subscription (and
therefore starting the apply worker). Even if we added a way to refresh
the subscription while enabling a publication, we still wouldn't know
which relations are new on the publication side, and therefore should be
fully synced, and which shouldn't.
To preserve the subscription relations, this patch teaches pg_dump to
restore the content of pg_subscription_rel from the old cluster by using
binary_upgrade_add_sub_rel_state SQL function. This is supported only
in binary upgrade mode.
The subscription's replication origin is needed to ensure that we don't
replicate anything twice.
To preserve the replication origins, this patch teaches pg_dump to update
the replication origin along with creating a subscription by using
binary_upgrade_replorigin_advance SQL function to restore the
underlying replication origin remote LSN. This is supported only in
binary upgrade mode.
pg_upgrade will check that all the subscription relations are in 'i'
(init) or in 'r' (ready) state and will error out if that's not the case,
logging the reason for the failure. This helps to avoid the risk of any
dangling slot or origin after the upgrade.
Author: Vignesh C, Julien Rouhaud, Shlok Kyal
Reviewed-by: Peter Smith, Masahiko Sawada, Michael Paquier, Amit Kapila, Hayato Kuroda
Discussion: https://postgr.es/m/20230217075433.u5mjly4d5cr4hcfe@jrouhaud
2 years ago
|
|
|
|
|
|
|
|
check_new_cluster_subscription_configuration();
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
void
|
|
|
|
|
report_clusters_compatible(void)
|
|
|
|
|
{
|
|
|
|
|
if (user_opts.check)
|
|
|
|
|
{
|
Remove trailing newlines in pg_upgrade's message strings.
pg_upgrade does not use common/logging.c, which is unfortunate
but changing it to do so seems like more work than is justified.
However, we really need to make it work more like common/logging.c
in one respect: the latter expects supplied message strings to not
end with a newline, instead adding one internally. As it stands,
pg_upgrade's logging facilities expect a caller-supplied newline
in some cases and not others, which is already an invitation to bugs,
but the inconsistency with our other frontend code makes it worse.
There are already several places with missing or extra newlines,
and it's inevitable that there won't be more if we let this stand.
Hence, run around and get rid of all trailing newlines in message
strings, and add an Assert that there's not one, similar to the
existing Assert in common/logging.c. Adjust the logging functions
to supply a newline at the right places.
(Some of these strings also have a *leading* newline, which would
be a good thing to get rid of too; but this patch doesn't attempt
that.)
There are some consequent minor changes in output. The ones that
aren't outright bug fixes are generally removal of extra blank
lines that the original coding intentionally inserted. It didn't
seem worth being bug-compatible with that.
Patch by me, reviewed by Kyotaro Horiguchi and Peter Eisentraut
Discussion: https://postgr.es/m/113191.1655233060@sss.pgh.pa.us
3 years ago
|
|
|
pg_log(PG_REPORT, "\n*Clusters are compatible*");
|
|
|
|
|
/* stops new cluster */
|
|
|
|
|
stop_postmaster(false);
|
Restructure pg_upgrade output directories for better idempotence
38bfae3 has moved the contents written to files by pg_upgrade under a
new directory called pg_upgrade_output.d/ located in the new cluster's
data folder, and it used a simple structure made of two subdirectories
leading to a fixed structure: log/ and dump/. This design has made
weaker pg_upgrade on repeated calls, as we could get failures when
creating one or more of those directories, while potentially losing the
logs of a previous run (logs are retained automatically on failure, and
cleaned up on success unless --retain is specified). So a user would
need to clean up pg_upgrade_output.d/ as an extra step for any repeated
calls of pg_upgrade. The most common scenario here is --check followed
by the actual upgrade, but one could see a failure when specifying an
incorrect input argument value. Removing entirely the logs would have
the disadvantage of removing all the past information, even if --retain
was specified at some past step.
This result is annoying for a lot of users and automated upgrade flows.
So, rather than requiring a manual removal of pg_upgrade_output.d/, this
redesigns the set of output directories in a more dynamic way, based on
a suggestion from Tom Lane and Daniel Gustafsson. pg_upgrade_output.d/
is still the base path, but a second directory level is added, mostly
named after an ISO-8601-formatted timestamp (in short human-readable,
with milliseconds appended to the name to avoid any conflicts). The
logs and dumps are saved within the same subdirectories as previously,
as of log/ and dump/, but these are located inside the subdirectory
named after the timestamp.
The logs of a given run are removed only after a successful run if
--retain is not used, and pg_upgrade_output.d/ is kept if there are any
logs from a previous run. Note that previously, pg_upgrade would have
kept the logs even after a successful --check but that was inconsistent
compared to the case without --check when using --retain. The code in
charge of the removal of the output directories is now refactored into a
single routine.
Two TAP tests are added with some --check commands (one failure case and
one success case), to look after the issue fixed here. Note that the
tests had to be tweaked a bit to fit with the new directory structure so
as it can find any logs generated on failure. This is still going to
require a change in the buildfarm client for the case where pg_upgrade
is tested without the TAP test, though, but I'll tackle that with a
separate patch where needed.
Reported-by: Tushar Ahuja
Author: Michael Paquier
Reviewed-by: Daniel Gustafsson, Justin Pryzby
Discussion: https://postgr.es/m/77e6ecaa-2785-97aa-f229-4b6e047cbd2b@enterprisedb.com
4 years ago
|
|
|
|
|
|
|
|
cleanup_output_dirs();
|
|
|
|
|
exit(0);
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
pg_log(PG_REPORT, "\n"
|
|
|
|
|
"If pg_upgrade fails after this point, you must re-initdb the\n"
|
Remove trailing newlines in pg_upgrade's message strings.
pg_upgrade does not use common/logging.c, which is unfortunate
but changing it to do so seems like more work than is justified.
However, we really need to make it work more like common/logging.c
in one respect: the latter expects supplied message strings to not
end with a newline, instead adding one internally. As it stands,
pg_upgrade's logging facilities expect a caller-supplied newline
in some cases and not others, which is already an invitation to bugs,
but the inconsistency with our other frontend code makes it worse.
There are already several places with missing or extra newlines,
and it's inevitable that there won't be more if we let this stand.
Hence, run around and get rid of all trailing newlines in message
strings, and add an Assert that there's not one, similar to the
existing Assert in common/logging.c. Adjust the logging functions
to supply a newline at the right places.
(Some of these strings also have a *leading* newline, which would
be a good thing to get rid of too; but this patch doesn't attempt
that.)
There are some consequent minor changes in output. The ones that
aren't outright bug fixes are generally removal of extra blank
lines that the original coding intentionally inserted. It didn't
seem worth being bug-compatible with that.
Patch by me, reviewed by Kyotaro Horiguchi and Peter Eisentraut
Discussion: https://postgr.es/m/113191.1655233060@sss.pgh.pa.us
3 years ago
|
|
|
"new cluster before continuing.");
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
void
|
|
|
|
|
issue_warnings_and_set_wal_level(void)
|
|
|
|
|
{
|
|
|
|
|
/*
|
|
|
|
|
* We unconditionally start/stop the new server because pg_resetwal -o set
|
|
|
|
|
* wal_level to 'minimum'. If the user is upgrading standby servers using
|
|
|
|
|
* the rsync instructions, they will need pg_upgrade to write its final
|
|
|
|
|
* WAL record showing wal_level as 'replica'.
|
|
|
|
|
*/
|
|
|
|
|
start_postmaster(&new_cluster, true);
|
|
|
|
|
|
|
|
|
|
/* Reindex hash indexes for old < 10.0 */
|
|
|
|
|
if (GET_MAJOR_VERSION(old_cluster.major_version) <= 906)
|
|
|
|
|
old_9_6_invalidate_hash_indexes(&new_cluster, false);
|
|
|
|
|
|
|
|
|
|
report_extension_updates(&new_cluster);
|
|
|
|
|
|
|
|
|
|
stop_postmaster(false);
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
void
|
|
|
|
|
output_completion_banner(char *deletion_script_file_name)
|
|
|
|
|
{
|
|
|
|
|
PQExpBufferData user_specification;
|
|
|
|
|
|
|
|
|
|
initPQExpBuffer(&user_specification);
|
|
|
|
|
if (os_info.user_specified)
|
|
|
|
|
{
|
|
|
|
|
appendPQExpBufferStr(&user_specification, "-U ");
|
|
|
|
|
appendShellString(&user_specification, os_info.user);
|
|
|
|
|
appendPQExpBufferChar(&user_specification, ' ');
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
pg_log(PG_REPORT,
|
|
|
|
|
"Optimizer statistics are not transferred by pg_upgrade.\n"
|
|
|
|
|
"Once you start the new server, consider running:\n"
|
Remove trailing newlines in pg_upgrade's message strings.
pg_upgrade does not use common/logging.c, which is unfortunate
but changing it to do so seems like more work than is justified.
However, we really need to make it work more like common/logging.c
in one respect: the latter expects supplied message strings to not
end with a newline, instead adding one internally. As it stands,
pg_upgrade's logging facilities expect a caller-supplied newline
in some cases and not others, which is already an invitation to bugs,
but the inconsistency with our other frontend code makes it worse.
There are already several places with missing or extra newlines,
and it's inevitable that there won't be more if we let this stand.
Hence, run around and get rid of all trailing newlines in message
strings, and add an Assert that there's not one, similar to the
existing Assert in common/logging.c. Adjust the logging functions
to supply a newline at the right places.
(Some of these strings also have a *leading* newline, which would
be a good thing to get rid of too; but this patch doesn't attempt
that.)
There are some consequent minor changes in output. The ones that
aren't outright bug fixes are generally removal of extra blank
lines that the original coding intentionally inserted. It didn't
seem worth being bug-compatible with that.
Patch by me, reviewed by Kyotaro Horiguchi and Peter Eisentraut
Discussion: https://postgr.es/m/113191.1655233060@sss.pgh.pa.us
3 years ago
|
|
|
" %s/vacuumdb %s--all --analyze-in-stages", new_cluster.bindir, user_specification.data);
|
|
|
|
|
|
|
|
|
|
if (deletion_script_file_name)
|
|
|
|
|
pg_log(PG_REPORT,
|
Phase 3 of pgindent updates.
Don't move parenthesized lines to the left, even if that means they
flow past the right margin.
By default, BSD indent lines up statement continuation lines that are
within parentheses so that they start just to the right of the preceding
left parenthesis. However, traditionally, if that resulted in the
continuation line extending to the right of the desired right margin,
then indent would push it left just far enough to not overrun the margin,
if it could do so without making the continuation line start to the left of
the current statement indent. That makes for a weird mix of indentations
unless one has been completely rigid about never violating the 80-column
limit.
This behavior has been pretty universally panned by Postgres developers.
Hence, disable it with indent's new -lpl switch, so that parenthesized
lines are always lined up with the preceding left paren.
This patch is much less interesting than the first round of indent
changes, but also bulkier, so I thought it best to separate the effects.
Discussion: https://postgr.es/m/E1dAmxK-0006EE-1r@gemulon.postgresql.org
Discussion: https://postgr.es/m/30527.1495162840@sss.pgh.pa.us
9 years ago
|
|
|
"Running this script will delete the old cluster's data files:\n"
|
Remove trailing newlines in pg_upgrade's message strings.
pg_upgrade does not use common/logging.c, which is unfortunate
but changing it to do so seems like more work than is justified.
However, we really need to make it work more like common/logging.c
in one respect: the latter expects supplied message strings to not
end with a newline, instead adding one internally. As it stands,
pg_upgrade's logging facilities expect a caller-supplied newline
in some cases and not others, which is already an invitation to bugs,
but the inconsistency with our other frontend code makes it worse.
There are already several places with missing or extra newlines,
and it's inevitable that there won't be more if we let this stand.
Hence, run around and get rid of all trailing newlines in message
strings, and add an Assert that there's not one, similar to the
existing Assert in common/logging.c. Adjust the logging functions
to supply a newline at the right places.
(Some of these strings also have a *leading* newline, which would
be a good thing to get rid of too; but this patch doesn't attempt
that.)
There are some consequent minor changes in output. The ones that
aren't outright bug fixes are generally removal of extra blank
lines that the original coding intentionally inserted. It didn't
seem worth being bug-compatible with that.
Patch by me, reviewed by Kyotaro Horiguchi and Peter Eisentraut
Discussion: https://postgr.es/m/113191.1655233060@sss.pgh.pa.us
3 years ago
|
|
|
" %s",
|
|
|
|
|
deletion_script_file_name);
|
|
|
|
|
else
|
|
|
|
|
pg_log(PG_REPORT,
|
Phase 3 of pgindent updates.
Don't move parenthesized lines to the left, even if that means they
flow past the right margin.
By default, BSD indent lines up statement continuation lines that are
within parentheses so that they start just to the right of the preceding
left parenthesis. However, traditionally, if that resulted in the
continuation line extending to the right of the desired right margin,
then indent would push it left just far enough to not overrun the margin,
if it could do so without making the continuation line start to the left of
the current statement indent. That makes for a weird mix of indentations
unless one has been completely rigid about never violating the 80-column
limit.
This behavior has been pretty universally panned by Postgres developers.
Hence, disable it with indent's new -lpl switch, so that parenthesized
lines are always lined up with the preceding left paren.
This patch is much less interesting than the first round of indent
changes, but also bulkier, so I thought it best to separate the effects.
Discussion: https://postgr.es/m/E1dAmxK-0006EE-1r@gemulon.postgresql.org
Discussion: https://postgr.es/m/30527.1495162840@sss.pgh.pa.us
9 years ago
|
|
|
"Could not create a script to delete the old cluster's data files\n"
|
|
|
|
|
"because user-defined tablespaces or the new cluster's data directory\n"
|
|
|
|
|
"exist in the old cluster directory. The old cluster's contents must\n"
|
Remove trailing newlines in pg_upgrade's message strings.
pg_upgrade does not use common/logging.c, which is unfortunate
but changing it to do so seems like more work than is justified.
However, we really need to make it work more like common/logging.c
in one respect: the latter expects supplied message strings to not
end with a newline, instead adding one internally. As it stands,
pg_upgrade's logging facilities expect a caller-supplied newline
in some cases and not others, which is already an invitation to bugs,
but the inconsistency with our other frontend code makes it worse.
There are already several places with missing or extra newlines,
and it's inevitable that there won't be more if we let this stand.
Hence, run around and get rid of all trailing newlines in message
strings, and add an Assert that there's not one, similar to the
existing Assert in common/logging.c. Adjust the logging functions
to supply a newline at the right places.
(Some of these strings also have a *leading* newline, which would
be a good thing to get rid of too; but this patch doesn't attempt
that.)
There are some consequent minor changes in output. The ones that
aren't outright bug fixes are generally removal of extra blank
lines that the original coding intentionally inserted. It didn't
seem worth being bug-compatible with that.
Patch by me, reviewed by Kyotaro Horiguchi and Peter Eisentraut
Discussion: https://postgr.es/m/113191.1655233060@sss.pgh.pa.us
3 years ago
|
|
|
"be deleted manually.");
|
|
|
|
|
|
|
|
|
|
termPQExpBuffer(&user_specification);
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
void
|
|
|
|
|
check_cluster_versions(void)
|
|
|
|
|
{
|
|
|
|
|
prep_status("Checking cluster versions");
|
|
|
|
|
|
|
|
|
|
/* cluster versions should already have been obtained */
|
|
|
|
|
Assert(old_cluster.major_version != 0);
|
|
|
|
|
Assert(new_cluster.major_version != 0);
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* We allow upgrades from/to the same major version for alpha/beta
|
|
|
|
|
* upgrades
|
|
|
|
|
*/
|
|
|
|
|
|
|
|
|
|
if (GET_MAJOR_VERSION(old_cluster.major_version) < 902)
|
Remove trailing newlines in pg_upgrade's message strings.
pg_upgrade does not use common/logging.c, which is unfortunate
but changing it to do so seems like more work than is justified.
However, we really need to make it work more like common/logging.c
in one respect: the latter expects supplied message strings to not
end with a newline, instead adding one internally. As it stands,
pg_upgrade's logging facilities expect a caller-supplied newline
in some cases and not others, which is already an invitation to bugs,
but the inconsistency with our other frontend code makes it worse.
There are already several places with missing or extra newlines,
and it's inevitable that there won't be more if we let this stand.
Hence, run around and get rid of all trailing newlines in message
strings, and add an Assert that there's not one, similar to the
existing Assert in common/logging.c. Adjust the logging functions
to supply a newline at the right places.
(Some of these strings also have a *leading* newline, which would
be a good thing to get rid of too; but this patch doesn't attempt
that.)
There are some consequent minor changes in output. The ones that
aren't outright bug fixes are generally removal of extra blank
lines that the original coding intentionally inserted. It didn't
seem worth being bug-compatible with that.
Patch by me, reviewed by Kyotaro Horiguchi and Peter Eisentraut
Discussion: https://postgr.es/m/113191.1655233060@sss.pgh.pa.us
3 years ago
|
|
|
pg_fatal("This utility can only upgrade from PostgreSQL version %s and later.",
|
|
|
|
|
"9.2");
|
|
|
|
|
|
|
|
|
|
/* Only current PG version is supported as a target */
|
|
|
|
|
if (GET_MAJOR_VERSION(new_cluster.major_version) != GET_MAJOR_VERSION(PG_VERSION_NUM))
|
Remove trailing newlines in pg_upgrade's message strings.
pg_upgrade does not use common/logging.c, which is unfortunate
but changing it to do so seems like more work than is justified.
However, we really need to make it work more like common/logging.c
in one respect: the latter expects supplied message strings to not
end with a newline, instead adding one internally. As it stands,
pg_upgrade's logging facilities expect a caller-supplied newline
in some cases and not others, which is already an invitation to bugs,
but the inconsistency with our other frontend code makes it worse.
There are already several places with missing or extra newlines,
and it's inevitable that there won't be more if we let this stand.
Hence, run around and get rid of all trailing newlines in message
strings, and add an Assert that there's not one, similar to the
existing Assert in common/logging.c. Adjust the logging functions
to supply a newline at the right places.
(Some of these strings also have a *leading* newline, which would
be a good thing to get rid of too; but this patch doesn't attempt
that.)
There are some consequent minor changes in output. The ones that
aren't outright bug fixes are generally removal of extra blank
lines that the original coding intentionally inserted. It didn't
seem worth being bug-compatible with that.
Patch by me, reviewed by Kyotaro Horiguchi and Peter Eisentraut
Discussion: https://postgr.es/m/113191.1655233060@sss.pgh.pa.us
3 years ago
|
|
|
pg_fatal("This utility can only upgrade to PostgreSQL version %s.",
|
|
|
|
|
PG_MAJORVERSION);
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* We can't allow downgrading because we use the target pg_dump, and
|
|
|
|
|
* pg_dump cannot operate on newer database versions, only current and
|
|
|
|
|
* older versions.
|
|
|
|
|
*/
|
|
|
|
|
if (old_cluster.major_version > new_cluster.major_version)
|
Remove trailing newlines in pg_upgrade's message strings.
pg_upgrade does not use common/logging.c, which is unfortunate
but changing it to do so seems like more work than is justified.
However, we really need to make it work more like common/logging.c
in one respect: the latter expects supplied message strings to not
end with a newline, instead adding one internally. As it stands,
pg_upgrade's logging facilities expect a caller-supplied newline
in some cases and not others, which is already an invitation to bugs,
but the inconsistency with our other frontend code makes it worse.
There are already several places with missing or extra newlines,
and it's inevitable that there won't be more if we let this stand.
Hence, run around and get rid of all trailing newlines in message
strings, and add an Assert that there's not one, similar to the
existing Assert in common/logging.c. Adjust the logging functions
to supply a newline at the right places.
(Some of these strings also have a *leading* newline, which would
be a good thing to get rid of too; but this patch doesn't attempt
that.)
There are some consequent minor changes in output. The ones that
aren't outright bug fixes are generally removal of extra blank
lines that the original coding intentionally inserted. It didn't
seem worth being bug-compatible with that.
Patch by me, reviewed by Kyotaro Horiguchi and Peter Eisentraut
Discussion: https://postgr.es/m/113191.1655233060@sss.pgh.pa.us
3 years ago
|
|
|
pg_fatal("This utility cannot be used to downgrade to older major PostgreSQL versions.");
|
|
|
|
|
|
|
|
|
|
/* Ensure binaries match the designated data directories */
|
|
|
|
|
if (GET_MAJOR_VERSION(old_cluster.major_version) !=
|
|
|
|
|
GET_MAJOR_VERSION(old_cluster.bin_version))
|
Remove trailing newlines in pg_upgrade's message strings.
pg_upgrade does not use common/logging.c, which is unfortunate
but changing it to do so seems like more work than is justified.
However, we really need to make it work more like common/logging.c
in one respect: the latter expects supplied message strings to not
end with a newline, instead adding one internally. As it stands,
pg_upgrade's logging facilities expect a caller-supplied newline
in some cases and not others, which is already an invitation to bugs,
but the inconsistency with our other frontend code makes it worse.
There are already several places with missing or extra newlines,
and it's inevitable that there won't be more if we let this stand.
Hence, run around and get rid of all trailing newlines in message
strings, and add an Assert that there's not one, similar to the
existing Assert in common/logging.c. Adjust the logging functions
to supply a newline at the right places.
(Some of these strings also have a *leading* newline, which would
be a good thing to get rid of too; but this patch doesn't attempt
that.)
There are some consequent minor changes in output. The ones that
aren't outright bug fixes are generally removal of extra blank
lines that the original coding intentionally inserted. It didn't
seem worth being bug-compatible with that.
Patch by me, reviewed by Kyotaro Horiguchi and Peter Eisentraut
Discussion: https://postgr.es/m/113191.1655233060@sss.pgh.pa.us
3 years ago
|
|
|
pg_fatal("Old cluster data and binary directories are from different major versions.");
|
|
|
|
|
if (GET_MAJOR_VERSION(new_cluster.major_version) !=
|
|
|
|
|
GET_MAJOR_VERSION(new_cluster.bin_version))
|
Remove trailing newlines in pg_upgrade's message strings.
pg_upgrade does not use common/logging.c, which is unfortunate
but changing it to do so seems like more work than is justified.
However, we really need to make it work more like common/logging.c
in one respect: the latter expects supplied message strings to not
end with a newline, instead adding one internally. As it stands,
pg_upgrade's logging facilities expect a caller-supplied newline
in some cases and not others, which is already an invitation to bugs,
but the inconsistency with our other frontend code makes it worse.
There are already several places with missing or extra newlines,
and it's inevitable that there won't be more if we let this stand.
Hence, run around and get rid of all trailing newlines in message
strings, and add an Assert that there's not one, similar to the
existing Assert in common/logging.c. Adjust the logging functions
to supply a newline at the right places.
(Some of these strings also have a *leading* newline, which would
be a good thing to get rid of too; but this patch doesn't attempt
that.)
There are some consequent minor changes in output. The ones that
aren't outright bug fixes are generally removal of extra blank
lines that the original coding intentionally inserted. It didn't
seem worth being bug-compatible with that.
Patch by me, reviewed by Kyotaro Horiguchi and Peter Eisentraut
Discussion: https://postgr.es/m/113191.1655233060@sss.pgh.pa.us
3 years ago
|
|
|
pg_fatal("New cluster data and binary directories are from different major versions.");
|
|
|
|
|
|
|
|
|
|
check_ok();
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
void
|
|
|
|
|
check_cluster_compatibility(void)
|
|
|
|
|
{
|
|
|
|
|
/* get/check pg_control data of servers */
|
|
|
|
|
get_control_data(&old_cluster);
|
|
|
|
|
get_control_data(&new_cluster);
|
|
|
|
|
check_control_data(&old_cluster.controldata, &new_cluster.controldata);
|
|
|
|
|
|
|
|
|
|
if (user_opts.live_check && old_cluster.port == new_cluster.port)
|
|
|
|
|
pg_fatal("When checking a live server, "
|
Remove trailing newlines in pg_upgrade's message strings.
pg_upgrade does not use common/logging.c, which is unfortunate
but changing it to do so seems like more work than is justified.
However, we really need to make it work more like common/logging.c
in one respect: the latter expects supplied message strings to not
end with a newline, instead adding one internally. As it stands,
pg_upgrade's logging facilities expect a caller-supplied newline
in some cases and not others, which is already an invitation to bugs,
but the inconsistency with our other frontend code makes it worse.
There are already several places with missing or extra newlines,
and it's inevitable that there won't be more if we let this stand.
Hence, run around and get rid of all trailing newlines in message
strings, and add an Assert that there's not one, similar to the
existing Assert in common/logging.c. Adjust the logging functions
to supply a newline at the right places.
(Some of these strings also have a *leading* newline, which would
be a good thing to get rid of too; but this patch doesn't attempt
that.)
There are some consequent minor changes in output. The ones that
aren't outright bug fixes are generally removal of extra blank
lines that the original coding intentionally inserted. It didn't
seem worth being bug-compatible with that.
Patch by me, reviewed by Kyotaro Horiguchi and Peter Eisentraut
Discussion: https://postgr.es/m/113191.1655233060@sss.pgh.pa.us
3 years ago
|
|
|
"the old and new port numbers must be different.");
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
static void
|
|
|
|
|
check_new_cluster_is_empty(void)
|
|
|
|
|
{
|
|
|
|
|
int dbnum;
|
|
|
|
|
|
|
|
|
|
for (dbnum = 0; dbnum < new_cluster.dbarr.ndbs; dbnum++)
|
|
|
|
|
{
|
|
|
|
|
int relnum;
|
|
|
|
|
RelInfoArr *rel_arr = &new_cluster.dbarr.dbs[dbnum].rel_arr;
|
|
|
|
|
|
|
|
|
|
for (relnum = 0; relnum < rel_arr->nrels;
|
|
|
|
|
relnum++)
|
|
|
|
|
{
|
|
|
|
|
/* pg_largeobject and its index should be skipped */
|
|
|
|
|
if (strcmp(rel_arr->rels[relnum].nspname, "pg_catalog") != 0)
|
Remove trailing newlines in pg_upgrade's message strings.
pg_upgrade does not use common/logging.c, which is unfortunate
but changing it to do so seems like more work than is justified.
However, we really need to make it work more like common/logging.c
in one respect: the latter expects supplied message strings to not
end with a newline, instead adding one internally. As it stands,
pg_upgrade's logging facilities expect a caller-supplied newline
in some cases and not others, which is already an invitation to bugs,
but the inconsistency with our other frontend code makes it worse.
There are already several places with missing or extra newlines,
and it's inevitable that there won't be more if we let this stand.
Hence, run around and get rid of all trailing newlines in message
strings, and add an Assert that there's not one, similar to the
existing Assert in common/logging.c. Adjust the logging functions
to supply a newline at the right places.
(Some of these strings also have a *leading* newline, which would
be a good thing to get rid of too; but this patch doesn't attempt
that.)
There are some consequent minor changes in output. The ones that
aren't outright bug fixes are generally removal of extra blank
lines that the original coding intentionally inserted. It didn't
seem worth being bug-compatible with that.
Patch by me, reviewed by Kyotaro Horiguchi and Peter Eisentraut
Discussion: https://postgr.es/m/113191.1655233060@sss.pgh.pa.us
3 years ago
|
|
|
pg_fatal("New cluster database \"%s\" is not empty: found relation \"%s.%s\"",
|
|
|
|
|
new_cluster.dbarr.dbs[dbnum].db_name,
|
|
|
|
|
rel_arr->rels[relnum].nspname,
|
|
|
|
|
rel_arr->rels[relnum].relname);
|
|
|
|
|
}
|
|
|
|
|
}
|
Change the way encoding and locale checks are done in pg_upgrade.
Lc_collate and lc_ctype have been per-database settings since server version
8.4, but pg_upgrade was still treating them as cluster-wide options. It
fetched the values for the template0 databases in old and new cluster, and
compared them. That's backwards; the encoding and locale of the template0
database doesn't matter, as template0 is guaranteed to contain only ASCII
characters. But if there are any other databases that exist on both clusters
(in particular template1 and postgres databases), their encodings and
locales must be compatible.
Also, make the locale comparison more lenient. If the locale names are not
equal, try to canonicalize both of them by passing them to setlocale(). We
used to do that only when upgrading from 9.1 or below, but it seems like a
good idea even with newer versions. If we change the canonical form of a
locale, this allows pg_upgrade to still work. I'm about to do just that to
fix bug #11431, by mapping a locale name that contains non-ASCII characters
to a pure-ASCII alias of the same locale.
No backpatching, because earlier versions of pg_upgrade still support
upgrading from 8.3 servers. That would be more complicated, so it doesn't
seem worth it, given that we haven't received any complaints about this
from users.
11 years ago
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* A previous run of pg_upgrade might have failed and the new cluster
|
|
|
|
|
* directory recreated, but they might have forgotten to remove
|
|
|
|
|
* the new cluster's tablespace directories. Therefore, check that
|
|
|
|
|
* new cluster tablespace directories do not already exist. If
|
|
|
|
|
* they do, it would cause an error while restoring global objects.
|
|
|
|
|
* This allows the failure to be detected at check time, rather than
|
|
|
|
|
* during schema restore.
|
|
|
|
|
*/
|
|
|
|
|
static void
|
|
|
|
|
check_for_new_tablespace_dir(void)
|
|
|
|
|
{
|
|
|
|
|
int tblnum;
|
|
|
|
|
char new_tablespace_dir[MAXPGPATH];
|
|
|
|
|
|
|
|
|
|
prep_status("Checking for new cluster tablespace directories");
|
|
|
|
|
|
|
|
|
|
for (tblnum = 0; tblnum < os_info.num_old_tablespaces; tblnum++)
|
|
|
|
|
{
|
|
|
|
|
struct stat statbuf;
|
|
|
|
|
|
|
|
|
|
snprintf(new_tablespace_dir, MAXPGPATH, "%s%s",
|
|
|
|
|
os_info.old_tablespaces[tblnum],
|
|
|
|
|
new_cluster.tablespace_suffix);
|
|
|
|
|
|
|
|
|
|
if (stat(new_tablespace_dir, &statbuf) == 0 || errno != ENOENT)
|
Remove trailing newlines in pg_upgrade's message strings.
pg_upgrade does not use common/logging.c, which is unfortunate
but changing it to do so seems like more work than is justified.
However, we really need to make it work more like common/logging.c
in one respect: the latter expects supplied message strings to not
end with a newline, instead adding one internally. As it stands,
pg_upgrade's logging facilities expect a caller-supplied newline
in some cases and not others, which is already an invitation to bugs,
but the inconsistency with our other frontend code makes it worse.
There are already several places with missing or extra newlines,
and it's inevitable that there won't be more if we let this stand.
Hence, run around and get rid of all trailing newlines in message
strings, and add an Assert that there's not one, similar to the
existing Assert in common/logging.c. Adjust the logging functions
to supply a newline at the right places.
(Some of these strings also have a *leading* newline, which would
be a good thing to get rid of too; but this patch doesn't attempt
that.)
There are some consequent minor changes in output. The ones that
aren't outright bug fixes are generally removal of extra blank
lines that the original coding intentionally inserted. It didn't
seem worth being bug-compatible with that.
Patch by me, reviewed by Kyotaro Horiguchi and Peter Eisentraut
Discussion: https://postgr.es/m/113191.1655233060@sss.pgh.pa.us
3 years ago
|
|
|
pg_fatal("new cluster tablespace directory already exists: \"%s\"",
|
|
|
|
|
new_tablespace_dir);
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
check_ok();
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* create_script_for_old_cluster_deletion()
|
|
|
|
|
*
|
|
|
|
|
* This is particularly useful for tablespace deletion.
|
|
|
|
|
*/
|
|
|
|
|
void
|
|
|
|
|
create_script_for_old_cluster_deletion(char **deletion_script_file_name)
|
|
|
|
|
{
|
|
|
|
|
FILE *script = NULL;
|
|
|
|
|
int tblnum;
|
|
|
|
|
char old_cluster_pgdata[MAXPGPATH],
|
|
|
|
|
new_cluster_pgdata[MAXPGPATH];
|
|
|
|
|
|
|
|
|
|
*deletion_script_file_name = psprintf("%sdelete_old_cluster.%s",
|
|
|
|
|
SCRIPT_PREFIX, SCRIPT_EXT);
|
|
|
|
|
|
|
|
|
|
strlcpy(old_cluster_pgdata, old_cluster.pgdata, MAXPGPATH);
|
|
|
|
|
canonicalize_path(old_cluster_pgdata);
|
|
|
|
|
|
|
|
|
|
strlcpy(new_cluster_pgdata, new_cluster.pgdata, MAXPGPATH);
|
|
|
|
|
canonicalize_path(new_cluster_pgdata);
|
|
|
|
|
|
|
|
|
|
/* Some people put the new data directory inside the old one. */
|
|
|
|
|
if (path_is_prefix_of_path(old_cluster_pgdata, new_cluster_pgdata))
|
|
|
|
|
{
|
|
|
|
|
pg_log(PG_WARNING,
|
|
|
|
|
"\nWARNING: new data directory should not be inside the old data directory, i.e. %s", old_cluster_pgdata);
|
|
|
|
|
|
|
|
|
|
/* Unlink file in case it is left over from a previous run. */
|
|
|
|
|
unlink(*deletion_script_file_name);
|
|
|
|
|
pg_free(*deletion_script_file_name);
|
|
|
|
|
*deletion_script_file_name = NULL;
|
|
|
|
|
return;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Some users (oddly) create tablespaces inside the cluster data
|
|
|
|
|
* directory. We can't create a proper old cluster delete script in that
|
|
|
|
|
* case.
|
|
|
|
|
*/
|
|
|
|
|
for (tblnum = 0; tblnum < os_info.num_old_tablespaces; tblnum++)
|
|
|
|
|
{
|
|
|
|
|
char old_tablespace_dir[MAXPGPATH];
|
|
|
|
|
|
|
|
|
|
strlcpy(old_tablespace_dir, os_info.old_tablespaces[tblnum], MAXPGPATH);
|
|
|
|
|
canonicalize_path(old_tablespace_dir);
|
|
|
|
|
if (path_is_prefix_of_path(old_cluster_pgdata, old_tablespace_dir))
|
|
|
|
|
{
|
|
|
|
|
/* reproduce warning from CREATE TABLESPACE that is in the log */
|
|
|
|
|
pg_log(PG_WARNING,
|
|
|
|
|
"\nWARNING: user-defined tablespace locations should not be inside the data directory, i.e. %s", old_tablespace_dir);
|
|
|
|
|
|
|
|
|
|
/* Unlink file in case it is left over from a previous run. */
|
|
|
|
|
unlink(*deletion_script_file_name);
|
|
|
|
|
pg_free(*deletion_script_file_name);
|
|
|
|
|
*deletion_script_file_name = NULL;
|
|
|
|
|
return;
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
prep_status("Creating script to delete old cluster");
|
|
|
|
|
|
|
|
|
|
if ((script = fopen_priv(*deletion_script_file_name, "w")) == NULL)
|
|
|
|
|
pg_fatal("could not open file \"%s\": %m",
|
|
|
|
|
*deletion_script_file_name);
|
|
|
|
|
|
|
|
|
|
#ifndef WIN32
|
|
|
|
|
/* add shebang header */
|
|
|
|
|
fprintf(script, "#!/bin/sh\n\n");
|
|
|
|
|
#endif
|
|
|
|
|
|
|
|
|
|
/* delete old cluster's default tablespace */
|
|
|
|
|
fprintf(script, RMDIR_CMD " %c%s%c\n", PATH_QUOTE,
|
|
|
|
|
fix_path_separator(old_cluster.pgdata), PATH_QUOTE);
|
|
|
|
|
|
|
|
|
|
/* delete old cluster's alternate tablespaces */
|
|
|
|
|
for (tblnum = 0; tblnum < os_info.num_old_tablespaces; tblnum++)
|
|
|
|
|
{
|
|
|
|
|
/*
|
|
|
|
|
* Do the old cluster's per-database directories share a directory
|
|
|
|
|
* with a new version-specific tablespace?
|
|
|
|
|
*/
|
|
|
|
|
if (strlen(old_cluster.tablespace_suffix) == 0)
|
|
|
|
|
{
|
|
|
|
|
/* delete per-database directories */
|
|
|
|
|
int dbnum;
|
|
|
|
|
|
|
|
|
|
fprintf(script, "\n");
|
|
|
|
|
|
|
|
|
|
for (dbnum = 0; dbnum < old_cluster.dbarr.ndbs; dbnum++)
|
|
|
|
|
fprintf(script, RMDIR_CMD " %c%s%c%u%c\n", PATH_QUOTE,
|
|
|
|
|
fix_path_separator(os_info.old_tablespaces[tblnum]),
|
|
|
|
|
PATH_SEPARATOR, old_cluster.dbarr.dbs[dbnum].db_oid,
|
|
|
|
|
PATH_QUOTE);
|
|
|
|
|
}
|
|
|
|
|
else
|
|
|
|
|
{
|
|
|
|
|
char *suffix_path = pg_strdup(old_cluster.tablespace_suffix);
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Simply delete the tablespace directory, which might be ".old"
|
|
|
|
|
* or a version-specific subdirectory.
|
|
|
|
|
*/
|
|
|
|
|
fprintf(script, RMDIR_CMD " %c%s%s%c\n", PATH_QUOTE,
|
|
|
|
|
fix_path_separator(os_info.old_tablespaces[tblnum]),
|
|
|
|
|
fix_path_separator(suffix_path), PATH_QUOTE);
|
|
|
|
|
pfree(suffix_path);
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
fclose(script);
|
|
|
|
|
|
|
|
|
|
#ifndef WIN32
|
|
|
|
|
if (chmod(*deletion_script_file_name, S_IRWXU) != 0)
|
|
|
|
|
pg_fatal("could not add execute permission to file \"%s\": %m",
|
|
|
|
|
*deletion_script_file_name);
|
|
|
|
|
#endif
|
|
|
|
|
|
|
|
|
|
check_ok();
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* check_is_install_user()
|
|
|
|
|
*
|
|
|
|
|
* Check we are the install user, and that the new cluster
|
|
|
|
|
* has no other users.
|
|
|
|
|
*/
|
|
|
|
|
static void
|
|
|
|
|
check_is_install_user(ClusterInfo *cluster)
|
|
|
|
|
{
|
|
|
|
|
PGresult *res;
|
|
|
|
|
PGconn *conn = connectToServer(cluster, "template1");
|
|
|
|
|
|
|
|
|
|
prep_status("Checking database user is the install user");
|
|
|
|
|
|
|
|
|
|
/* Can't use pg_authid because only superusers can view it. */
|
|
|
|
|
res = executeQueryOrDie(conn,
|
|
|
|
|
"SELECT rolsuper, oid "
|
|
|
|
|
"FROM pg_catalog.pg_roles "
|
|
|
|
|
"WHERE rolname = current_user "
|
|
|
|
|
"AND rolname !~ '^pg_'");
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* We only allow the install user in the new cluster (see comment below)
|
|
|
|
|
* and we preserve pg_authid.oid, so this must be the install user in the
|
|
|
|
|
* old cluster too.
|
|
|
|
|
*/
|
|
|
|
|
if (PQntuples(res) != 1 ||
|
|
|
|
|
atooid(PQgetvalue(res, 0, 1)) != BOOTSTRAP_SUPERUSERID)
|
Remove trailing newlines in pg_upgrade's message strings.
pg_upgrade does not use common/logging.c, which is unfortunate
but changing it to do so seems like more work than is justified.
However, we really need to make it work more like common/logging.c
in one respect: the latter expects supplied message strings to not
end with a newline, instead adding one internally. As it stands,
pg_upgrade's logging facilities expect a caller-supplied newline
in some cases and not others, which is already an invitation to bugs,
but the inconsistency with our other frontend code makes it worse.
There are already several places with missing or extra newlines,
and it's inevitable that there won't be more if we let this stand.
Hence, run around and get rid of all trailing newlines in message
strings, and add an Assert that there's not one, similar to the
existing Assert in common/logging.c. Adjust the logging functions
to supply a newline at the right places.
(Some of these strings also have a *leading* newline, which would
be a good thing to get rid of too; but this patch doesn't attempt
that.)
There are some consequent minor changes in output. The ones that
aren't outright bug fixes are generally removal of extra blank
lines that the original coding intentionally inserted. It didn't
seem worth being bug-compatible with that.
Patch by me, reviewed by Kyotaro Horiguchi and Peter Eisentraut
Discussion: https://postgr.es/m/113191.1655233060@sss.pgh.pa.us
3 years ago
|
|
|
pg_fatal("database user \"%s\" is not the install user",
|
|
|
|
|
os_info.user);
|
|
|
|
|
|
|
|
|
|
PQclear(res);
|
|
|
|
|
|
|
|
|
|
res = executeQueryOrDie(conn,
|
|
|
|
|
"SELECT COUNT(*) "
|
|
|
|
|
"FROM pg_catalog.pg_roles "
|
|
|
|
|
"WHERE rolname !~ '^pg_'");
|
|
|
|
|
|
|
|
|
|
if (PQntuples(res) != 1)
|
Remove trailing newlines in pg_upgrade's message strings.
pg_upgrade does not use common/logging.c, which is unfortunate
but changing it to do so seems like more work than is justified.
However, we really need to make it work more like common/logging.c
in one respect: the latter expects supplied message strings to not
end with a newline, instead adding one internally. As it stands,
pg_upgrade's logging facilities expect a caller-supplied newline
in some cases and not others, which is already an invitation to bugs,
but the inconsistency with our other frontend code makes it worse.
There are already several places with missing or extra newlines,
and it's inevitable that there won't be more if we let this stand.
Hence, run around and get rid of all trailing newlines in message
strings, and add an Assert that there's not one, similar to the
existing Assert in common/logging.c. Adjust the logging functions
to supply a newline at the right places.
(Some of these strings also have a *leading* newline, which would
be a good thing to get rid of too; but this patch doesn't attempt
that.)
There are some consequent minor changes in output. The ones that
aren't outright bug fixes are generally removal of extra blank
lines that the original coding intentionally inserted. It didn't
seem worth being bug-compatible with that.
Patch by me, reviewed by Kyotaro Horiguchi and Peter Eisentraut
Discussion: https://postgr.es/m/113191.1655233060@sss.pgh.pa.us
3 years ago
|
|
|
pg_fatal("could not determine the number of users");
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* We only allow the install user in the new cluster because other defined
|
|
|
|
|
* users might match users defined in the old cluster and generate an
|
|
|
|
|
* error during pg_dump restore.
|
|
|
|
|
*/
|
|
|
|
|
if (cluster == &new_cluster && strcmp(PQgetvalue(res, 0, 0), "1") != 0)
|
Remove trailing newlines in pg_upgrade's message strings.
pg_upgrade does not use common/logging.c, which is unfortunate
but changing it to do so seems like more work than is justified.
However, we really need to make it work more like common/logging.c
in one respect: the latter expects supplied message strings to not
end with a newline, instead adding one internally. As it stands,
pg_upgrade's logging facilities expect a caller-supplied newline
in some cases and not others, which is already an invitation to bugs,
but the inconsistency with our other frontend code makes it worse.
There are already several places with missing or extra newlines,
and it's inevitable that there won't be more if we let this stand.
Hence, run around and get rid of all trailing newlines in message
strings, and add an Assert that there's not one, similar to the
existing Assert in common/logging.c. Adjust the logging functions
to supply a newline at the right places.
(Some of these strings also have a *leading* newline, which would
be a good thing to get rid of too; but this patch doesn't attempt
that.)
There are some consequent minor changes in output. The ones that
aren't outright bug fixes are generally removal of extra blank
lines that the original coding intentionally inserted. It didn't
seem worth being bug-compatible with that.
Patch by me, reviewed by Kyotaro Horiguchi and Peter Eisentraut
Discussion: https://postgr.es/m/113191.1655233060@sss.pgh.pa.us
3 years ago
|
|
|
pg_fatal("Only the install user can be defined in the new cluster.");
|
|
|
|
|
|
|
|
|
|
PQclear(res);
|
|
|
|
|
|
|
|
|
|
PQfinish(conn);
|
|
|
|
|
|
|
|
|
|
check_ok();
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* check_proper_datallowconn
|
|
|
|
|
*
|
|
|
|
|
* Ensure that all non-template0 databases allow connections since they
|
|
|
|
|
* otherwise won't be restored; and that template0 explicitly doesn't allow
|
|
|
|
|
* connections since it would make pg_dumpall --globals restore fail.
|
|
|
|
|
*/
|
|
|
|
|
static void
|
|
|
|
|
check_proper_datallowconn(ClusterInfo *cluster)
|
|
|
|
|
{
|
|
|
|
|
int dbnum;
|
|
|
|
|
PGconn *conn_template1;
|
|
|
|
|
PGresult *dbres;
|
|
|
|
|
int ntups;
|
|
|
|
|
int i_datname;
|
|
|
|
|
int i_datallowconn;
|
|
|
|
|
FILE *script = NULL;
|
|
|
|
|
char output_path[MAXPGPATH];
|
|
|
|
|
|
|
|
|
|
prep_status("Checking database connection settings");
|
|
|
|
|
|
|
|
|
|
snprintf(output_path, sizeof(output_path), "%s/%s",
|
|
|
|
|
log_opts.basedir,
|
|
|
|
|
"databases_with_datallowconn_false.txt");
|
|
|
|
|
|
|
|
|
|
conn_template1 = connectToServer(cluster, "template1");
|
|
|
|
|
|
|
|
|
|
/* get database names */
|
|
|
|
|
dbres = executeQueryOrDie(conn_template1,
|
|
|
|
|
"SELECT datname, datallowconn "
|
|
|
|
|
"FROM pg_catalog.pg_database");
|
|
|
|
|
|
|
|
|
|
i_datname = PQfnumber(dbres, "datname");
|
|
|
|
|
i_datallowconn = PQfnumber(dbres, "datallowconn");
|
|
|
|
|
|
|
|
|
|
ntups = PQntuples(dbres);
|
|
|
|
|
for (dbnum = 0; dbnum < ntups; dbnum++)
|
|
|
|
|
{
|
|
|
|
|
char *datname = PQgetvalue(dbres, dbnum, i_datname);
|
|
|
|
|
char *datallowconn = PQgetvalue(dbres, dbnum, i_datallowconn);
|
|
|
|
|
|
|
|
|
|
if (strcmp(datname, "template0") == 0)
|
|
|
|
|
{
|
|
|
|
|
/* avoid restore failure when pg_dumpall tries to create template0 */
|
|
|
|
|
if (strcmp(datallowconn, "t") == 0)
|
|
|
|
|
pg_fatal("template0 must not allow connections, "
|
Remove trailing newlines in pg_upgrade's message strings.
pg_upgrade does not use common/logging.c, which is unfortunate
but changing it to do so seems like more work than is justified.
However, we really need to make it work more like common/logging.c
in one respect: the latter expects supplied message strings to not
end with a newline, instead adding one internally. As it stands,
pg_upgrade's logging facilities expect a caller-supplied newline
in some cases and not others, which is already an invitation to bugs,
but the inconsistency with our other frontend code makes it worse.
There are already several places with missing or extra newlines,
and it's inevitable that there won't be more if we let this stand.
Hence, run around and get rid of all trailing newlines in message
strings, and add an Assert that there's not one, similar to the
existing Assert in common/logging.c. Adjust the logging functions
to supply a newline at the right places.
(Some of these strings also have a *leading* newline, which would
be a good thing to get rid of too; but this patch doesn't attempt
that.)
There are some consequent minor changes in output. The ones that
aren't outright bug fixes are generally removal of extra blank
lines that the original coding intentionally inserted. It didn't
seem worth being bug-compatible with that.
Patch by me, reviewed by Kyotaro Horiguchi and Peter Eisentraut
Discussion: https://postgr.es/m/113191.1655233060@sss.pgh.pa.us
3 years ago
|
|
|
"i.e. its pg_database.datallowconn must be false");
|
|
|
|
|
}
|
|
|
|
|
else
|
|
|
|
|
{
|
|
|
|
|
/*
|
|
|
|
|
* avoid datallowconn == false databases from being skipped on
|
|
|
|
|
* restore
|
|
|
|
|
*/
|
|
|
|
|
if (strcmp(datallowconn, "f") == 0)
|
|
|
|
|
{
|
|
|
|
|
if (script == NULL && (script = fopen_priv(output_path, "w")) == NULL)
|
|
|
|
|
pg_fatal("could not open file \"%s\": %m", output_path);
|
|
|
|
|
|
|
|
|
|
fprintf(script, "%s\n", datname);
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
PQclear(dbres);
|
|
|
|
|
|
|
|
|
|
PQfinish(conn_template1);
|
|
|
|
|
|
|
|
|
|
if (script)
|
|
|
|
|
{
|
|
|
|
|
fclose(script);
|
Remove trailing newlines in pg_upgrade's message strings.
pg_upgrade does not use common/logging.c, which is unfortunate
but changing it to do so seems like more work than is justified.
However, we really need to make it work more like common/logging.c
in one respect: the latter expects supplied message strings to not
end with a newline, instead adding one internally. As it stands,
pg_upgrade's logging facilities expect a caller-supplied newline
in some cases and not others, which is already an invitation to bugs,
but the inconsistency with our other frontend code makes it worse.
There are already several places with missing or extra newlines,
and it's inevitable that there won't be more if we let this stand.
Hence, run around and get rid of all trailing newlines in message
strings, and add an Assert that there's not one, similar to the
existing Assert in common/logging.c. Adjust the logging functions
to supply a newline at the right places.
(Some of these strings also have a *leading* newline, which would
be a good thing to get rid of too; but this patch doesn't attempt
that.)
There are some consequent minor changes in output. The ones that
aren't outright bug fixes are generally removal of extra blank
lines that the original coding intentionally inserted. It didn't
seem worth being bug-compatible with that.
Patch by me, reviewed by Kyotaro Horiguchi and Peter Eisentraut
Discussion: https://postgr.es/m/113191.1655233060@sss.pgh.pa.us
3 years ago
|
|
|
pg_log(PG_REPORT, "fatal");
|
|
|
|
|
pg_fatal("All non-template0 databases must allow connections, i.e. their\n"
|
|
|
|
|
"pg_database.datallowconn must be true. Your installation contains\n"
|
|
|
|
|
"non-template0 databases with their pg_database.datallowconn set to\n"
|
|
|
|
|
"false. Consider allowing connection for all non-template0 databases\n"
|
|
|
|
|
"or drop the databases which do not allow connections. A list of\n"
|
|
|
|
|
"databases with the problem is in the file:\n"
|
Remove trailing newlines in pg_upgrade's message strings.
pg_upgrade does not use common/logging.c, which is unfortunate
but changing it to do so seems like more work than is justified.
However, we really need to make it work more like common/logging.c
in one respect: the latter expects supplied message strings to not
end with a newline, instead adding one internally. As it stands,
pg_upgrade's logging facilities expect a caller-supplied newline
in some cases and not others, which is already an invitation to bugs,
but the inconsistency with our other frontend code makes it worse.
There are already several places with missing or extra newlines,
and it's inevitable that there won't be more if we let this stand.
Hence, run around and get rid of all trailing newlines in message
strings, and add an Assert that there's not one, similar to the
existing Assert in common/logging.c. Adjust the logging functions
to supply a newline at the right places.
(Some of these strings also have a *leading* newline, which would
be a good thing to get rid of too; but this patch doesn't attempt
that.)
There are some consequent minor changes in output. The ones that
aren't outright bug fixes are generally removal of extra blank
lines that the original coding intentionally inserted. It didn't
seem worth being bug-compatible with that.
Patch by me, reviewed by Kyotaro Horiguchi and Peter Eisentraut
Discussion: https://postgr.es/m/113191.1655233060@sss.pgh.pa.us
3 years ago
|
|
|
" %s", output_path);
|
|
|
|
|
}
|
|
|
|
|
else
|
|
|
|
|
check_ok();
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* check_for_prepared_transactions()
|
|
|
|
|
*
|
|
|
|
|
* Make sure there are no prepared transactions because the storage format
|
|
|
|
|
* might have changed.
|
|
|
|
|
*/
|
|
|
|
|
static void
|
|
|
|
|
check_for_prepared_transactions(ClusterInfo *cluster)
|
|
|
|
|
{
|
|
|
|
|
PGresult *res;
|
|
|
|
|
PGconn *conn = connectToServer(cluster, "template1");
|
|
|
|
|
|
|
|
|
|
prep_status("Checking for prepared transactions");
|
|
|
|
|
|
|
|
|
|
res = executeQueryOrDie(conn,
|
|
|
|
|
"SELECT * "
|
|
|
|
|
"FROM pg_catalog.pg_prepared_xacts");
|
|
|
|
|
|
|
|
|
|
if (PQntuples(res) != 0)
|
|
|
|
|
{
|
|
|
|
|
if (cluster == &old_cluster)
|
Remove trailing newlines in pg_upgrade's message strings.
pg_upgrade does not use common/logging.c, which is unfortunate
but changing it to do so seems like more work than is justified.
However, we really need to make it work more like common/logging.c
in one respect: the latter expects supplied message strings to not
end with a newline, instead adding one internally. As it stands,
pg_upgrade's logging facilities expect a caller-supplied newline
in some cases and not others, which is already an invitation to bugs,
but the inconsistency with our other frontend code makes it worse.
There are already several places with missing or extra newlines,
and it's inevitable that there won't be more if we let this stand.
Hence, run around and get rid of all trailing newlines in message
strings, and add an Assert that there's not one, similar to the
existing Assert in common/logging.c. Adjust the logging functions
to supply a newline at the right places.
(Some of these strings also have a *leading* newline, which would
be a good thing to get rid of too; but this patch doesn't attempt
that.)
There are some consequent minor changes in output. The ones that
aren't outright bug fixes are generally removal of extra blank
lines that the original coding intentionally inserted. It didn't
seem worth being bug-compatible with that.
Patch by me, reviewed by Kyotaro Horiguchi and Peter Eisentraut
Discussion: https://postgr.es/m/113191.1655233060@sss.pgh.pa.us
3 years ago
|
|
|
pg_fatal("The source cluster contains prepared transactions");
|
|
|
|
|
else
|
Remove trailing newlines in pg_upgrade's message strings.
pg_upgrade does not use common/logging.c, which is unfortunate
but changing it to do so seems like more work than is justified.
However, we really need to make it work more like common/logging.c
in one respect: the latter expects supplied message strings to not
end with a newline, instead adding one internally. As it stands,
pg_upgrade's logging facilities expect a caller-supplied newline
in some cases and not others, which is already an invitation to bugs,
but the inconsistency with our other frontend code makes it worse.
There are already several places with missing or extra newlines,
and it's inevitable that there won't be more if we let this stand.
Hence, run around and get rid of all trailing newlines in message
strings, and add an Assert that there's not one, similar to the
existing Assert in common/logging.c. Adjust the logging functions
to supply a newline at the right places.
(Some of these strings also have a *leading* newline, which would
be a good thing to get rid of too; but this patch doesn't attempt
that.)
There are some consequent minor changes in output. The ones that
aren't outright bug fixes are generally removal of extra blank
lines that the original coding intentionally inserted. It didn't
seem worth being bug-compatible with that.
Patch by me, reviewed by Kyotaro Horiguchi and Peter Eisentraut
Discussion: https://postgr.es/m/113191.1655233060@sss.pgh.pa.us
3 years ago
|
|
|
pg_fatal("The target cluster contains prepared transactions");
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
PQclear(res);
|
|
|
|
|
|
|
|
|
|
PQfinish(conn);
|
|
|
|
|
|
|
|
|
|
check_ok();
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Callback function for processing result of query for
|
|
|
|
|
* check_for_isn_and_int8_passing_mismatch()'s UpgradeTask. If the query
|
|
|
|
|
* returned any rows (i.e., the check failed), write the details to the report
|
|
|
|
|
* file.
|
|
|
|
|
*/
|
|
|
|
|
static void
|
|
|
|
|
process_isn_and_int8_passing_mismatch(DbInfo *dbinfo, PGresult *res, void *arg)
|
|
|
|
|
{
|
|
|
|
|
bool db_used = false;
|
|
|
|
|
int ntups = PQntuples(res);
|
|
|
|
|
int i_nspname = PQfnumber(res, "nspname");
|
|
|
|
|
int i_proname = PQfnumber(res, "proname");
|
|
|
|
|
UpgradeTaskReport *report = (UpgradeTaskReport *) arg;
|
|
|
|
|
|
|
|
|
|
AssertVariableIsOfType(&process_isn_and_int8_passing_mismatch,
|
|
|
|
|
UpgradeTaskProcessCB);
|
|
|
|
|
|
|
|
|
|
for (int rowno = 0; rowno < ntups; rowno++)
|
|
|
|
|
{
|
|
|
|
|
if (report->file == NULL &&
|
|
|
|
|
(report->file = fopen_priv(report->path, "w")) == NULL)
|
|
|
|
|
pg_fatal("could not open file \"%s\": %m", report->path);
|
|
|
|
|
if (!db_used)
|
|
|
|
|
{
|
|
|
|
|
fprintf(report->file, "In database: %s\n", dbinfo->db_name);
|
|
|
|
|
db_used = true;
|
|
|
|
|
}
|
|
|
|
|
fprintf(report->file, " %s.%s\n",
|
|
|
|
|
PQgetvalue(res, rowno, i_nspname),
|
|
|
|
|
PQgetvalue(res, rowno, i_proname));
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* check_for_isn_and_int8_passing_mismatch()
|
|
|
|
|
*
|
|
|
|
|
* contrib/isn relies on data type int8, and in 8.4 int8 can now be passed
|
|
|
|
|
* by value. The schema dumps the CREATE TYPE PASSEDBYVALUE setting so
|
|
|
|
|
* it must match for the old and new servers.
|
|
|
|
|
*/
|
|
|
|
|
static void
|
|
|
|
|
check_for_isn_and_int8_passing_mismatch(ClusterInfo *cluster)
|
|
|
|
|
{
|
|
|
|
|
UpgradeTask *task;
|
|
|
|
|
UpgradeTaskReport report;
|
|
|
|
|
const char *query = "SELECT n.nspname, p.proname "
|
|
|
|
|
"FROM pg_catalog.pg_proc p, "
|
|
|
|
|
" pg_catalog.pg_namespace n "
|
|
|
|
|
"WHERE p.pronamespace = n.oid AND "
|
|
|
|
|
" p.probin = '$libdir/isn'";
|
|
|
|
|
|
|
|
|
|
prep_status("Checking for contrib/isn with bigint-passing mismatch");
|
|
|
|
|
|
|
|
|
|
if (old_cluster.controldata.float8_pass_by_value ==
|
|
|
|
|
new_cluster.controldata.float8_pass_by_value)
|
|
|
|
|
{
|
|
|
|
|
/* no mismatch */
|
|
|
|
|
check_ok();
|
|
|
|
|
return;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
report.file = NULL;
|
|
|
|
|
snprintf(report.path, sizeof(report.path), "%s/%s",
|
pg_upgrade: Move all the files generated internally to a subdirectory
Historically, the location of any files generated by pg_upgrade, as of
the per-database logs and internal dumps, has been the current working
directory, leaving all those files behind when using --retain or on a
failure.
Putting all those contents in a targeted subdirectory makes the whole
easier to debug, and simplifies the code in charge of cleaning up the
logs. Note that another reason is that this facilitates the move of
pg_upgrade to TAP with a fixed location for all the logs to grab if the
test fails repeatedly.
Initially, we thought about being able to specify the output directory
with a new option, but we have settled on using a subdirectory located
at the root of the new cluster's data folder, "pg_upgrade_output.d",
instead, as at the end the new data directory is the location of all the
data generated by pg_upgrade. There is a take with group permissions
here though: if the new data folder has been initialized with this
option, we need to create all the files and paths with the correct
permissions or a base backup taken after a pg_upgrade --retain would
fail, meaning that GetDataDirectoryCreatePerm() has to be called before
creating the log paths, before a couple of sanity checks on the clusters
and before getting the socket directory for the cluster's host settings.
The idea of the new location is based on a suggestion from Peter
Eisentraut.
Also thanks to Andrew Dunstan, Peter Eisentraut, Daniel Gustafsson, Tom
Lane and Bruce Momjian for the discussion (in alphabetical order).
Author: Justin Pryzby
Discussion: https://postgr.es/m/20211212025017.GN17618@telsasoft.com
4 years ago
|
|
|
log_opts.basedir,
|
|
|
|
|
"contrib_isn_and_int8_pass_by_value.txt");
|
|
|
|
|
|
|
|
|
|
task = upgrade_task_create();
|
|
|
|
|
upgrade_task_add_step(task, query, process_isn_and_int8_passing_mismatch,
|
|
|
|
|
true, &report);
|
|
|
|
|
upgrade_task_run(task, cluster);
|
|
|
|
|
upgrade_task_free(task);
|
|
|
|
|
|
|
|
|
|
if (report.file)
|
|
|
|
|
{
|
|
|
|
|
fclose(report.file);
|
Remove trailing newlines in pg_upgrade's message strings.
pg_upgrade does not use common/logging.c, which is unfortunate
but changing it to do so seems like more work than is justified.
However, we really need to make it work more like common/logging.c
in one respect: the latter expects supplied message strings to not
end with a newline, instead adding one internally. As it stands,
pg_upgrade's logging facilities expect a caller-supplied newline
in some cases and not others, which is already an invitation to bugs,
but the inconsistency with our other frontend code makes it worse.
There are already several places with missing or extra newlines,
and it's inevitable that there won't be more if we let this stand.
Hence, run around and get rid of all trailing newlines in message
strings, and add an Assert that there's not one, similar to the
existing Assert in common/logging.c. Adjust the logging functions
to supply a newline at the right places.
(Some of these strings also have a *leading* newline, which would
be a good thing to get rid of too; but this patch doesn't attempt
that.)
There are some consequent minor changes in output. The ones that
aren't outright bug fixes are generally removal of extra blank
lines that the original coding intentionally inserted. It didn't
seem worth being bug-compatible with that.
Patch by me, reviewed by Kyotaro Horiguchi and Peter Eisentraut
Discussion: https://postgr.es/m/113191.1655233060@sss.pgh.pa.us
3 years ago
|
|
|
pg_log(PG_REPORT, "fatal");
|
|
|
|
|
pg_fatal("Your installation contains \"contrib/isn\" functions which rely on the\n"
|
Phase 3 of pgindent updates.
Don't move parenthesized lines to the left, even if that means they
flow past the right margin.
By default, BSD indent lines up statement continuation lines that are
within parentheses so that they start just to the right of the preceding
left parenthesis. However, traditionally, if that resulted in the
continuation line extending to the right of the desired right margin,
then indent would push it left just far enough to not overrun the margin,
if it could do so without making the continuation line start to the left of
the current statement indent. That makes for a weird mix of indentations
unless one has been completely rigid about never violating the 80-column
limit.
This behavior has been pretty universally panned by Postgres developers.
Hence, disable it with indent's new -lpl switch, so that parenthesized
lines are always lined up with the preceding left paren.
This patch is much less interesting than the first round of indent
changes, but also bulkier, so I thought it best to separate the effects.
Discussion: https://postgr.es/m/E1dAmxK-0006EE-1r@gemulon.postgresql.org
Discussion: https://postgr.es/m/30527.1495162840@sss.pgh.pa.us
9 years ago
|
|
|
"bigint data type. Your old and new clusters pass bigint values\n"
|
|
|
|
|
"differently so this cluster cannot currently be upgraded. You can\n"
|
|
|
|
|
"manually dump databases in the old cluster that use \"contrib/isn\"\n"
|
|
|
|
|
"facilities, drop them, perform the upgrade, and then restore them. A\n"
|
|
|
|
|
"list of the problem functions is in the file:\n"
|
|
|
|
|
" %s", report.path);
|
|
|
|
|
}
|
|
|
|
|
else
|
|
|
|
|
check_ok();
|
|
|
|
|
}
|
|
|
|
|
|
Remove support for postfix (right-unary) operators.
This feature has been a thorn in our sides for a long time, causing
many grammatical ambiguity problems. It doesn't seem worth the
pain to continue to support it, so remove it.
There are some follow-on improvements we can make in the grammar,
but this commit only removes the bare minimum number of productions,
plus assorted backend support code.
Note that pg_dump and psql continue to have full support, since
they may be used against older servers. However, pg_dump warns
about postfix operators. There is also a check in pg_upgrade.
Documentation-wise, I (tgl) largely removed the "left unary"
terminology in favor of saying "prefix operator", which is
a more standard and IMO less confusing term.
I included a catversion bump, although no initial catalog data
changes here, to mark the boundary at which oprkind = 'r'
stopped being valid in pg_operator.
Mark Dilger, based on work by myself and Robert Haas;
review by John Naylor
Discussion: https://postgr.es/m/38ca86db-42ab-9b48-2902-337a0d6b8311@2ndquadrant.com
5 years ago
|
|
|
/*
|
|
|
|
|
* Callback function for processing result of query for
|
|
|
|
|
* check_for_user_defined_postfix_ops()'s UpgradeTask. If the query returned
|
|
|
|
|
* any rows (i.e., the check failed), write the details to the report file.
|
Remove support for postfix (right-unary) operators.
This feature has been a thorn in our sides for a long time, causing
many grammatical ambiguity problems. It doesn't seem worth the
pain to continue to support it, so remove it.
There are some follow-on improvements we can make in the grammar,
but this commit only removes the bare minimum number of productions,
plus assorted backend support code.
Note that pg_dump and psql continue to have full support, since
they may be used against older servers. However, pg_dump warns
about postfix operators. There is also a check in pg_upgrade.
Documentation-wise, I (tgl) largely removed the "left unary"
terminology in favor of saying "prefix operator", which is
a more standard and IMO less confusing term.
I included a catversion bump, although no initial catalog data
changes here, to mark the boundary at which oprkind = 'r'
stopped being valid in pg_operator.
Mark Dilger, based on work by myself and Robert Haas;
review by John Naylor
Discussion: https://postgr.es/m/38ca86db-42ab-9b48-2902-337a0d6b8311@2ndquadrant.com
5 years ago
|
|
|
*/
|
|
|
|
|
static void
|
|
|
|
|
process_user_defined_postfix_ops(DbInfo *dbinfo, PGresult *res, void *arg)
|
Remove support for postfix (right-unary) operators.
This feature has been a thorn in our sides for a long time, causing
many grammatical ambiguity problems. It doesn't seem worth the
pain to continue to support it, so remove it.
There are some follow-on improvements we can make in the grammar,
but this commit only removes the bare minimum number of productions,
plus assorted backend support code.
Note that pg_dump and psql continue to have full support, since
they may be used against older servers. However, pg_dump warns
about postfix operators. There is also a check in pg_upgrade.
Documentation-wise, I (tgl) largely removed the "left unary"
terminology in favor of saying "prefix operator", which is
a more standard and IMO less confusing term.
I included a catversion bump, although no initial catalog data
changes here, to mark the boundary at which oprkind = 'r'
stopped being valid in pg_operator.
Mark Dilger, based on work by myself and Robert Haas;
review by John Naylor
Discussion: https://postgr.es/m/38ca86db-42ab-9b48-2902-337a0d6b8311@2ndquadrant.com
5 years ago
|
|
|
{
|
|
|
|
|
UpgradeTaskReport *report = (UpgradeTaskReport *) arg;
|
|
|
|
|
int ntups = PQntuples(res);
|
|
|
|
|
bool db_used = false;
|
|
|
|
|
int i_oproid = PQfnumber(res, "oproid");
|
|
|
|
|
int i_oprnsp = PQfnumber(res, "oprnsp");
|
|
|
|
|
int i_oprname = PQfnumber(res, "oprname");
|
|
|
|
|
int i_typnsp = PQfnumber(res, "typnsp");
|
|
|
|
|
int i_typname = PQfnumber(res, "typname");
|
Remove support for postfix (right-unary) operators.
This feature has been a thorn in our sides for a long time, causing
many grammatical ambiguity problems. It doesn't seem worth the
pain to continue to support it, so remove it.
There are some follow-on improvements we can make in the grammar,
but this commit only removes the bare minimum number of productions,
plus assorted backend support code.
Note that pg_dump and psql continue to have full support, since
they may be used against older servers. However, pg_dump warns
about postfix operators. There is also a check in pg_upgrade.
Documentation-wise, I (tgl) largely removed the "left unary"
terminology in favor of saying "prefix operator", which is
a more standard and IMO less confusing term.
I included a catversion bump, although no initial catalog data
changes here, to mark the boundary at which oprkind = 'r'
stopped being valid in pg_operator.
Mark Dilger, based on work by myself and Robert Haas;
review by John Naylor
Discussion: https://postgr.es/m/38ca86db-42ab-9b48-2902-337a0d6b8311@2ndquadrant.com
5 years ago
|
|
|
|
|
|
|
|
AssertVariableIsOfType(&process_user_defined_postfix_ops,
|
|
|
|
|
UpgradeTaskProcessCB);
|
Remove support for postfix (right-unary) operators.
This feature has been a thorn in our sides for a long time, causing
many grammatical ambiguity problems. It doesn't seem worth the
pain to continue to support it, so remove it.
There are some follow-on improvements we can make in the grammar,
but this commit only removes the bare minimum number of productions,
plus assorted backend support code.
Note that pg_dump and psql continue to have full support, since
they may be used against older servers. However, pg_dump warns
about postfix operators. There is also a check in pg_upgrade.
Documentation-wise, I (tgl) largely removed the "left unary"
terminology in favor of saying "prefix operator", which is
a more standard and IMO less confusing term.
I included a catversion bump, although no initial catalog data
changes here, to mark the boundary at which oprkind = 'r'
stopped being valid in pg_operator.
Mark Dilger, based on work by myself and Robert Haas;
review by John Naylor
Discussion: https://postgr.es/m/38ca86db-42ab-9b48-2902-337a0d6b8311@2ndquadrant.com
5 years ago
|
|
|
|
|
|
|
|
if (!ntups)
|
|
|
|
|
return;
|
Remove support for postfix (right-unary) operators.
This feature has been a thorn in our sides for a long time, causing
many grammatical ambiguity problems. It doesn't seem worth the
pain to continue to support it, so remove it.
There are some follow-on improvements we can make in the grammar,
but this commit only removes the bare minimum number of productions,
plus assorted backend support code.
Note that pg_dump and psql continue to have full support, since
they may be used against older servers. However, pg_dump warns
about postfix operators. There is also a check in pg_upgrade.
Documentation-wise, I (tgl) largely removed the "left unary"
terminology in favor of saying "prefix operator", which is
a more standard and IMO less confusing term.
I included a catversion bump, although no initial catalog data
changes here, to mark the boundary at which oprkind = 'r'
stopped being valid in pg_operator.
Mark Dilger, based on work by myself and Robert Haas;
review by John Naylor
Discussion: https://postgr.es/m/38ca86db-42ab-9b48-2902-337a0d6b8311@2ndquadrant.com
5 years ago
|
|
|
|
|
|
|
|
for (int rowno = 0; rowno < ntups; rowno++)
|
Remove support for postfix (right-unary) operators.
This feature has been a thorn in our sides for a long time, causing
many grammatical ambiguity problems. It doesn't seem worth the
pain to continue to support it, so remove it.
There are some follow-on improvements we can make in the grammar,
but this commit only removes the bare minimum number of productions,
plus assorted backend support code.
Note that pg_dump and psql continue to have full support, since
they may be used against older servers. However, pg_dump warns
about postfix operators. There is also a check in pg_upgrade.
Documentation-wise, I (tgl) largely removed the "left unary"
terminology in favor of saying "prefix operator", which is
a more standard and IMO less confusing term.
I included a catversion bump, although no initial catalog data
changes here, to mark the boundary at which oprkind = 'r'
stopped being valid in pg_operator.
Mark Dilger, based on work by myself and Robert Haas;
review by John Naylor
Discussion: https://postgr.es/m/38ca86db-42ab-9b48-2902-337a0d6b8311@2ndquadrant.com
5 years ago
|
|
|
{
|
|
|
|
|
if (report->file == NULL &&
|
|
|
|
|
(report->file = fopen_priv(report->path, "w")) == NULL)
|
|
|
|
|
pg_fatal("could not open file \"%s\": %m", report->path);
|
|
|
|
|
if (!db_used)
|
Remove support for postfix (right-unary) operators.
This feature has been a thorn in our sides for a long time, causing
many grammatical ambiguity problems. It doesn't seem worth the
pain to continue to support it, so remove it.
There are some follow-on improvements we can make in the grammar,
but this commit only removes the bare minimum number of productions,
plus assorted backend support code.
Note that pg_dump and psql continue to have full support, since
they may be used against older servers. However, pg_dump warns
about postfix operators. There is also a check in pg_upgrade.
Documentation-wise, I (tgl) largely removed the "left unary"
terminology in favor of saying "prefix operator", which is
a more standard and IMO less confusing term.
I included a catversion bump, although no initial catalog data
changes here, to mark the boundary at which oprkind = 'r'
stopped being valid in pg_operator.
Mark Dilger, based on work by myself and Robert Haas;
review by John Naylor
Discussion: https://postgr.es/m/38ca86db-42ab-9b48-2902-337a0d6b8311@2ndquadrant.com
5 years ago
|
|
|
{
|
|
|
|
|
fprintf(report->file, "In database: %s\n", dbinfo->db_name);
|
|
|
|
|
db_used = true;
|
Remove support for postfix (right-unary) operators.
This feature has been a thorn in our sides for a long time, causing
many grammatical ambiguity problems. It doesn't seem worth the
pain to continue to support it, so remove it.
There are some follow-on improvements we can make in the grammar,
but this commit only removes the bare minimum number of productions,
plus assorted backend support code.
Note that pg_dump and psql continue to have full support, since
they may be used against older servers. However, pg_dump warns
about postfix operators. There is also a check in pg_upgrade.
Documentation-wise, I (tgl) largely removed the "left unary"
terminology in favor of saying "prefix operator", which is
a more standard and IMO less confusing term.
I included a catversion bump, although no initial catalog data
changes here, to mark the boundary at which oprkind = 'r'
stopped being valid in pg_operator.
Mark Dilger, based on work by myself and Robert Haas;
review by John Naylor
Discussion: https://postgr.es/m/38ca86db-42ab-9b48-2902-337a0d6b8311@2ndquadrant.com
5 years ago
|
|
|
}
|
|
|
|
|
fprintf(report->file, " (oid=%s) %s.%s (%s.%s, NONE)\n",
|
|
|
|
|
PQgetvalue(res, rowno, i_oproid),
|
|
|
|
|
PQgetvalue(res, rowno, i_oprnsp),
|
|
|
|
|
PQgetvalue(res, rowno, i_oprname),
|
|
|
|
|
PQgetvalue(res, rowno, i_typnsp),
|
|
|
|
|
PQgetvalue(res, rowno, i_typname));
|
|
|
|
|
}
|
|
|
|
|
}
|
Remove support for postfix (right-unary) operators.
This feature has been a thorn in our sides for a long time, causing
many grammatical ambiguity problems. It doesn't seem worth the
pain to continue to support it, so remove it.
There are some follow-on improvements we can make in the grammar,
but this commit only removes the bare minimum number of productions,
plus assorted backend support code.
Note that pg_dump and psql continue to have full support, since
they may be used against older servers. However, pg_dump warns
about postfix operators. There is also a check in pg_upgrade.
Documentation-wise, I (tgl) largely removed the "left unary"
terminology in favor of saying "prefix operator", which is
a more standard and IMO less confusing term.
I included a catversion bump, although no initial catalog data
changes here, to mark the boundary at which oprkind = 'r'
stopped being valid in pg_operator.
Mark Dilger, based on work by myself and Robert Haas;
review by John Naylor
Discussion: https://postgr.es/m/38ca86db-42ab-9b48-2902-337a0d6b8311@2ndquadrant.com
5 years ago
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Verify that no user defined postfix operators exist.
|
|
|
|
|
*/
|
|
|
|
|
static void
|
|
|
|
|
check_for_user_defined_postfix_ops(ClusterInfo *cluster)
|
|
|
|
|
{
|
|
|
|
|
UpgradeTaskReport report;
|
|
|
|
|
UpgradeTask *task = upgrade_task_create();
|
|
|
|
|
const char *query;
|
Remove support for postfix (right-unary) operators.
This feature has been a thorn in our sides for a long time, causing
many grammatical ambiguity problems. It doesn't seem worth the
pain to continue to support it, so remove it.
There are some follow-on improvements we can make in the grammar,
but this commit only removes the bare minimum number of productions,
plus assorted backend support code.
Note that pg_dump and psql continue to have full support, since
they may be used against older servers. However, pg_dump warns
about postfix operators. There is also a check in pg_upgrade.
Documentation-wise, I (tgl) largely removed the "left unary"
terminology in favor of saying "prefix operator", which is
a more standard and IMO less confusing term.
I included a catversion bump, although no initial catalog data
changes here, to mark the boundary at which oprkind = 'r'
stopped being valid in pg_operator.
Mark Dilger, based on work by myself and Robert Haas;
review by John Naylor
Discussion: https://postgr.es/m/38ca86db-42ab-9b48-2902-337a0d6b8311@2ndquadrant.com
5 years ago
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* The query below hardcodes FirstNormalObjectId as 16384 rather than
|
|
|
|
|
* interpolating that C #define into the query because, if that #define is
|
|
|
|
|
* ever changed, the cutoff we want to use is the value used by
|
|
|
|
|
* pre-version 14 servers, not that of some future version.
|
|
|
|
|
*/
|
|
|
|
|
query = "SELECT o.oid AS oproid, "
|
|
|
|
|
" n.nspname AS oprnsp, "
|
|
|
|
|
" o.oprname, "
|
|
|
|
|
" tn.nspname AS typnsp, "
|
|
|
|
|
" t.typname "
|
|
|
|
|
"FROM pg_catalog.pg_operator o, "
|
|
|
|
|
" pg_catalog.pg_namespace n, "
|
|
|
|
|
" pg_catalog.pg_type t, "
|
|
|
|
|
" pg_catalog.pg_namespace tn "
|
|
|
|
|
"WHERE o.oprnamespace = n.oid AND "
|
|
|
|
|
" o.oprleft = t.oid AND "
|
|
|
|
|
" t.typnamespace = tn.oid AND "
|
|
|
|
|
" o.oprright = 0 AND "
|
|
|
|
|
" o.oid >= 16384";
|
Remove support for postfix (right-unary) operators.
This feature has been a thorn in our sides for a long time, causing
many grammatical ambiguity problems. It doesn't seem worth the
pain to continue to support it, so remove it.
There are some follow-on improvements we can make in the grammar,
but this commit only removes the bare minimum number of productions,
plus assorted backend support code.
Note that pg_dump and psql continue to have full support, since
they may be used against older servers. However, pg_dump warns
about postfix operators. There is also a check in pg_upgrade.
Documentation-wise, I (tgl) largely removed the "left unary"
terminology in favor of saying "prefix operator", which is
a more standard and IMO less confusing term.
I included a catversion bump, although no initial catalog data
changes here, to mark the boundary at which oprkind = 'r'
stopped being valid in pg_operator.
Mark Dilger, based on work by myself and Robert Haas;
review by John Naylor
Discussion: https://postgr.es/m/38ca86db-42ab-9b48-2902-337a0d6b8311@2ndquadrant.com
5 years ago
|
|
|
|
|
|
|
|
prep_status("Checking for user-defined postfix operators");
|
|
|
|
|
|
|
|
|
|
report.file = NULL;
|
|
|
|
|
snprintf(report.path, sizeof(report.path), "%s/%s",
|
|
|
|
|
log_opts.basedir,
|
|
|
|
|
"postfix_ops.txt");
|
|
|
|
|
|
|
|
|
|
upgrade_task_add_step(task, query, process_user_defined_postfix_ops,
|
|
|
|
|
true, &report);
|
|
|
|
|
upgrade_task_run(task, cluster);
|
|
|
|
|
upgrade_task_free(task);
|
|
|
|
|
|
|
|
|
|
if (report.file)
|
Remove support for postfix (right-unary) operators.
This feature has been a thorn in our sides for a long time, causing
many grammatical ambiguity problems. It doesn't seem worth the
pain to continue to support it, so remove it.
There are some follow-on improvements we can make in the grammar,
but this commit only removes the bare minimum number of productions,
plus assorted backend support code.
Note that pg_dump and psql continue to have full support, since
they may be used against older servers. However, pg_dump warns
about postfix operators. There is also a check in pg_upgrade.
Documentation-wise, I (tgl) largely removed the "left unary"
terminology in favor of saying "prefix operator", which is
a more standard and IMO less confusing term.
I included a catversion bump, although no initial catalog data
changes here, to mark the boundary at which oprkind = 'r'
stopped being valid in pg_operator.
Mark Dilger, based on work by myself and Robert Haas;
review by John Naylor
Discussion: https://postgr.es/m/38ca86db-42ab-9b48-2902-337a0d6b8311@2ndquadrant.com
5 years ago
|
|
|
{
|
|
|
|
|
fclose(report.file);
|
Remove trailing newlines in pg_upgrade's message strings.
pg_upgrade does not use common/logging.c, which is unfortunate
but changing it to do so seems like more work than is justified.
However, we really need to make it work more like common/logging.c
in one respect: the latter expects supplied message strings to not
end with a newline, instead adding one internally. As it stands,
pg_upgrade's logging facilities expect a caller-supplied newline
in some cases and not others, which is already an invitation to bugs,
but the inconsistency with our other frontend code makes it worse.
There are already several places with missing or extra newlines,
and it's inevitable that there won't be more if we let this stand.
Hence, run around and get rid of all trailing newlines in message
strings, and add an Assert that there's not one, similar to the
existing Assert in common/logging.c. Adjust the logging functions
to supply a newline at the right places.
(Some of these strings also have a *leading* newline, which would
be a good thing to get rid of too; but this patch doesn't attempt
that.)
There are some consequent minor changes in output. The ones that
aren't outright bug fixes are generally removal of extra blank
lines that the original coding intentionally inserted. It didn't
seem worth being bug-compatible with that.
Patch by me, reviewed by Kyotaro Horiguchi and Peter Eisentraut
Discussion: https://postgr.es/m/113191.1655233060@sss.pgh.pa.us
3 years ago
|
|
|
pg_log(PG_REPORT, "fatal");
|
Remove support for postfix (right-unary) operators.
This feature has been a thorn in our sides for a long time, causing
many grammatical ambiguity problems. It doesn't seem worth the
pain to continue to support it, so remove it.
There are some follow-on improvements we can make in the grammar,
but this commit only removes the bare minimum number of productions,
plus assorted backend support code.
Note that pg_dump and psql continue to have full support, since
they may be used against older servers. However, pg_dump warns
about postfix operators. There is also a check in pg_upgrade.
Documentation-wise, I (tgl) largely removed the "left unary"
terminology in favor of saying "prefix operator", which is
a more standard and IMO less confusing term.
I included a catversion bump, although no initial catalog data
changes here, to mark the boundary at which oprkind = 'r'
stopped being valid in pg_operator.
Mark Dilger, based on work by myself and Robert Haas;
review by John Naylor
Discussion: https://postgr.es/m/38ca86db-42ab-9b48-2902-337a0d6b8311@2ndquadrant.com
5 years ago
|
|
|
pg_fatal("Your installation contains user-defined postfix operators, which are not\n"
|
|
|
|
|
"supported anymore. Consider dropping the postfix operators and replacing\n"
|
|
|
|
|
"them with prefix operators or function calls.\n"
|
|
|
|
|
"A list of user-defined postfix operators is in the file:\n"
|
|
|
|
|
" %s", report.path);
|
Remove support for postfix (right-unary) operators.
This feature has been a thorn in our sides for a long time, causing
many grammatical ambiguity problems. It doesn't seem worth the
pain to continue to support it, so remove it.
There are some follow-on improvements we can make in the grammar,
but this commit only removes the bare minimum number of productions,
plus assorted backend support code.
Note that pg_dump and psql continue to have full support, since
they may be used against older servers. However, pg_dump warns
about postfix operators. There is also a check in pg_upgrade.
Documentation-wise, I (tgl) largely removed the "left unary"
terminology in favor of saying "prefix operator", which is
a more standard and IMO less confusing term.
I included a catversion bump, although no initial catalog data
changes here, to mark the boundary at which oprkind = 'r'
stopped being valid in pg_operator.
Mark Dilger, based on work by myself and Robert Haas;
review by John Naylor
Discussion: https://postgr.es/m/38ca86db-42ab-9b48-2902-337a0d6b8311@2ndquadrant.com
5 years ago
|
|
|
}
|
|
|
|
|
else
|
|
|
|
|
check_ok();
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* check_for_incompatible_polymorphics()
|
|
|
|
|
*
|
|
|
|
|
* Make sure nothing is using old polymorphic functions with
|
|
|
|
|
* anyarray/anyelement rather than the new anycompatible variants.
|
|
|
|
|
*/
|
|
|
|
|
static void
|
|
|
|
|
check_for_incompatible_polymorphics(ClusterInfo *cluster)
|
|
|
|
|
{
|
|
|
|
|
PGresult *res;
|
|
|
|
|
FILE *script = NULL;
|
|
|
|
|
char output_path[MAXPGPATH];
|
|
|
|
|
PQExpBufferData old_polymorphics;
|
|
|
|
|
|
|
|
|
|
prep_status("Checking for incompatible polymorphic functions");
|
|
|
|
|
|
|
|
|
|
snprintf(output_path, sizeof(output_path), "%s/%s",
|
|
|
|
|
log_opts.basedir,
|
|
|
|
|
"incompatible_polymorphics.txt");
|
|
|
|
|
|
|
|
|
|
/* The set of problematic functions varies a bit in different versions */
|
|
|
|
|
initPQExpBuffer(&old_polymorphics);
|
|
|
|
|
|
|
|
|
|
appendPQExpBufferStr(&old_polymorphics,
|
|
|
|
|
"'array_append(anyarray,anyelement)'"
|
|
|
|
|
", 'array_cat(anyarray,anyarray)'"
|
|
|
|
|
", 'array_prepend(anyelement,anyarray)'");
|
|
|
|
|
|
|
|
|
|
if (GET_MAJOR_VERSION(cluster->major_version) >= 903)
|
|
|
|
|
appendPQExpBufferStr(&old_polymorphics,
|
|
|
|
|
", 'array_remove(anyarray,anyelement)'"
|
|
|
|
|
", 'array_replace(anyarray,anyelement,anyelement)'");
|
|
|
|
|
|
|
|
|
|
if (GET_MAJOR_VERSION(cluster->major_version) >= 905)
|
|
|
|
|
appendPQExpBufferStr(&old_polymorphics,
|
|
|
|
|
", 'array_position(anyarray,anyelement)'"
|
|
|
|
|
", 'array_position(anyarray,anyelement,integer)'"
|
|
|
|
|
", 'array_positions(anyarray,anyelement)'"
|
|
|
|
|
", 'width_bucket(anyelement,anyarray)'");
|
|
|
|
|
|
|
|
|
|
for (int dbnum = 0; dbnum < cluster->dbarr.ndbs; dbnum++)
|
|
|
|
|
{
|
|
|
|
|
bool db_used = false;
|
|
|
|
|
DbInfo *active_db = &cluster->dbarr.dbs[dbnum];
|
|
|
|
|
PGconn *conn = connectToServer(cluster, active_db->db_name);
|
|
|
|
|
int ntups;
|
|
|
|
|
int i_objkind,
|
|
|
|
|
i_objname;
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* The query below hardcodes FirstNormalObjectId as 16384 rather than
|
|
|
|
|
* interpolating that C #define into the query because, if that
|
|
|
|
|
* #define is ever changed, the cutoff we want to use is the value
|
|
|
|
|
* used by pre-version 14 servers, not that of some future version.
|
|
|
|
|
*/
|
|
|
|
|
res = executeQueryOrDie(conn,
|
|
|
|
|
/* Aggregate transition functions */
|
|
|
|
|
"SELECT 'aggregate' AS objkind, p.oid::regprocedure::text AS objname "
|
|
|
|
|
"FROM pg_proc AS p "
|
|
|
|
|
"JOIN pg_aggregate AS a ON a.aggfnoid=p.oid "
|
|
|
|
|
"JOIN pg_proc AS transfn ON transfn.oid=a.aggtransfn "
|
|
|
|
|
"WHERE p.oid >= 16384 "
|
|
|
|
|
"AND a.aggtransfn = ANY(ARRAY[%s]::regprocedure[]) "
|
|
|
|
|
"AND a.aggtranstype = ANY(ARRAY['anyarray', 'anyelement']::regtype[]) "
|
|
|
|
|
|
|
|
|
|
/* Aggregate final functions */
|
|
|
|
|
"UNION ALL "
|
|
|
|
|
"SELECT 'aggregate' AS objkind, p.oid::regprocedure::text AS objname "
|
|
|
|
|
"FROM pg_proc AS p "
|
|
|
|
|
"JOIN pg_aggregate AS a ON a.aggfnoid=p.oid "
|
|
|
|
|
"JOIN pg_proc AS finalfn ON finalfn.oid=a.aggfinalfn "
|
|
|
|
|
"WHERE p.oid >= 16384 "
|
|
|
|
|
"AND a.aggfinalfn = ANY(ARRAY[%s]::regprocedure[]) "
|
|
|
|
|
"AND a.aggtranstype = ANY(ARRAY['anyarray', 'anyelement']::regtype[]) "
|
|
|
|
|
|
|
|
|
|
/* Operators */
|
|
|
|
|
"UNION ALL "
|
|
|
|
|
"SELECT 'operator' AS objkind, op.oid::regoperator::text AS objname "
|
|
|
|
|
"FROM pg_operator AS op "
|
|
|
|
|
"WHERE op.oid >= 16384 "
|
|
|
|
|
"AND oprcode = ANY(ARRAY[%s]::regprocedure[]) "
|
|
|
|
|
"AND oprleft = ANY(ARRAY['anyarray', 'anyelement']::regtype[]);",
|
|
|
|
|
old_polymorphics.data,
|
|
|
|
|
old_polymorphics.data,
|
|
|
|
|
old_polymorphics.data);
|
|
|
|
|
|
|
|
|
|
ntups = PQntuples(res);
|
|
|
|
|
|
|
|
|
|
i_objkind = PQfnumber(res, "objkind");
|
|
|
|
|
i_objname = PQfnumber(res, "objname");
|
|
|
|
|
|
|
|
|
|
for (int rowno = 0; rowno < ntups; rowno++)
|
|
|
|
|
{
|
|
|
|
|
if (script == NULL &&
|
|
|
|
|
(script = fopen_priv(output_path, "w")) == NULL)
|
|
|
|
|
pg_fatal("could not open file \"%s\": %m", output_path);
|
|
|
|
|
if (!db_used)
|
|
|
|
|
{
|
|
|
|
|
fprintf(script, "In database: %s\n", active_db->db_name);
|
|
|
|
|
db_used = true;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
fprintf(script, " %s: %s\n",
|
|
|
|
|
PQgetvalue(res, rowno, i_objkind),
|
|
|
|
|
PQgetvalue(res, rowno, i_objname));
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
PQclear(res);
|
|
|
|
|
PQfinish(conn);
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
if (script)
|
|
|
|
|
{
|
|
|
|
|
fclose(script);
|
Remove trailing newlines in pg_upgrade's message strings.
pg_upgrade does not use common/logging.c, which is unfortunate
but changing it to do so seems like more work than is justified.
However, we really need to make it work more like common/logging.c
in one respect: the latter expects supplied message strings to not
end with a newline, instead adding one internally. As it stands,
pg_upgrade's logging facilities expect a caller-supplied newline
in some cases and not others, which is already an invitation to bugs,
but the inconsistency with our other frontend code makes it worse.
There are already several places with missing or extra newlines,
and it's inevitable that there won't be more if we let this stand.
Hence, run around and get rid of all trailing newlines in message
strings, and add an Assert that there's not one, similar to the
existing Assert in common/logging.c. Adjust the logging functions
to supply a newline at the right places.
(Some of these strings also have a *leading* newline, which would
be a good thing to get rid of too; but this patch doesn't attempt
that.)
There are some consequent minor changes in output. The ones that
aren't outright bug fixes are generally removal of extra blank
lines that the original coding intentionally inserted. It didn't
seem worth being bug-compatible with that.
Patch by me, reviewed by Kyotaro Horiguchi and Peter Eisentraut
Discussion: https://postgr.es/m/113191.1655233060@sss.pgh.pa.us
3 years ago
|
|
|
pg_log(PG_REPORT, "fatal");
|
|
|
|
|
pg_fatal("Your installation contains user-defined objects that refer to internal\n"
|
|
|
|
|
"polymorphic functions with arguments of type \"anyarray\" or \"anyelement\".\n"
|
|
|
|
|
"These user-defined objects must be dropped before upgrading and restored\n"
|
|
|
|
|
"afterwards, changing them to refer to the new corresponding functions with\n"
|
|
|
|
|
"arguments of type \"anycompatiblearray\" and \"anycompatible\".\n"
|
|
|
|
|
"A list of the problematic objects is in the file:\n"
|
Remove trailing newlines in pg_upgrade's message strings.
pg_upgrade does not use common/logging.c, which is unfortunate
but changing it to do so seems like more work than is justified.
However, we really need to make it work more like common/logging.c
in one respect: the latter expects supplied message strings to not
end with a newline, instead adding one internally. As it stands,
pg_upgrade's logging facilities expect a caller-supplied newline
in some cases and not others, which is already an invitation to bugs,
but the inconsistency with our other frontend code makes it worse.
There are already several places with missing or extra newlines,
and it's inevitable that there won't be more if we let this stand.
Hence, run around and get rid of all trailing newlines in message
strings, and add an Assert that there's not one, similar to the
existing Assert in common/logging.c. Adjust the logging functions
to supply a newline at the right places.
(Some of these strings also have a *leading* newline, which would
be a good thing to get rid of too; but this patch doesn't attempt
that.)
There are some consequent minor changes in output. The ones that
aren't outright bug fixes are generally removal of extra blank
lines that the original coding intentionally inserted. It didn't
seem worth being bug-compatible with that.
Patch by me, reviewed by Kyotaro Horiguchi and Peter Eisentraut
Discussion: https://postgr.es/m/113191.1655233060@sss.pgh.pa.us
3 years ago
|
|
|
" %s", output_path);
|
|
|
|
|
}
|
|
|
|
|
else
|
|
|
|
|
check_ok();
|
|
|
|
|
|
|
|
|
|
termPQExpBuffer(&old_polymorphics);
|
|
|
|
|
}
|
|
|
|
|
|
Remove WITH OIDS support, change oid catalog column visibility.
Previously tables declared WITH OIDS, including a significant fraction
of the catalog tables, stored the oid column not as a normal column,
but as part of the tuple header.
This special column was not shown by default, which was somewhat odd,
as it's often (consider e.g. pg_class.oid) one of the more important
parts of a row. Neither pg_dump nor COPY included the contents of the
oid column by default.
The fact that the oid column was not an ordinary column necessitated a
significant amount of special case code to support oid columns. That
already was painful for the existing, but upcoming work aiming to make
table storage pluggable, would have required expanding and duplicating
that "specialness" significantly.
WITH OIDS has been deprecated since 2005 (commit ff02d0a05280e0).
Remove it.
Removing includes:
- CREATE TABLE and ALTER TABLE syntax for declaring the table to be
WITH OIDS has been removed (WITH (oids[ = true]) will error out)
- pg_dump does not support dumping tables declared WITH OIDS and will
issue a warning when dumping one (and ignore the oid column).
- restoring an pg_dump archive with pg_restore will warn when
restoring a table with oid contents (and ignore the oid column)
- COPY will refuse to load binary dump that includes oids.
- pg_upgrade will error out when encountering tables declared WITH
OIDS, they have to be altered to remove the oid column first.
- Functionality to access the oid of the last inserted row (like
plpgsql's RESULT_OID, spi's SPI_lastoid, ...) has been removed.
The syntax for declaring a table WITHOUT OIDS (or WITH (oids = false)
for CREATE TABLE) is still supported. While that requires a bit of
support code, it seems unnecessary to break applications / dumps that
do not use oids, and are explicit about not using them.
The biggest user of WITH OID columns was postgres' catalog. This
commit changes all 'magic' oid columns to be columns that are normally
declared and stored. To reduce unnecessary query breakage all the
newly added columns are still named 'oid', even if a table's column
naming scheme would indicate 'reloid' or such. This obviously
requires adapting a lot code, mostly replacing oid access via
HeapTupleGetOid() with access to the underlying Form_pg_*->oid column.
The bootstrap process now assigns oids for all oid columns in
genbki.pl that do not have an explicit value (starting at the largest
oid previously used), only oids assigned later by oids will be above
FirstBootstrapObjectId. As the oid column now is a normal column the
special bootstrap syntax for oids has been removed.
Oids are not automatically assigned during insertion anymore, all
backend code explicitly assigns oids with GetNewOidWithIndex(). For
the rare case that insertions into the catalog via SQL are called for
the new pg_nextoid() function can be used (which only works on catalog
tables).
The fact that oid columns on system tables are now normal columns
means that they will be included in the set of columns expanded
by * (i.e. SELECT * FROM pg_class will now include the table's oid,
previously it did not). It'd not technically be hard to hide oid
column by default, but that'd mean confusing behavior would either
have to be carried forward forever, or it'd cause breakage down the
line.
While it's not unlikely that further adjustments are needed, the
scope/invasiveness of the patch makes it worthwhile to get merge this
now. It's painful to maintain externally, too complicated to commit
after the code code freeze, and a dependency of a number of other
patches.
Catversion bump, for obvious reasons.
Author: Andres Freund, with contributions by John Naylor
Discussion: https://postgr.es/m/20180930034810.ywp2c7awz7opzcfr@alap3.anarazel.de
7 years ago
|
|
|
/*
|
|
|
|
|
* Verify that no tables are declared WITH OIDS.
|
|
|
|
|
*/
|
|
|
|
|
static void
|
|
|
|
|
check_for_tables_with_oids(ClusterInfo *cluster)
|
|
|
|
|
{
|
|
|
|
|
int dbnum;
|
|
|
|
|
FILE *script = NULL;
|
|
|
|
|
char output_path[MAXPGPATH];
|
|
|
|
|
|
|
|
|
|
prep_status("Checking for tables WITH OIDS");
|
Remove WITH OIDS support, change oid catalog column visibility.
Previously tables declared WITH OIDS, including a significant fraction
of the catalog tables, stored the oid column not as a normal column,
but as part of the tuple header.
This special column was not shown by default, which was somewhat odd,
as it's often (consider e.g. pg_class.oid) one of the more important
parts of a row. Neither pg_dump nor COPY included the contents of the
oid column by default.
The fact that the oid column was not an ordinary column necessitated a
significant amount of special case code to support oid columns. That
already was painful for the existing, but upcoming work aiming to make
table storage pluggable, would have required expanding and duplicating
that "specialness" significantly.
WITH OIDS has been deprecated since 2005 (commit ff02d0a05280e0).
Remove it.
Removing includes:
- CREATE TABLE and ALTER TABLE syntax for declaring the table to be
WITH OIDS has been removed (WITH (oids[ = true]) will error out)
- pg_dump does not support dumping tables declared WITH OIDS and will
issue a warning when dumping one (and ignore the oid column).
- restoring an pg_dump archive with pg_restore will warn when
restoring a table with oid contents (and ignore the oid column)
- COPY will refuse to load binary dump that includes oids.
- pg_upgrade will error out when encountering tables declared WITH
OIDS, they have to be altered to remove the oid column first.
- Functionality to access the oid of the last inserted row (like
plpgsql's RESULT_OID, spi's SPI_lastoid, ...) has been removed.
The syntax for declaring a table WITHOUT OIDS (or WITH (oids = false)
for CREATE TABLE) is still supported. While that requires a bit of
support code, it seems unnecessary to break applications / dumps that
do not use oids, and are explicit about not using them.
The biggest user of WITH OID columns was postgres' catalog. This
commit changes all 'magic' oid columns to be columns that are normally
declared and stored. To reduce unnecessary query breakage all the
newly added columns are still named 'oid', even if a table's column
naming scheme would indicate 'reloid' or such. This obviously
requires adapting a lot code, mostly replacing oid access via
HeapTupleGetOid() with access to the underlying Form_pg_*->oid column.
The bootstrap process now assigns oids for all oid columns in
genbki.pl that do not have an explicit value (starting at the largest
oid previously used), only oids assigned later by oids will be above
FirstBootstrapObjectId. As the oid column now is a normal column the
special bootstrap syntax for oids has been removed.
Oids are not automatically assigned during insertion anymore, all
backend code explicitly assigns oids with GetNewOidWithIndex(). For
the rare case that insertions into the catalog via SQL are called for
the new pg_nextoid() function can be used (which only works on catalog
tables).
The fact that oid columns on system tables are now normal columns
means that they will be included in the set of columns expanded
by * (i.e. SELECT * FROM pg_class will now include the table's oid,
previously it did not). It'd not technically be hard to hide oid
column by default, but that'd mean confusing behavior would either
have to be carried forward forever, or it'd cause breakage down the
line.
While it's not unlikely that further adjustments are needed, the
scope/invasiveness of the patch makes it worthwhile to get merge this
now. It's painful to maintain externally, too complicated to commit
after the code code freeze, and a dependency of a number of other
patches.
Catversion bump, for obvious reasons.
Author: Andres Freund, with contributions by John Naylor
Discussion: https://postgr.es/m/20180930034810.ywp2c7awz7opzcfr@alap3.anarazel.de
7 years ago
|
|
|
|
pg_upgrade: Move all the files generated internally to a subdirectory
Historically, the location of any files generated by pg_upgrade, as of
the per-database logs and internal dumps, has been the current working
directory, leaving all those files behind when using --retain or on a
failure.
Putting all those contents in a targeted subdirectory makes the whole
easier to debug, and simplifies the code in charge of cleaning up the
logs. Note that another reason is that this facilitates the move of
pg_upgrade to TAP with a fixed location for all the logs to grab if the
test fails repeatedly.
Initially, we thought about being able to specify the output directory
with a new option, but we have settled on using a subdirectory located
at the root of the new cluster's data folder, "pg_upgrade_output.d",
instead, as at the end the new data directory is the location of all the
data generated by pg_upgrade. There is a take with group permissions
here though: if the new data folder has been initialized with this
option, we need to create all the files and paths with the correct
permissions or a base backup taken after a pg_upgrade --retain would
fail, meaning that GetDataDirectoryCreatePerm() has to be called before
creating the log paths, before a couple of sanity checks on the clusters
and before getting the socket directory for the cluster's host settings.
The idea of the new location is based on a suggestion from Peter
Eisentraut.
Also thanks to Andrew Dunstan, Peter Eisentraut, Daniel Gustafsson, Tom
Lane and Bruce Momjian for the discussion (in alphabetical order).
Author: Justin Pryzby
Discussion: https://postgr.es/m/20211212025017.GN17618@telsasoft.com
4 years ago
|
|
|
snprintf(output_path, sizeof(output_path), "%s/%s",
|
|
|
|
|
log_opts.basedir,
|
Remove WITH OIDS support, change oid catalog column visibility.
Previously tables declared WITH OIDS, including a significant fraction
of the catalog tables, stored the oid column not as a normal column,
but as part of the tuple header.
This special column was not shown by default, which was somewhat odd,
as it's often (consider e.g. pg_class.oid) one of the more important
parts of a row. Neither pg_dump nor COPY included the contents of the
oid column by default.
The fact that the oid column was not an ordinary column necessitated a
significant amount of special case code to support oid columns. That
already was painful for the existing, but upcoming work aiming to make
table storage pluggable, would have required expanding and duplicating
that "specialness" significantly.
WITH OIDS has been deprecated since 2005 (commit ff02d0a05280e0).
Remove it.
Removing includes:
- CREATE TABLE and ALTER TABLE syntax for declaring the table to be
WITH OIDS has been removed (WITH (oids[ = true]) will error out)
- pg_dump does not support dumping tables declared WITH OIDS and will
issue a warning when dumping one (and ignore the oid column).
- restoring an pg_dump archive with pg_restore will warn when
restoring a table with oid contents (and ignore the oid column)
- COPY will refuse to load binary dump that includes oids.
- pg_upgrade will error out when encountering tables declared WITH
OIDS, they have to be altered to remove the oid column first.
- Functionality to access the oid of the last inserted row (like
plpgsql's RESULT_OID, spi's SPI_lastoid, ...) has been removed.
The syntax for declaring a table WITHOUT OIDS (or WITH (oids = false)
for CREATE TABLE) is still supported. While that requires a bit of
support code, it seems unnecessary to break applications / dumps that
do not use oids, and are explicit about not using them.
The biggest user of WITH OID columns was postgres' catalog. This
commit changes all 'magic' oid columns to be columns that are normally
declared and stored. To reduce unnecessary query breakage all the
newly added columns are still named 'oid', even if a table's column
naming scheme would indicate 'reloid' or such. This obviously
requires adapting a lot code, mostly replacing oid access via
HeapTupleGetOid() with access to the underlying Form_pg_*->oid column.
The bootstrap process now assigns oids for all oid columns in
genbki.pl that do not have an explicit value (starting at the largest
oid previously used), only oids assigned later by oids will be above
FirstBootstrapObjectId. As the oid column now is a normal column the
special bootstrap syntax for oids has been removed.
Oids are not automatically assigned during insertion anymore, all
backend code explicitly assigns oids with GetNewOidWithIndex(). For
the rare case that insertions into the catalog via SQL are called for
the new pg_nextoid() function can be used (which only works on catalog
tables).
The fact that oid columns on system tables are now normal columns
means that they will be included in the set of columns expanded
by * (i.e. SELECT * FROM pg_class will now include the table's oid,
previously it did not). It'd not technically be hard to hide oid
column by default, but that'd mean confusing behavior would either
have to be carried forward forever, or it'd cause breakage down the
line.
While it's not unlikely that further adjustments are needed, the
scope/invasiveness of the patch makes it worthwhile to get merge this
now. It's painful to maintain externally, too complicated to commit
after the code code freeze, and a dependency of a number of other
patches.
Catversion bump, for obvious reasons.
Author: Andres Freund, with contributions by John Naylor
Discussion: https://postgr.es/m/20180930034810.ywp2c7awz7opzcfr@alap3.anarazel.de
7 years ago
|
|
|
"tables_with_oids.txt");
|
|
|
|
|
|
|
|
|
|
/* Find any tables declared WITH OIDS */
|
|
|
|
|
for (dbnum = 0; dbnum < cluster->dbarr.ndbs; dbnum++)
|
|
|
|
|
{
|
|
|
|
|
PGresult *res;
|
|
|
|
|
bool db_used = false;
|
|
|
|
|
int ntups;
|
|
|
|
|
int rowno;
|
|
|
|
|
int i_nspname,
|
|
|
|
|
i_relname;
|
|
|
|
|
DbInfo *active_db = &cluster->dbarr.dbs[dbnum];
|
|
|
|
|
PGconn *conn = connectToServer(cluster, active_db->db_name);
|
|
|
|
|
|
|
|
|
|
res = executeQueryOrDie(conn,
|
|
|
|
|
"SELECT n.nspname, c.relname "
|
|
|
|
|
"FROM pg_catalog.pg_class c, "
|
|
|
|
|
" pg_catalog.pg_namespace n "
|
|
|
|
|
"WHERE c.relnamespace = n.oid AND "
|
|
|
|
|
" c.relhasoids AND"
|
|
|
|
|
" n.nspname NOT IN ('pg_catalog')");
|
|
|
|
|
|
|
|
|
|
ntups = PQntuples(res);
|
|
|
|
|
i_nspname = PQfnumber(res, "nspname");
|
|
|
|
|
i_relname = PQfnumber(res, "relname");
|
|
|
|
|
for (rowno = 0; rowno < ntups; rowno++)
|
|
|
|
|
{
|
|
|
|
|
if (script == NULL && (script = fopen_priv(output_path, "w")) == NULL)
|
|
|
|
|
pg_fatal("could not open file \"%s\": %m", output_path);
|
Remove WITH OIDS support, change oid catalog column visibility.
Previously tables declared WITH OIDS, including a significant fraction
of the catalog tables, stored the oid column not as a normal column,
but as part of the tuple header.
This special column was not shown by default, which was somewhat odd,
as it's often (consider e.g. pg_class.oid) one of the more important
parts of a row. Neither pg_dump nor COPY included the contents of the
oid column by default.
The fact that the oid column was not an ordinary column necessitated a
significant amount of special case code to support oid columns. That
already was painful for the existing, but upcoming work aiming to make
table storage pluggable, would have required expanding and duplicating
that "specialness" significantly.
WITH OIDS has been deprecated since 2005 (commit ff02d0a05280e0).
Remove it.
Removing includes:
- CREATE TABLE and ALTER TABLE syntax for declaring the table to be
WITH OIDS has been removed (WITH (oids[ = true]) will error out)
- pg_dump does not support dumping tables declared WITH OIDS and will
issue a warning when dumping one (and ignore the oid column).
- restoring an pg_dump archive with pg_restore will warn when
restoring a table with oid contents (and ignore the oid column)
- COPY will refuse to load binary dump that includes oids.
- pg_upgrade will error out when encountering tables declared WITH
OIDS, they have to be altered to remove the oid column first.
- Functionality to access the oid of the last inserted row (like
plpgsql's RESULT_OID, spi's SPI_lastoid, ...) has been removed.
The syntax for declaring a table WITHOUT OIDS (or WITH (oids = false)
for CREATE TABLE) is still supported. While that requires a bit of
support code, it seems unnecessary to break applications / dumps that
do not use oids, and are explicit about not using them.
The biggest user of WITH OID columns was postgres' catalog. This
commit changes all 'magic' oid columns to be columns that are normally
declared and stored. To reduce unnecessary query breakage all the
newly added columns are still named 'oid', even if a table's column
naming scheme would indicate 'reloid' or such. This obviously
requires adapting a lot code, mostly replacing oid access via
HeapTupleGetOid() with access to the underlying Form_pg_*->oid column.
The bootstrap process now assigns oids for all oid columns in
genbki.pl that do not have an explicit value (starting at the largest
oid previously used), only oids assigned later by oids will be above
FirstBootstrapObjectId. As the oid column now is a normal column the
special bootstrap syntax for oids has been removed.
Oids are not automatically assigned during insertion anymore, all
backend code explicitly assigns oids with GetNewOidWithIndex(). For
the rare case that insertions into the catalog via SQL are called for
the new pg_nextoid() function can be used (which only works on catalog
tables).
The fact that oid columns on system tables are now normal columns
means that they will be included in the set of columns expanded
by * (i.e. SELECT * FROM pg_class will now include the table's oid,
previously it did not). It'd not technically be hard to hide oid
column by default, but that'd mean confusing behavior would either
have to be carried forward forever, or it'd cause breakage down the
line.
While it's not unlikely that further adjustments are needed, the
scope/invasiveness of the patch makes it worthwhile to get merge this
now. It's painful to maintain externally, too complicated to commit
after the code code freeze, and a dependency of a number of other
patches.
Catversion bump, for obvious reasons.
Author: Andres Freund, with contributions by John Naylor
Discussion: https://postgr.es/m/20180930034810.ywp2c7awz7opzcfr@alap3.anarazel.de
7 years ago
|
|
|
if (!db_used)
|
|
|
|
|
{
|
|
|
|
|
fprintf(script, "In database: %s\n", active_db->db_name);
|
Remove WITH OIDS support, change oid catalog column visibility.
Previously tables declared WITH OIDS, including a significant fraction
of the catalog tables, stored the oid column not as a normal column,
but as part of the tuple header.
This special column was not shown by default, which was somewhat odd,
as it's often (consider e.g. pg_class.oid) one of the more important
parts of a row. Neither pg_dump nor COPY included the contents of the
oid column by default.
The fact that the oid column was not an ordinary column necessitated a
significant amount of special case code to support oid columns. That
already was painful for the existing, but upcoming work aiming to make
table storage pluggable, would have required expanding and duplicating
that "specialness" significantly.
WITH OIDS has been deprecated since 2005 (commit ff02d0a05280e0).
Remove it.
Removing includes:
- CREATE TABLE and ALTER TABLE syntax for declaring the table to be
WITH OIDS has been removed (WITH (oids[ = true]) will error out)
- pg_dump does not support dumping tables declared WITH OIDS and will
issue a warning when dumping one (and ignore the oid column).
- restoring an pg_dump archive with pg_restore will warn when
restoring a table with oid contents (and ignore the oid column)
- COPY will refuse to load binary dump that includes oids.
- pg_upgrade will error out when encountering tables declared WITH
OIDS, they have to be altered to remove the oid column first.
- Functionality to access the oid of the last inserted row (like
plpgsql's RESULT_OID, spi's SPI_lastoid, ...) has been removed.
The syntax for declaring a table WITHOUT OIDS (or WITH (oids = false)
for CREATE TABLE) is still supported. While that requires a bit of
support code, it seems unnecessary to break applications / dumps that
do not use oids, and are explicit about not using them.
The biggest user of WITH OID columns was postgres' catalog. This
commit changes all 'magic' oid columns to be columns that are normally
declared and stored. To reduce unnecessary query breakage all the
newly added columns are still named 'oid', even if a table's column
naming scheme would indicate 'reloid' or such. This obviously
requires adapting a lot code, mostly replacing oid access via
HeapTupleGetOid() with access to the underlying Form_pg_*->oid column.
The bootstrap process now assigns oids for all oid columns in
genbki.pl that do not have an explicit value (starting at the largest
oid previously used), only oids assigned later by oids will be above
FirstBootstrapObjectId. As the oid column now is a normal column the
special bootstrap syntax for oids has been removed.
Oids are not automatically assigned during insertion anymore, all
backend code explicitly assigns oids with GetNewOidWithIndex(). For
the rare case that insertions into the catalog via SQL are called for
the new pg_nextoid() function can be used (which only works on catalog
tables).
The fact that oid columns on system tables are now normal columns
means that they will be included in the set of columns expanded
by * (i.e. SELECT * FROM pg_class will now include the table's oid,
previously it did not). It'd not technically be hard to hide oid
column by default, but that'd mean confusing behavior would either
have to be carried forward forever, or it'd cause breakage down the
line.
While it's not unlikely that further adjustments are needed, the
scope/invasiveness of the patch makes it worthwhile to get merge this
now. It's painful to maintain externally, too complicated to commit
after the code code freeze, and a dependency of a number of other
patches.
Catversion bump, for obvious reasons.
Author: Andres Freund, with contributions by John Naylor
Discussion: https://postgr.es/m/20180930034810.ywp2c7awz7opzcfr@alap3.anarazel.de
7 years ago
|
|
|
db_used = true;
|
|
|
|
|
}
|
|
|
|
|
fprintf(script, " %s.%s\n",
|
|
|
|
|
PQgetvalue(res, rowno, i_nspname),
|
|
|
|
|
PQgetvalue(res, rowno, i_relname));
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
PQclear(res);
|
|
|
|
|
|
|
|
|
|
PQfinish(conn);
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
if (script)
|
|
|
|
|
{
|
|
|
|
|
fclose(script);
|
Remove trailing newlines in pg_upgrade's message strings.
pg_upgrade does not use common/logging.c, which is unfortunate
but changing it to do so seems like more work than is justified.
However, we really need to make it work more like common/logging.c
in one respect: the latter expects supplied message strings to not
end with a newline, instead adding one internally. As it stands,
pg_upgrade's logging facilities expect a caller-supplied newline
in some cases and not others, which is already an invitation to bugs,
but the inconsistency with our other frontend code makes it worse.
There are already several places with missing or extra newlines,
and it's inevitable that there won't be more if we let this stand.
Hence, run around and get rid of all trailing newlines in message
strings, and add an Assert that there's not one, similar to the
existing Assert in common/logging.c. Adjust the logging functions
to supply a newline at the right places.
(Some of these strings also have a *leading* newline, which would
be a good thing to get rid of too; but this patch doesn't attempt
that.)
There are some consequent minor changes in output. The ones that
aren't outright bug fixes are generally removal of extra blank
lines that the original coding intentionally inserted. It didn't
seem worth being bug-compatible with that.
Patch by me, reviewed by Kyotaro Horiguchi and Peter Eisentraut
Discussion: https://postgr.es/m/113191.1655233060@sss.pgh.pa.us
3 years ago
|
|
|
pg_log(PG_REPORT, "fatal");
|
|
|
|
|
pg_fatal("Your installation contains tables declared WITH OIDS, which is not\n"
|
|
|
|
|
"supported anymore. Consider removing the oid column using\n"
|
Remove WITH OIDS support, change oid catalog column visibility.
Previously tables declared WITH OIDS, including a significant fraction
of the catalog tables, stored the oid column not as a normal column,
but as part of the tuple header.
This special column was not shown by default, which was somewhat odd,
as it's often (consider e.g. pg_class.oid) one of the more important
parts of a row. Neither pg_dump nor COPY included the contents of the
oid column by default.
The fact that the oid column was not an ordinary column necessitated a
significant amount of special case code to support oid columns. That
already was painful for the existing, but upcoming work aiming to make
table storage pluggable, would have required expanding and duplicating
that "specialness" significantly.
WITH OIDS has been deprecated since 2005 (commit ff02d0a05280e0).
Remove it.
Removing includes:
- CREATE TABLE and ALTER TABLE syntax for declaring the table to be
WITH OIDS has been removed (WITH (oids[ = true]) will error out)
- pg_dump does not support dumping tables declared WITH OIDS and will
issue a warning when dumping one (and ignore the oid column).
- restoring an pg_dump archive with pg_restore will warn when
restoring a table with oid contents (and ignore the oid column)
- COPY will refuse to load binary dump that includes oids.
- pg_upgrade will error out when encountering tables declared WITH
OIDS, they have to be altered to remove the oid column first.
- Functionality to access the oid of the last inserted row (like
plpgsql's RESULT_OID, spi's SPI_lastoid, ...) has been removed.
The syntax for declaring a table WITHOUT OIDS (or WITH (oids = false)
for CREATE TABLE) is still supported. While that requires a bit of
support code, it seems unnecessary to break applications / dumps that
do not use oids, and are explicit about not using them.
The biggest user of WITH OID columns was postgres' catalog. This
commit changes all 'magic' oid columns to be columns that are normally
declared and stored. To reduce unnecessary query breakage all the
newly added columns are still named 'oid', even if a table's column
naming scheme would indicate 'reloid' or such. This obviously
requires adapting a lot code, mostly replacing oid access via
HeapTupleGetOid() with access to the underlying Form_pg_*->oid column.
The bootstrap process now assigns oids for all oid columns in
genbki.pl that do not have an explicit value (starting at the largest
oid previously used), only oids assigned later by oids will be above
FirstBootstrapObjectId. As the oid column now is a normal column the
special bootstrap syntax for oids has been removed.
Oids are not automatically assigned during insertion anymore, all
backend code explicitly assigns oids with GetNewOidWithIndex(). For
the rare case that insertions into the catalog via SQL are called for
the new pg_nextoid() function can be used (which only works on catalog
tables).
The fact that oid columns on system tables are now normal columns
means that they will be included in the set of columns expanded
by * (i.e. SELECT * FROM pg_class will now include the table's oid,
previously it did not). It'd not technically be hard to hide oid
column by default, but that'd mean confusing behavior would either
have to be carried forward forever, or it'd cause breakage down the
line.
While it's not unlikely that further adjustments are needed, the
scope/invasiveness of the patch makes it worthwhile to get merge this
now. It's painful to maintain externally, too complicated to commit
after the code code freeze, and a dependency of a number of other
patches.
Catversion bump, for obvious reasons.
Author: Andres Freund, with contributions by John Naylor
Discussion: https://postgr.es/m/20180930034810.ywp2c7awz7opzcfr@alap3.anarazel.de
7 years ago
|
|
|
" ALTER TABLE ... SET WITHOUT OIDS;\n"
|
|
|
|
|
"A list of tables with the problem is in the file:\n"
|
Remove trailing newlines in pg_upgrade's message strings.
pg_upgrade does not use common/logging.c, which is unfortunate
but changing it to do so seems like more work than is justified.
However, we really need to make it work more like common/logging.c
in one respect: the latter expects supplied message strings to not
end with a newline, instead adding one internally. As it stands,
pg_upgrade's logging facilities expect a caller-supplied newline
in some cases and not others, which is already an invitation to bugs,
but the inconsistency with our other frontend code makes it worse.
There are already several places with missing or extra newlines,
and it's inevitable that there won't be more if we let this stand.
Hence, run around and get rid of all trailing newlines in message
strings, and add an Assert that there's not one, similar to the
existing Assert in common/logging.c. Adjust the logging functions
to supply a newline at the right places.
(Some of these strings also have a *leading* newline, which would
be a good thing to get rid of too; but this patch doesn't attempt
that.)
There are some consequent minor changes in output. The ones that
aren't outright bug fixes are generally removal of extra blank
lines that the original coding intentionally inserted. It didn't
seem worth being bug-compatible with that.
Patch by me, reviewed by Kyotaro Horiguchi and Peter Eisentraut
Discussion: https://postgr.es/m/113191.1655233060@sss.pgh.pa.us
3 years ago
|
|
|
" %s", output_path);
|
Remove WITH OIDS support, change oid catalog column visibility.
Previously tables declared WITH OIDS, including a significant fraction
of the catalog tables, stored the oid column not as a normal column,
but as part of the tuple header.
This special column was not shown by default, which was somewhat odd,
as it's often (consider e.g. pg_class.oid) one of the more important
parts of a row. Neither pg_dump nor COPY included the contents of the
oid column by default.
The fact that the oid column was not an ordinary column necessitated a
significant amount of special case code to support oid columns. That
already was painful for the existing, but upcoming work aiming to make
table storage pluggable, would have required expanding and duplicating
that "specialness" significantly.
WITH OIDS has been deprecated since 2005 (commit ff02d0a05280e0).
Remove it.
Removing includes:
- CREATE TABLE and ALTER TABLE syntax for declaring the table to be
WITH OIDS has been removed (WITH (oids[ = true]) will error out)
- pg_dump does not support dumping tables declared WITH OIDS and will
issue a warning when dumping one (and ignore the oid column).
- restoring an pg_dump archive with pg_restore will warn when
restoring a table with oid contents (and ignore the oid column)
- COPY will refuse to load binary dump that includes oids.
- pg_upgrade will error out when encountering tables declared WITH
OIDS, they have to be altered to remove the oid column first.
- Functionality to access the oid of the last inserted row (like
plpgsql's RESULT_OID, spi's SPI_lastoid, ...) has been removed.
The syntax for declaring a table WITHOUT OIDS (or WITH (oids = false)
for CREATE TABLE) is still supported. While that requires a bit of
support code, it seems unnecessary to break applications / dumps that
do not use oids, and are explicit about not using them.
The biggest user of WITH OID columns was postgres' catalog. This
commit changes all 'magic' oid columns to be columns that are normally
declared and stored. To reduce unnecessary query breakage all the
newly added columns are still named 'oid', even if a table's column
naming scheme would indicate 'reloid' or such. This obviously
requires adapting a lot code, mostly replacing oid access via
HeapTupleGetOid() with access to the underlying Form_pg_*->oid column.
The bootstrap process now assigns oids for all oid columns in
genbki.pl that do not have an explicit value (starting at the largest
oid previously used), only oids assigned later by oids will be above
FirstBootstrapObjectId. As the oid column now is a normal column the
special bootstrap syntax for oids has been removed.
Oids are not automatically assigned during insertion anymore, all
backend code explicitly assigns oids with GetNewOidWithIndex(). For
the rare case that insertions into the catalog via SQL are called for
the new pg_nextoid() function can be used (which only works on catalog
tables).
The fact that oid columns on system tables are now normal columns
means that they will be included in the set of columns expanded
by * (i.e. SELECT * FROM pg_class will now include the table's oid,
previously it did not). It'd not technically be hard to hide oid
column by default, but that'd mean confusing behavior would either
have to be carried forward forever, or it'd cause breakage down the
line.
While it's not unlikely that further adjustments are needed, the
scope/invasiveness of the patch makes it worthwhile to get merge this
now. It's painful to maintain externally, too complicated to commit
after the code code freeze, and a dependency of a number of other
patches.
Catversion bump, for obvious reasons.
Author: Andres Freund, with contributions by John Naylor
Discussion: https://postgr.es/m/20180930034810.ywp2c7awz7opzcfr@alap3.anarazel.de
7 years ago
|
|
|
}
|
|
|
|
|
else
|
|
|
|
|
check_ok();
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* check_for_pg_role_prefix()
|
|
|
|
|
*
|
|
|
|
|
* Versions older than 9.6 should not have any pg_* roles
|
|
|
|
|
*/
|
|
|
|
|
static void
|
|
|
|
|
check_for_pg_role_prefix(ClusterInfo *cluster)
|
|
|
|
|
{
|
|
|
|
|
PGresult *res;
|
|
|
|
|
PGconn *conn = connectToServer(cluster, "template1");
|
|
|
|
|
int ntups;
|
|
|
|
|
int i_roloid;
|
|
|
|
|
int i_rolname;
|
|
|
|
|
FILE *script = NULL;
|
|
|
|
|
char output_path[MAXPGPATH];
|
|
|
|
|
|
|
|
|
|
prep_status("Checking for roles starting with \"pg_\"");
|
|
|
|
|
|
|
|
|
|
snprintf(output_path, sizeof(output_path), "%s/%s",
|
|
|
|
|
log_opts.basedir,
|
|
|
|
|
"pg_role_prefix.txt");
|
|
|
|
|
|
|
|
|
|
res = executeQueryOrDie(conn,
|
|
|
|
|
"SELECT oid AS roloid, rolname "
|
|
|
|
|
"FROM pg_catalog.pg_roles "
|
|
|
|
|
"WHERE rolname ~ '^pg_'");
|
|
|
|
|
|
|
|
|
|
ntups = PQntuples(res);
|
|
|
|
|
i_roloid = PQfnumber(res, "roloid");
|
|
|
|
|
i_rolname = PQfnumber(res, "rolname");
|
|
|
|
|
for (int rowno = 0; rowno < ntups; rowno++)
|
|
|
|
|
{
|
|
|
|
|
if (script == NULL && (script = fopen_priv(output_path, "w")) == NULL)
|
|
|
|
|
pg_fatal("could not open file \"%s\": %m", output_path);
|
|
|
|
|
fprintf(script, "%s (oid=%s)\n",
|
|
|
|
|
PQgetvalue(res, rowno, i_rolname),
|
|
|
|
|
PQgetvalue(res, rowno, i_roloid));
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
PQclear(res);
|
|
|
|
|
|
|
|
|
|
PQfinish(conn);
|
|
|
|
|
|
|
|
|
|
if (script)
|
|
|
|
|
{
|
|
|
|
|
fclose(script);
|
|
|
|
|
pg_log(PG_REPORT, "fatal");
|
|
|
|
|
pg_fatal("Your installation contains roles starting with \"pg_\".\n"
|
|
|
|
|
"\"pg_\" is a reserved prefix for system roles. The cluster\n"
|
|
|
|
|
"cannot be upgraded until these roles are renamed.\n"
|
|
|
|
|
"A list of roles starting with \"pg_\" is in the file:\n"
|
|
|
|
|
" %s", output_path);
|
|
|
|
|
}
|
|
|
|
|
else
|
|
|
|
|
check_ok();
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Verify that no user-defined encoding conversions exist.
|
|
|
|
|
*/
|
|
|
|
|
static void
|
|
|
|
|
check_for_user_defined_encoding_conversions(ClusterInfo *cluster)
|
|
|
|
|
{
|
|
|
|
|
int dbnum;
|
|
|
|
|
FILE *script = NULL;
|
|
|
|
|
char output_path[MAXPGPATH];
|
|
|
|
|
|
|
|
|
|
prep_status("Checking for user-defined encoding conversions");
|
|
|
|
|
|
pg_upgrade: Move all the files generated internally to a subdirectory
Historically, the location of any files generated by pg_upgrade, as of
the per-database logs and internal dumps, has been the current working
directory, leaving all those files behind when using --retain or on a
failure.
Putting all those contents in a targeted subdirectory makes the whole
easier to debug, and simplifies the code in charge of cleaning up the
logs. Note that another reason is that this facilitates the move of
pg_upgrade to TAP with a fixed location for all the logs to grab if the
test fails repeatedly.
Initially, we thought about being able to specify the output directory
with a new option, but we have settled on using a subdirectory located
at the root of the new cluster's data folder, "pg_upgrade_output.d",
instead, as at the end the new data directory is the location of all the
data generated by pg_upgrade. There is a take with group permissions
here though: if the new data folder has been initialized with this
option, we need to create all the files and paths with the correct
permissions or a base backup taken after a pg_upgrade --retain would
fail, meaning that GetDataDirectoryCreatePerm() has to be called before
creating the log paths, before a couple of sanity checks on the clusters
and before getting the socket directory for the cluster's host settings.
The idea of the new location is based on a suggestion from Peter
Eisentraut.
Also thanks to Andrew Dunstan, Peter Eisentraut, Daniel Gustafsson, Tom
Lane and Bruce Momjian for the discussion (in alphabetical order).
Author: Justin Pryzby
Discussion: https://postgr.es/m/20211212025017.GN17618@telsasoft.com
4 years ago
|
|
|
snprintf(output_path, sizeof(output_path), "%s/%s",
|
|
|
|
|
log_opts.basedir,
|
|
|
|
|
"encoding_conversions.txt");
|
|
|
|
|
|
|
|
|
|
/* Find any user defined encoding conversions */
|
|
|
|
|
for (dbnum = 0; dbnum < cluster->dbarr.ndbs; dbnum++)
|
|
|
|
|
{
|
|
|
|
|
PGresult *res;
|
|
|
|
|
bool db_used = false;
|
|
|
|
|
int ntups;
|
|
|
|
|
int rowno;
|
|
|
|
|
int i_conoid,
|
|
|
|
|
i_conname,
|
|
|
|
|
i_nspname;
|
|
|
|
|
DbInfo *active_db = &cluster->dbarr.dbs[dbnum];
|
|
|
|
|
PGconn *conn = connectToServer(cluster, active_db->db_name);
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* The query below hardcodes FirstNormalObjectId as 16384 rather than
|
|
|
|
|
* interpolating that C #define into the query because, if that
|
|
|
|
|
* #define is ever changed, the cutoff we want to use is the value
|
|
|
|
|
* used by pre-version 14 servers, not that of some future version.
|
|
|
|
|
*/
|
|
|
|
|
res = executeQueryOrDie(conn,
|
|
|
|
|
"SELECT c.oid as conoid, c.conname, n.nspname "
|
|
|
|
|
"FROM pg_catalog.pg_conversion c, "
|
|
|
|
|
" pg_catalog.pg_namespace n "
|
|
|
|
|
"WHERE c.connamespace = n.oid AND "
|
|
|
|
|
" c.oid >= 16384");
|
|
|
|
|
ntups = PQntuples(res);
|
|
|
|
|
i_conoid = PQfnumber(res, "conoid");
|
|
|
|
|
i_conname = PQfnumber(res, "conname");
|
|
|
|
|
i_nspname = PQfnumber(res, "nspname");
|
|
|
|
|
for (rowno = 0; rowno < ntups; rowno++)
|
|
|
|
|
{
|
|
|
|
|
if (script == NULL &&
|
|
|
|
|
(script = fopen_priv(output_path, "w")) == NULL)
|
|
|
|
|
pg_fatal("could not open file \"%s\": %m", output_path);
|
|
|
|
|
if (!db_used)
|
|
|
|
|
{
|
|
|
|
|
fprintf(script, "In database: %s\n", active_db->db_name);
|
|
|
|
|
db_used = true;
|
|
|
|
|
}
|
|
|
|
|
fprintf(script, " (oid=%s) %s.%s\n",
|
|
|
|
|
PQgetvalue(res, rowno, i_conoid),
|
|
|
|
|
PQgetvalue(res, rowno, i_nspname),
|
|
|
|
|
PQgetvalue(res, rowno, i_conname));
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
PQclear(res);
|
|
|
|
|
|
|
|
|
|
PQfinish(conn);
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
if (script)
|
|
|
|
|
{
|
|
|
|
|
fclose(script);
|
Remove trailing newlines in pg_upgrade's message strings.
pg_upgrade does not use common/logging.c, which is unfortunate
but changing it to do so seems like more work than is justified.
However, we really need to make it work more like common/logging.c
in one respect: the latter expects supplied message strings to not
end with a newline, instead adding one internally. As it stands,
pg_upgrade's logging facilities expect a caller-supplied newline
in some cases and not others, which is already an invitation to bugs,
but the inconsistency with our other frontend code makes it worse.
There are already several places with missing or extra newlines,
and it's inevitable that there won't be more if we let this stand.
Hence, run around and get rid of all trailing newlines in message
strings, and add an Assert that there's not one, similar to the
existing Assert in common/logging.c. Adjust the logging functions
to supply a newline at the right places.
(Some of these strings also have a *leading* newline, which would
be a good thing to get rid of too; but this patch doesn't attempt
that.)
There are some consequent minor changes in output. The ones that
aren't outright bug fixes are generally removal of extra blank
lines that the original coding intentionally inserted. It didn't
seem worth being bug-compatible with that.
Patch by me, reviewed by Kyotaro Horiguchi and Peter Eisentraut
Discussion: https://postgr.es/m/113191.1655233060@sss.pgh.pa.us
3 years ago
|
|
|
pg_log(PG_REPORT, "fatal");
|
|
|
|
|
pg_fatal("Your installation contains user-defined encoding conversions.\n"
|
|
|
|
|
"The conversion function parameters changed in PostgreSQL version 14\n"
|
|
|
|
|
"so this cluster cannot currently be upgraded. You can remove the\n"
|
|
|
|
|
"encoding conversions in the old cluster and restart the upgrade.\n"
|
|
|
|
|
"A list of user-defined encoding conversions is in the file:\n"
|
Remove trailing newlines in pg_upgrade's message strings.
pg_upgrade does not use common/logging.c, which is unfortunate
but changing it to do so seems like more work than is justified.
However, we really need to make it work more like common/logging.c
in one respect: the latter expects supplied message strings to not
end with a newline, instead adding one internally. As it stands,
pg_upgrade's logging facilities expect a caller-supplied newline
in some cases and not others, which is already an invitation to bugs,
but the inconsistency with our other frontend code makes it worse.
There are already several places with missing or extra newlines,
and it's inevitable that there won't be more if we let this stand.
Hence, run around and get rid of all trailing newlines in message
strings, and add an Assert that there's not one, similar to the
existing Assert in common/logging.c. Adjust the logging functions
to supply a newline at the right places.
(Some of these strings also have a *leading* newline, which would
be a good thing to get rid of too; but this patch doesn't attempt
that.)
There are some consequent minor changes in output. The ones that
aren't outright bug fixes are generally removal of extra blank
lines that the original coding intentionally inserted. It didn't
seem worth being bug-compatible with that.
Patch by me, reviewed by Kyotaro Horiguchi and Peter Eisentraut
Discussion: https://postgr.es/m/113191.1655233060@sss.pgh.pa.us
3 years ago
|
|
|
" %s", output_path);
|
|
|
|
|
}
|
|
|
|
|
else
|
|
|
|
|
check_ok();
|
|
|
|
|
}
|
Migrate logical slots to the new node during an upgrade.
While reading information from the old cluster, a list of logical
slots is fetched. At the later part of upgrading, pg_upgrade revisits the
list and restores slots by executing pg_create_logical_replication_slot()
on the new cluster. Migration of logical replication slots is only
supported when the old cluster is version 17.0 or later.
If the old node has invalid slots or slots with unconsumed WAL records,
the pg_upgrade fails. These checks are needed to prevent data loss.
The significant advantage of this commit is that it makes it easy to
continue logical replication even after upgrading the publisher node.
Previously, pg_upgrade allowed copying publications to a new node. With
this patch, adjusting the connection string to the new publisher will
cause the apply worker on the subscriber to connect to the new publisher
automatically. This enables seamless continuation of logical replication,
even after an upgrade.
Author: Hayato Kuroda, Hou Zhijie
Reviewed-by: Peter Smith, Bharath Rupireddy, Dilip Kumar, Vignesh C, Shlok Kyal
Discussion: http://postgr.es/m/TYAPR01MB58664C81887B3AF2EB6B16E3F5939@TYAPR01MB5866.jpnprd01.prod.outlook.com
Discussion: http://postgr.es/m/CAA4eK1+t7xYcfa0rEQw839=b2MzsfvYDPz3xbD+ZqOdP3zpKYg@mail.gmail.com
2 years ago
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* check_new_cluster_logical_replication_slots()
|
|
|
|
|
*
|
|
|
|
|
* Verify that there are no logical replication slots on the new cluster and
|
|
|
|
|
* that the parameter settings necessary for creating slots are sufficient.
|
|
|
|
|
*/
|
|
|
|
|
static void
|
|
|
|
|
check_new_cluster_logical_replication_slots(void)
|
|
|
|
|
{
|
|
|
|
|
PGresult *res;
|
|
|
|
|
PGconn *conn;
|
|
|
|
|
int nslots_on_old;
|
|
|
|
|
int nslots_on_new;
|
|
|
|
|
int max_replication_slots;
|
|
|
|
|
char *wal_level;
|
|
|
|
|
|
|
|
|
|
/* Logical slots can be migrated since PG17. */
|
|
|
|
|
if (GET_MAJOR_VERSION(old_cluster.major_version) <= 1600)
|
|
|
|
|
return;
|
|
|
|
|
|
|
|
|
|
nslots_on_old = count_old_cluster_logical_slots();
|
|
|
|
|
|
|
|
|
|
/* Quick return if there are no logical slots to be migrated. */
|
|
|
|
|
if (nslots_on_old == 0)
|
|
|
|
|
return;
|
|
|
|
|
|
|
|
|
|
conn = connectToServer(&new_cluster, "template1");
|
|
|
|
|
|
|
|
|
|
prep_status("Checking for new cluster logical replication slots");
|
|
|
|
|
|
|
|
|
|
res = executeQueryOrDie(conn, "SELECT count(*) "
|
|
|
|
|
"FROM pg_catalog.pg_replication_slots "
|
|
|
|
|
"WHERE slot_type = 'logical' AND "
|
|
|
|
|
"temporary IS FALSE;");
|
|
|
|
|
|
|
|
|
|
if (PQntuples(res) != 1)
|
|
|
|
|
pg_fatal("could not count the number of logical replication slots");
|
|
|
|
|
|
|
|
|
|
nslots_on_new = atoi(PQgetvalue(res, 0, 0));
|
|
|
|
|
|
|
|
|
|
if (nslots_on_new)
|
|
|
|
|
pg_fatal("expected 0 logical replication slots but found %d",
|
Migrate logical slots to the new node during an upgrade.
While reading information from the old cluster, a list of logical
slots is fetched. At the later part of upgrading, pg_upgrade revisits the
list and restores slots by executing pg_create_logical_replication_slot()
on the new cluster. Migration of logical replication slots is only
supported when the old cluster is version 17.0 or later.
If the old node has invalid slots or slots with unconsumed WAL records,
the pg_upgrade fails. These checks are needed to prevent data loss.
The significant advantage of this commit is that it makes it easy to
continue logical replication even after upgrading the publisher node.
Previously, pg_upgrade allowed copying publications to a new node. With
this patch, adjusting the connection string to the new publisher will
cause the apply worker on the subscriber to connect to the new publisher
automatically. This enables seamless continuation of logical replication,
even after an upgrade.
Author: Hayato Kuroda, Hou Zhijie
Reviewed-by: Peter Smith, Bharath Rupireddy, Dilip Kumar, Vignesh C, Shlok Kyal
Discussion: http://postgr.es/m/TYAPR01MB58664C81887B3AF2EB6B16E3F5939@TYAPR01MB5866.jpnprd01.prod.outlook.com
Discussion: http://postgr.es/m/CAA4eK1+t7xYcfa0rEQw839=b2MzsfvYDPz3xbD+ZqOdP3zpKYg@mail.gmail.com
2 years ago
|
|
|
nslots_on_new);
|
|
|
|
|
|
|
|
|
|
PQclear(res);
|
|
|
|
|
|
|
|
|
|
res = executeQueryOrDie(conn, "SELECT setting FROM pg_settings "
|
|
|
|
|
"WHERE name IN ('wal_level', 'max_replication_slots') "
|
|
|
|
|
"ORDER BY name DESC;");
|
|
|
|
|
|
|
|
|
|
if (PQntuples(res) != 2)
|
|
|
|
|
pg_fatal("could not determine parameter settings on new cluster");
|
|
|
|
|
|
|
|
|
|
wal_level = PQgetvalue(res, 0, 0);
|
|
|
|
|
|
|
|
|
|
if (strcmp(wal_level, "logical") != 0)
|
|
|
|
|
pg_fatal("\"wal_level\" must be \"logical\" but is set to \"%s\"",
|
Migrate logical slots to the new node during an upgrade.
While reading information from the old cluster, a list of logical
slots is fetched. At the later part of upgrading, pg_upgrade revisits the
list and restores slots by executing pg_create_logical_replication_slot()
on the new cluster. Migration of logical replication slots is only
supported when the old cluster is version 17.0 or later.
If the old node has invalid slots or slots with unconsumed WAL records,
the pg_upgrade fails. These checks are needed to prevent data loss.
The significant advantage of this commit is that it makes it easy to
continue logical replication even after upgrading the publisher node.
Previously, pg_upgrade allowed copying publications to a new node. With
this patch, adjusting the connection string to the new publisher will
cause the apply worker on the subscriber to connect to the new publisher
automatically. This enables seamless continuation of logical replication,
even after an upgrade.
Author: Hayato Kuroda, Hou Zhijie
Reviewed-by: Peter Smith, Bharath Rupireddy, Dilip Kumar, Vignesh C, Shlok Kyal
Discussion: http://postgr.es/m/TYAPR01MB58664C81887B3AF2EB6B16E3F5939@TYAPR01MB5866.jpnprd01.prod.outlook.com
Discussion: http://postgr.es/m/CAA4eK1+t7xYcfa0rEQw839=b2MzsfvYDPz3xbD+ZqOdP3zpKYg@mail.gmail.com
2 years ago
|
|
|
wal_level);
|
|
|
|
|
|
|
|
|
|
max_replication_slots = atoi(PQgetvalue(res, 1, 0));
|
|
|
|
|
|
|
|
|
|
if (nslots_on_old > max_replication_slots)
|
|
|
|
|
pg_fatal("\"max_replication_slots\" (%d) must be greater than or equal to the number of "
|
Migrate logical slots to the new node during an upgrade.
While reading information from the old cluster, a list of logical
slots is fetched. At the later part of upgrading, pg_upgrade revisits the
list and restores slots by executing pg_create_logical_replication_slot()
on the new cluster. Migration of logical replication slots is only
supported when the old cluster is version 17.0 or later.
If the old node has invalid slots or slots with unconsumed WAL records,
the pg_upgrade fails. These checks are needed to prevent data loss.
The significant advantage of this commit is that it makes it easy to
continue logical replication even after upgrading the publisher node.
Previously, pg_upgrade allowed copying publications to a new node. With
this patch, adjusting the connection string to the new publisher will
cause the apply worker on the subscriber to connect to the new publisher
automatically. This enables seamless continuation of logical replication,
even after an upgrade.
Author: Hayato Kuroda, Hou Zhijie
Reviewed-by: Peter Smith, Bharath Rupireddy, Dilip Kumar, Vignesh C, Shlok Kyal
Discussion: http://postgr.es/m/TYAPR01MB58664C81887B3AF2EB6B16E3F5939@TYAPR01MB5866.jpnprd01.prod.outlook.com
Discussion: http://postgr.es/m/CAA4eK1+t7xYcfa0rEQw839=b2MzsfvYDPz3xbD+ZqOdP3zpKYg@mail.gmail.com
2 years ago
|
|
|
"logical replication slots (%d) on the old cluster",
|
|
|
|
|
max_replication_slots, nslots_on_old);
|
|
|
|
|
|
|
|
|
|
PQclear(res);
|
|
|
|
|
PQfinish(conn);
|
|
|
|
|
|
|
|
|
|
check_ok();
|
|
|
|
|
}
|
|
|
|
|
|
Allow upgrades to preserve the full subscription's state.
This feature will allow us to replicate the changes on subscriber nodes
after the upgrade.
Previously, only the subscription metadata information was preserved.
Without the list of relations and their state, it's not possible to
re-enable the subscriptions without missing some records as the list of
relations can only be refreshed after enabling the subscription (and
therefore starting the apply worker). Even if we added a way to refresh
the subscription while enabling a publication, we still wouldn't know
which relations are new on the publication side, and therefore should be
fully synced, and which shouldn't.
To preserve the subscription relations, this patch teaches pg_dump to
restore the content of pg_subscription_rel from the old cluster by using
binary_upgrade_add_sub_rel_state SQL function. This is supported only
in binary upgrade mode.
The subscription's replication origin is needed to ensure that we don't
replicate anything twice.
To preserve the replication origins, this patch teaches pg_dump to update
the replication origin along with creating a subscription by using
binary_upgrade_replorigin_advance SQL function to restore the
underlying replication origin remote LSN. This is supported only in
binary upgrade mode.
pg_upgrade will check that all the subscription relations are in 'i'
(init) or in 'r' (ready) state and will error out if that's not the case,
logging the reason for the failure. This helps to avoid the risk of any
dangling slot or origin after the upgrade.
Author: Vignesh C, Julien Rouhaud, Shlok Kyal
Reviewed-by: Peter Smith, Masahiko Sawada, Michael Paquier, Amit Kapila, Hayato Kuroda
Discussion: https://postgr.es/m/20230217075433.u5mjly4d5cr4hcfe@jrouhaud
2 years ago
|
|
|
/*
|
|
|
|
|
* check_new_cluster_subscription_configuration()
|
|
|
|
|
*
|
|
|
|
|
* Verify that the max_replication_slots configuration specified is enough for
|
|
|
|
|
* creating the subscriptions. This is required to create the replication
|
|
|
|
|
* origin for each subscription.
|
|
|
|
|
*/
|
|
|
|
|
static void
|
|
|
|
|
check_new_cluster_subscription_configuration(void)
|
|
|
|
|
{
|
|
|
|
|
PGresult *res;
|
|
|
|
|
PGconn *conn;
|
|
|
|
|
int max_replication_slots;
|
|
|
|
|
|
|
|
|
|
/* Subscriptions and their dependencies can be migrated since PG17. */
|
|
|
|
|
if (GET_MAJOR_VERSION(old_cluster.major_version) < 1700)
|
|
|
|
|
return;
|
|
|
|
|
|
|
|
|
|
/* Quick return if there are no subscriptions to be migrated. */
|
pg_upgrade: Retrieve subscription count more efficiently.
Presently, pg_upgrade obtains the number of subscriptions in the
to-be-upgraded cluster by first querying pg_subscription in every
database for the number of subscriptions in only that database.
Then, in count_old_cluster_subscriptions(), it adds all the values
collected in the first step. This is expensive, especially when
there are many databases.
Fortunately, there is a better way to retrieve the subscription
count. Since pg_subscription is a shared catalog, we only need to
connect to a single database and query it once. This commit
modifies pg_upgrade to use that approach, which also allows us to
trim several lines of code. In passing, move the call to
get_db_subscription_count(), which has been renamed to
get_subscription_count(), from get_db_rel_and_slot_infos() to the
dedicated >= v17 section in check_and_dump_old_cluster().
We may be able to make similar improvements to
get_old_cluster_logical_slot_infos(), but that is left as a future
exercise.
Reviewed-by: Michael Paquier, Amit Kapila
Discussion: https://postgr.es/m/ZprQJv_TxccN3tkr%40nathan
Backpatch-through: 17
1 year ago
|
|
|
if (old_cluster.nsubs == 0)
|
Allow upgrades to preserve the full subscription's state.
This feature will allow us to replicate the changes on subscriber nodes
after the upgrade.
Previously, only the subscription metadata information was preserved.
Without the list of relations and their state, it's not possible to
re-enable the subscriptions without missing some records as the list of
relations can only be refreshed after enabling the subscription (and
therefore starting the apply worker). Even if we added a way to refresh
the subscription while enabling a publication, we still wouldn't know
which relations are new on the publication side, and therefore should be
fully synced, and which shouldn't.
To preserve the subscription relations, this patch teaches pg_dump to
restore the content of pg_subscription_rel from the old cluster by using
binary_upgrade_add_sub_rel_state SQL function. This is supported only
in binary upgrade mode.
The subscription's replication origin is needed to ensure that we don't
replicate anything twice.
To preserve the replication origins, this patch teaches pg_dump to update
the replication origin along with creating a subscription by using
binary_upgrade_replorigin_advance SQL function to restore the
underlying replication origin remote LSN. This is supported only in
binary upgrade mode.
pg_upgrade will check that all the subscription relations are in 'i'
(init) or in 'r' (ready) state and will error out if that's not the case,
logging the reason for the failure. This helps to avoid the risk of any
dangling slot or origin after the upgrade.
Author: Vignesh C, Julien Rouhaud, Shlok Kyal
Reviewed-by: Peter Smith, Masahiko Sawada, Michael Paquier, Amit Kapila, Hayato Kuroda
Discussion: https://postgr.es/m/20230217075433.u5mjly4d5cr4hcfe@jrouhaud
2 years ago
|
|
|
return;
|
|
|
|
|
|
|
|
|
|
prep_status("Checking for new cluster configuration for subscriptions");
|
|
|
|
|
|
|
|
|
|
conn = connectToServer(&new_cluster, "template1");
|
|
|
|
|
|
|
|
|
|
res = executeQueryOrDie(conn, "SELECT setting FROM pg_settings "
|
|
|
|
|
"WHERE name = 'max_replication_slots';");
|
|
|
|
|
|
|
|
|
|
if (PQntuples(res) != 1)
|
|
|
|
|
pg_fatal("could not determine parameter settings on new cluster");
|
|
|
|
|
|
|
|
|
|
max_replication_slots = atoi(PQgetvalue(res, 0, 0));
|
pg_upgrade: Retrieve subscription count more efficiently.
Presently, pg_upgrade obtains the number of subscriptions in the
to-be-upgraded cluster by first querying pg_subscription in every
database for the number of subscriptions in only that database.
Then, in count_old_cluster_subscriptions(), it adds all the values
collected in the first step. This is expensive, especially when
there are many databases.
Fortunately, there is a better way to retrieve the subscription
count. Since pg_subscription is a shared catalog, we only need to
connect to a single database and query it once. This commit
modifies pg_upgrade to use that approach, which also allows us to
trim several lines of code. In passing, move the call to
get_db_subscription_count(), which has been renamed to
get_subscription_count(), from get_db_rel_and_slot_infos() to the
dedicated >= v17 section in check_and_dump_old_cluster().
We may be able to make similar improvements to
get_old_cluster_logical_slot_infos(), but that is left as a future
exercise.
Reviewed-by: Michael Paquier, Amit Kapila
Discussion: https://postgr.es/m/ZprQJv_TxccN3tkr%40nathan
Backpatch-through: 17
1 year ago
|
|
|
if (old_cluster.nsubs > max_replication_slots)
|
|
|
|
|
pg_fatal("\"max_replication_slots\" (%d) must be greater than or equal to the number of "
|
Allow upgrades to preserve the full subscription's state.
This feature will allow us to replicate the changes on subscriber nodes
after the upgrade.
Previously, only the subscription metadata information was preserved.
Without the list of relations and their state, it's not possible to
re-enable the subscriptions without missing some records as the list of
relations can only be refreshed after enabling the subscription (and
therefore starting the apply worker). Even if we added a way to refresh
the subscription while enabling a publication, we still wouldn't know
which relations are new on the publication side, and therefore should be
fully synced, and which shouldn't.
To preserve the subscription relations, this patch teaches pg_dump to
restore the content of pg_subscription_rel from the old cluster by using
binary_upgrade_add_sub_rel_state SQL function. This is supported only
in binary upgrade mode.
The subscription's replication origin is needed to ensure that we don't
replicate anything twice.
To preserve the replication origins, this patch teaches pg_dump to update
the replication origin along with creating a subscription by using
binary_upgrade_replorigin_advance SQL function to restore the
underlying replication origin remote LSN. This is supported only in
binary upgrade mode.
pg_upgrade will check that all the subscription relations are in 'i'
(init) or in 'r' (ready) state and will error out if that's not the case,
logging the reason for the failure. This helps to avoid the risk of any
dangling slot or origin after the upgrade.
Author: Vignesh C, Julien Rouhaud, Shlok Kyal
Reviewed-by: Peter Smith, Masahiko Sawada, Michael Paquier, Amit Kapila, Hayato Kuroda
Discussion: https://postgr.es/m/20230217075433.u5mjly4d5cr4hcfe@jrouhaud
2 years ago
|
|
|
"subscriptions (%d) on the old cluster",
|
pg_upgrade: Retrieve subscription count more efficiently.
Presently, pg_upgrade obtains the number of subscriptions in the
to-be-upgraded cluster by first querying pg_subscription in every
database for the number of subscriptions in only that database.
Then, in count_old_cluster_subscriptions(), it adds all the values
collected in the first step. This is expensive, especially when
there are many databases.
Fortunately, there is a better way to retrieve the subscription
count. Since pg_subscription is a shared catalog, we only need to
connect to a single database and query it once. This commit
modifies pg_upgrade to use that approach, which also allows us to
trim several lines of code. In passing, move the call to
get_db_subscription_count(), which has been renamed to
get_subscription_count(), from get_db_rel_and_slot_infos() to the
dedicated >= v17 section in check_and_dump_old_cluster().
We may be able to make similar improvements to
get_old_cluster_logical_slot_infos(), but that is left as a future
exercise.
Reviewed-by: Michael Paquier, Amit Kapila
Discussion: https://postgr.es/m/ZprQJv_TxccN3tkr%40nathan
Backpatch-through: 17
1 year ago
|
|
|
max_replication_slots, old_cluster.nsubs);
|
Allow upgrades to preserve the full subscription's state.
This feature will allow us to replicate the changes on subscriber nodes
after the upgrade.
Previously, only the subscription metadata information was preserved.
Without the list of relations and their state, it's not possible to
re-enable the subscriptions without missing some records as the list of
relations can only be refreshed after enabling the subscription (and
therefore starting the apply worker). Even if we added a way to refresh
the subscription while enabling a publication, we still wouldn't know
which relations are new on the publication side, and therefore should be
fully synced, and which shouldn't.
To preserve the subscription relations, this patch teaches pg_dump to
restore the content of pg_subscription_rel from the old cluster by using
binary_upgrade_add_sub_rel_state SQL function. This is supported only
in binary upgrade mode.
The subscription's replication origin is needed to ensure that we don't
replicate anything twice.
To preserve the replication origins, this patch teaches pg_dump to update
the replication origin along with creating a subscription by using
binary_upgrade_replorigin_advance SQL function to restore the
underlying replication origin remote LSN. This is supported only in
binary upgrade mode.
pg_upgrade will check that all the subscription relations are in 'i'
(init) or in 'r' (ready) state and will error out if that's not the case,
logging the reason for the failure. This helps to avoid the risk of any
dangling slot or origin after the upgrade.
Author: Vignesh C, Julien Rouhaud, Shlok Kyal
Reviewed-by: Peter Smith, Masahiko Sawada, Michael Paquier, Amit Kapila, Hayato Kuroda
Discussion: https://postgr.es/m/20230217075433.u5mjly4d5cr4hcfe@jrouhaud
2 years ago
|
|
|
|
|
|
|
|
PQclear(res);
|
|
|
|
|
PQfinish(conn);
|
|
|
|
|
|
|
|
|
|
check_ok();
|
|
|
|
|
}
|
|
|
|
|
|
Migrate logical slots to the new node during an upgrade.
While reading information from the old cluster, a list of logical
slots is fetched. At the later part of upgrading, pg_upgrade revisits the
list and restores slots by executing pg_create_logical_replication_slot()
on the new cluster. Migration of logical replication slots is only
supported when the old cluster is version 17.0 or later.
If the old node has invalid slots or slots with unconsumed WAL records,
the pg_upgrade fails. These checks are needed to prevent data loss.
The significant advantage of this commit is that it makes it easy to
continue logical replication even after upgrading the publisher node.
Previously, pg_upgrade allowed copying publications to a new node. With
this patch, adjusting the connection string to the new publisher will
cause the apply worker on the subscriber to connect to the new publisher
automatically. This enables seamless continuation of logical replication,
even after an upgrade.
Author: Hayato Kuroda, Hou Zhijie
Reviewed-by: Peter Smith, Bharath Rupireddy, Dilip Kumar, Vignesh C, Shlok Kyal
Discussion: http://postgr.es/m/TYAPR01MB58664C81887B3AF2EB6B16E3F5939@TYAPR01MB5866.jpnprd01.prod.outlook.com
Discussion: http://postgr.es/m/CAA4eK1+t7xYcfa0rEQw839=b2MzsfvYDPz3xbD+ZqOdP3zpKYg@mail.gmail.com
2 years ago
|
|
|
/*
|
|
|
|
|
* check_old_cluster_for_valid_slots()
|
|
|
|
|
*
|
|
|
|
|
* Verify that all the logical slots are valid and have consumed all the WAL
|
|
|
|
|
* before shutdown.
|
|
|
|
|
*/
|
|
|
|
|
static void
|
|
|
|
|
check_old_cluster_for_valid_slots(void)
|
Migrate logical slots to the new node during an upgrade.
While reading information from the old cluster, a list of logical
slots is fetched. At the later part of upgrading, pg_upgrade revisits the
list and restores slots by executing pg_create_logical_replication_slot()
on the new cluster. Migration of logical replication slots is only
supported when the old cluster is version 17.0 or later.
If the old node has invalid slots or slots with unconsumed WAL records,
the pg_upgrade fails. These checks are needed to prevent data loss.
The significant advantage of this commit is that it makes it easy to
continue logical replication even after upgrading the publisher node.
Previously, pg_upgrade allowed copying publications to a new node. With
this patch, adjusting the connection string to the new publisher will
cause the apply worker on the subscriber to connect to the new publisher
automatically. This enables seamless continuation of logical replication,
even after an upgrade.
Author: Hayato Kuroda, Hou Zhijie
Reviewed-by: Peter Smith, Bharath Rupireddy, Dilip Kumar, Vignesh C, Shlok Kyal
Discussion: http://postgr.es/m/TYAPR01MB58664C81887B3AF2EB6B16E3F5939@TYAPR01MB5866.jpnprd01.prod.outlook.com
Discussion: http://postgr.es/m/CAA4eK1+t7xYcfa0rEQw839=b2MzsfvYDPz3xbD+ZqOdP3zpKYg@mail.gmail.com
2 years ago
|
|
|
{
|
|
|
|
|
char output_path[MAXPGPATH];
|
|
|
|
|
FILE *script = NULL;
|
|
|
|
|
|
|
|
|
|
prep_status("Checking for valid logical replication slots");
|
|
|
|
|
|
|
|
|
|
snprintf(output_path, sizeof(output_path), "%s/%s",
|
|
|
|
|
log_opts.basedir,
|
|
|
|
|
"invalid_logical_slots.txt");
|
Migrate logical slots to the new node during an upgrade.
While reading information from the old cluster, a list of logical
slots is fetched. At the later part of upgrading, pg_upgrade revisits the
list and restores slots by executing pg_create_logical_replication_slot()
on the new cluster. Migration of logical replication slots is only
supported when the old cluster is version 17.0 or later.
If the old node has invalid slots or slots with unconsumed WAL records,
the pg_upgrade fails. These checks are needed to prevent data loss.
The significant advantage of this commit is that it makes it easy to
continue logical replication even after upgrading the publisher node.
Previously, pg_upgrade allowed copying publications to a new node. With
this patch, adjusting the connection string to the new publisher will
cause the apply worker on the subscriber to connect to the new publisher
automatically. This enables seamless continuation of logical replication,
even after an upgrade.
Author: Hayato Kuroda, Hou Zhijie
Reviewed-by: Peter Smith, Bharath Rupireddy, Dilip Kumar, Vignesh C, Shlok Kyal
Discussion: http://postgr.es/m/TYAPR01MB58664C81887B3AF2EB6B16E3F5939@TYAPR01MB5866.jpnprd01.prod.outlook.com
Discussion: http://postgr.es/m/CAA4eK1+t7xYcfa0rEQw839=b2MzsfvYDPz3xbD+ZqOdP3zpKYg@mail.gmail.com
2 years ago
|
|
|
|
|
|
|
|
for (int dbnum = 0; dbnum < old_cluster.dbarr.ndbs; dbnum++)
|
|
|
|
|
{
|
|
|
|
|
LogicalSlotInfoArr *slot_arr = &old_cluster.dbarr.dbs[dbnum].slot_arr;
|
|
|
|
|
|
|
|
|
|
for (int slotnum = 0; slotnum < slot_arr->nslots; slotnum++)
|
|
|
|
|
{
|
|
|
|
|
LogicalSlotInfo *slot = &slot_arr->slots[slotnum];
|
|
|
|
|
|
|
|
|
|
/* Is the slot usable? */
|
|
|
|
|
if (slot->invalid)
|
|
|
|
|
{
|
|
|
|
|
if (script == NULL &&
|
|
|
|
|
(script = fopen_priv(output_path, "w")) == NULL)
|
|
|
|
|
pg_fatal("could not open file \"%s\": %m", output_path);
|
Migrate logical slots to the new node during an upgrade.
While reading information from the old cluster, a list of logical
slots is fetched. At the later part of upgrading, pg_upgrade revisits the
list and restores slots by executing pg_create_logical_replication_slot()
on the new cluster. Migration of logical replication slots is only
supported when the old cluster is version 17.0 or later.
If the old node has invalid slots or slots with unconsumed WAL records,
the pg_upgrade fails. These checks are needed to prevent data loss.
The significant advantage of this commit is that it makes it easy to
continue logical replication even after upgrading the publisher node.
Previously, pg_upgrade allowed copying publications to a new node. With
this patch, adjusting the connection string to the new publisher will
cause the apply worker on the subscriber to connect to the new publisher
automatically. This enables seamless continuation of logical replication,
even after an upgrade.
Author: Hayato Kuroda, Hou Zhijie
Reviewed-by: Peter Smith, Bharath Rupireddy, Dilip Kumar, Vignesh C, Shlok Kyal
Discussion: http://postgr.es/m/TYAPR01MB58664C81887B3AF2EB6B16E3F5939@TYAPR01MB5866.jpnprd01.prod.outlook.com
Discussion: http://postgr.es/m/CAA4eK1+t7xYcfa0rEQw839=b2MzsfvYDPz3xbD+ZqOdP3zpKYg@mail.gmail.com
2 years ago
|
|
|
|
|
|
|
|
fprintf(script, "The slot \"%s\" is invalid\n",
|
|
|
|
|
slot->slotname);
|
|
|
|
|
|
|
|
|
|
continue;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Do additional check to ensure that all logical replication
|
|
|
|
|
* slots have consumed all the WAL before shutdown.
|
|
|
|
|
*
|
|
|
|
|
* Note: This can be satisfied only when the old cluster has been
|
|
|
|
|
* shut down, so we skip this for live checks.
|
|
|
|
|
*/
|
|
|
|
|
if (!user_opts.live_check && !slot->caught_up)
|
Migrate logical slots to the new node during an upgrade.
While reading information from the old cluster, a list of logical
slots is fetched. At the later part of upgrading, pg_upgrade revisits the
list and restores slots by executing pg_create_logical_replication_slot()
on the new cluster. Migration of logical replication slots is only
supported when the old cluster is version 17.0 or later.
If the old node has invalid slots or slots with unconsumed WAL records,
the pg_upgrade fails. These checks are needed to prevent data loss.
The significant advantage of this commit is that it makes it easy to
continue logical replication even after upgrading the publisher node.
Previously, pg_upgrade allowed copying publications to a new node. With
this patch, adjusting the connection string to the new publisher will
cause the apply worker on the subscriber to connect to the new publisher
automatically. This enables seamless continuation of logical replication,
even after an upgrade.
Author: Hayato Kuroda, Hou Zhijie
Reviewed-by: Peter Smith, Bharath Rupireddy, Dilip Kumar, Vignesh C, Shlok Kyal
Discussion: http://postgr.es/m/TYAPR01MB58664C81887B3AF2EB6B16E3F5939@TYAPR01MB5866.jpnprd01.prod.outlook.com
Discussion: http://postgr.es/m/CAA4eK1+t7xYcfa0rEQw839=b2MzsfvYDPz3xbD+ZqOdP3zpKYg@mail.gmail.com
2 years ago
|
|
|
{
|
|
|
|
|
if (script == NULL &&
|
|
|
|
|
(script = fopen_priv(output_path, "w")) == NULL)
|
|
|
|
|
pg_fatal("could not open file \"%s\": %m", output_path);
|
Migrate logical slots to the new node during an upgrade.
While reading information from the old cluster, a list of logical
slots is fetched. At the later part of upgrading, pg_upgrade revisits the
list and restores slots by executing pg_create_logical_replication_slot()
on the new cluster. Migration of logical replication slots is only
supported when the old cluster is version 17.0 or later.
If the old node has invalid slots or slots with unconsumed WAL records,
the pg_upgrade fails. These checks are needed to prevent data loss.
The significant advantage of this commit is that it makes it easy to
continue logical replication even after upgrading the publisher node.
Previously, pg_upgrade allowed copying publications to a new node. With
this patch, adjusting the connection string to the new publisher will
cause the apply worker on the subscriber to connect to the new publisher
automatically. This enables seamless continuation of logical replication,
even after an upgrade.
Author: Hayato Kuroda, Hou Zhijie
Reviewed-by: Peter Smith, Bharath Rupireddy, Dilip Kumar, Vignesh C, Shlok Kyal
Discussion: http://postgr.es/m/TYAPR01MB58664C81887B3AF2EB6B16E3F5939@TYAPR01MB5866.jpnprd01.prod.outlook.com
Discussion: http://postgr.es/m/CAA4eK1+t7xYcfa0rEQw839=b2MzsfvYDPz3xbD+ZqOdP3zpKYg@mail.gmail.com
2 years ago
|
|
|
|
|
|
|
|
fprintf(script,
|
|
|
|
|
"The slot \"%s\" has not consumed the WAL yet\n",
|
|
|
|
|
slot->slotname);
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
if (script)
|
|
|
|
|
{
|
|
|
|
|
fclose(script);
|
|
|
|
|
|
|
|
|
|
pg_log(PG_REPORT, "fatal");
|
|
|
|
|
pg_fatal("Your installation contains logical replication slots that cannot be upgraded.\n"
|
Migrate logical slots to the new node during an upgrade.
While reading information from the old cluster, a list of logical
slots is fetched. At the later part of upgrading, pg_upgrade revisits the
list and restores slots by executing pg_create_logical_replication_slot()
on the new cluster. Migration of logical replication slots is only
supported when the old cluster is version 17.0 or later.
If the old node has invalid slots or slots with unconsumed WAL records,
the pg_upgrade fails. These checks are needed to prevent data loss.
The significant advantage of this commit is that it makes it easy to
continue logical replication even after upgrading the publisher node.
Previously, pg_upgrade allowed copying publications to a new node. With
this patch, adjusting the connection string to the new publisher will
cause the apply worker on the subscriber to connect to the new publisher
automatically. This enables seamless continuation of logical replication,
even after an upgrade.
Author: Hayato Kuroda, Hou Zhijie
Reviewed-by: Peter Smith, Bharath Rupireddy, Dilip Kumar, Vignesh C, Shlok Kyal
Discussion: http://postgr.es/m/TYAPR01MB58664C81887B3AF2EB6B16E3F5939@TYAPR01MB5866.jpnprd01.prod.outlook.com
Discussion: http://postgr.es/m/CAA4eK1+t7xYcfa0rEQw839=b2MzsfvYDPz3xbD+ZqOdP3zpKYg@mail.gmail.com
2 years ago
|
|
|
"You can remove invalid slots and/or consume the pending WAL for other slots,\n"
|
|
|
|
|
"and then restart the upgrade.\n"
|
|
|
|
|
"A list of the problematic slots is in the file:\n"
|
|
|
|
|
" %s", output_path);
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
check_ok();
|
|
|
|
|
}
|
Allow upgrades to preserve the full subscription's state.
This feature will allow us to replicate the changes on subscriber nodes
after the upgrade.
Previously, only the subscription metadata information was preserved.
Without the list of relations and their state, it's not possible to
re-enable the subscriptions without missing some records as the list of
relations can only be refreshed after enabling the subscription (and
therefore starting the apply worker). Even if we added a way to refresh
the subscription while enabling a publication, we still wouldn't know
which relations are new on the publication side, and therefore should be
fully synced, and which shouldn't.
To preserve the subscription relations, this patch teaches pg_dump to
restore the content of pg_subscription_rel from the old cluster by using
binary_upgrade_add_sub_rel_state SQL function. This is supported only
in binary upgrade mode.
The subscription's replication origin is needed to ensure that we don't
replicate anything twice.
To preserve the replication origins, this patch teaches pg_dump to update
the replication origin along with creating a subscription by using
binary_upgrade_replorigin_advance SQL function to restore the
underlying replication origin remote LSN. This is supported only in
binary upgrade mode.
pg_upgrade will check that all the subscription relations are in 'i'
(init) or in 'r' (ready) state and will error out if that's not the case,
logging the reason for the failure. This helps to avoid the risk of any
dangling slot or origin after the upgrade.
Author: Vignesh C, Julien Rouhaud, Shlok Kyal
Reviewed-by: Peter Smith, Masahiko Sawada, Michael Paquier, Amit Kapila, Hayato Kuroda
Discussion: https://postgr.es/m/20230217075433.u5mjly4d5cr4hcfe@jrouhaud
2 years ago
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Callback function for processing results of query for
|
|
|
|
|
* check_old_cluster_subscription_state()'s UpgradeTask. If the query returned
|
|
|
|
|
* any rows (i.e., the check failed), write the details to the report file.
|
|
|
|
|
*/
|
|
|
|
|
static void
|
|
|
|
|
process_old_sub_state_check(DbInfo *dbinfo, PGresult *res, void *arg)
|
|
|
|
|
{
|
|
|
|
|
UpgradeTaskReport *report = (UpgradeTaskReport *) arg;
|
|
|
|
|
int ntup = PQntuples(res);
|
|
|
|
|
int i_srsubstate = PQfnumber(res, "srsubstate");
|
|
|
|
|
int i_subname = PQfnumber(res, "subname");
|
|
|
|
|
int i_nspname = PQfnumber(res, "nspname");
|
|
|
|
|
int i_relname = PQfnumber(res, "relname");
|
|
|
|
|
|
|
|
|
|
AssertVariableIsOfType(&process_old_sub_state_check, UpgradeTaskProcessCB);
|
|
|
|
|
|
|
|
|
|
for (int i = 0; i < ntup; i++)
|
|
|
|
|
{
|
|
|
|
|
if (report->file == NULL &&
|
|
|
|
|
(report->file = fopen_priv(report->path, "w")) == NULL)
|
|
|
|
|
pg_fatal("could not open file \"%s\": %m", report->path);
|
|
|
|
|
|
|
|
|
|
fprintf(report->file, "The table sync state \"%s\" is not allowed for database:\"%s\" subscription:\"%s\" schema:\"%s\" relation:\"%s\"\n",
|
|
|
|
|
PQgetvalue(res, i, i_srsubstate),
|
|
|
|
|
dbinfo->db_name,
|
|
|
|
|
PQgetvalue(res, i, i_subname),
|
|
|
|
|
PQgetvalue(res, i, i_nspname),
|
|
|
|
|
PQgetvalue(res, i, i_relname));
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
|
Allow upgrades to preserve the full subscription's state.
This feature will allow us to replicate the changes on subscriber nodes
after the upgrade.
Previously, only the subscription metadata information was preserved.
Without the list of relations and their state, it's not possible to
re-enable the subscriptions without missing some records as the list of
relations can only be refreshed after enabling the subscription (and
therefore starting the apply worker). Even if we added a way to refresh
the subscription while enabling a publication, we still wouldn't know
which relations are new on the publication side, and therefore should be
fully synced, and which shouldn't.
To preserve the subscription relations, this patch teaches pg_dump to
restore the content of pg_subscription_rel from the old cluster by using
binary_upgrade_add_sub_rel_state SQL function. This is supported only
in binary upgrade mode.
The subscription's replication origin is needed to ensure that we don't
replicate anything twice.
To preserve the replication origins, this patch teaches pg_dump to update
the replication origin along with creating a subscription by using
binary_upgrade_replorigin_advance SQL function to restore the
underlying replication origin remote LSN. This is supported only in
binary upgrade mode.
pg_upgrade will check that all the subscription relations are in 'i'
(init) or in 'r' (ready) state and will error out if that's not the case,
logging the reason for the failure. This helps to avoid the risk of any
dangling slot or origin after the upgrade.
Author: Vignesh C, Julien Rouhaud, Shlok Kyal
Reviewed-by: Peter Smith, Masahiko Sawada, Michael Paquier, Amit Kapila, Hayato Kuroda
Discussion: https://postgr.es/m/20230217075433.u5mjly4d5cr4hcfe@jrouhaud
2 years ago
|
|
|
/*
|
|
|
|
|
* check_old_cluster_subscription_state()
|
|
|
|
|
*
|
|
|
|
|
* Verify that the replication origin corresponding to each of the
|
|
|
|
|
* subscriptions are present and each of the subscribed tables is in
|
|
|
|
|
* 'i' (initialize) or 'r' (ready) state.
|
|
|
|
|
*/
|
|
|
|
|
static void
|
|
|
|
|
check_old_cluster_subscription_state(void)
|
|
|
|
|
{
|
|
|
|
|
UpgradeTask *task = upgrade_task_create();
|
|
|
|
|
UpgradeTaskReport report;
|
|
|
|
|
const char *query;
|
|
|
|
|
PGresult *res;
|
|
|
|
|
PGconn *conn;
|
Allow upgrades to preserve the full subscription's state.
This feature will allow us to replicate the changes on subscriber nodes
after the upgrade.
Previously, only the subscription metadata information was preserved.
Without the list of relations and their state, it's not possible to
re-enable the subscriptions without missing some records as the list of
relations can only be refreshed after enabling the subscription (and
therefore starting the apply worker). Even if we added a way to refresh
the subscription while enabling a publication, we still wouldn't know
which relations are new on the publication side, and therefore should be
fully synced, and which shouldn't.
To preserve the subscription relations, this patch teaches pg_dump to
restore the content of pg_subscription_rel from the old cluster by using
binary_upgrade_add_sub_rel_state SQL function. This is supported only
in binary upgrade mode.
The subscription's replication origin is needed to ensure that we don't
replicate anything twice.
To preserve the replication origins, this patch teaches pg_dump to update
the replication origin along with creating a subscription by using
binary_upgrade_replorigin_advance SQL function to restore the
underlying replication origin remote LSN. This is supported only in
binary upgrade mode.
pg_upgrade will check that all the subscription relations are in 'i'
(init) or in 'r' (ready) state and will error out if that's not the case,
logging the reason for the failure. This helps to avoid the risk of any
dangling slot or origin after the upgrade.
Author: Vignesh C, Julien Rouhaud, Shlok Kyal
Reviewed-by: Peter Smith, Masahiko Sawada, Michael Paquier, Amit Kapila, Hayato Kuroda
Discussion: https://postgr.es/m/20230217075433.u5mjly4d5cr4hcfe@jrouhaud
2 years ago
|
|
|
int ntup;
|
|
|
|
|
|
|
|
|
|
prep_status("Checking for subscription state");
|
|
|
|
|
|
|
|
|
|
report.file = NULL;
|
|
|
|
|
snprintf(report.path, sizeof(report.path), "%s/%s",
|
Allow upgrades to preserve the full subscription's state.
This feature will allow us to replicate the changes on subscriber nodes
after the upgrade.
Previously, only the subscription metadata information was preserved.
Without the list of relations and their state, it's not possible to
re-enable the subscriptions without missing some records as the list of
relations can only be refreshed after enabling the subscription (and
therefore starting the apply worker). Even if we added a way to refresh
the subscription while enabling a publication, we still wouldn't know
which relations are new on the publication side, and therefore should be
fully synced, and which shouldn't.
To preserve the subscription relations, this patch teaches pg_dump to
restore the content of pg_subscription_rel from the old cluster by using
binary_upgrade_add_sub_rel_state SQL function. This is supported only
in binary upgrade mode.
The subscription's replication origin is needed to ensure that we don't
replicate anything twice.
To preserve the replication origins, this patch teaches pg_dump to update
the replication origin along with creating a subscription by using
binary_upgrade_replorigin_advance SQL function to restore the
underlying replication origin remote LSN. This is supported only in
binary upgrade mode.
pg_upgrade will check that all the subscription relations are in 'i'
(init) or in 'r' (ready) state and will error out if that's not the case,
logging the reason for the failure. This helps to avoid the risk of any
dangling slot or origin after the upgrade.
Author: Vignesh C, Julien Rouhaud, Shlok Kyal
Reviewed-by: Peter Smith, Masahiko Sawada, Michael Paquier, Amit Kapila, Hayato Kuroda
Discussion: https://postgr.es/m/20230217075433.u5mjly4d5cr4hcfe@jrouhaud
2 years ago
|
|
|
log_opts.basedir,
|
|
|
|
|
"subs_invalid.txt");
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Check that all the subscriptions have their respective replication
|
|
|
|
|
* origin. This check only needs to run once.
|
|
|
|
|
*/
|
|
|
|
|
conn = connectToServer(&old_cluster, old_cluster.dbarr.dbs[0].db_name);
|
|
|
|
|
res = executeQueryOrDie(conn,
|
|
|
|
|
"SELECT d.datname, s.subname "
|
|
|
|
|
"FROM pg_catalog.pg_subscription s "
|
|
|
|
|
"LEFT OUTER JOIN pg_catalog.pg_replication_origin o "
|
|
|
|
|
" ON o.roname = 'pg_' || s.oid "
|
|
|
|
|
"INNER JOIN pg_catalog.pg_database d "
|
|
|
|
|
" ON d.oid = s.subdbid "
|
|
|
|
|
"WHERE o.roname IS NULL;");
|
|
|
|
|
ntup = PQntuples(res);
|
|
|
|
|
for (int i = 0; i < ntup; i++)
|
|
|
|
|
{
|
|
|
|
|
if (report.file == NULL &&
|
|
|
|
|
(report.file = fopen_priv(report.path, "w")) == NULL)
|
|
|
|
|
pg_fatal("could not open file \"%s\": %m", report.path);
|
|
|
|
|
fprintf(report.file, "The replication origin is missing for database:\"%s\" subscription:\"%s\"\n",
|
|
|
|
|
PQgetvalue(res, i, 0),
|
|
|
|
|
PQgetvalue(res, i, 1));
|
Allow upgrades to preserve the full subscription's state.
This feature will allow us to replicate the changes on subscriber nodes
after the upgrade.
Previously, only the subscription metadata information was preserved.
Without the list of relations and their state, it's not possible to
re-enable the subscriptions without missing some records as the list of
relations can only be refreshed after enabling the subscription (and
therefore starting the apply worker). Even if we added a way to refresh
the subscription while enabling a publication, we still wouldn't know
which relations are new on the publication side, and therefore should be
fully synced, and which shouldn't.
To preserve the subscription relations, this patch teaches pg_dump to
restore the content of pg_subscription_rel from the old cluster by using
binary_upgrade_add_sub_rel_state SQL function. This is supported only
in binary upgrade mode.
The subscription's replication origin is needed to ensure that we don't
replicate anything twice.
To preserve the replication origins, this patch teaches pg_dump to update
the replication origin along with creating a subscription by using
binary_upgrade_replorigin_advance SQL function to restore the
underlying replication origin remote LSN. This is supported only in
binary upgrade mode.
pg_upgrade will check that all the subscription relations are in 'i'
(init) or in 'r' (ready) state and will error out if that's not the case,
logging the reason for the failure. This helps to avoid the risk of any
dangling slot or origin after the upgrade.
Author: Vignesh C, Julien Rouhaud, Shlok Kyal
Reviewed-by: Peter Smith, Masahiko Sawada, Michael Paquier, Amit Kapila, Hayato Kuroda
Discussion: https://postgr.es/m/20230217075433.u5mjly4d5cr4hcfe@jrouhaud
2 years ago
|
|
|
}
|
|
|
|
|
PQclear(res);
|
|
|
|
|
PQfinish(conn);
|
Allow upgrades to preserve the full subscription's state.
This feature will allow us to replicate the changes on subscriber nodes
after the upgrade.
Previously, only the subscription metadata information was preserved.
Without the list of relations and their state, it's not possible to
re-enable the subscriptions without missing some records as the list of
relations can only be refreshed after enabling the subscription (and
therefore starting the apply worker). Even if we added a way to refresh
the subscription while enabling a publication, we still wouldn't know
which relations are new on the publication side, and therefore should be
fully synced, and which shouldn't.
To preserve the subscription relations, this patch teaches pg_dump to
restore the content of pg_subscription_rel from the old cluster by using
binary_upgrade_add_sub_rel_state SQL function. This is supported only
in binary upgrade mode.
The subscription's replication origin is needed to ensure that we don't
replicate anything twice.
To preserve the replication origins, this patch teaches pg_dump to update
the replication origin along with creating a subscription by using
binary_upgrade_replorigin_advance SQL function to restore the
underlying replication origin remote LSN. This is supported only in
binary upgrade mode.
pg_upgrade will check that all the subscription relations are in 'i'
(init) or in 'r' (ready) state and will error out if that's not the case,
logging the reason for the failure. This helps to avoid the risk of any
dangling slot or origin after the upgrade.
Author: Vignesh C, Julien Rouhaud, Shlok Kyal
Reviewed-by: Peter Smith, Masahiko Sawada, Michael Paquier, Amit Kapila, Hayato Kuroda
Discussion: https://postgr.es/m/20230217075433.u5mjly4d5cr4hcfe@jrouhaud
2 years ago
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* We don't allow upgrade if there is a risk of dangling slot or origin
|
|
|
|
|
* corresponding to initial sync after upgrade.
|
|
|
|
|
*
|
|
|
|
|
* A slot/origin not created yet refers to the 'i' (initialize) state,
|
|
|
|
|
* while 'r' (ready) state refers to a slot/origin created previously but
|
|
|
|
|
* already dropped. These states are supported for pg_upgrade. The other
|
|
|
|
|
* states listed below are not supported:
|
|
|
|
|
*
|
|
|
|
|
* a) SUBREL_STATE_DATASYNC: A relation upgraded while in this state would
|
|
|
|
|
* retain a replication slot, which could not be dropped by the sync
|
|
|
|
|
* worker spawned after the upgrade because the subscription ID used for
|
|
|
|
|
* the slot name won't match anymore.
|
|
|
|
|
*
|
|
|
|
|
* b) SUBREL_STATE_SYNCDONE: A relation upgraded while in this state would
|
|
|
|
|
* retain the replication origin when there is a failure in tablesync
|
|
|
|
|
* worker immediately after dropping the replication slot in the
|
|
|
|
|
* publisher.
|
|
|
|
|
*
|
|
|
|
|
* c) SUBREL_STATE_FINISHEDCOPY: A tablesync worker spawned to work on a
|
|
|
|
|
* relation upgraded while in this state would expect an origin ID with
|
|
|
|
|
* the OID of the subscription used before the upgrade, causing it to
|
|
|
|
|
* fail.
|
|
|
|
|
*
|
|
|
|
|
* d) SUBREL_STATE_SYNCWAIT, SUBREL_STATE_CATCHUP and
|
|
|
|
|
* SUBREL_STATE_UNKNOWN: These states are not stored in the catalog, so we
|
|
|
|
|
* need not allow these states.
|
|
|
|
|
*/
|
|
|
|
|
query = "SELECT r.srsubstate, s.subname, n.nspname, c.relname "
|
|
|
|
|
"FROM pg_catalog.pg_subscription_rel r "
|
|
|
|
|
"LEFT JOIN pg_catalog.pg_subscription s"
|
|
|
|
|
" ON r.srsubid = s.oid "
|
|
|
|
|
"LEFT JOIN pg_catalog.pg_class c"
|
|
|
|
|
" ON r.srrelid = c.oid "
|
|
|
|
|
"LEFT JOIN pg_catalog.pg_namespace n"
|
|
|
|
|
" ON c.relnamespace = n.oid "
|
|
|
|
|
"WHERE r.srsubstate NOT IN ('i', 'r') "
|
|
|
|
|
"ORDER BY s.subname";
|
|
|
|
|
|
|
|
|
|
upgrade_task_add_step(task, query, process_old_sub_state_check,
|
|
|
|
|
true, &report);
|
|
|
|
|
|
|
|
|
|
upgrade_task_run(task, &old_cluster);
|
|
|
|
|
upgrade_task_free(task);
|
|
|
|
|
|
|
|
|
|
if (report.file)
|
Allow upgrades to preserve the full subscription's state.
This feature will allow us to replicate the changes on subscriber nodes
after the upgrade.
Previously, only the subscription metadata information was preserved.
Without the list of relations and their state, it's not possible to
re-enable the subscriptions without missing some records as the list of
relations can only be refreshed after enabling the subscription (and
therefore starting the apply worker). Even if we added a way to refresh
the subscription while enabling a publication, we still wouldn't know
which relations are new on the publication side, and therefore should be
fully synced, and which shouldn't.
To preserve the subscription relations, this patch teaches pg_dump to
restore the content of pg_subscription_rel from the old cluster by using
binary_upgrade_add_sub_rel_state SQL function. This is supported only
in binary upgrade mode.
The subscription's replication origin is needed to ensure that we don't
replicate anything twice.
To preserve the replication origins, this patch teaches pg_dump to update
the replication origin along with creating a subscription by using
binary_upgrade_replorigin_advance SQL function to restore the
underlying replication origin remote LSN. This is supported only in
binary upgrade mode.
pg_upgrade will check that all the subscription relations are in 'i'
(init) or in 'r' (ready) state and will error out if that's not the case,
logging the reason for the failure. This helps to avoid the risk of any
dangling slot or origin after the upgrade.
Author: Vignesh C, Julien Rouhaud, Shlok Kyal
Reviewed-by: Peter Smith, Masahiko Sawada, Michael Paquier, Amit Kapila, Hayato Kuroda
Discussion: https://postgr.es/m/20230217075433.u5mjly4d5cr4hcfe@jrouhaud
2 years ago
|
|
|
{
|
|
|
|
|
fclose(report.file);
|
Allow upgrades to preserve the full subscription's state.
This feature will allow us to replicate the changes on subscriber nodes
after the upgrade.
Previously, only the subscription metadata information was preserved.
Without the list of relations and their state, it's not possible to
re-enable the subscriptions without missing some records as the list of
relations can only be refreshed after enabling the subscription (and
therefore starting the apply worker). Even if we added a way to refresh
the subscription while enabling a publication, we still wouldn't know
which relations are new on the publication side, and therefore should be
fully synced, and which shouldn't.
To preserve the subscription relations, this patch teaches pg_dump to
restore the content of pg_subscription_rel from the old cluster by using
binary_upgrade_add_sub_rel_state SQL function. This is supported only
in binary upgrade mode.
The subscription's replication origin is needed to ensure that we don't
replicate anything twice.
To preserve the replication origins, this patch teaches pg_dump to update
the replication origin along with creating a subscription by using
binary_upgrade_replorigin_advance SQL function to restore the
underlying replication origin remote LSN. This is supported only in
binary upgrade mode.
pg_upgrade will check that all the subscription relations are in 'i'
(init) or in 'r' (ready) state and will error out if that's not the case,
logging the reason for the failure. This helps to avoid the risk of any
dangling slot or origin after the upgrade.
Author: Vignesh C, Julien Rouhaud, Shlok Kyal
Reviewed-by: Peter Smith, Masahiko Sawada, Michael Paquier, Amit Kapila, Hayato Kuroda
Discussion: https://postgr.es/m/20230217075433.u5mjly4d5cr4hcfe@jrouhaud
2 years ago
|
|
|
pg_log(PG_REPORT, "fatal");
|
|
|
|
|
pg_fatal("Your installation contains subscriptions without origin or having relations not in i (initialize) or r (ready) state.\n"
|
|
|
|
|
"You can allow the initial sync to finish for all relations and then restart the upgrade.\n"
|
|
|
|
|
"A list of the problematic subscriptions is in the file:\n"
|
|
|
|
|
" %s", report.path);
|
Allow upgrades to preserve the full subscription's state.
This feature will allow us to replicate the changes on subscriber nodes
after the upgrade.
Previously, only the subscription metadata information was preserved.
Without the list of relations and their state, it's not possible to
re-enable the subscriptions without missing some records as the list of
relations can only be refreshed after enabling the subscription (and
therefore starting the apply worker). Even if we added a way to refresh
the subscription while enabling a publication, we still wouldn't know
which relations are new on the publication side, and therefore should be
fully synced, and which shouldn't.
To preserve the subscription relations, this patch teaches pg_dump to
restore the content of pg_subscription_rel from the old cluster by using
binary_upgrade_add_sub_rel_state SQL function. This is supported only
in binary upgrade mode.
The subscription's replication origin is needed to ensure that we don't
replicate anything twice.
To preserve the replication origins, this patch teaches pg_dump to update
the replication origin along with creating a subscription by using
binary_upgrade_replorigin_advance SQL function to restore the
underlying replication origin remote LSN. This is supported only in
binary upgrade mode.
pg_upgrade will check that all the subscription relations are in 'i'
(init) or in 'r' (ready) state and will error out if that's not the case,
logging the reason for the failure. This helps to avoid the risk of any
dangling slot or origin after the upgrade.
Author: Vignesh C, Julien Rouhaud, Shlok Kyal
Reviewed-by: Peter Smith, Masahiko Sawada, Michael Paquier, Amit Kapila, Hayato Kuroda
Discussion: https://postgr.es/m/20230217075433.u5mjly4d5cr4hcfe@jrouhaud
2 years ago
|
|
|
}
|
|
|
|
|
else
|
|
|
|
|
check_ok();
|
|
|
|
|
}
|
