|
|
|
|
/*-------------------------------------------------------------------------
|
|
|
|
|
*
|
|
|
|
|
* tupdesc.c
|
|
|
|
|
* POSTGRES tuple descriptor support code
|
|
|
|
|
*
|
|
|
|
|
* Portions Copyright (c) 1996-2025, PostgreSQL Global Development Group
|
|
|
|
|
* Portions Copyright (c) 1994, Regents of the University of California
|
|
|
|
|
*
|
|
|
|
|
*
|
|
|
|
|
* IDENTIFICATION
|
|
|
|
|
* src/backend/access/common/tupdesc.c
|
|
|
|
|
*
|
|
|
|
|
* NOTES
|
|
|
|
|
* some of the executor utility code such as "ExecTypeFromTL" should be
|
|
|
|
|
* moved here.
|
|
|
|
|
*
|
|
|
|
|
*-------------------------------------------------------------------------
|
|
|
|
|
*/
|
|
|
|
|
|
|
|
|
|
#include "postgres.h"
|
|
|
|
|
|
|
|
|
|
#include "access/htup_details.h"
|
Allow configurable LZ4 TOAST compression.
There is now a per-column COMPRESSION option which can be set to pglz
(the default, and the only option in up until now) or lz4. Or, if you
like, you can set the new default_toast_compression GUC to lz4, and
then that will be the default for new table columns for which no value
is specified. We don't have lz4 support in the PostgreSQL code, so
to use lz4 compression, PostgreSQL must be built --with-lz4.
In general, TOAST compression means compression of individual column
values, not the whole tuple, and those values can either be compressed
inline within the tuple or compressed and then stored externally in
the TOAST table, so those properties also apply to this feature.
Prior to this commit, a TOAST pointer has two unused bits as part of
the va_extsize field, and a compessed datum has two unused bits as
part of the va_rawsize field. These bits are unused because the length
of a varlena is limited to 1GB; we now use them to indicate the
compression type that was used. This means we only have bit space for
2 more built-in compresison types, but we could work around that
problem, if necessary, by introducing a new vartag_external value for
any further types we end up wanting to add. Hopefully, it won't be
too important to offer a wide selection of algorithms here, since
each one we add not only takes more coding but also adds a build
dependency for every packager. Nevertheless, it seems worth doing
at least this much, because LZ4 gets better compression than PGLZ
with less CPU usage.
It's possible for LZ4-compressed datums to leak into composite type
values stored on disk, just as it is for PGLZ. It's also possible for
LZ4-compressed attributes to be copied into a different table via SQL
commands such as CREATE TABLE AS or INSERT .. SELECT. It would be
expensive to force such values to be decompressed, so PostgreSQL has
never done so. For the same reasons, we also don't force recompression
of already-compressed values even if the target table prefers a
different compression method than was used for the source data. These
architectural decisions are perhaps arguable but revisiting them is
well beyond the scope of what seemed possible to do as part of this
project. However, it's relatively cheap to recompress as part of
VACUUM FULL or CLUSTER, so this commit adjusts those commands to do
so, if the configured compression method of the table happens not to
match what was used for some column value stored therein.
Dilip Kumar. The original patches on which this work was based were
written by Ildus Kurbangaliev, and those were patches were based on
even earlier work by Nikita Glukhov, but the design has since changed
very substantially, since allow a potentially large number of
compression methods that could be added and dropped on a running
system proved too problematic given some of the architectural issues
mentioned above; the choice of which specific compression method to
add first is now different; and a lot of the code has been heavily
refactored. More recently, Justin Przyby helped quite a bit with
testing and reviewing and this version also includes some code
contributions from him. Other design input and review from Tomas
Vondra, Álvaro Herrera, Andres Freund, Oleg Bartunov, Alexander
Korotkov, and me.
Discussion: http://postgr.es/m/20170907194236.4cefce96%40wp.localdomain
Discussion: http://postgr.es/m/CAFiTN-uUpX3ck%3DK0mLEk-G_kUQY%3DSNOTeqdaNRR9FMdQrHKebw%40mail.gmail.com
5 years ago
|
|
|
#include "access/toast_compression.h"
|
|
|
|
|
#include "access/tupdesc_details.h"
|
Allow NOT NULL constraints to be added as NOT VALID
This allows them to be added without scanning the table, and validating
them afterwards without holding access exclusive lock on the table after
any violating rows have been deleted or fixed.
Doing ALTER TABLE ... SET NOT NULL for a column that has an invalid
not-null constraint validates that constraint. ALTER TABLE .. VALIDATE
CONSTRAINT is also supported. There are various checks on whether an
invalid constraint is allowed in a child table when the parent table has
a valid constraint; this should match what we do for enforced/not
enforced constraints.
pg_attribute.attnotnull is now only an indicator for whether a not-null
constraint exists for the column; whether it's valid or invalid must be
queried in pg_constraint. Applications can continue to query
pg_attribute.attnotnull as before, but now it's possible that NULL rows
are present in the column even when that's set to true.
For backend internal purposes, we cache the nullability status in
CompactAttribute->attnullability that each tuple descriptor carries
(replacing CompactAttribute.attnotnull, which was a mirror of
Form_pg_attribute.attnotnull). During the initial tuple descriptor
creation, based on the pg_attribute scan, we set this to UNRESTRICTED if
pg_attribute.attnotnull is false, or to UNKNOWN if it's true; then we
update the latter to VALID or INVALID depending on the pg_constraint
scan. This flag is also copied when tupledescs are copied.
Comparing tuple descs for equality must also compare the
CompactAttribute.attnullability flag and return false in case of a
mismatch.
pg_dump deals with these constraints by storing the OIDs of invalid
not-null constraints in a separate array, and running a query to obtain
their properties. The regular table creation SQL omits them entirely.
They are then dealt with in the same way as "separate" CHECK
constraints, and dumped after the data has been loaded. Because no
additional pg_dump infrastructure was required, we don't bump its
version number.
I decided not to bump catversion either, because the old catalog state
works perfectly in the new world. (Trying to run with new catalog state
and the old server version would likely run into issues, however.)
System catalogs do not support invalid not-null constraints (because
commit 14e87ffa5c54 didn't allow them to have pg_constraint rows
anyway.)
Author: Rushabh Lathia <rushabh.lathia@gmail.com>
Author: Jian He <jian.universality@gmail.com>
Reviewed-by: Álvaro Herrera <alvherre@alvh.no-ip.org>
Tested-by: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
Discussion: https://postgr.es/m/CAGPqQf0KitkNack4F5CFkFi-9Dqvp29Ro=EpcWt=4_hs-Rt+bQ@mail.gmail.com
9 months ago
|
|
|
#include "catalog/catalog.h"
|
|
|
|
|
#include "catalog/pg_collation.h"
|
|
|
|
|
#include "catalog/pg_type.h"
|
|
|
|
|
#include "common/hashfn.h"
|
|
|
|
|
#include "utils/builtins.h"
|
|
|
|
|
#include "utils/datum.h"
|
Make ResourceOwners more easily extensible.
Instead of having a separate array/hash for each resource kind, use a
single array and hash to hold all kinds of resources. This makes it
possible to introduce new resource "kinds" without having to modify
the ResourceOwnerData struct. In particular, this makes it possible
for extensions to register custom resource kinds.
The old approach was to have a small array of resources of each kind,
and if it fills up, switch to a hash table. The new approach also uses
an array and a hash, but now the array and the hash are used at the
same time. The array is used to hold the recently added resources, and
when it fills up, they are moved to the hash. This keeps the access to
recent entries fast, even when there are a lot of long-held resources.
All the resource-specific ResourceOwnerEnlarge*(),
ResourceOwnerRemember*(), and ResourceOwnerForget*() functions have
been replaced with three generic functions that take resource kind as
argument. For convenience, we still define resource-specific wrapper
macros around the generic functions with the old names, but they are
now defined in the source files that use those resource kinds.
The release callback no longer needs to call ResourceOwnerForget on
the resource being released. ResourceOwnerRelease unregisters the
resource from the owner before calling the callback. That needed some
changes in bufmgr.c and some other files, where releasing the
resources previously always called ResourceOwnerForget.
Each resource kind specifies a release priority, and
ResourceOwnerReleaseAll releases the resources in priority order. To
make that possible, we have to restrict what you can do between
phases. After calling ResourceOwnerRelease(), you are no longer
allowed to remember any more resources in it or to forget any
previously remembered resources by calling ResourceOwnerForget. There
was one case where that was done previously. At subtransaction commit,
AtEOSubXact_Inval() would handle the invalidation messages and call
RelationFlushRelation(), which temporarily increased the reference
count on the relation being flushed. We now switch to the parent
subtransaction's resource owner before calling AtEOSubXact_Inval(), so
that there is a valid ResourceOwner to temporarily hold that relcache
reference.
Other end-of-xact routines make similar calls to AtEOXact_Inval()
between release phases, but I didn't see any regression test failures
from those, so I'm not sure if they could reach a codepath that needs
remembering extra resources.
There were two exceptions to how the resource leak WARNINGs on commit
were printed previously: llvmjit silently released the context without
printing the warning, and a leaked buffer io triggered a PANIC. Now
everything prints a WARNING, including those cases.
Add tests in src/test/modules/test_resowner.
Reviewed-by: Aleksander Alekseev, Michael Paquier, Julien Rouhaud
Reviewed-by: Kyotaro Horiguchi, Hayato Kuroda, Álvaro Herrera, Zhihong Yu
Reviewed-by: Peter Eisentraut, Andres Freund
Discussion: https://www.postgresql.org/message-id/cbfabeb0-cd3c-e951-a572-19b365ed314d%40iki.fi
2 years ago
|
|
|
#include "utils/resowner.h"
|
|
|
|
|
#include "utils/syscache.h"
|
|
|
|
|
|
Make ResourceOwners more easily extensible.
Instead of having a separate array/hash for each resource kind, use a
single array and hash to hold all kinds of resources. This makes it
possible to introduce new resource "kinds" without having to modify
the ResourceOwnerData struct. In particular, this makes it possible
for extensions to register custom resource kinds.
The old approach was to have a small array of resources of each kind,
and if it fills up, switch to a hash table. The new approach also uses
an array and a hash, but now the array and the hash are used at the
same time. The array is used to hold the recently added resources, and
when it fills up, they are moved to the hash. This keeps the access to
recent entries fast, even when there are a lot of long-held resources.
All the resource-specific ResourceOwnerEnlarge*(),
ResourceOwnerRemember*(), and ResourceOwnerForget*() functions have
been replaced with three generic functions that take resource kind as
argument. For convenience, we still define resource-specific wrapper
macros around the generic functions with the old names, but they are
now defined in the source files that use those resource kinds.
The release callback no longer needs to call ResourceOwnerForget on
the resource being released. ResourceOwnerRelease unregisters the
resource from the owner before calling the callback. That needed some
changes in bufmgr.c and some other files, where releasing the
resources previously always called ResourceOwnerForget.
Each resource kind specifies a release priority, and
ResourceOwnerReleaseAll releases the resources in priority order. To
make that possible, we have to restrict what you can do between
phases. After calling ResourceOwnerRelease(), you are no longer
allowed to remember any more resources in it or to forget any
previously remembered resources by calling ResourceOwnerForget. There
was one case where that was done previously. At subtransaction commit,
AtEOSubXact_Inval() would handle the invalidation messages and call
RelationFlushRelation(), which temporarily increased the reference
count on the relation being flushed. We now switch to the parent
subtransaction's resource owner before calling AtEOSubXact_Inval(), so
that there is a valid ResourceOwner to temporarily hold that relcache
reference.
Other end-of-xact routines make similar calls to AtEOXact_Inval()
between release phases, but I didn't see any regression test failures
from those, so I'm not sure if they could reach a codepath that needs
remembering extra resources.
There were two exceptions to how the resource leak WARNINGs on commit
were printed previously: llvmjit silently released the context without
printing the warning, and a leaked buffer io triggered a PANIC. Now
everything prints a WARNING, including those cases.
Add tests in src/test/modules/test_resowner.
Reviewed-by: Aleksander Alekseev, Michael Paquier, Julien Rouhaud
Reviewed-by: Kyotaro Horiguchi, Hayato Kuroda, Álvaro Herrera, Zhihong Yu
Reviewed-by: Peter Eisentraut, Andres Freund
Discussion: https://www.postgresql.org/message-id/cbfabeb0-cd3c-e951-a572-19b365ed314d%40iki.fi
2 years ago
|
|
|
/* ResourceOwner callbacks to hold tupledesc references */
|
|
|
|
|
static void ResOwnerReleaseTupleDesc(Datum res);
|
|
|
|
|
static char *ResOwnerPrintTupleDesc(Datum res);
|
|
|
|
|
|
|
|
|
|
static const ResourceOwnerDesc tupdesc_resowner_desc =
|
|
|
|
|
{
|
|
|
|
|
.name = "tupdesc reference",
|
|
|
|
|
.release_phase = RESOURCE_RELEASE_AFTER_LOCKS,
|
|
|
|
|
.release_priority = RELEASE_PRIO_TUPDESC_REFS,
|
|
|
|
|
.ReleaseResource = ResOwnerReleaseTupleDesc,
|
|
|
|
|
.DebugPrint = ResOwnerPrintTupleDesc
|
|
|
|
|
};
|
|
|
|
|
|
|
|
|
|
/* Convenience wrappers over ResourceOwnerRemember/Forget */
|
|
|
|
|
static inline void
|
|
|
|
|
ResourceOwnerRememberTupleDesc(ResourceOwner owner, TupleDesc tupdesc)
|
|
|
|
|
{
|
|
|
|
|
ResourceOwnerRemember(owner, PointerGetDatum(tupdesc), &tupdesc_resowner_desc);
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
static inline void
|
|
|
|
|
ResourceOwnerForgetTupleDesc(ResourceOwner owner, TupleDesc tupdesc)
|
|
|
|
|
{
|
|
|
|
|
ResourceOwnerForget(owner, PointerGetDatum(tupdesc), &tupdesc_resowner_desc);
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* populate_compact_attribute_internal
|
|
|
|
|
* Helper function for populate_compact_attribute()
|
|
|
|
|
*/
|
|
|
|
|
static inline void
|
|
|
|
|
populate_compact_attribute_internal(Form_pg_attribute src,
|
|
|
|
|
CompactAttribute *dst)
|
|
|
|
|
{
|
|
|
|
|
memset(dst, 0, sizeof(CompactAttribute));
|
|
|
|
|
|
|
|
|
|
dst->attcacheoff = -1;
|
|
|
|
|
dst->attlen = src->attlen;
|
|
|
|
|
|
|
|
|
|
dst->attbyval = src->attbyval;
|
|
|
|
|
dst->attispackable = (src->attstorage != TYPSTORAGE_PLAIN);
|
|
|
|
|
dst->atthasmissing = src->atthasmissing;
|
|
|
|
|
dst->attisdropped = src->attisdropped;
|
|
|
|
|
dst->attgenerated = (src->attgenerated != '\0');
|
Allow NOT NULL constraints to be added as NOT VALID
This allows them to be added without scanning the table, and validating
them afterwards without holding access exclusive lock on the table after
any violating rows have been deleted or fixed.
Doing ALTER TABLE ... SET NOT NULL for a column that has an invalid
not-null constraint validates that constraint. ALTER TABLE .. VALIDATE
CONSTRAINT is also supported. There are various checks on whether an
invalid constraint is allowed in a child table when the parent table has
a valid constraint; this should match what we do for enforced/not
enforced constraints.
pg_attribute.attnotnull is now only an indicator for whether a not-null
constraint exists for the column; whether it's valid or invalid must be
queried in pg_constraint. Applications can continue to query
pg_attribute.attnotnull as before, but now it's possible that NULL rows
are present in the column even when that's set to true.
For backend internal purposes, we cache the nullability status in
CompactAttribute->attnullability that each tuple descriptor carries
(replacing CompactAttribute.attnotnull, which was a mirror of
Form_pg_attribute.attnotnull). During the initial tuple descriptor
creation, based on the pg_attribute scan, we set this to UNRESTRICTED if
pg_attribute.attnotnull is false, or to UNKNOWN if it's true; then we
update the latter to VALID or INVALID depending on the pg_constraint
scan. This flag is also copied when tupledescs are copied.
Comparing tuple descs for equality must also compare the
CompactAttribute.attnullability flag and return false in case of a
mismatch.
pg_dump deals with these constraints by storing the OIDs of invalid
not-null constraints in a separate array, and running a query to obtain
their properties. The regular table creation SQL omits them entirely.
They are then dealt with in the same way as "separate" CHECK
constraints, and dumped after the data has been loaded. Because no
additional pg_dump infrastructure was required, we don't bump its
version number.
I decided not to bump catversion either, because the old catalog state
works perfectly in the new world. (Trying to run with new catalog state
and the old server version would likely run into issues, however.)
System catalogs do not support invalid not-null constraints (because
commit 14e87ffa5c54 didn't allow them to have pg_constraint rows
anyway.)
Author: Rushabh Lathia <rushabh.lathia@gmail.com>
Author: Jian He <jian.universality@gmail.com>
Reviewed-by: Álvaro Herrera <alvherre@alvh.no-ip.org>
Tested-by: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
Discussion: https://postgr.es/m/CAGPqQf0KitkNack4F5CFkFi-9Dqvp29Ro=EpcWt=4_hs-Rt+bQ@mail.gmail.com
9 months ago
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Assign nullability status for this column. Assuming that a constraint
|
|
|
|
|
* exists, at this point we don't know if a not-null constraint is valid,
|
|
|
|
|
* so we assign UNKNOWN unless the table is a catalog, in which case we
|
|
|
|
|
* know it's valid.
|
|
|
|
|
*/
|
|
|
|
|
dst->attnullability = !src->attnotnull ? ATTNULLABLE_UNRESTRICTED :
|
|
|
|
|
IsCatalogRelationOid(src->attrelid) ? ATTNULLABLE_VALID :
|
|
|
|
|
ATTNULLABLE_UNKNOWN;
|
|
|
|
|
|
|
|
|
|
switch (src->attalign)
|
|
|
|
|
{
|
|
|
|
|
case TYPALIGN_INT:
|
|
|
|
|
dst->attalignby = ALIGNOF_INT;
|
|
|
|
|
break;
|
|
|
|
|
case TYPALIGN_CHAR:
|
|
|
|
|
dst->attalignby = sizeof(char);
|
|
|
|
|
break;
|
|
|
|
|
case TYPALIGN_DOUBLE:
|
|
|
|
|
dst->attalignby = ALIGNOF_DOUBLE;
|
|
|
|
|
break;
|
|
|
|
|
case TYPALIGN_SHORT:
|
|
|
|
|
dst->attalignby = ALIGNOF_SHORT;
|
|
|
|
|
break;
|
|
|
|
|
default:
|
|
|
|
|
dst->attalignby = 0;
|
|
|
|
|
elog(ERROR, "invalid attalign value: %c", src->attalign);
|
|
|
|
|
break;
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* populate_compact_attribute
|
|
|
|
|
* Fill in the corresponding CompactAttribute element from the
|
|
|
|
|
* Form_pg_attribute for the given attribute number. This must be called
|
|
|
|
|
* whenever a change is made to a Form_pg_attribute in the TupleDesc.
|
|
|
|
|
*/
|
|
|
|
|
void
|
|
|
|
|
populate_compact_attribute(TupleDesc tupdesc, int attnum)
|
|
|
|
|
{
|
|
|
|
|
Form_pg_attribute src = TupleDescAttr(tupdesc, attnum);
|
|
|
|
|
CompactAttribute *dst;
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Don't use TupleDescCompactAttr to prevent infinite recursion in assert
|
|
|
|
|
* builds.
|
|
|
|
|
*/
|
|
|
|
|
dst = &tupdesc->compact_attrs[attnum];
|
|
|
|
|
|
|
|
|
|
populate_compact_attribute_internal(src, dst);
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* verify_compact_attribute
|
|
|
|
|
* In Assert enabled builds, we verify that the CompactAttribute is
|
|
|
|
|
* populated correctly. This helps find bugs in places such as ALTER
|
|
|
|
|
* TABLE where code makes changes to the FormData_pg_attribute but
|
|
|
|
|
* forgets to call populate_compact_attribute().
|
|
|
|
|
*
|
|
|
|
|
* This is used in TupleDescCompactAttr(), but declared here to allow access
|
|
|
|
|
* to populate_compact_attribute_internal().
|
|
|
|
|
*/
|
|
|
|
|
void
|
|
|
|
|
verify_compact_attribute(TupleDesc tupdesc, int attnum)
|
|
|
|
|
{
|
|
|
|
|
#ifdef USE_ASSERT_CHECKING
|
|
|
|
|
CompactAttribute *cattr = &tupdesc->compact_attrs[attnum];
|
|
|
|
|
Form_pg_attribute attr = TupleDescAttr(tupdesc, attnum);
|
|
|
|
|
CompactAttribute tmp;
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Populate the temporary CompactAttribute from the corresponding
|
|
|
|
|
* Form_pg_attribute
|
|
|
|
|
*/
|
|
|
|
|
populate_compact_attribute_internal(attr, &tmp);
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Make the attcacheoff match since it's been reset to -1 by
|
Allow NOT NULL constraints to be added as NOT VALID
This allows them to be added without scanning the table, and validating
them afterwards without holding access exclusive lock on the table after
any violating rows have been deleted or fixed.
Doing ALTER TABLE ... SET NOT NULL for a column that has an invalid
not-null constraint validates that constraint. ALTER TABLE .. VALIDATE
CONSTRAINT is also supported. There are various checks on whether an
invalid constraint is allowed in a child table when the parent table has
a valid constraint; this should match what we do for enforced/not
enforced constraints.
pg_attribute.attnotnull is now only an indicator for whether a not-null
constraint exists for the column; whether it's valid or invalid must be
queried in pg_constraint. Applications can continue to query
pg_attribute.attnotnull as before, but now it's possible that NULL rows
are present in the column even when that's set to true.
For backend internal purposes, we cache the nullability status in
CompactAttribute->attnullability that each tuple descriptor carries
(replacing CompactAttribute.attnotnull, which was a mirror of
Form_pg_attribute.attnotnull). During the initial tuple descriptor
creation, based on the pg_attribute scan, we set this to UNRESTRICTED if
pg_attribute.attnotnull is false, or to UNKNOWN if it's true; then we
update the latter to VALID or INVALID depending on the pg_constraint
scan. This flag is also copied when tupledescs are copied.
Comparing tuple descs for equality must also compare the
CompactAttribute.attnullability flag and return false in case of a
mismatch.
pg_dump deals with these constraints by storing the OIDs of invalid
not-null constraints in a separate array, and running a query to obtain
their properties. The regular table creation SQL omits them entirely.
They are then dealt with in the same way as "separate" CHECK
constraints, and dumped after the data has been loaded. Because no
additional pg_dump infrastructure was required, we don't bump its
version number.
I decided not to bump catversion either, because the old catalog state
works perfectly in the new world. (Trying to run with new catalog state
and the old server version would likely run into issues, however.)
System catalogs do not support invalid not-null constraints (because
commit 14e87ffa5c54 didn't allow them to have pg_constraint rows
anyway.)
Author: Rushabh Lathia <rushabh.lathia@gmail.com>
Author: Jian He <jian.universality@gmail.com>
Reviewed-by: Álvaro Herrera <alvherre@alvh.no-ip.org>
Tested-by: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
Discussion: https://postgr.es/m/CAGPqQf0KitkNack4F5CFkFi-9Dqvp29Ro=EpcWt=4_hs-Rt+bQ@mail.gmail.com
9 months ago
|
|
|
* populate_compact_attribute_internal. Same with attnullability.
|
|
|
|
|
*/
|
|
|
|
|
tmp.attcacheoff = cattr->attcacheoff;
|
Allow NOT NULL constraints to be added as NOT VALID
This allows them to be added without scanning the table, and validating
them afterwards without holding access exclusive lock on the table after
any violating rows have been deleted or fixed.
Doing ALTER TABLE ... SET NOT NULL for a column that has an invalid
not-null constraint validates that constraint. ALTER TABLE .. VALIDATE
CONSTRAINT is also supported. There are various checks on whether an
invalid constraint is allowed in a child table when the parent table has
a valid constraint; this should match what we do for enforced/not
enforced constraints.
pg_attribute.attnotnull is now only an indicator for whether a not-null
constraint exists for the column; whether it's valid or invalid must be
queried in pg_constraint. Applications can continue to query
pg_attribute.attnotnull as before, but now it's possible that NULL rows
are present in the column even when that's set to true.
For backend internal purposes, we cache the nullability status in
CompactAttribute->attnullability that each tuple descriptor carries
(replacing CompactAttribute.attnotnull, which was a mirror of
Form_pg_attribute.attnotnull). During the initial tuple descriptor
creation, based on the pg_attribute scan, we set this to UNRESTRICTED if
pg_attribute.attnotnull is false, or to UNKNOWN if it's true; then we
update the latter to VALID or INVALID depending on the pg_constraint
scan. This flag is also copied when tupledescs are copied.
Comparing tuple descs for equality must also compare the
CompactAttribute.attnullability flag and return false in case of a
mismatch.
pg_dump deals with these constraints by storing the OIDs of invalid
not-null constraints in a separate array, and running a query to obtain
their properties. The regular table creation SQL omits them entirely.
They are then dealt with in the same way as "separate" CHECK
constraints, and dumped after the data has been loaded. Because no
additional pg_dump infrastructure was required, we don't bump its
version number.
I decided not to bump catversion either, because the old catalog state
works perfectly in the new world. (Trying to run with new catalog state
and the old server version would likely run into issues, however.)
System catalogs do not support invalid not-null constraints (because
commit 14e87ffa5c54 didn't allow them to have pg_constraint rows
anyway.)
Author: Rushabh Lathia <rushabh.lathia@gmail.com>
Author: Jian He <jian.universality@gmail.com>
Reviewed-by: Álvaro Herrera <alvherre@alvh.no-ip.org>
Tested-by: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
Discussion: https://postgr.es/m/CAGPqQf0KitkNack4F5CFkFi-9Dqvp29Ro=EpcWt=4_hs-Rt+bQ@mail.gmail.com
9 months ago
|
|
|
tmp.attnullability = cattr->attnullability;
|
|
|
|
|
|
|
|
|
|
/* Check the freshly populated CompactAttribute matches the TupleDesc's */
|
|
|
|
|
Assert(memcmp(&tmp, cattr, sizeof(CompactAttribute)) == 0);
|
|
|
|
|
#endif
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* CreateTemplateTupleDesc
|
|
|
|
|
* This function allocates an empty tuple descriptor structure.
|
|
|
|
|
*
|
|
|
|
|
* Tuple type ID information is initially set for an anonymous record type;
|
|
|
|
|
* caller can overwrite this if needed.
|
|
|
|
|
*/
|
|
|
|
|
TupleDesc
|
Remove WITH OIDS support, change oid catalog column visibility.
Previously tables declared WITH OIDS, including a significant fraction
of the catalog tables, stored the oid column not as a normal column,
but as part of the tuple header.
This special column was not shown by default, which was somewhat odd,
as it's often (consider e.g. pg_class.oid) one of the more important
parts of a row. Neither pg_dump nor COPY included the contents of the
oid column by default.
The fact that the oid column was not an ordinary column necessitated a
significant amount of special case code to support oid columns. That
already was painful for the existing, but upcoming work aiming to make
table storage pluggable, would have required expanding and duplicating
that "specialness" significantly.
WITH OIDS has been deprecated since 2005 (commit ff02d0a05280e0).
Remove it.
Removing includes:
- CREATE TABLE and ALTER TABLE syntax for declaring the table to be
WITH OIDS has been removed (WITH (oids[ = true]) will error out)
- pg_dump does not support dumping tables declared WITH OIDS and will
issue a warning when dumping one (and ignore the oid column).
- restoring an pg_dump archive with pg_restore will warn when
restoring a table with oid contents (and ignore the oid column)
- COPY will refuse to load binary dump that includes oids.
- pg_upgrade will error out when encountering tables declared WITH
OIDS, they have to be altered to remove the oid column first.
- Functionality to access the oid of the last inserted row (like
plpgsql's RESULT_OID, spi's SPI_lastoid, ...) has been removed.
The syntax for declaring a table WITHOUT OIDS (or WITH (oids = false)
for CREATE TABLE) is still supported. While that requires a bit of
support code, it seems unnecessary to break applications / dumps that
do not use oids, and are explicit about not using them.
The biggest user of WITH OID columns was postgres' catalog. This
commit changes all 'magic' oid columns to be columns that are normally
declared and stored. To reduce unnecessary query breakage all the
newly added columns are still named 'oid', even if a table's column
naming scheme would indicate 'reloid' or such. This obviously
requires adapting a lot code, mostly replacing oid access via
HeapTupleGetOid() with access to the underlying Form_pg_*->oid column.
The bootstrap process now assigns oids for all oid columns in
genbki.pl that do not have an explicit value (starting at the largest
oid previously used), only oids assigned later by oids will be above
FirstBootstrapObjectId. As the oid column now is a normal column the
special bootstrap syntax for oids has been removed.
Oids are not automatically assigned during insertion anymore, all
backend code explicitly assigns oids with GetNewOidWithIndex(). For
the rare case that insertions into the catalog via SQL are called for
the new pg_nextoid() function can be used (which only works on catalog
tables).
The fact that oid columns on system tables are now normal columns
means that they will be included in the set of columns expanded
by * (i.e. SELECT * FROM pg_class will now include the table's oid,
previously it did not). It'd not technically be hard to hide oid
column by default, but that'd mean confusing behavior would either
have to be carried forward forever, or it'd cause breakage down the
line.
While it's not unlikely that further adjustments are needed, the
scope/invasiveness of the patch makes it worthwhile to get merge this
now. It's painful to maintain externally, too complicated to commit
after the code code freeze, and a dependency of a number of other
patches.
Catversion bump, for obvious reasons.
Author: Andres Freund, with contributions by John Naylor
Discussion: https://postgr.es/m/20180930034810.ywp2c7awz7opzcfr@alap3.anarazel.de
7 years ago
|
|
|
CreateTemplateTupleDesc(int natts)
|
|
|
|
|
{
|
|
|
|
|
TupleDesc desc;
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* sanity checks
|
|
|
|
|
*/
|
|
|
|
|
Assert(natts >= 0);
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Allocate enough memory for the tuple descriptor, the CompactAttribute
|
|
|
|
|
* array and also an array of FormData_pg_attribute.
|
|
|
|
|
*
|
|
|
|
|
* Note: the FormData_pg_attribute array stride is
|
|
|
|
|
* sizeof(FormData_pg_attribute), since we declare the array elements as
|
|
|
|
|
* FormData_pg_attribute for notational convenience. However, we only
|
|
|
|
|
* guarantee that the first ATTRIBUTE_FIXED_PART_SIZE bytes of each entry
|
|
|
|
|
* are valid; most code that copies tupdesc entries around copies just
|
|
|
|
|
* that much. In principle that could be less due to trailing padding,
|
|
|
|
|
* although with the current definition of pg_attribute there probably
|
|
|
|
|
* isn't any padding.
|
|
|
|
|
*/
|
|
|
|
|
desc = (TupleDesc) palloc(offsetof(struct TupleDescData, compact_attrs) +
|
|
|
|
|
natts * sizeof(CompactAttribute) +
|
|
|
|
|
natts * sizeof(FormData_pg_attribute));
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Initialize other fields of the tupdesc.
|
|
|
|
|
*/
|
|
|
|
|
desc->natts = natts;
|
|
|
|
|
desc->constr = NULL;
|
|
|
|
|
desc->tdtypeid = RECORDOID;
|
|
|
|
|
desc->tdtypmod = -1;
|
|
|
|
|
desc->tdrefcount = -1; /* assume not reference-counted */
|
|
|
|
|
|
|
|
|
|
return desc;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* CreateTupleDesc
|
|
|
|
|
* This function allocates a new TupleDesc by copying a given
|
|
|
|
|
* Form_pg_attribute array.
|
|
|
|
|
*
|
|
|
|
|
* Tuple type ID information is initially set for an anonymous record type;
|
|
|
|
|
* caller can overwrite this if needed.
|
|
|
|
|
*/
|
|
|
|
|
TupleDesc
|
Remove WITH OIDS support, change oid catalog column visibility.
Previously tables declared WITH OIDS, including a significant fraction
of the catalog tables, stored the oid column not as a normal column,
but as part of the tuple header.
This special column was not shown by default, which was somewhat odd,
as it's often (consider e.g. pg_class.oid) one of the more important
parts of a row. Neither pg_dump nor COPY included the contents of the
oid column by default.
The fact that the oid column was not an ordinary column necessitated a
significant amount of special case code to support oid columns. That
already was painful for the existing, but upcoming work aiming to make
table storage pluggable, would have required expanding and duplicating
that "specialness" significantly.
WITH OIDS has been deprecated since 2005 (commit ff02d0a05280e0).
Remove it.
Removing includes:
- CREATE TABLE and ALTER TABLE syntax for declaring the table to be
WITH OIDS has been removed (WITH (oids[ = true]) will error out)
- pg_dump does not support dumping tables declared WITH OIDS and will
issue a warning when dumping one (and ignore the oid column).
- restoring an pg_dump archive with pg_restore will warn when
restoring a table with oid contents (and ignore the oid column)
- COPY will refuse to load binary dump that includes oids.
- pg_upgrade will error out when encountering tables declared WITH
OIDS, they have to be altered to remove the oid column first.
- Functionality to access the oid of the last inserted row (like
plpgsql's RESULT_OID, spi's SPI_lastoid, ...) has been removed.
The syntax for declaring a table WITHOUT OIDS (or WITH (oids = false)
for CREATE TABLE) is still supported. While that requires a bit of
support code, it seems unnecessary to break applications / dumps that
do not use oids, and are explicit about not using them.
The biggest user of WITH OID columns was postgres' catalog. This
commit changes all 'magic' oid columns to be columns that are normally
declared and stored. To reduce unnecessary query breakage all the
newly added columns are still named 'oid', even if a table's column
naming scheme would indicate 'reloid' or such. This obviously
requires adapting a lot code, mostly replacing oid access via
HeapTupleGetOid() with access to the underlying Form_pg_*->oid column.
The bootstrap process now assigns oids for all oid columns in
genbki.pl that do not have an explicit value (starting at the largest
oid previously used), only oids assigned later by oids will be above
FirstBootstrapObjectId. As the oid column now is a normal column the
special bootstrap syntax for oids has been removed.
Oids are not automatically assigned during insertion anymore, all
backend code explicitly assigns oids with GetNewOidWithIndex(). For
the rare case that insertions into the catalog via SQL are called for
the new pg_nextoid() function can be used (which only works on catalog
tables).
The fact that oid columns on system tables are now normal columns
means that they will be included in the set of columns expanded
by * (i.e. SELECT * FROM pg_class will now include the table's oid,
previously it did not). It'd not technically be hard to hide oid
column by default, but that'd mean confusing behavior would either
have to be carried forward forever, or it'd cause breakage down the
line.
While it's not unlikely that further adjustments are needed, the
scope/invasiveness of the patch makes it worthwhile to get merge this
now. It's painful to maintain externally, too complicated to commit
after the code code freeze, and a dependency of a number of other
patches.
Catversion bump, for obvious reasons.
Author: Andres Freund, with contributions by John Naylor
Discussion: https://postgr.es/m/20180930034810.ywp2c7awz7opzcfr@alap3.anarazel.de
7 years ago
|
|
|
CreateTupleDesc(int natts, Form_pg_attribute *attrs)
|
|
|
|
|
{
|
|
|
|
|
TupleDesc desc;
|
|
|
|
|
int i;
|
|
|
|
|
|
Remove WITH OIDS support, change oid catalog column visibility.
Previously tables declared WITH OIDS, including a significant fraction
of the catalog tables, stored the oid column not as a normal column,
but as part of the tuple header.
This special column was not shown by default, which was somewhat odd,
as it's often (consider e.g. pg_class.oid) one of the more important
parts of a row. Neither pg_dump nor COPY included the contents of the
oid column by default.
The fact that the oid column was not an ordinary column necessitated a
significant amount of special case code to support oid columns. That
already was painful for the existing, but upcoming work aiming to make
table storage pluggable, would have required expanding and duplicating
that "specialness" significantly.
WITH OIDS has been deprecated since 2005 (commit ff02d0a05280e0).
Remove it.
Removing includes:
- CREATE TABLE and ALTER TABLE syntax for declaring the table to be
WITH OIDS has been removed (WITH (oids[ = true]) will error out)
- pg_dump does not support dumping tables declared WITH OIDS and will
issue a warning when dumping one (and ignore the oid column).
- restoring an pg_dump archive with pg_restore will warn when
restoring a table with oid contents (and ignore the oid column)
- COPY will refuse to load binary dump that includes oids.
- pg_upgrade will error out when encountering tables declared WITH
OIDS, they have to be altered to remove the oid column first.
- Functionality to access the oid of the last inserted row (like
plpgsql's RESULT_OID, spi's SPI_lastoid, ...) has been removed.
The syntax for declaring a table WITHOUT OIDS (or WITH (oids = false)
for CREATE TABLE) is still supported. While that requires a bit of
support code, it seems unnecessary to break applications / dumps that
do not use oids, and are explicit about not using them.
The biggest user of WITH OID columns was postgres' catalog. This
commit changes all 'magic' oid columns to be columns that are normally
declared and stored. To reduce unnecessary query breakage all the
newly added columns are still named 'oid', even if a table's column
naming scheme would indicate 'reloid' or such. This obviously
requires adapting a lot code, mostly replacing oid access via
HeapTupleGetOid() with access to the underlying Form_pg_*->oid column.
The bootstrap process now assigns oids for all oid columns in
genbki.pl that do not have an explicit value (starting at the largest
oid previously used), only oids assigned later by oids will be above
FirstBootstrapObjectId. As the oid column now is a normal column the
special bootstrap syntax for oids has been removed.
Oids are not automatically assigned during insertion anymore, all
backend code explicitly assigns oids with GetNewOidWithIndex(). For
the rare case that insertions into the catalog via SQL are called for
the new pg_nextoid() function can be used (which only works on catalog
tables).
The fact that oid columns on system tables are now normal columns
means that they will be included in the set of columns expanded
by * (i.e. SELECT * FROM pg_class will now include the table's oid,
previously it did not). It'd not technically be hard to hide oid
column by default, but that'd mean confusing behavior would either
have to be carried forward forever, or it'd cause breakage down the
line.
While it's not unlikely that further adjustments are needed, the
scope/invasiveness of the patch makes it worthwhile to get merge this
now. It's painful to maintain externally, too complicated to commit
after the code code freeze, and a dependency of a number of other
patches.
Catversion bump, for obvious reasons.
Author: Andres Freund, with contributions by John Naylor
Discussion: https://postgr.es/m/20180930034810.ywp2c7awz7opzcfr@alap3.anarazel.de
7 years ago
|
|
|
desc = CreateTemplateTupleDesc(natts);
|
|
|
|
|
|
|
|
|
|
for (i = 0; i < natts; ++i)
|
|
|
|
|
{
|
|
|
|
|
memcpy(TupleDescAttr(desc, i), attrs[i], ATTRIBUTE_FIXED_PART_SIZE);
|
|
|
|
|
populate_compact_attribute(desc, i);
|
|
|
|
|
}
|
|
|
|
|
return desc;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* CreateTupleDescCopy
|
|
|
|
|
* This function creates a new TupleDesc by copying from an existing
|
|
|
|
|
* TupleDesc.
|
|
|
|
|
*
|
|
|
|
|
* !!! Constraints and defaults are not copied !!!
|
|
|
|
|
*/
|
|
|
|
|
TupleDesc
|
|
|
|
|
CreateTupleDescCopy(TupleDesc tupdesc)
|
|
|
|
|
{
|
|
|
|
|
TupleDesc desc;
|
|
|
|
|
int i;
|
|
|
|
|
|
Remove WITH OIDS support, change oid catalog column visibility.
Previously tables declared WITH OIDS, including a significant fraction
of the catalog tables, stored the oid column not as a normal column,
but as part of the tuple header.
This special column was not shown by default, which was somewhat odd,
as it's often (consider e.g. pg_class.oid) one of the more important
parts of a row. Neither pg_dump nor COPY included the contents of the
oid column by default.
The fact that the oid column was not an ordinary column necessitated a
significant amount of special case code to support oid columns. That
already was painful for the existing, but upcoming work aiming to make
table storage pluggable, would have required expanding and duplicating
that "specialness" significantly.
WITH OIDS has been deprecated since 2005 (commit ff02d0a05280e0).
Remove it.
Removing includes:
- CREATE TABLE and ALTER TABLE syntax for declaring the table to be
WITH OIDS has been removed (WITH (oids[ = true]) will error out)
- pg_dump does not support dumping tables declared WITH OIDS and will
issue a warning when dumping one (and ignore the oid column).
- restoring an pg_dump archive with pg_restore will warn when
restoring a table with oid contents (and ignore the oid column)
- COPY will refuse to load binary dump that includes oids.
- pg_upgrade will error out when encountering tables declared WITH
OIDS, they have to be altered to remove the oid column first.
- Functionality to access the oid of the last inserted row (like
plpgsql's RESULT_OID, spi's SPI_lastoid, ...) has been removed.
The syntax for declaring a table WITHOUT OIDS (or WITH (oids = false)
for CREATE TABLE) is still supported. While that requires a bit of
support code, it seems unnecessary to break applications / dumps that
do not use oids, and are explicit about not using them.
The biggest user of WITH OID columns was postgres' catalog. This
commit changes all 'magic' oid columns to be columns that are normally
declared and stored. To reduce unnecessary query breakage all the
newly added columns are still named 'oid', even if a table's column
naming scheme would indicate 'reloid' or such. This obviously
requires adapting a lot code, mostly replacing oid access via
HeapTupleGetOid() with access to the underlying Form_pg_*->oid column.
The bootstrap process now assigns oids for all oid columns in
genbki.pl that do not have an explicit value (starting at the largest
oid previously used), only oids assigned later by oids will be above
FirstBootstrapObjectId. As the oid column now is a normal column the
special bootstrap syntax for oids has been removed.
Oids are not automatically assigned during insertion anymore, all
backend code explicitly assigns oids with GetNewOidWithIndex(). For
the rare case that insertions into the catalog via SQL are called for
the new pg_nextoid() function can be used (which only works on catalog
tables).
The fact that oid columns on system tables are now normal columns
means that they will be included in the set of columns expanded
by * (i.e. SELECT * FROM pg_class will now include the table's oid,
previously it did not). It'd not technically be hard to hide oid
column by default, but that'd mean confusing behavior would either
have to be carried forward forever, or it'd cause breakage down the
line.
While it's not unlikely that further adjustments are needed, the
scope/invasiveness of the patch makes it worthwhile to get merge this
now. It's painful to maintain externally, too complicated to commit
after the code code freeze, and a dependency of a number of other
patches.
Catversion bump, for obvious reasons.
Author: Andres Freund, with contributions by John Naylor
Discussion: https://postgr.es/m/20180930034810.ywp2c7awz7opzcfr@alap3.anarazel.de
7 years ago
|
|
|
desc = CreateTemplateTupleDesc(tupdesc->natts);
|
|
|
|
|
|
|
|
|
|
/* Flat-copy the attribute array */
|
|
|
|
|
memcpy(TupleDescAttr(desc, 0),
|
|
|
|
|
TupleDescAttr(tupdesc, 0),
|
|
|
|
|
desc->natts * sizeof(FormData_pg_attribute));
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Since we're not copying constraints and defaults, clear fields
|
|
|
|
|
* associated with them.
|
|
|
|
|
*/
|
|
|
|
|
for (i = 0; i < desc->natts; i++)
|
|
|
|
|
{
|
|
|
|
|
Form_pg_attribute att = TupleDescAttr(desc, i);
|
|
|
|
|
|
|
|
|
|
att->attnotnull = false;
|
|
|
|
|
att->atthasdef = false;
|
|
|
|
|
att->atthasmissing = false;
|
|
|
|
|
att->attidentity = '\0';
|
|
|
|
|
att->attgenerated = '\0';
|
|
|
|
|
|
|
|
|
|
populate_compact_attribute(desc, i);
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/* We can copy the tuple type identification, too */
|
|
|
|
|
desc->tdtypeid = tupdesc->tdtypeid;
|
|
|
|
|
desc->tdtypmod = tupdesc->tdtypmod;
|
|
|
|
|
|
|
|
|
|
return desc;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* CreateTupleDescTruncatedCopy
|
|
|
|
|
* This function creates a new TupleDesc with only the first 'natts'
|
|
|
|
|
* attributes from an existing TupleDesc
|
|
|
|
|
*
|
|
|
|
|
* !!! Constraints and defaults are not copied !!!
|
|
|
|
|
*/
|
|
|
|
|
TupleDesc
|
|
|
|
|
CreateTupleDescTruncatedCopy(TupleDesc tupdesc, int natts)
|
|
|
|
|
{
|
|
|
|
|
TupleDesc desc;
|
|
|
|
|
int i;
|
|
|
|
|
|
|
|
|
|
Assert(natts <= tupdesc->natts);
|
|
|
|
|
|
|
|
|
|
desc = CreateTemplateTupleDesc(natts);
|
|
|
|
|
|
|
|
|
|
/* Flat-copy the attribute array */
|
|
|
|
|
memcpy(TupleDescAttr(desc, 0),
|
|
|
|
|
TupleDescAttr(tupdesc, 0),
|
|
|
|
|
desc->natts * sizeof(FormData_pg_attribute));
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Since we're not copying constraints and defaults, clear fields
|
|
|
|
|
* associated with them.
|
|
|
|
|
*/
|
|
|
|
|
for (i = 0; i < desc->natts; i++)
|
|
|
|
|
{
|
|
|
|
|
Form_pg_attribute att = TupleDescAttr(desc, i);
|
|
|
|
|
|
|
|
|
|
att->attnotnull = false;
|
|
|
|
|
att->atthasdef = false;
|
|
|
|
|
att->atthasmissing = false;
|
|
|
|
|
att->attidentity = '\0';
|
|
|
|
|
att->attgenerated = '\0';
|
|
|
|
|
|
|
|
|
|
populate_compact_attribute(desc, i);
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/* We can copy the tuple type identification, too */
|
|
|
|
|
desc->tdtypeid = tupdesc->tdtypeid;
|
|
|
|
|
desc->tdtypmod = tupdesc->tdtypmod;
|
|
|
|
|
|
|
|
|
|
return desc;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* CreateTupleDescCopyConstr
|
|
|
|
|
* This function creates a new TupleDesc by copying from an existing
|
|
|
|
|
* TupleDesc (including its constraints and defaults).
|
|
|
|
|
*/
|
|
|
|
|
TupleDesc
|
|
|
|
|
CreateTupleDescCopyConstr(TupleDesc tupdesc)
|
|
|
|
|
{
|
|
|
|
|
TupleDesc desc;
|
|
|
|
|
TupleConstr *constr = tupdesc->constr;
|
|
|
|
|
int i;
|
|
|
|
|
|
Remove WITH OIDS support, change oid catalog column visibility.
Previously tables declared WITH OIDS, including a significant fraction
of the catalog tables, stored the oid column not as a normal column,
but as part of the tuple header.
This special column was not shown by default, which was somewhat odd,
as it's often (consider e.g. pg_class.oid) one of the more important
parts of a row. Neither pg_dump nor COPY included the contents of the
oid column by default.
The fact that the oid column was not an ordinary column necessitated a
significant amount of special case code to support oid columns. That
already was painful for the existing, but upcoming work aiming to make
table storage pluggable, would have required expanding and duplicating
that "specialness" significantly.
WITH OIDS has been deprecated since 2005 (commit ff02d0a05280e0).
Remove it.
Removing includes:
- CREATE TABLE and ALTER TABLE syntax for declaring the table to be
WITH OIDS has been removed (WITH (oids[ = true]) will error out)
- pg_dump does not support dumping tables declared WITH OIDS and will
issue a warning when dumping one (and ignore the oid column).
- restoring an pg_dump archive with pg_restore will warn when
restoring a table with oid contents (and ignore the oid column)
- COPY will refuse to load binary dump that includes oids.
- pg_upgrade will error out when encountering tables declared WITH
OIDS, they have to be altered to remove the oid column first.
- Functionality to access the oid of the last inserted row (like
plpgsql's RESULT_OID, spi's SPI_lastoid, ...) has been removed.
The syntax for declaring a table WITHOUT OIDS (or WITH (oids = false)
for CREATE TABLE) is still supported. While that requires a bit of
support code, it seems unnecessary to break applications / dumps that
do not use oids, and are explicit about not using them.
The biggest user of WITH OID columns was postgres' catalog. This
commit changes all 'magic' oid columns to be columns that are normally
declared and stored. To reduce unnecessary query breakage all the
newly added columns are still named 'oid', even if a table's column
naming scheme would indicate 'reloid' or such. This obviously
requires adapting a lot code, mostly replacing oid access via
HeapTupleGetOid() with access to the underlying Form_pg_*->oid column.
The bootstrap process now assigns oids for all oid columns in
genbki.pl that do not have an explicit value (starting at the largest
oid previously used), only oids assigned later by oids will be above
FirstBootstrapObjectId. As the oid column now is a normal column the
special bootstrap syntax for oids has been removed.
Oids are not automatically assigned during insertion anymore, all
backend code explicitly assigns oids with GetNewOidWithIndex(). For
the rare case that insertions into the catalog via SQL are called for
the new pg_nextoid() function can be used (which only works on catalog
tables).
The fact that oid columns on system tables are now normal columns
means that they will be included in the set of columns expanded
by * (i.e. SELECT * FROM pg_class will now include the table's oid,
previously it did not). It'd not technically be hard to hide oid
column by default, but that'd mean confusing behavior would either
have to be carried forward forever, or it'd cause breakage down the
line.
While it's not unlikely that further adjustments are needed, the
scope/invasiveness of the patch makes it worthwhile to get merge this
now. It's painful to maintain externally, too complicated to commit
after the code code freeze, and a dependency of a number of other
patches.
Catversion bump, for obvious reasons.
Author: Andres Freund, with contributions by John Naylor
Discussion: https://postgr.es/m/20180930034810.ywp2c7awz7opzcfr@alap3.anarazel.de
7 years ago
|
|
|
desc = CreateTemplateTupleDesc(tupdesc->natts);
|
|
|
|
|
|
|
|
|
|
/* Flat-copy the attribute array */
|
|
|
|
|
memcpy(TupleDescAttr(desc, 0),
|
|
|
|
|
TupleDescAttr(tupdesc, 0),
|
|
|
|
|
desc->natts * sizeof(FormData_pg_attribute));
|
|
|
|
|
|
|
|
|
|
for (i = 0; i < desc->natts; i++)
|
Allow NOT NULL constraints to be added as NOT VALID
This allows them to be added without scanning the table, and validating
them afterwards without holding access exclusive lock on the table after
any violating rows have been deleted or fixed.
Doing ALTER TABLE ... SET NOT NULL for a column that has an invalid
not-null constraint validates that constraint. ALTER TABLE .. VALIDATE
CONSTRAINT is also supported. There are various checks on whether an
invalid constraint is allowed in a child table when the parent table has
a valid constraint; this should match what we do for enforced/not
enforced constraints.
pg_attribute.attnotnull is now only an indicator for whether a not-null
constraint exists for the column; whether it's valid or invalid must be
queried in pg_constraint. Applications can continue to query
pg_attribute.attnotnull as before, but now it's possible that NULL rows
are present in the column even when that's set to true.
For backend internal purposes, we cache the nullability status in
CompactAttribute->attnullability that each tuple descriptor carries
(replacing CompactAttribute.attnotnull, which was a mirror of
Form_pg_attribute.attnotnull). During the initial tuple descriptor
creation, based on the pg_attribute scan, we set this to UNRESTRICTED if
pg_attribute.attnotnull is false, or to UNKNOWN if it's true; then we
update the latter to VALID or INVALID depending on the pg_constraint
scan. This flag is also copied when tupledescs are copied.
Comparing tuple descs for equality must also compare the
CompactAttribute.attnullability flag and return false in case of a
mismatch.
pg_dump deals with these constraints by storing the OIDs of invalid
not-null constraints in a separate array, and running a query to obtain
their properties. The regular table creation SQL omits them entirely.
They are then dealt with in the same way as "separate" CHECK
constraints, and dumped after the data has been loaded. Because no
additional pg_dump infrastructure was required, we don't bump its
version number.
I decided not to bump catversion either, because the old catalog state
works perfectly in the new world. (Trying to run with new catalog state
and the old server version would likely run into issues, however.)
System catalogs do not support invalid not-null constraints (because
commit 14e87ffa5c54 didn't allow them to have pg_constraint rows
anyway.)
Author: Rushabh Lathia <rushabh.lathia@gmail.com>
Author: Jian He <jian.universality@gmail.com>
Reviewed-by: Álvaro Herrera <alvherre@alvh.no-ip.org>
Tested-by: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
Discussion: https://postgr.es/m/CAGPqQf0KitkNack4F5CFkFi-9Dqvp29Ro=EpcWt=4_hs-Rt+bQ@mail.gmail.com
9 months ago
|
|
|
{
|
|
|
|
|
populate_compact_attribute(desc, i);
|
|
|
|
|
|
Allow NOT NULL constraints to be added as NOT VALID
This allows them to be added without scanning the table, and validating
them afterwards without holding access exclusive lock on the table after
any violating rows have been deleted or fixed.
Doing ALTER TABLE ... SET NOT NULL for a column that has an invalid
not-null constraint validates that constraint. ALTER TABLE .. VALIDATE
CONSTRAINT is also supported. There are various checks on whether an
invalid constraint is allowed in a child table when the parent table has
a valid constraint; this should match what we do for enforced/not
enforced constraints.
pg_attribute.attnotnull is now only an indicator for whether a not-null
constraint exists for the column; whether it's valid or invalid must be
queried in pg_constraint. Applications can continue to query
pg_attribute.attnotnull as before, but now it's possible that NULL rows
are present in the column even when that's set to true.
For backend internal purposes, we cache the nullability status in
CompactAttribute->attnullability that each tuple descriptor carries
(replacing CompactAttribute.attnotnull, which was a mirror of
Form_pg_attribute.attnotnull). During the initial tuple descriptor
creation, based on the pg_attribute scan, we set this to UNRESTRICTED if
pg_attribute.attnotnull is false, or to UNKNOWN if it's true; then we
update the latter to VALID or INVALID depending on the pg_constraint
scan. This flag is also copied when tupledescs are copied.
Comparing tuple descs for equality must also compare the
CompactAttribute.attnullability flag and return false in case of a
mismatch.
pg_dump deals with these constraints by storing the OIDs of invalid
not-null constraints in a separate array, and running a query to obtain
their properties. The regular table creation SQL omits them entirely.
They are then dealt with in the same way as "separate" CHECK
constraints, and dumped after the data has been loaded. Because no
additional pg_dump infrastructure was required, we don't bump its
version number.
I decided not to bump catversion either, because the old catalog state
works perfectly in the new world. (Trying to run with new catalog state
and the old server version would likely run into issues, however.)
System catalogs do not support invalid not-null constraints (because
commit 14e87ffa5c54 didn't allow them to have pg_constraint rows
anyway.)
Author: Rushabh Lathia <rushabh.lathia@gmail.com>
Author: Jian He <jian.universality@gmail.com>
Reviewed-by: Álvaro Herrera <alvherre@alvh.no-ip.org>
Tested-by: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
Discussion: https://postgr.es/m/CAGPqQf0KitkNack4F5CFkFi-9Dqvp29Ro=EpcWt=4_hs-Rt+bQ@mail.gmail.com
9 months ago
|
|
|
TupleDescCompactAttr(desc, i)->attnullability =
|
|
|
|
|
TupleDescCompactAttr(tupdesc, i)->attnullability;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/* Copy the TupleConstr data structure, if any */
|
|
|
|
|
if (constr)
|
|
|
|
|
{
|
|
|
|
|
TupleConstr *cpy = (TupleConstr *) palloc0(sizeof(TupleConstr));
|
|
|
|
|
|
|
|
|
|
cpy->has_not_null = constr->has_not_null;
|
|
|
|
|
cpy->has_generated_stored = constr->has_generated_stored;
|
Virtual generated columns
This adds a new variant of generated columns that are computed on read
(like a view, unlike the existing stored generated columns, which are
computed on write, like a materialized view).
The syntax for the column definition is
... GENERATED ALWAYS AS (...) VIRTUAL
and VIRTUAL is also optional. VIRTUAL is the default rather than
STORED to match various other SQL products. (The SQL standard makes
no specification about this, but it also doesn't know about VIRTUAL or
STORED.) (Also, virtual views are the default, rather than
materialized views.)
Virtual generated columns are stored in tuples as null values. (A
very early version of this patch had the ambition to not store them at
all. But so much stuff breaks or gets confused if you have tuples
where a column in the middle is completely missing. This is a
compromise, and it still saves space over being forced to use stored
generated columns. If we ever find a way to improve this, a bit of
pg_upgrade cleverness could allow for upgrades to a newer scheme.)
The capabilities and restrictions of virtual generated columns are
mostly the same as for stored generated columns. In some cases, this
patch keeps virtual generated columns more restricted than they might
technically need to be, to keep the two kinds consistent. Some of
that could maybe be relaxed later after separate careful
considerations.
Some functionality that is currently not supported, but could possibly
be added as incremental features, some easier than others:
- index on or using a virtual column
- hence also no unique constraints on virtual columns
- extended statistics on virtual columns
- foreign-key constraints on virtual columns
- not-null constraints on virtual columns (check constraints are supported)
- ALTER TABLE / DROP EXPRESSION
- virtual column cannot have domain type
- virtual columns are not supported in logical replication
The tests in generated_virtual.sql have been copied over from
generated_stored.sql with the keyword replaced. This way we can make
sure the behavior is mostly aligned, and the differences can be
visible. Some tests for currently not supported features are
currently commented out.
Reviewed-by: Jian He <jian.universality@gmail.com>
Reviewed-by: Dean Rasheed <dean.a.rasheed@gmail.com>
Tested-by: Shlok Kyal <shlok.kyal.oss@gmail.com>
Discussion: https://www.postgresql.org/message-id/flat/a368248e-69e4-40be-9c07-6c3b5880b0a6@eisentraut.org
10 months ago
|
|
|
cpy->has_generated_virtual = constr->has_generated_virtual;
|
|
|
|
|
|
|
|
|
|
if ((cpy->num_defval = constr->num_defval) > 0)
|
|
|
|
|
{
|
|
|
|
|
cpy->defval = (AttrDefault *) palloc(cpy->num_defval * sizeof(AttrDefault));
|
|
|
|
|
memcpy(cpy->defval, constr->defval, cpy->num_defval * sizeof(AttrDefault));
|
|
|
|
|
for (i = cpy->num_defval - 1; i >= 0; i--)
|
Clean up treatment of missing default and CHECK-constraint records.
Andrew Gierth reported that it's possible to crash the backend if no
pg_attrdef record is found to match an attribute that has atthasdef set.
AttrDefaultFetch warns about this situation, but then leaves behind
a relation tupdesc that has null "adbin" pointer(s), which most places
don't guard against.
We considered promoting the warning to an error, but throwing errors
during relcache load is pretty drastic: it effectively locks one out
of using the relation at all. What seems better is to leave the
load-time behavior as a warning, but then throw an error in any code
path that wants to use a default and can't find it. This confines
the error to a subset of INSERT/UPDATE operations on the table, and
in particular will at least allow a pg_dump to succeed.
Also, we should fix AttrDefaultFetch to not leave any null pointers
in the tupdesc, because that just creates an untested bug hazard.
While at it, apply the same philosophy of "warn at load, throw error
only upon use of the known-missing info" to CHECK constraints.
CheckConstraintFetch is very nearly the same logic as AttrDefaultFetch,
but for reasons lost in the mists of time, it was throwing ERROR for
the same cases that AttrDefaultFetch treats as WARNING. Make the two
functions more nearly alike.
In passing, get rid of potentially-O(N^2) loops in equalTupleDesc
by making AttrDefaultFetch sort the entries after fetching them,
so that equalTupleDesc can assume that entries in two equal tupdescs
must be in matching order. (CheckConstraintFetch already was sorting
CHECK constraints, but equalTupleDesc hadn't been told about it.)
There's some argument for back-patching this, but with such a small
number of field reports, I'm content to fix it in HEAD.
Discussion: https://postgr.es/m/87pmzaq4gx.fsf@news-spur.riddles.org.uk
5 years ago
|
|
|
cpy->defval[i].adbin = pstrdup(constr->defval[i].adbin);
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
if (constr->missing)
|
|
|
|
|
{
|
|
|
|
|
cpy->missing = (AttrMissing *) palloc(tupdesc->natts * sizeof(AttrMissing));
|
|
|
|
|
memcpy(cpy->missing, constr->missing, tupdesc->natts * sizeof(AttrMissing));
|
|
|
|
|
for (i = tupdesc->natts - 1; i >= 0; i--)
|
|
|
|
|
{
|
|
|
|
|
if (constr->missing[i].am_present)
|
|
|
|
|
{
|
|
|
|
|
CompactAttribute *attr = TupleDescCompactAttr(tupdesc, i);
|
|
|
|
|
|
|
|
|
|
cpy->missing[i].am_value = datumCopy(constr->missing[i].am_value,
|
|
|
|
|
attr->attbyval,
|
|
|
|
|
attr->attlen);
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
if ((cpy->num_check = constr->num_check) > 0)
|
|
|
|
|
{
|
|
|
|
|
cpy->check = (ConstrCheck *) palloc(cpy->num_check * sizeof(ConstrCheck));
|
|
|
|
|
memcpy(cpy->check, constr->check, cpy->num_check * sizeof(ConstrCheck));
|
|
|
|
|
for (i = cpy->num_check - 1; i >= 0; i--)
|
|
|
|
|
{
|
Clean up treatment of missing default and CHECK-constraint records.
Andrew Gierth reported that it's possible to crash the backend if no
pg_attrdef record is found to match an attribute that has atthasdef set.
AttrDefaultFetch warns about this situation, but then leaves behind
a relation tupdesc that has null "adbin" pointer(s), which most places
don't guard against.
We considered promoting the warning to an error, but throwing errors
during relcache load is pretty drastic: it effectively locks one out
of using the relation at all. What seems better is to leave the
load-time behavior as a warning, but then throw an error in any code
path that wants to use a default and can't find it. This confines
the error to a subset of INSERT/UPDATE operations on the table, and
in particular will at least allow a pg_dump to succeed.
Also, we should fix AttrDefaultFetch to not leave any null pointers
in the tupdesc, because that just creates an untested bug hazard.
While at it, apply the same philosophy of "warn at load, throw error
only upon use of the known-missing info" to CHECK constraints.
CheckConstraintFetch is very nearly the same logic as AttrDefaultFetch,
but for reasons lost in the mists of time, it was throwing ERROR for
the same cases that AttrDefaultFetch treats as WARNING. Make the two
functions more nearly alike.
In passing, get rid of potentially-O(N^2) loops in equalTupleDesc
by making AttrDefaultFetch sort the entries after fetching them,
so that equalTupleDesc can assume that entries in two equal tupdescs
must be in matching order. (CheckConstraintFetch already was sorting
CHECK constraints, but equalTupleDesc hadn't been told about it.)
There's some argument for back-patching this, but with such a small
number of field reports, I'm content to fix it in HEAD.
Discussion: https://postgr.es/m/87pmzaq4gx.fsf@news-spur.riddles.org.uk
5 years ago
|
|
|
cpy->check[i].ccname = pstrdup(constr->check[i].ccname);
|
|
|
|
|
cpy->check[i].ccbin = pstrdup(constr->check[i].ccbin);
|
|
|
|
|
cpy->check[i].ccenforced = constr->check[i].ccenforced;
|
|
|
|
|
cpy->check[i].ccvalid = constr->check[i].ccvalid;
|
|
|
|
|
cpy->check[i].ccnoinherit = constr->check[i].ccnoinherit;
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
desc->constr = cpy;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/* We can copy the tuple type identification, too */
|
|
|
|
|
desc->tdtypeid = tupdesc->tdtypeid;
|
|
|
|
|
desc->tdtypmod = tupdesc->tdtypmod;
|
|
|
|
|
|
|
|
|
|
return desc;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* TupleDescCopy
|
|
|
|
|
* Copy a tuple descriptor into caller-supplied memory.
|
|
|
|
|
* The memory may be shared memory mapped at any address, and must
|
|
|
|
|
* be sufficient to hold TupleDescSize(src) bytes.
|
|
|
|
|
*
|
|
|
|
|
* !!! Constraints and defaults are not copied !!!
|
|
|
|
|
*/
|
|
|
|
|
void
|
|
|
|
|
TupleDescCopy(TupleDesc dst, TupleDesc src)
|
|
|
|
|
{
|
|
|
|
|
int i;
|
|
|
|
|
|
|
|
|
|
/* Flat-copy the header and attribute arrays */
|
|
|
|
|
memcpy(dst, src, TupleDescSize(src));
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Since we're not copying constraints and defaults, clear fields
|
|
|
|
|
* associated with them.
|
|
|
|
|
*/
|
|
|
|
|
for (i = 0; i < dst->natts; i++)
|
|
|
|
|
{
|
|
|
|
|
Form_pg_attribute att = TupleDescAttr(dst, i);
|
|
|
|
|
|
|
|
|
|
att->attnotnull = false;
|
|
|
|
|
att->atthasdef = false;
|
|
|
|
|
att->atthasmissing = false;
|
|
|
|
|
att->attidentity = '\0';
|
|
|
|
|
att->attgenerated = '\0';
|
|
|
|
|
|
|
|
|
|
populate_compact_attribute(dst, i);
|
|
|
|
|
}
|
|
|
|
|
dst->constr = NULL;
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Also, assume the destination is not to be ref-counted. (Copying the
|
|
|
|
|
* source's refcount would be wrong in any case.)
|
|
|
|
|
*/
|
|
|
|
|
dst->tdrefcount = -1;
|
|
|
|
|
}
|
|
|
|
|
|
Support multi-argument UNNEST(), and TABLE() syntax for multiple functions.
This patch adds the ability to write TABLE( function1(), function2(), ...)
as a single FROM-clause entry. The result is the concatenation of the
first row from each function, followed by the second row from each
function, etc; with NULLs inserted if any function produces fewer rows than
others. This is believed to be a much more useful behavior than what
Postgres currently does with multiple SRFs in a SELECT list.
This syntax also provides a reasonable way to combine use of column
definition lists with WITH ORDINALITY: put the column definition list
inside TABLE(), where it's clear that it doesn't control the ordinality
column as well.
Also implement SQL-compliant multiple-argument UNNEST(), by turning
UNNEST(a,b,c) into TABLE(unnest(a), unnest(b), unnest(c)).
The SQL standard specifies TABLE() with only a single function, not
multiple functions, and it seems to require an implicit UNNEST() which is
not what this patch does. There may be something wrong with that reading
of the spec, though, because if it's right then the spec's TABLE() is just
a pointless alternative spelling of UNNEST(). After further review of
that, we might choose to adopt a different syntax for what this patch does,
but in any case this functionality seems clearly worthwhile.
Andrew Gierth, reviewed by Zoltán Böszörményi and Heikki Linnakangas, and
significantly revised by me
12 years ago
|
|
|
/*
|
|
|
|
|
* TupleDescCopyEntry
|
|
|
|
|
* This function copies a single attribute structure from one tuple
|
|
|
|
|
* descriptor to another.
|
|
|
|
|
*
|
|
|
|
|
* !!! Constraints and defaults are not copied !!!
|
|
|
|
|
*/
|
|
|
|
|
void
|
|
|
|
|
TupleDescCopyEntry(TupleDesc dst, AttrNumber dstAttno,
|
|
|
|
|
TupleDesc src, AttrNumber srcAttno)
|
|
|
|
|
{
|
|
|
|
|
Form_pg_attribute dstAtt = TupleDescAttr(dst, dstAttno - 1);
|
|
|
|
|
Form_pg_attribute srcAtt = TupleDescAttr(src, srcAttno - 1);
|
|
|
|
|
|
Support multi-argument UNNEST(), and TABLE() syntax for multiple functions.
This patch adds the ability to write TABLE( function1(), function2(), ...)
as a single FROM-clause entry. The result is the concatenation of the
first row from each function, followed by the second row from each
function, etc; with NULLs inserted if any function produces fewer rows than
others. This is believed to be a much more useful behavior than what
Postgres currently does with multiple SRFs in a SELECT list.
This syntax also provides a reasonable way to combine use of column
definition lists with WITH ORDINALITY: put the column definition list
inside TABLE(), where it's clear that it doesn't control the ordinality
column as well.
Also implement SQL-compliant multiple-argument UNNEST(), by turning
UNNEST(a,b,c) into TABLE(unnest(a), unnest(b), unnest(c)).
The SQL standard specifies TABLE() with only a single function, not
multiple functions, and it seems to require an implicit UNNEST() which is
not what this patch does. There may be something wrong with that reading
of the spec, though, because if it's right then the spec's TABLE() is just
a pointless alternative spelling of UNNEST(). After further review of
that, we might choose to adopt a different syntax for what this patch does,
but in any case this functionality seems clearly worthwhile.
Andrew Gierth, reviewed by Zoltán Böszörményi and Heikki Linnakangas, and
significantly revised by me
12 years ago
|
|
|
/*
|
|
|
|
|
* sanity checks
|
|
|
|
|
*/
|
|
|
|
|
Assert(PointerIsValid(src));
|
|
|
|
|
Assert(PointerIsValid(dst));
|
|
|
|
|
Assert(srcAttno >= 1);
|
|
|
|
|
Assert(srcAttno <= src->natts);
|
|
|
|
|
Assert(dstAttno >= 1);
|
|
|
|
|
Assert(dstAttno <= dst->natts);
|
Support multi-argument UNNEST(), and TABLE() syntax for multiple functions.
This patch adds the ability to write TABLE( function1(), function2(), ...)
as a single FROM-clause entry. The result is the concatenation of the
first row from each function, followed by the second row from each
function, etc; with NULLs inserted if any function produces fewer rows than
others. This is believed to be a much more useful behavior than what
Postgres currently does with multiple SRFs in a SELECT list.
This syntax also provides a reasonable way to combine use of column
definition lists with WITH ORDINALITY: put the column definition list
inside TABLE(), where it's clear that it doesn't control the ordinality
column as well.
Also implement SQL-compliant multiple-argument UNNEST(), by turning
UNNEST(a,b,c) into TABLE(unnest(a), unnest(b), unnest(c)).
The SQL standard specifies TABLE() with only a single function, not
multiple functions, and it seems to require an implicit UNNEST() which is
not what this patch does. There may be something wrong with that reading
of the spec, though, because if it's right then the spec's TABLE() is just
a pointless alternative spelling of UNNEST(). After further review of
that, we might choose to adopt a different syntax for what this patch does,
but in any case this functionality seems clearly worthwhile.
Andrew Gierth, reviewed by Zoltán Böszörményi and Heikki Linnakangas, and
significantly revised by me
12 years ago
|
|
|
|
|
|
|
|
memcpy(dstAtt, srcAtt, ATTRIBUTE_FIXED_PART_SIZE);
|
Support multi-argument UNNEST(), and TABLE() syntax for multiple functions.
This patch adds the ability to write TABLE( function1(), function2(), ...)
as a single FROM-clause entry. The result is the concatenation of the
first row from each function, followed by the second row from each
function, etc; with NULLs inserted if any function produces fewer rows than
others. This is believed to be a much more useful behavior than what
Postgres currently does with multiple SRFs in a SELECT list.
This syntax also provides a reasonable way to combine use of column
definition lists with WITH ORDINALITY: put the column definition list
inside TABLE(), where it's clear that it doesn't control the ordinality
column as well.
Also implement SQL-compliant multiple-argument UNNEST(), by turning
UNNEST(a,b,c) into TABLE(unnest(a), unnest(b), unnest(c)).
The SQL standard specifies TABLE() with only a single function, not
multiple functions, and it seems to require an implicit UNNEST() which is
not what this patch does. There may be something wrong with that reading
of the spec, though, because if it's right then the spec's TABLE() is just
a pointless alternative spelling of UNNEST(). After further review of
that, we might choose to adopt a different syntax for what this patch does,
but in any case this functionality seems clearly worthwhile.
Andrew Gierth, reviewed by Zoltán Böszörményi and Heikki Linnakangas, and
significantly revised by me
12 years ago
|
|
|
|
|
|
|
|
dstAtt->attnum = dstAttno;
|
Support multi-argument UNNEST(), and TABLE() syntax for multiple functions.
This patch adds the ability to write TABLE( function1(), function2(), ...)
as a single FROM-clause entry. The result is the concatenation of the
first row from each function, followed by the second row from each
function, etc; with NULLs inserted if any function produces fewer rows than
others. This is believed to be a much more useful behavior than what
Postgres currently does with multiple SRFs in a SELECT list.
This syntax also provides a reasonable way to combine use of column
definition lists with WITH ORDINALITY: put the column definition list
inside TABLE(), where it's clear that it doesn't control the ordinality
column as well.
Also implement SQL-compliant multiple-argument UNNEST(), by turning
UNNEST(a,b,c) into TABLE(unnest(a), unnest(b), unnest(c)).
The SQL standard specifies TABLE() with only a single function, not
multiple functions, and it seems to require an implicit UNNEST() which is
not what this patch does. There may be something wrong with that reading
of the spec, though, because if it's right then the spec's TABLE() is just
a pointless alternative spelling of UNNEST(). After further review of
that, we might choose to adopt a different syntax for what this patch does,
but in any case this functionality seems clearly worthwhile.
Andrew Gierth, reviewed by Zoltán Böszörményi and Heikki Linnakangas, and
significantly revised by me
12 years ago
|
|
|
|
|
|
|
|
/* since we're not copying constraints or defaults, clear these */
|
|
|
|
|
dstAtt->attnotnull = false;
|
|
|
|
|
dstAtt->atthasdef = false;
|
|
|
|
|
dstAtt->atthasmissing = false;
|
|
|
|
|
dstAtt->attidentity = '\0';
|
|
|
|
|
dstAtt->attgenerated = '\0';
|
|
|
|
|
|
|
|
|
|
populate_compact_attribute(dst, dstAttno - 1);
|
Support multi-argument UNNEST(), and TABLE() syntax for multiple functions.
This patch adds the ability to write TABLE( function1(), function2(), ...)
as a single FROM-clause entry. The result is the concatenation of the
first row from each function, followed by the second row from each
function, etc; with NULLs inserted if any function produces fewer rows than
others. This is believed to be a much more useful behavior than what
Postgres currently does with multiple SRFs in a SELECT list.
This syntax also provides a reasonable way to combine use of column
definition lists with WITH ORDINALITY: put the column definition list
inside TABLE(), where it's clear that it doesn't control the ordinality
column as well.
Also implement SQL-compliant multiple-argument UNNEST(), by turning
UNNEST(a,b,c) into TABLE(unnest(a), unnest(b), unnest(c)).
The SQL standard specifies TABLE() with only a single function, not
multiple functions, and it seems to require an implicit UNNEST() which is
not what this patch does. There may be something wrong with that reading
of the spec, though, because if it's right then the spec's TABLE() is just
a pointless alternative spelling of UNNEST(). After further review of
that, we might choose to adopt a different syntax for what this patch does,
but in any case this functionality seems clearly worthwhile.
Andrew Gierth, reviewed by Zoltán Böszörményi and Heikki Linnakangas, and
significantly revised by me
12 years ago
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Free a TupleDesc including all substructure
|
|
|
|
|
*/
|
|
|
|
|
void
|
|
|
|
|
FreeTupleDesc(TupleDesc tupdesc)
|
|
|
|
|
{
|
|
|
|
|
int i;
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Possibly this should assert tdrefcount == 0, to disallow explicit
|
|
|
|
|
* freeing of un-refcounted tupdescs?
|
|
|
|
|
*/
|
|
|
|
|
Assert(tupdesc->tdrefcount <= 0);
|
|
|
|
|
|
|
|
|
|
if (tupdesc->constr)
|
|
|
|
|
{
|
|
|
|
|
if (tupdesc->constr->num_defval > 0)
|
|
|
|
|
{
|
|
|
|
|
AttrDefault *attrdef = tupdesc->constr->defval;
|
|
|
|
|
|
|
|
|
|
for (i = tupdesc->constr->num_defval - 1; i >= 0; i--)
|
Clean up treatment of missing default and CHECK-constraint records.
Andrew Gierth reported that it's possible to crash the backend if no
pg_attrdef record is found to match an attribute that has atthasdef set.
AttrDefaultFetch warns about this situation, but then leaves behind
a relation tupdesc that has null "adbin" pointer(s), which most places
don't guard against.
We considered promoting the warning to an error, but throwing errors
during relcache load is pretty drastic: it effectively locks one out
of using the relation at all. What seems better is to leave the
load-time behavior as a warning, but then throw an error in any code
path that wants to use a default and can't find it. This confines
the error to a subset of INSERT/UPDATE operations on the table, and
in particular will at least allow a pg_dump to succeed.
Also, we should fix AttrDefaultFetch to not leave any null pointers
in the tupdesc, because that just creates an untested bug hazard.
While at it, apply the same philosophy of "warn at load, throw error
only upon use of the known-missing info" to CHECK constraints.
CheckConstraintFetch is very nearly the same logic as AttrDefaultFetch,
but for reasons lost in the mists of time, it was throwing ERROR for
the same cases that AttrDefaultFetch treats as WARNING. Make the two
functions more nearly alike.
In passing, get rid of potentially-O(N^2) loops in equalTupleDesc
by making AttrDefaultFetch sort the entries after fetching them,
so that equalTupleDesc can assume that entries in two equal tupdescs
must be in matching order. (CheckConstraintFetch already was sorting
CHECK constraints, but equalTupleDesc hadn't been told about it.)
There's some argument for back-patching this, but with such a small
number of field reports, I'm content to fix it in HEAD.
Discussion: https://postgr.es/m/87pmzaq4gx.fsf@news-spur.riddles.org.uk
5 years ago
|
|
|
pfree(attrdef[i].adbin);
|
|
|
|
|
pfree(attrdef);
|
|
|
|
|
}
|
|
|
|
|
if (tupdesc->constr->missing)
|
|
|
|
|
{
|
|
|
|
|
AttrMissing *attrmiss = tupdesc->constr->missing;
|
|
|
|
|
|
|
|
|
|
for (i = tupdesc->natts - 1; i >= 0; i--)
|
|
|
|
|
{
|
|
|
|
|
if (attrmiss[i].am_present
|
|
|
|
|
&& !TupleDescAttr(tupdesc, i)->attbyval)
|
|
|
|
|
pfree(DatumGetPointer(attrmiss[i].am_value));
|
|
|
|
|
}
|
|
|
|
|
pfree(attrmiss);
|
|
|
|
|
}
|
|
|
|
|
if (tupdesc->constr->num_check > 0)
|
|
|
|
|
{
|
|
|
|
|
ConstrCheck *check = tupdesc->constr->check;
|
|
|
|
|
|
|
|
|
|
for (i = tupdesc->constr->num_check - 1; i >= 0; i--)
|
|
|
|
|
{
|
Clean up treatment of missing default and CHECK-constraint records.
Andrew Gierth reported that it's possible to crash the backend if no
pg_attrdef record is found to match an attribute that has atthasdef set.
AttrDefaultFetch warns about this situation, but then leaves behind
a relation tupdesc that has null "adbin" pointer(s), which most places
don't guard against.
We considered promoting the warning to an error, but throwing errors
during relcache load is pretty drastic: it effectively locks one out
of using the relation at all. What seems better is to leave the
load-time behavior as a warning, but then throw an error in any code
path that wants to use a default and can't find it. This confines
the error to a subset of INSERT/UPDATE operations on the table, and
in particular will at least allow a pg_dump to succeed.
Also, we should fix AttrDefaultFetch to not leave any null pointers
in the tupdesc, because that just creates an untested bug hazard.
While at it, apply the same philosophy of "warn at load, throw error
only upon use of the known-missing info" to CHECK constraints.
CheckConstraintFetch is very nearly the same logic as AttrDefaultFetch,
but for reasons lost in the mists of time, it was throwing ERROR for
the same cases that AttrDefaultFetch treats as WARNING. Make the two
functions more nearly alike.
In passing, get rid of potentially-O(N^2) loops in equalTupleDesc
by making AttrDefaultFetch sort the entries after fetching them,
so that equalTupleDesc can assume that entries in two equal tupdescs
must be in matching order. (CheckConstraintFetch already was sorting
CHECK constraints, but equalTupleDesc hadn't been told about it.)
There's some argument for back-patching this, but with such a small
number of field reports, I'm content to fix it in HEAD.
Discussion: https://postgr.es/m/87pmzaq4gx.fsf@news-spur.riddles.org.uk
5 years ago
|
|
|
pfree(check[i].ccname);
|
|
|
|
|
pfree(check[i].ccbin);
|
|
|
|
|
}
|
|
|
|
|
pfree(check);
|
|
|
|
|
}
|
|
|
|
|
pfree(tupdesc->constr);
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
pfree(tupdesc);
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Increment the reference count of a tupdesc, and log the reference in
|
|
|
|
|
* CurrentResourceOwner.
|
|
|
|
|
*
|
|
|
|
|
* Do not apply this to tupdescs that are not being refcounted. (Use the
|
|
|
|
|
* macro PinTupleDesc for tupdescs of uncertain status.)
|
|
|
|
|
*/
|
|
|
|
|
void
|
|
|
|
|
IncrTupleDescRefCount(TupleDesc tupdesc)
|
|
|
|
|
{
|
|
|
|
|
Assert(tupdesc->tdrefcount >= 0);
|
|
|
|
|
|
Make ResourceOwners more easily extensible.
Instead of having a separate array/hash for each resource kind, use a
single array and hash to hold all kinds of resources. This makes it
possible to introduce new resource "kinds" without having to modify
the ResourceOwnerData struct. In particular, this makes it possible
for extensions to register custom resource kinds.
The old approach was to have a small array of resources of each kind,
and if it fills up, switch to a hash table. The new approach also uses
an array and a hash, but now the array and the hash are used at the
same time. The array is used to hold the recently added resources, and
when it fills up, they are moved to the hash. This keeps the access to
recent entries fast, even when there are a lot of long-held resources.
All the resource-specific ResourceOwnerEnlarge*(),
ResourceOwnerRemember*(), and ResourceOwnerForget*() functions have
been replaced with three generic functions that take resource kind as
argument. For convenience, we still define resource-specific wrapper
macros around the generic functions with the old names, but they are
now defined in the source files that use those resource kinds.
The release callback no longer needs to call ResourceOwnerForget on
the resource being released. ResourceOwnerRelease unregisters the
resource from the owner before calling the callback. That needed some
changes in bufmgr.c and some other files, where releasing the
resources previously always called ResourceOwnerForget.
Each resource kind specifies a release priority, and
ResourceOwnerReleaseAll releases the resources in priority order. To
make that possible, we have to restrict what you can do between
phases. After calling ResourceOwnerRelease(), you are no longer
allowed to remember any more resources in it or to forget any
previously remembered resources by calling ResourceOwnerForget. There
was one case where that was done previously. At subtransaction commit,
AtEOSubXact_Inval() would handle the invalidation messages and call
RelationFlushRelation(), which temporarily increased the reference
count on the relation being flushed. We now switch to the parent
subtransaction's resource owner before calling AtEOSubXact_Inval(), so
that there is a valid ResourceOwner to temporarily hold that relcache
reference.
Other end-of-xact routines make similar calls to AtEOXact_Inval()
between release phases, but I didn't see any regression test failures
from those, so I'm not sure if they could reach a codepath that needs
remembering extra resources.
There were two exceptions to how the resource leak WARNINGs on commit
were printed previously: llvmjit silently released the context without
printing the warning, and a leaked buffer io triggered a PANIC. Now
everything prints a WARNING, including those cases.
Add tests in src/test/modules/test_resowner.
Reviewed-by: Aleksander Alekseev, Michael Paquier, Julien Rouhaud
Reviewed-by: Kyotaro Horiguchi, Hayato Kuroda, Álvaro Herrera, Zhihong Yu
Reviewed-by: Peter Eisentraut, Andres Freund
Discussion: https://www.postgresql.org/message-id/cbfabeb0-cd3c-e951-a572-19b365ed314d%40iki.fi
2 years ago
|
|
|
ResourceOwnerEnlarge(CurrentResourceOwner);
|
|
|
|
|
tupdesc->tdrefcount++;
|
|
|
|
|
ResourceOwnerRememberTupleDesc(CurrentResourceOwner, tupdesc);
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Decrement the reference count of a tupdesc, remove the corresponding
|
|
|
|
|
* reference from CurrentResourceOwner, and free the tupdesc if no more
|
|
|
|
|
* references remain.
|
|
|
|
|
*
|
|
|
|
|
* Do not apply this to tupdescs that are not being refcounted. (Use the
|
|
|
|
|
* macro ReleaseTupleDesc for tupdescs of uncertain status.)
|
|
|
|
|
*/
|
|
|
|
|
void
|
|
|
|
|
DecrTupleDescRefCount(TupleDesc tupdesc)
|
|
|
|
|
{
|
|
|
|
|
Assert(tupdesc->tdrefcount > 0);
|
|
|
|
|
|
|
|
|
|
ResourceOwnerForgetTupleDesc(CurrentResourceOwner, tupdesc);
|
|
|
|
|
if (--tupdesc->tdrefcount == 0)
|
|
|
|
|
FreeTupleDesc(tupdesc);
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Compare two TupleDesc structures for logical equality
|
|
|
|
|
*/
|
|
|
|
|
bool
|
|
|
|
|
equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
|
|
|
|
|
{
|
|
|
|
|
int i,
|
|
|
|
|
n;
|
|
|
|
|
|
|
|
|
|
if (tupdesc1->natts != tupdesc2->natts)
|
|
|
|
|
return false;
|
|
|
|
|
if (tupdesc1->tdtypeid != tupdesc2->tdtypeid)
|
|
|
|
|
return false;
|
|
|
|
|
|
|
|
|
|
/* tdtypmod and tdrefcount are not checked */
|
|
|
|
|
|
|
|
|
|
for (i = 0; i < tupdesc1->natts; i++)
|
|
|
|
|
{
|
|
|
|
|
Form_pg_attribute attr1 = TupleDescAttr(tupdesc1, i);
|
|
|
|
|
Form_pg_attribute attr2 = TupleDescAttr(tupdesc2, i);
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* We do not need to check every single field here: we can disregard
|
|
|
|
|
* attrelid and attnum (which were used to place the row in the attrs
|
|
|
|
|
* array in the first place). It might look like we could dispense
|
|
|
|
|
* with checking attlen/attbyval/attalign, since these are derived
|
|
|
|
|
* from atttypid; but in the case of dropped columns we must check
|
|
|
|
|
* them (since atttypid will be zero for all dropped columns) and in
|
|
|
|
|
* general it seems safer to check them always.
|
|
|
|
|
*
|
|
|
|
|
* We intentionally ignore atthasmissing, since that's not very
|
|
|
|
|
* relevant in tupdescs, which lack the attmissingval field.
|
|
|
|
|
*/
|
|
|
|
|
if (strcmp(NameStr(attr1->attname), NameStr(attr2->attname)) != 0)
|
|
|
|
|
return false;
|
|
|
|
|
if (attr1->atttypid != attr2->atttypid)
|
|
|
|
|
return false;
|
|
|
|
|
if (attr1->attlen != attr2->attlen)
|
|
|
|
|
return false;
|
|
|
|
|
if (attr1->attndims != attr2->attndims)
|
|
|
|
|
return false;
|
|
|
|
|
if (attr1->atttypmod != attr2->atttypmod)
|
|
|
|
|
return false;
|
|
|
|
|
if (attr1->attbyval != attr2->attbyval)
|
|
|
|
|
return false;
|
|
|
|
|
if (attr1->attalign != attr2->attalign)
|
|
|
|
|
return false;
|
|
|
|
|
if (attr1->attstorage != attr2->attstorage)
|
|
|
|
|
return false;
|
|
|
|
|
if (attr1->attcompression != attr2->attcompression)
|
|
|
|
|
return false;
|
|
|
|
|
if (attr1->attnotnull != attr2->attnotnull)
|
|
|
|
|
return false;
|
Allow NOT NULL constraints to be added as NOT VALID
This allows them to be added without scanning the table, and validating
them afterwards without holding access exclusive lock on the table after
any violating rows have been deleted or fixed.
Doing ALTER TABLE ... SET NOT NULL for a column that has an invalid
not-null constraint validates that constraint. ALTER TABLE .. VALIDATE
CONSTRAINT is also supported. There are various checks on whether an
invalid constraint is allowed in a child table when the parent table has
a valid constraint; this should match what we do for enforced/not
enforced constraints.
pg_attribute.attnotnull is now only an indicator for whether a not-null
constraint exists for the column; whether it's valid or invalid must be
queried in pg_constraint. Applications can continue to query
pg_attribute.attnotnull as before, but now it's possible that NULL rows
are present in the column even when that's set to true.
For backend internal purposes, we cache the nullability status in
CompactAttribute->attnullability that each tuple descriptor carries
(replacing CompactAttribute.attnotnull, which was a mirror of
Form_pg_attribute.attnotnull). During the initial tuple descriptor
creation, based on the pg_attribute scan, we set this to UNRESTRICTED if
pg_attribute.attnotnull is false, or to UNKNOWN if it's true; then we
update the latter to VALID or INVALID depending on the pg_constraint
scan. This flag is also copied when tupledescs are copied.
Comparing tuple descs for equality must also compare the
CompactAttribute.attnullability flag and return false in case of a
mismatch.
pg_dump deals with these constraints by storing the OIDs of invalid
not-null constraints in a separate array, and running a query to obtain
their properties. The regular table creation SQL omits them entirely.
They are then dealt with in the same way as "separate" CHECK
constraints, and dumped after the data has been loaded. Because no
additional pg_dump infrastructure was required, we don't bump its
version number.
I decided not to bump catversion either, because the old catalog state
works perfectly in the new world. (Trying to run with new catalog state
and the old server version would likely run into issues, however.)
System catalogs do not support invalid not-null constraints (because
commit 14e87ffa5c54 didn't allow them to have pg_constraint rows
anyway.)
Author: Rushabh Lathia <rushabh.lathia@gmail.com>
Author: Jian He <jian.universality@gmail.com>
Reviewed-by: Álvaro Herrera <alvherre@alvh.no-ip.org>
Tested-by: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
Discussion: https://postgr.es/m/CAGPqQf0KitkNack4F5CFkFi-9Dqvp29Ro=EpcWt=4_hs-Rt+bQ@mail.gmail.com
9 months ago
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* When the column has a not-null constraint, we also need to consider
|
|
|
|
|
* its validity aspect, which only manifests in CompactAttribute->
|
|
|
|
|
* attnullability, so verify that.
|
|
|
|
|
*/
|
|
|
|
|
if (attr1->attnotnull)
|
|
|
|
|
{
|
|
|
|
|
CompactAttribute *cattr1 = TupleDescCompactAttr(tupdesc1, i);
|
|
|
|
|
CompactAttribute *cattr2 = TupleDescCompactAttr(tupdesc2, i);
|
|
|
|
|
|
|
|
|
|
Assert(cattr1->attnullability != ATTNULLABLE_UNKNOWN);
|
|
|
|
|
Assert((cattr1->attnullability == ATTNULLABLE_UNKNOWN) ==
|
|
|
|
|
(cattr2->attnullability == ATTNULLABLE_UNKNOWN));
|
|
|
|
|
|
|
|
|
|
if (cattr1->attnullability != cattr2->attnullability)
|
|
|
|
|
return false;
|
|
|
|
|
}
|
|
|
|
|
if (attr1->atthasdef != attr2->atthasdef)
|
|
|
|
|
return false;
|
|
|
|
|
if (attr1->attidentity != attr2->attidentity)
|
|
|
|
|
return false;
|
|
|
|
|
if (attr1->attgenerated != attr2->attgenerated)
|
|
|
|
|
return false;
|
|
|
|
|
if (attr1->attisdropped != attr2->attisdropped)
|
|
|
|
|
return false;
|
|
|
|
|
if (attr1->attislocal != attr2->attislocal)
|
|
|
|
|
return false;
|
|
|
|
|
if (attr1->attinhcount != attr2->attinhcount)
|
|
|
|
|
return false;
|
|
|
|
|
if (attr1->attcollation != attr2->attcollation)
|
|
|
|
|
return false;
|
|
|
|
|
/* variable-length fields are not even present... */
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
if (tupdesc1->constr != NULL)
|
|
|
|
|
{
|
|
|
|
|
TupleConstr *constr1 = tupdesc1->constr;
|
|
|
|
|
TupleConstr *constr2 = tupdesc2->constr;
|
|
|
|
|
|
|
|
|
|
if (constr2 == NULL)
|
|
|
|
|
return false;
|
|
|
|
|
if (constr1->has_not_null != constr2->has_not_null)
|
|
|
|
|
return false;
|
|
|
|
|
if (constr1->has_generated_stored != constr2->has_generated_stored)
|
|
|
|
|
return false;
|
Virtual generated columns
This adds a new variant of generated columns that are computed on read
(like a view, unlike the existing stored generated columns, which are
computed on write, like a materialized view).
The syntax for the column definition is
... GENERATED ALWAYS AS (...) VIRTUAL
and VIRTUAL is also optional. VIRTUAL is the default rather than
STORED to match various other SQL products. (The SQL standard makes
no specification about this, but it also doesn't know about VIRTUAL or
STORED.) (Also, virtual views are the default, rather than
materialized views.)
Virtual generated columns are stored in tuples as null values. (A
very early version of this patch had the ambition to not store them at
all. But so much stuff breaks or gets confused if you have tuples
where a column in the middle is completely missing. This is a
compromise, and it still saves space over being forced to use stored
generated columns. If we ever find a way to improve this, a bit of
pg_upgrade cleverness could allow for upgrades to a newer scheme.)
The capabilities and restrictions of virtual generated columns are
mostly the same as for stored generated columns. In some cases, this
patch keeps virtual generated columns more restricted than they might
technically need to be, to keep the two kinds consistent. Some of
that could maybe be relaxed later after separate careful
considerations.
Some functionality that is currently not supported, but could possibly
be added as incremental features, some easier than others:
- index on or using a virtual column
- hence also no unique constraints on virtual columns
- extended statistics on virtual columns
- foreign-key constraints on virtual columns
- not-null constraints on virtual columns (check constraints are supported)
- ALTER TABLE / DROP EXPRESSION
- virtual column cannot have domain type
- virtual columns are not supported in logical replication
The tests in generated_virtual.sql have been copied over from
generated_stored.sql with the keyword replaced. This way we can make
sure the behavior is mostly aligned, and the differences can be
visible. Some tests for currently not supported features are
currently commented out.
Reviewed-by: Jian He <jian.universality@gmail.com>
Reviewed-by: Dean Rasheed <dean.a.rasheed@gmail.com>
Tested-by: Shlok Kyal <shlok.kyal.oss@gmail.com>
Discussion: https://www.postgresql.org/message-id/flat/a368248e-69e4-40be-9c07-6c3b5880b0a6@eisentraut.org
10 months ago
|
|
|
if (constr1->has_generated_virtual != constr2->has_generated_virtual)
|
|
|
|
|
return false;
|
|
|
|
|
n = constr1->num_defval;
|
|
|
|
|
if (n != (int) constr2->num_defval)
|
|
|
|
|
return false;
|
Clean up treatment of missing default and CHECK-constraint records.
Andrew Gierth reported that it's possible to crash the backend if no
pg_attrdef record is found to match an attribute that has atthasdef set.
AttrDefaultFetch warns about this situation, but then leaves behind
a relation tupdesc that has null "adbin" pointer(s), which most places
don't guard against.
We considered promoting the warning to an error, but throwing errors
during relcache load is pretty drastic: it effectively locks one out
of using the relation at all. What seems better is to leave the
load-time behavior as a warning, but then throw an error in any code
path that wants to use a default and can't find it. This confines
the error to a subset of INSERT/UPDATE operations on the table, and
in particular will at least allow a pg_dump to succeed.
Also, we should fix AttrDefaultFetch to not leave any null pointers
in the tupdesc, because that just creates an untested bug hazard.
While at it, apply the same philosophy of "warn at load, throw error
only upon use of the known-missing info" to CHECK constraints.
CheckConstraintFetch is very nearly the same logic as AttrDefaultFetch,
but for reasons lost in the mists of time, it was throwing ERROR for
the same cases that AttrDefaultFetch treats as WARNING. Make the two
functions more nearly alike.
In passing, get rid of potentially-O(N^2) loops in equalTupleDesc
by making AttrDefaultFetch sort the entries after fetching them,
so that equalTupleDesc can assume that entries in two equal tupdescs
must be in matching order. (CheckConstraintFetch already was sorting
CHECK constraints, but equalTupleDesc hadn't been told about it.)
There's some argument for back-patching this, but with such a small
number of field reports, I'm content to fix it in HEAD.
Discussion: https://postgr.es/m/87pmzaq4gx.fsf@news-spur.riddles.org.uk
5 years ago
|
|
|
/* We assume here that both AttrDefault arrays are in adnum order */
|
|
|
|
|
for (i = 0; i < n; i++)
|
|
|
|
|
{
|
|
|
|
|
AttrDefault *defval1 = constr1->defval + i;
|
Clean up treatment of missing default and CHECK-constraint records.
Andrew Gierth reported that it's possible to crash the backend if no
pg_attrdef record is found to match an attribute that has atthasdef set.
AttrDefaultFetch warns about this situation, but then leaves behind
a relation tupdesc that has null "adbin" pointer(s), which most places
don't guard against.
We considered promoting the warning to an error, but throwing errors
during relcache load is pretty drastic: it effectively locks one out
of using the relation at all. What seems better is to leave the
load-time behavior as a warning, but then throw an error in any code
path that wants to use a default and can't find it. This confines
the error to a subset of INSERT/UPDATE operations on the table, and
in particular will at least allow a pg_dump to succeed.
Also, we should fix AttrDefaultFetch to not leave any null pointers
in the tupdesc, because that just creates an untested bug hazard.
While at it, apply the same philosophy of "warn at load, throw error
only upon use of the known-missing info" to CHECK constraints.
CheckConstraintFetch is very nearly the same logic as AttrDefaultFetch,
but for reasons lost in the mists of time, it was throwing ERROR for
the same cases that AttrDefaultFetch treats as WARNING. Make the two
functions more nearly alike.
In passing, get rid of potentially-O(N^2) loops in equalTupleDesc
by making AttrDefaultFetch sort the entries after fetching them,
so that equalTupleDesc can assume that entries in two equal tupdescs
must be in matching order. (CheckConstraintFetch already was sorting
CHECK constraints, but equalTupleDesc hadn't been told about it.)
There's some argument for back-patching this, but with such a small
number of field reports, I'm content to fix it in HEAD.
Discussion: https://postgr.es/m/87pmzaq4gx.fsf@news-spur.riddles.org.uk
5 years ago
|
|
|
AttrDefault *defval2 = constr2->defval + i;
|
|
|
|
|
|
|
|
|
|
if (defval1->adnum != defval2->adnum)
|
|
|
|
|
return false;
|
|
|
|
|
if (strcmp(defval1->adbin, defval2->adbin) != 0)
|
|
|
|
|
return false;
|
|
|
|
|
}
|
|
|
|
|
if (constr1->missing)
|
|
|
|
|
{
|
|
|
|
|
if (!constr2->missing)
|
|
|
|
|
return false;
|
|
|
|
|
for (i = 0; i < tupdesc1->natts; i++)
|
|
|
|
|
{
|
|
|
|
|
AttrMissing *missval1 = constr1->missing + i;
|
|
|
|
|
AttrMissing *missval2 = constr2->missing + i;
|
|
|
|
|
|
|
|
|
|
if (missval1->am_present != missval2->am_present)
|
|
|
|
|
return false;
|
|
|
|
|
if (missval1->am_present)
|
|
|
|
|
{
|
|
|
|
|
CompactAttribute *missatt1 = TupleDescCompactAttr(tupdesc1, i);
|
|
|
|
|
|
|
|
|
|
if (!datumIsEqual(missval1->am_value, missval2->am_value,
|
|
|
|
|
missatt1->attbyval, missatt1->attlen))
|
|
|
|
|
return false;
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
else if (constr2->missing)
|
|
|
|
|
return false;
|
|
|
|
|
n = constr1->num_check;
|
|
|
|
|
if (n != (int) constr2->num_check)
|
|
|
|
|
return false;
|
Clean up treatment of missing default and CHECK-constraint records.
Andrew Gierth reported that it's possible to crash the backend if no
pg_attrdef record is found to match an attribute that has atthasdef set.
AttrDefaultFetch warns about this situation, but then leaves behind
a relation tupdesc that has null "adbin" pointer(s), which most places
don't guard against.
We considered promoting the warning to an error, but throwing errors
during relcache load is pretty drastic: it effectively locks one out
of using the relation at all. What seems better is to leave the
load-time behavior as a warning, but then throw an error in any code
path that wants to use a default and can't find it. This confines
the error to a subset of INSERT/UPDATE operations on the table, and
in particular will at least allow a pg_dump to succeed.
Also, we should fix AttrDefaultFetch to not leave any null pointers
in the tupdesc, because that just creates an untested bug hazard.
While at it, apply the same philosophy of "warn at load, throw error
only upon use of the known-missing info" to CHECK constraints.
CheckConstraintFetch is very nearly the same logic as AttrDefaultFetch,
but for reasons lost in the mists of time, it was throwing ERROR for
the same cases that AttrDefaultFetch treats as WARNING. Make the two
functions more nearly alike.
In passing, get rid of potentially-O(N^2) loops in equalTupleDesc
by making AttrDefaultFetch sort the entries after fetching them,
so that equalTupleDesc can assume that entries in two equal tupdescs
must be in matching order. (CheckConstraintFetch already was sorting
CHECK constraints, but equalTupleDesc hadn't been told about it.)
There's some argument for back-patching this, but with such a small
number of field reports, I'm content to fix it in HEAD.
Discussion: https://postgr.es/m/87pmzaq4gx.fsf@news-spur.riddles.org.uk
5 years ago
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Similarly, we rely here on the ConstrCheck entries being sorted by
|
|
|
|
|
* name. If there are duplicate names, the outcome of the comparison
|
|
|
|
|
* is uncertain, but that should not happen.
|
|
|
|
|
*/
|
|
|
|
|
for (i = 0; i < n; i++)
|
|
|
|
|
{
|
|
|
|
|
ConstrCheck *check1 = constr1->check + i;
|
Clean up treatment of missing default and CHECK-constraint records.
Andrew Gierth reported that it's possible to crash the backend if no
pg_attrdef record is found to match an attribute that has atthasdef set.
AttrDefaultFetch warns about this situation, but then leaves behind
a relation tupdesc that has null "adbin" pointer(s), which most places
don't guard against.
We considered promoting the warning to an error, but throwing errors
during relcache load is pretty drastic: it effectively locks one out
of using the relation at all. What seems better is to leave the
load-time behavior as a warning, but then throw an error in any code
path that wants to use a default and can't find it. This confines
the error to a subset of INSERT/UPDATE operations on the table, and
in particular will at least allow a pg_dump to succeed.
Also, we should fix AttrDefaultFetch to not leave any null pointers
in the tupdesc, because that just creates an untested bug hazard.
While at it, apply the same philosophy of "warn at load, throw error
only upon use of the known-missing info" to CHECK constraints.
CheckConstraintFetch is very nearly the same logic as AttrDefaultFetch,
but for reasons lost in the mists of time, it was throwing ERROR for
the same cases that AttrDefaultFetch treats as WARNING. Make the two
functions more nearly alike.
In passing, get rid of potentially-O(N^2) loops in equalTupleDesc
by making AttrDefaultFetch sort the entries after fetching them,
so that equalTupleDesc can assume that entries in two equal tupdescs
must be in matching order. (CheckConstraintFetch already was sorting
CHECK constraints, but equalTupleDesc hadn't been told about it.)
There's some argument for back-patching this, but with such a small
number of field reports, I'm content to fix it in HEAD.
Discussion: https://postgr.es/m/87pmzaq4gx.fsf@news-spur.riddles.org.uk
5 years ago
|
|
|
ConstrCheck *check2 = constr2->check + i;
|
|
|
|
|
|
|
|
|
|
if (!(strcmp(check1->ccname, check2->ccname) == 0 &&
|
|
|
|
|
strcmp(check1->ccbin, check2->ccbin) == 0 &&
|
|
|
|
|
check1->ccenforced == check2->ccenforced &&
|
Clean up treatment of missing default and CHECK-constraint records.
Andrew Gierth reported that it's possible to crash the backend if no
pg_attrdef record is found to match an attribute that has atthasdef set.
AttrDefaultFetch warns about this situation, but then leaves behind
a relation tupdesc that has null "adbin" pointer(s), which most places
don't guard against.
We considered promoting the warning to an error, but throwing errors
during relcache load is pretty drastic: it effectively locks one out
of using the relation at all. What seems better is to leave the
load-time behavior as a warning, but then throw an error in any code
path that wants to use a default and can't find it. This confines
the error to a subset of INSERT/UPDATE operations on the table, and
in particular will at least allow a pg_dump to succeed.
Also, we should fix AttrDefaultFetch to not leave any null pointers
in the tupdesc, because that just creates an untested bug hazard.
While at it, apply the same philosophy of "warn at load, throw error
only upon use of the known-missing info" to CHECK constraints.
CheckConstraintFetch is very nearly the same logic as AttrDefaultFetch,
but for reasons lost in the mists of time, it was throwing ERROR for
the same cases that AttrDefaultFetch treats as WARNING. Make the two
functions more nearly alike.
In passing, get rid of potentially-O(N^2) loops in equalTupleDesc
by making AttrDefaultFetch sort the entries after fetching them,
so that equalTupleDesc can assume that entries in two equal tupdescs
must be in matching order. (CheckConstraintFetch already was sorting
CHECK constraints, but equalTupleDesc hadn't been told about it.)
There's some argument for back-patching this, but with such a small
number of field reports, I'm content to fix it in HEAD.
Discussion: https://postgr.es/m/87pmzaq4gx.fsf@news-spur.riddles.org.uk
5 years ago
|
|
|
check1->ccvalid == check2->ccvalid &&
|
|
|
|
|
check1->ccnoinherit == check2->ccnoinherit))
|
|
|
|
|
return false;
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
else if (tupdesc2->constr != NULL)
|
|
|
|
|
return false;
|
|
|
|
|
return true;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* equalRowTypes
|
|
|
|
|
*
|
|
|
|
|
* This determines whether two tuple descriptors have equal row types. This
|
|
|
|
|
* only checks those fields in pg_attribute that are applicable for row types,
|
|
|
|
|
* while ignoring those fields that define the physical row storage or those
|
|
|
|
|
* that define table column metadata.
|
|
|
|
|
*
|
|
|
|
|
* Specifically, this checks:
|
|
|
|
|
*
|
|
|
|
|
* - same number of attributes
|
|
|
|
|
* - same composite type ID (but could both be zero)
|
|
|
|
|
* - corresponding attributes (in order) have same the name, type, typmod,
|
|
|
|
|
* collation
|
|
|
|
|
*
|
|
|
|
|
* This is used to check whether two record types are compatible, whether
|
|
|
|
|
* function return row types are the same, and other similar situations.
|
|
|
|
|
*
|
|
|
|
|
* (XXX There was some discussion whether attndims should be checked here, but
|
|
|
|
|
* for now it has been decided not to.)
|
|
|
|
|
*
|
|
|
|
|
* Note: We deliberately do not check the tdtypmod field. This allows
|
|
|
|
|
* typcache.c to use this routine to see if a cached record type matches a
|
|
|
|
|
* requested type.
|
|
|
|
|
*/
|
|
|
|
|
bool
|
|
|
|
|
equalRowTypes(TupleDesc tupdesc1, TupleDesc tupdesc2)
|
|
|
|
|
{
|
|
|
|
|
if (tupdesc1->natts != tupdesc2->natts)
|
|
|
|
|
return false;
|
|
|
|
|
if (tupdesc1->tdtypeid != tupdesc2->tdtypeid)
|
|
|
|
|
return false;
|
|
|
|
|
|
|
|
|
|
for (int i = 0; i < tupdesc1->natts; i++)
|
|
|
|
|
{
|
|
|
|
|
Form_pg_attribute attr1 = TupleDescAttr(tupdesc1, i);
|
|
|
|
|
Form_pg_attribute attr2 = TupleDescAttr(tupdesc2, i);
|
|
|
|
|
|
|
|
|
|
if (strcmp(NameStr(attr1->attname), NameStr(attr2->attname)) != 0)
|
|
|
|
|
return false;
|
|
|
|
|
if (attr1->atttypid != attr2->atttypid)
|
|
|
|
|
return false;
|
|
|
|
|
if (attr1->atttypmod != attr2->atttypmod)
|
|
|
|
|
return false;
|
|
|
|
|
if (attr1->attcollation != attr2->attcollation)
|
|
|
|
|
return false;
|
|
|
|
|
|
|
|
|
|
/* Record types derived from tables could have dropped fields. */
|
|
|
|
|
if (attr1->attisdropped != attr2->attisdropped)
|
|
|
|
|
return false;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
return true;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* hashRowType
|
|
|
|
|
*
|
|
|
|
|
* If two tuple descriptors would be considered equal by equalRowTypes()
|
|
|
|
|
* then their hash value will be equal according to this function.
|
|
|
|
|
*/
|
|
|
|
|
uint32
|
|
|
|
|
hashRowType(TupleDesc desc)
|
|
|
|
|
{
|
|
|
|
|
uint32 s;
|
|
|
|
|
int i;
|
|
|
|
|
|
|
|
|
|
s = hash_combine(0, hash_uint32(desc->natts));
|
|
|
|
|
s = hash_combine(s, hash_uint32(desc->tdtypeid));
|
|
|
|
|
for (i = 0; i < desc->natts; ++i)
|
|
|
|
|
s = hash_combine(s, hash_uint32(TupleDescAttr(desc, i)->atttypid));
|
|
|
|
|
|
|
|
|
|
return s;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* TupleDescInitEntry
|
|
|
|
|
* This function initializes a single attribute structure in
|
|
|
|
|
* a previously allocated tuple descriptor.
|
|
|
|
|
*
|
|
|
|
|
* If attributeName is NULL, the attname field is set to an empty string
|
|
|
|
|
* (this is for cases where we don't know or need a name for the field).
|
|
|
|
|
* Also, some callers use this function to change the datatype-related fields
|
|
|
|
|
* in an existing tupdesc; they pass attributeName = NameStr(att->attname)
|
|
|
|
|
* to indicate that the attname field shouldn't be modified.
|
|
|
|
|
*
|
|
|
|
|
* Note that attcollation is set to the default for the specified datatype.
|
|
|
|
|
* If a nondefault collation is needed, insert it afterwards using
|
|
|
|
|
* TupleDescInitEntryCollation.
|
|
|
|
|
*/
|
|
|
|
|
void
|
|
|
|
|
TupleDescInitEntry(TupleDesc desc,
|
|
|
|
|
AttrNumber attributeNumber,
|
|
|
|
|
const char *attributeName,
|
|
|
|
|
Oid oidtypeid,
|
|
|
|
|
int32 typmod,
|
|
|
|
|
int attdim)
|
|
|
|
|
{
|
|
|
|
|
HeapTuple tuple;
|
|
|
|
|
Form_pg_type typeForm;
|
|
|
|
|
Form_pg_attribute att;
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* sanity checks
|
|
|
|
|
*/
|
|
|
|
|
Assert(PointerIsValid(desc));
|
|
|
|
|
Assert(attributeNumber >= 1);
|
|
|
|
|
Assert(attributeNumber <= desc->natts);
|
|
|
|
|
Assert(attdim >= 0);
|
|
|
|
|
Assert(attdim <= PG_INT16_MAX);
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* initialize the attribute fields
|
|
|
|
|
*/
|
|
|
|
|
att = TupleDescAttr(desc, attributeNumber - 1);
|
|
|
|
|
|
|
|
|
|
att->attrelid = 0; /* dummy value */
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Note: attributeName can be NULL, because the planner doesn't always
|
|
|
|
|
* fill in valid resname values in targetlists, particularly for resjunk
|
|
|
|
|
* attributes. Also, do nothing if caller wants to re-use the old attname.
|
|
|
|
|
*/
|
|
|
|
|
if (attributeName == NULL)
|
|
|
|
|
MemSet(NameStr(att->attname), 0, NAMEDATALEN);
|
|
|
|
|
else if (attributeName != NameStr(att->attname))
|
|
|
|
|
namestrcpy(&(att->attname), attributeName);
|
|
|
|
|
|
|
|
|
|
att->atttypmod = typmod;
|
|
|
|
|
|
|
|
|
|
att->attnum = attributeNumber;
|
|
|
|
|
att->attndims = attdim;
|
|
|
|
|
|
|
|
|
|
att->attnotnull = false;
|
|
|
|
|
att->atthasdef = false;
|
|
|
|
|
att->atthasmissing = false;
|
|
|
|
|
att->attidentity = '\0';
|
|
|
|
|
att->attgenerated = '\0';
|
|
|
|
|
att->attisdropped = false;
|
|
|
|
|
att->attislocal = true;
|
|
|
|
|
att->attinhcount = 0;
|
|
|
|
|
/* variable-length fields are not present in tupledescs */
|
|
|
|
|
|
|
|
|
|
tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(oidtypeid));
|
|
|
|
|
if (!HeapTupleIsValid(tuple))
|
|
|
|
|
elog(ERROR, "cache lookup failed for type %u", oidtypeid);
|
|
|
|
|
typeForm = (Form_pg_type) GETSTRUCT(tuple);
|
|
|
|
|
|
|
|
|
|
att->atttypid = oidtypeid;
|
|
|
|
|
att->attlen = typeForm->typlen;
|
|
|
|
|
att->attbyval = typeForm->typbyval;
|
|
|
|
|
att->attalign = typeForm->typalign;
|
|
|
|
|
att->attstorage = typeForm->typstorage;
|
|
|
|
|
att->attcompression = InvalidCompressionMethod;
|
|
|
|
|
att->attcollation = typeForm->typcollation;
|
Allow configurable LZ4 TOAST compression.
There is now a per-column COMPRESSION option which can be set to pglz
(the default, and the only option in up until now) or lz4. Or, if you
like, you can set the new default_toast_compression GUC to lz4, and
then that will be the default for new table columns for which no value
is specified. We don't have lz4 support in the PostgreSQL code, so
to use lz4 compression, PostgreSQL must be built --with-lz4.
In general, TOAST compression means compression of individual column
values, not the whole tuple, and those values can either be compressed
inline within the tuple or compressed and then stored externally in
the TOAST table, so those properties also apply to this feature.
Prior to this commit, a TOAST pointer has two unused bits as part of
the va_extsize field, and a compessed datum has two unused bits as
part of the va_rawsize field. These bits are unused because the length
of a varlena is limited to 1GB; we now use them to indicate the
compression type that was used. This means we only have bit space for
2 more built-in compresison types, but we could work around that
problem, if necessary, by introducing a new vartag_external value for
any further types we end up wanting to add. Hopefully, it won't be
too important to offer a wide selection of algorithms here, since
each one we add not only takes more coding but also adds a build
dependency for every packager. Nevertheless, it seems worth doing
at least this much, because LZ4 gets better compression than PGLZ
with less CPU usage.
It's possible for LZ4-compressed datums to leak into composite type
values stored on disk, just as it is for PGLZ. It's also possible for
LZ4-compressed attributes to be copied into a different table via SQL
commands such as CREATE TABLE AS or INSERT .. SELECT. It would be
expensive to force such values to be decompressed, so PostgreSQL has
never done so. For the same reasons, we also don't force recompression
of already-compressed values even if the target table prefers a
different compression method than was used for the source data. These
architectural decisions are perhaps arguable but revisiting them is
well beyond the scope of what seemed possible to do as part of this
project. However, it's relatively cheap to recompress as part of
VACUUM FULL or CLUSTER, so this commit adjusts those commands to do
so, if the configured compression method of the table happens not to
match what was used for some column value stored therein.
Dilip Kumar. The original patches on which this work was based were
written by Ildus Kurbangaliev, and those were patches were based on
even earlier work by Nikita Glukhov, but the design has since changed
very substantially, since allow a potentially large number of
compression methods that could be added and dropped on a running
system proved too problematic given some of the architectural issues
mentioned above; the choice of which specific compression method to
add first is now different; and a lot of the code has been heavily
refactored. More recently, Justin Przyby helped quite a bit with
testing and reviewing and this version also includes some code
contributions from him. Other design input and review from Tomas
Vondra, Álvaro Herrera, Andres Freund, Oleg Bartunov, Alexander
Korotkov, and me.
Discussion: http://postgr.es/m/20170907194236.4cefce96%40wp.localdomain
Discussion: http://postgr.es/m/CAFiTN-uUpX3ck%3DK0mLEk-G_kUQY%3DSNOTeqdaNRR9FMdQrHKebw%40mail.gmail.com
5 years ago
|
|
|
|
|
|
|
|
populate_compact_attribute(desc, attributeNumber - 1);
|
|
|
|
|
|
|
|
|
|
ReleaseSysCache(tuple);
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* TupleDescInitBuiltinEntry
|
|
|
|
|
* Initialize a tuple descriptor without catalog access. Only
|
|
|
|
|
* a limited range of builtin types are supported.
|
|
|
|
|
*/
|
|
|
|
|
void
|
|
|
|
|
TupleDescInitBuiltinEntry(TupleDesc desc,
|
|
|
|
|
AttrNumber attributeNumber,
|
|
|
|
|
const char *attributeName,
|
|
|
|
|
Oid oidtypeid,
|
|
|
|
|
int32 typmod,
|
|
|
|
|
int attdim)
|
|
|
|
|
{
|
|
|
|
|
Form_pg_attribute att;
|
|
|
|
|
|
|
|
|
|
/* sanity checks */
|
|
|
|
|
Assert(PointerIsValid(desc));
|
|
|
|
|
Assert(attributeNumber >= 1);
|
|
|
|
|
Assert(attributeNumber <= desc->natts);
|
|
|
|
|
Assert(attdim >= 0);
|
|
|
|
|
Assert(attdim <= PG_INT16_MAX);
|
|
|
|
|
|
|
|
|
|
/* initialize the attribute fields */
|
|
|
|
|
att = TupleDescAttr(desc, attributeNumber - 1);
|
|
|
|
|
att->attrelid = 0; /* dummy value */
|
|
|
|
|
|
|
|
|
|
/* unlike TupleDescInitEntry, we require an attribute name */
|
|
|
|
|
Assert(attributeName != NULL);
|
|
|
|
|
namestrcpy(&(att->attname), attributeName);
|
|
|
|
|
|
|
|
|
|
att->atttypmod = typmod;
|
|
|
|
|
|
|
|
|
|
att->attnum = attributeNumber;
|
|
|
|
|
att->attndims = attdim;
|
|
|
|
|
|
|
|
|
|
att->attnotnull = false;
|
|
|
|
|
att->atthasdef = false;
|
|
|
|
|
att->atthasmissing = false;
|
|
|
|
|
att->attidentity = '\0';
|
|
|
|
|
att->attgenerated = '\0';
|
|
|
|
|
att->attisdropped = false;
|
|
|
|
|
att->attislocal = true;
|
|
|
|
|
att->attinhcount = 0;
|
|
|
|
|
/* variable-length fields are not present in tupledescs */
|
|
|
|
|
|
|
|
|
|
att->atttypid = oidtypeid;
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Our goal here is to support just enough types to let basic builtin
|
|
|
|
|
* commands work without catalog access - e.g. so that we can do certain
|
|
|
|
|
* things even in processes that are not connected to a database.
|
|
|
|
|
*/
|
|
|
|
|
switch (oidtypeid)
|
|
|
|
|
{
|
|
|
|
|
case TEXTOID:
|
|
|
|
|
case TEXTARRAYOID:
|
|
|
|
|
att->attlen = -1;
|
|
|
|
|
att->attbyval = false;
|
|
|
|
|
att->attalign = TYPALIGN_INT;
|
|
|
|
|
att->attstorage = TYPSTORAGE_EXTENDED;
|
|
|
|
|
att->attcompression = InvalidCompressionMethod;
|
|
|
|
|
att->attcollation = DEFAULT_COLLATION_OID;
|
|
|
|
|
break;
|
|
|
|
|
|
|
|
|
|
case BOOLOID:
|
|
|
|
|
att->attlen = 1;
|
|
|
|
|
att->attbyval = true;
|
|
|
|
|
att->attalign = TYPALIGN_CHAR;
|
|
|
|
|
att->attstorage = TYPSTORAGE_PLAIN;
|
|
|
|
|
att->attcompression = InvalidCompressionMethod;
|
|
|
|
|
att->attcollation = InvalidOid;
|
|
|
|
|
break;
|
|
|
|
|
|
|
|
|
|
case INT4OID:
|
|
|
|
|
att->attlen = 4;
|
|
|
|
|
att->attbyval = true;
|
|
|
|
|
att->attalign = TYPALIGN_INT;
|
|
|
|
|
att->attstorage = TYPSTORAGE_PLAIN;
|
|
|
|
|
att->attcompression = InvalidCompressionMethod;
|
|
|
|
|
att->attcollation = InvalidOid;
|
|
|
|
|
break;
|
|
|
|
|
|
|
|
|
|
case INT8OID:
|
|
|
|
|
att->attlen = 8;
|
|
|
|
|
att->attbyval = FLOAT8PASSBYVAL;
|
|
|
|
|
att->attalign = TYPALIGN_DOUBLE;
|
|
|
|
|
att->attstorage = TYPSTORAGE_PLAIN;
|
|
|
|
|
att->attcompression = InvalidCompressionMethod;
|
|
|
|
|
att->attcollation = InvalidOid;
|
|
|
|
|
break;
|
|
|
|
|
|
|
|
|
|
case OIDOID:
|
|
|
|
|
att->attlen = 4;
|
|
|
|
|
att->attbyval = true;
|
|
|
|
|
att->attalign = TYPALIGN_INT;
|
|
|
|
|
att->attstorage = TYPSTORAGE_PLAIN;
|
|
|
|
|
att->attcompression = InvalidCompressionMethod;
|
|
|
|
|
att->attcollation = InvalidOid;
|
|
|
|
|
break;
|
|
|
|
|
|
|
|
|
|
default:
|
|
|
|
|
elog(ERROR, "unsupported type %u", oidtypeid);
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
populate_compact_attribute(desc, attributeNumber - 1);
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* TupleDescInitEntryCollation
|
|
|
|
|
*
|
|
|
|
|
* Assign a nondefault collation to a previously initialized tuple descriptor
|
|
|
|
|
* entry.
|
|
|
|
|
*/
|
|
|
|
|
void
|
|
|
|
|
TupleDescInitEntryCollation(TupleDesc desc,
|
|
|
|
|
AttrNumber attributeNumber,
|
|
|
|
|
Oid collationid)
|
|
|
|
|
{
|
|
|
|
|
/*
|
|
|
|
|
* sanity checks
|
|
|
|
|
*/
|
|
|
|
|
Assert(PointerIsValid(desc));
|
|
|
|
|
Assert(attributeNumber >= 1);
|
|
|
|
|
Assert(attributeNumber <= desc->natts);
|
|
|
|
|
|
|
|
|
|
TupleDescAttr(desc, attributeNumber - 1)->attcollation = collationid;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* BuildDescFromLists
|
|
|
|
|
*
|
|
|
|
|
* Build a TupleDesc given lists of column names (as String nodes),
|
|
|
|
|
* column type OIDs, typmods, and collation OIDs.
|
|
|
|
|
*
|
|
|
|
|
* No constraints are generated.
|
|
|
|
|
*
|
|
|
|
|
* This is for use with functions returning RECORD.
|
|
|
|
|
*/
|
|
|
|
|
TupleDesc
|
|
|
|
|
BuildDescFromLists(const List *names, const List *types, const List *typmods, const List *collations)
|
|
|
|
|
{
|
|
|
|
|
int natts;
|
|
|
|
|
AttrNumber attnum;
|
|
|
|
|
ListCell *l1;
|
|
|
|
|
ListCell *l2;
|
|
|
|
|
ListCell *l3;
|
|
|
|
|
ListCell *l4;
|
|
|
|
|
TupleDesc desc;
|
|
|
|
|
|
|
|
|
|
natts = list_length(names);
|
|
|
|
|
Assert(natts == list_length(types));
|
|
|
|
|
Assert(natts == list_length(typmods));
|
|
|
|
|
Assert(natts == list_length(collations));
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* allocate a new tuple descriptor
|
|
|
|
|
*/
|
Remove WITH OIDS support, change oid catalog column visibility.
Previously tables declared WITH OIDS, including a significant fraction
of the catalog tables, stored the oid column not as a normal column,
but as part of the tuple header.
This special column was not shown by default, which was somewhat odd,
as it's often (consider e.g. pg_class.oid) one of the more important
parts of a row. Neither pg_dump nor COPY included the contents of the
oid column by default.
The fact that the oid column was not an ordinary column necessitated a
significant amount of special case code to support oid columns. That
already was painful for the existing, but upcoming work aiming to make
table storage pluggable, would have required expanding and duplicating
that "specialness" significantly.
WITH OIDS has been deprecated since 2005 (commit ff02d0a05280e0).
Remove it.
Removing includes:
- CREATE TABLE and ALTER TABLE syntax for declaring the table to be
WITH OIDS has been removed (WITH (oids[ = true]) will error out)
- pg_dump does not support dumping tables declared WITH OIDS and will
issue a warning when dumping one (and ignore the oid column).
- restoring an pg_dump archive with pg_restore will warn when
restoring a table with oid contents (and ignore the oid column)
- COPY will refuse to load binary dump that includes oids.
- pg_upgrade will error out when encountering tables declared WITH
OIDS, they have to be altered to remove the oid column first.
- Functionality to access the oid of the last inserted row (like
plpgsql's RESULT_OID, spi's SPI_lastoid, ...) has been removed.
The syntax for declaring a table WITHOUT OIDS (or WITH (oids = false)
for CREATE TABLE) is still supported. While that requires a bit of
support code, it seems unnecessary to break applications / dumps that
do not use oids, and are explicit about not using them.
The biggest user of WITH OID columns was postgres' catalog. This
commit changes all 'magic' oid columns to be columns that are normally
declared and stored. To reduce unnecessary query breakage all the
newly added columns are still named 'oid', even if a table's column
naming scheme would indicate 'reloid' or such. This obviously
requires adapting a lot code, mostly replacing oid access via
HeapTupleGetOid() with access to the underlying Form_pg_*->oid column.
The bootstrap process now assigns oids for all oid columns in
genbki.pl that do not have an explicit value (starting at the largest
oid previously used), only oids assigned later by oids will be above
FirstBootstrapObjectId. As the oid column now is a normal column the
special bootstrap syntax for oids has been removed.
Oids are not automatically assigned during insertion anymore, all
backend code explicitly assigns oids with GetNewOidWithIndex(). For
the rare case that insertions into the catalog via SQL are called for
the new pg_nextoid() function can be used (which only works on catalog
tables).
The fact that oid columns on system tables are now normal columns
means that they will be included in the set of columns expanded
by * (i.e. SELECT * FROM pg_class will now include the table's oid,
previously it did not). It'd not technically be hard to hide oid
column by default, but that'd mean confusing behavior would either
have to be carried forward forever, or it'd cause breakage down the
line.
While it's not unlikely that further adjustments are needed, the
scope/invasiveness of the patch makes it worthwhile to get merge this
now. It's painful to maintain externally, too complicated to commit
after the code code freeze, and a dependency of a number of other
patches.
Catversion bump, for obvious reasons.
Author: Andres Freund, with contributions by John Naylor
Discussion: https://postgr.es/m/20180930034810.ywp2c7awz7opzcfr@alap3.anarazel.de
7 years ago
|
|
|
desc = CreateTemplateTupleDesc(natts);
|
|
|
|
|
|
|
|
|
|
attnum = 0;
|
|
|
|
|
forfour(l1, names, l2, types, l3, typmods, l4, collations)
|
|
|
|
|
{
|
|
|
|
|
char *attname = strVal(lfirst(l1));
|
|
|
|
|
Oid atttypid = lfirst_oid(l2);
|
|
|
|
|
int32 atttypmod = lfirst_int(l3);
|
|
|
|
|
Oid attcollation = lfirst_oid(l4);
|
|
|
|
|
|
|
|
|
|
attnum++;
|
|
|
|
|
|
|
|
|
|
TupleDescInitEntry(desc, attnum, attname, atttypid, atttypmod, 0);
|
|
|
|
|
TupleDescInitEntryCollation(desc, attnum, attcollation);
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
return desc;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Get default expression (or NULL if none) for the given attribute number.
|
|
|
|
|
*/
|
|
|
|
|
Node *
|
|
|
|
|
TupleDescGetDefault(TupleDesc tupdesc, AttrNumber attnum)
|
|
|
|
|
{
|
|
|
|
|
Node *result = NULL;
|
|
|
|
|
|
|
|
|
|
if (tupdesc->constr)
|
|
|
|
|
{
|
|
|
|
|
AttrDefault *attrdef = tupdesc->constr->defval;
|
|
|
|
|
|
|
|
|
|
for (int i = 0; i < tupdesc->constr->num_defval; i++)
|
|
|
|
|
{
|
|
|
|
|
if (attrdef[i].adnum == attnum)
|
|
|
|
|
{
|
|
|
|
|
result = stringToNode(attrdef[i].adbin);
|
|
|
|
|
break;
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
return result;
|
|
|
|
|
}
|
Make ResourceOwners more easily extensible.
Instead of having a separate array/hash for each resource kind, use a
single array and hash to hold all kinds of resources. This makes it
possible to introduce new resource "kinds" without having to modify
the ResourceOwnerData struct. In particular, this makes it possible
for extensions to register custom resource kinds.
The old approach was to have a small array of resources of each kind,
and if it fills up, switch to a hash table. The new approach also uses
an array and a hash, but now the array and the hash are used at the
same time. The array is used to hold the recently added resources, and
when it fills up, they are moved to the hash. This keeps the access to
recent entries fast, even when there are a lot of long-held resources.
All the resource-specific ResourceOwnerEnlarge*(),
ResourceOwnerRemember*(), and ResourceOwnerForget*() functions have
been replaced with three generic functions that take resource kind as
argument. For convenience, we still define resource-specific wrapper
macros around the generic functions with the old names, but they are
now defined in the source files that use those resource kinds.
The release callback no longer needs to call ResourceOwnerForget on
the resource being released. ResourceOwnerRelease unregisters the
resource from the owner before calling the callback. That needed some
changes in bufmgr.c and some other files, where releasing the
resources previously always called ResourceOwnerForget.
Each resource kind specifies a release priority, and
ResourceOwnerReleaseAll releases the resources in priority order. To
make that possible, we have to restrict what you can do between
phases. After calling ResourceOwnerRelease(), you are no longer
allowed to remember any more resources in it or to forget any
previously remembered resources by calling ResourceOwnerForget. There
was one case where that was done previously. At subtransaction commit,
AtEOSubXact_Inval() would handle the invalidation messages and call
RelationFlushRelation(), which temporarily increased the reference
count on the relation being flushed. We now switch to the parent
subtransaction's resource owner before calling AtEOSubXact_Inval(), so
that there is a valid ResourceOwner to temporarily hold that relcache
reference.
Other end-of-xact routines make similar calls to AtEOXact_Inval()
between release phases, but I didn't see any regression test failures
from those, so I'm not sure if they could reach a codepath that needs
remembering extra resources.
There were two exceptions to how the resource leak WARNINGs on commit
were printed previously: llvmjit silently released the context without
printing the warning, and a leaked buffer io triggered a PANIC. Now
everything prints a WARNING, including those cases.
Add tests in src/test/modules/test_resowner.
Reviewed-by: Aleksander Alekseev, Michael Paquier, Julien Rouhaud
Reviewed-by: Kyotaro Horiguchi, Hayato Kuroda, Álvaro Herrera, Zhihong Yu
Reviewed-by: Peter Eisentraut, Andres Freund
Discussion: https://www.postgresql.org/message-id/cbfabeb0-cd3c-e951-a572-19b365ed314d%40iki.fi
2 years ago
|
|
|
|
|
|
|
|
/* ResourceOwner callbacks */
|
|
|
|
|
|
|
|
|
|
static void
|
|
|
|
|
ResOwnerReleaseTupleDesc(Datum res)
|
|
|
|
|
{
|
|
|
|
|
TupleDesc tupdesc = (TupleDesc) DatumGetPointer(res);
|
|
|
|
|
|
|
|
|
|
/* Like DecrTupleDescRefCount, but don't call ResourceOwnerForget() */
|
|
|
|
|
Assert(tupdesc->tdrefcount > 0);
|
|
|
|
|
if (--tupdesc->tdrefcount == 0)
|
|
|
|
|
FreeTupleDesc(tupdesc);
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
static char *
|
|
|
|
|
ResOwnerPrintTupleDesc(Datum res)
|
|
|
|
|
{
|
|
|
|
|
TupleDesc tupdesc = (TupleDesc) DatumGetPointer(res);
|
|
|
|
|
|
|
|
|
|
return psprintf("TupleDesc %p (%u,%d)",
|
|
|
|
|
tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod);
|
|
|
|
|
}
|