open-dsi
/
postgres
mirror of https://github.com/postgres/postgres
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Teach genbki.pl to auto-generate pg_type entries for array types.

Browse Source 
This eliminates some more tedium in adding new catalog entries,
specifically the need to set up an array type when adding a new
built-in data type.  Now it's sufficient to assign an OID for the
array type and write it in an "array_type_oid" metadata field.
You don't have to fill the base type's typarray link explicitly, either.

No catversion bump since the contents of pg_type aren't changed.
(Well, their order might be different, but that doesn't matter.)

John Naylor, reviewed and whacked around a bit by
Dagfinn Ilmari Mannsåker, and some more by me.

Discussion: https://postgr.es/m/CAJVSVGVTb6m9pJF49b3SuA8J+T-THO9c0hxOmoyv-yGKh-FbNg@mail.gmail.com
pull/34/head
Tom Lane 7 years ago
parent 09e99ce86e
commit 3dc820c43e
7 changed files with 415 additions and 642 deletions
Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL
Show Stats Download Patch File Download Diff File
  1. 52
      doc/src/sgml/bki.sgml
  2. 78
      src/backend/catalog/Catalog.pm
  3. 2
      src/backend/catalog/genbki.pl
  4. 2
      src/include/catalog/genbki.h
  5. 864
      src/include/catalog/pg_type.dat
  6. 30
      src/include/catalog/pg_type.h
  7. 29
      src/include/catalog/reformat_dat_file.pl

52
doc/src/sgml/bki.sgml
Unescape View File

@ -217,12 +217,14 @@
<replaceable>value</replaceable> pairs. The
allowed <replaceable>key</replaceable>s are the names of the catalog's
columns, plus the metadata keys <literal>oid</literal>,
<literal>oid_symbol</literal>, and <literal>descr</literal>.
<literal>oid_symbol</literal>,
<literal>array_type_oid</literal>, and <literal>descr</literal>.
(The use of <literal>oid</literal> and <literal>oid_symbol</literal>
is described in <xref linkend="system-catalog-oid-assignment"/>
below. <literal>descr</literal> supplies a description string for
the object, which will be inserted
into <structname>pg_description</structname>
is described in <xref linkend="system-catalog-oid-assignment"/> below,
while <literal>array_type_oid</literal> is described in
<xref linkend="system-catalog-auto-array-types"/>.
<literal>descr</literal> supplies a description string for the object,
which will be inserted into <structname>pg_description</structname>
or <structname>pg_shdescription</structname> as appropriate.)
While the metadata keys are optional, the catalog's defined columns
must all be provided, except when the catalog's <literal>.h</literal>
@ -282,8 +284,9 @@
<para>
Within each pair of curly braces, the metadata
fields <literal>oid</literal>, <literal>oid_symbol</literal>,
and <literal>descr</literal> (if present) come first, in that
order, then the catalog's own fields appear in their defined order.
<literal>array_type_oid</literal>, and <literal>descr</literal>
(if present) come first, in that order, then the catalog's own
fields appear in their defined order.
</para>
</listitem>
@ -498,6 +501,41 @@
</para>
</sect2>
<sect2 id="system-catalog-auto-array-types">
<title>Automatic Creation of Array Types</title>
<para>
Most scalar data types should have a corresponding array type (that is,
a standard varlena array type whose element type is the scalar type, and
which is referenced by the <structfield>typarray</structfield> field of
the scalar type's <structname>pg_type</structname>
entry). <filename>genbki.pl</filename> is able to generate
the <structname>pg_type</structname> entry for the array type
automatically in most cases.
</para>
<para>
To use this facility, just write an <literal>array_type_oid
=&gt; <replaceable>nnnn</replaceable></literal> metadata field in the
scalar type's <structname>pg_type</structname> entry, specifying the OID
to use for the array type. You may then omit
the <structfield>typarray</structfield> field, since it will be filled
automatically with that OID.
</para>
<para>
The generated array type's name is the scalar type's name with an
underscore prepended. The array entry's other fields are filled from
<literal>BKI_ARRAY_DEFAULT(<replaceable>value</replaceable>)</literal>
annotations in <filename>pg_type.h</filename>, or if there isn't one,
copied from the scalar type. (There's also a special case
for <structfield>typalign</structfield>.) Then
the <structfield>typelem</structfield>
and <structfield>typarray</structfield> fields of the two entries are
set to cross-reference each other.
</para>
</sect2>
<sect2 id="system-catalog-recipes">
<title>Recipes for Editing Data Files</title>

78
src/backend/catalog/Catalog.pm
Unescape View File

@ -191,6 +191,11 @@ sub ParseHeader
{
$column{default} = $1;
}
elsif (
$attopt =~ /BKI_ARRAY_DEFAULT\(['"]?([^'"]+)['"]?\)/)
{
$column{array_default} = $1;
}
elsif ($attopt =~ /BKI_LOOKUP\((\w+)\)/)
{
$column{lookup} = $1;
@ -277,12 +282,16 @@ sub ParseData
}
}
# If we found a hash reference, keep it.
# Only keep non-data strings if we
# are told to preserve formatting.
# If we found a hash reference, keep it, unless it is marked as
# autogenerated; in that case it'd duplicate an entry we'll
# autogenerate below. (This makes it safe for reformat_dat_file.pl
# with --full-tuples to print autogenerated entries, which seems like
# useful behavior for debugging.)
#
# Only keep non-data strings if we are told to preserve formatting.
if (defined $hash_ref)
{
push @$data, $hash_ref;
push @$data, $hash_ref if !$hash_ref->{autogenerated};
}
elsif ($preserve_formatting)
{
@ -290,6 +299,10 @@ sub ParseData
}
}
close $ifd;
# If this is pg_type, auto-generate array types too.
GenerateArrayTypes($schema, $data) if $catname eq 'pg_type';
return $data;
}
@ -343,6 +356,63 @@ sub AddDefaultValues
}
}
# If a pg_type entry has an array_type_oid metadata field,
# auto-generate an entry for its array type.
sub GenerateArrayTypes
{
my $pgtype_schema = shift;
my $types = shift;
my @array_types;
foreach my $elem_type (@$types)
{
next if !(ref $elem_type eq 'HASH');
next if !defined($elem_type->{array_type_oid});
my %array_type;
# Set up metadata fields for array type.
$array_type{oid} = $elem_type->{array_type_oid};
$array_type{autogenerated} = 1;
$array_type{line_number} = $elem_type->{line_number};
# Set up column values derived from the element type.
$array_type{typname} = '_' . $elem_type->{typname};
$array_type{typelem} = $elem_type->{typname};
# Arrays require INT alignment, unless the element type requires
# DOUBLE alignment.
$array_type{typalign} = $elem_type->{typalign} eq 'd' ? 'd' : 'i';
# Fill in the rest of the array entry's fields.
foreach my $column (@$pgtype_schema)
{
my $attname = $column->{name};
# Skip if we already set it above.
next if defined $array_type{$attname};
# Apply the BKI_ARRAY_DEFAULT setting if there is one,
# otherwise copy the field from the element type.
if (defined $column->{array_default})
{
$array_type{$attname} = $column->{array_default};
}
else
{
$array_type{$attname} = $elem_type->{$attname};
}
}
# Lastly, cross-link the array to the element type.
$elem_type->{typarray} = $array_type{typname};
push @array_types, \%array_type;
}
push @$types, @array_types;
}
# Rename temporary files to final names.
# Call this function with the final file name and the .tmp extension.
#

2
src/backend/catalog/genbki.pl
Unescape View File

@ -394,7 +394,9 @@ EOM
next
if $key eq "oid"
|| $key eq "oid_symbol"
|| $key eq "array_type_oid"
|| $key eq "descr"
|| $key eq "autogenerated"
|| $key eq "line_number";
die sprintf "unrecognized field name \"%s\" in %s.dat line %s\n",
$key, $catname, $bki_values{line_number}

2
src/include/catalog/genbki.h
Unescape View File

@ -34,6 +34,8 @@
#define BKI_FORCE_NOT_NULL
/* Specifies a default value for a catalog field */
#define BKI_DEFAULT(value)
/* Specifies a default value for auto-generated array types */
#define BKI_ARRAY_DEFAULT(value)
/* Indicates how to perform name lookups for an OID or OID-array field */
#define BKI_LOOKUP(catalog)

864
src/include/catalog/pg_type.dat
View File

File diff suppressed because it is too large Load Diff

30
src/include/catalog/pg_type.h
Unescape View File

@ -52,7 +52,7 @@ CATALOG(pg_type,1247,TypeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(71,TypeRelati
* "varlena" type (one that has a length word), -2 to indicate a
* null-terminated C string.
*/
int16 typlen;
int16 typlen BKI_ARRAY_DEFAULT(-1);
/*
* typbyval determines whether internal Postgres routines pass a value of
@ -62,7 +62,7 @@ CATALOG(pg_type,1247,TypeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(71,TypeRelati
* typbyval can be false even if the length would allow pass-by-value; for
* example, type macaddr8 is pass-by-ref even when Datum is 8 bytes.
*/
bool typbyval;
bool typbyval BKI_ARRAY_DEFAULT(f);
/*
* typtype is 'b' for a base type, 'c' for a composite type (e.g., a
@ -71,7 +71,7 @@ CATALOG(pg_type,1247,TypeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(71,TypeRelati
*
* If typtype is 'c', typrelid is the OID of the class' entry in pg_class.
*/
char typtype BKI_DEFAULT(b);
char typtype BKI_DEFAULT(b) BKI_ARRAY_DEFAULT(b);
/*
* typcategory and typispreferred help the parser distinguish preferred
@ -81,10 +81,10 @@ CATALOG(pg_type,1247,TypeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(71,TypeRelati
*/
/* arbitrary type classification */
char typcategory;
char typcategory BKI_ARRAY_DEFAULT(A);
/* is type "preferred" within its category? */
bool typispreferred BKI_DEFAULT(f);
bool typispreferred BKI_DEFAULT(f) BKI_ARRAY_DEFAULT(f);
/*
* If typisdefined is false, the entry is only a placeholder (forward
@ -97,7 +97,7 @@ CATALOG(pg_type,1247,TypeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(71,TypeRelati
char typdelim BKI_DEFAULT(',');
/* associated pg_class OID if a composite type, else 0 */
Oid typrelid BKI_DEFAULT(0);
Oid typrelid BKI_DEFAULT(0) BKI_ARRAY_DEFAULT(0);
/*
* If typelem is not 0 then it identifies another row in pg_type. The
@ -116,19 +116,19 @@ CATALOG(pg_type,1247,TypeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(71,TypeRelati
* If there is a "true" array type having this type as element type,
* typarray links to it. Zero if no associated "true" array type.
*/
Oid typarray BKI_DEFAULT(0) BKI_LOOKUP(pg_type);
Oid typarray BKI_DEFAULT(0) BKI_ARRAY_DEFAULT(0) BKI_LOOKUP(pg_type);
/*
* I/O conversion procedures for the datatype.
*/
/* text format (required) */
regproc typinput BKI_LOOKUP(pg_proc);
regproc typoutput BKI_LOOKUP(pg_proc);
regproc typinput BKI_ARRAY_DEFAULT(array_in) BKI_LOOKUP(pg_proc);
regproc typoutput BKI_ARRAY_DEFAULT(array_out) BKI_LOOKUP(pg_proc);
/* binary format (optional) */
regproc typreceive BKI_LOOKUP(pg_proc);
regproc typsend BKI_LOOKUP(pg_proc);
regproc typreceive BKI_ARRAY_DEFAULT(array_recv) BKI_LOOKUP(pg_proc);
regproc typsend BKI_ARRAY_DEFAULT(array_send) BKI_LOOKUP(pg_proc);
/*
* I/O functions for optional type modifiers.
@ -139,7 +139,7 @@ CATALOG(pg_type,1247,TypeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(71,TypeRelati
/*
* Custom ANALYZE procedure for the datatype (0 selects the default).
*/
regproc typanalyze BKI_DEFAULT(-) BKI_LOOKUP(pg_proc);
regproc typanalyze BKI_DEFAULT(-) BKI_ARRAY_DEFAULT(array_typanalyze) BKI_LOOKUP(pg_proc);
/* ----------------
* typalign is the alignment required when storing a value of this
@ -177,7 +177,7 @@ CATALOG(pg_type,1247,TypeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(71,TypeRelati
* 'm' MAIN like 'x' but try to keep in main tuple
* ----------------
*/
char typstorage BKI_DEFAULT(p);
char typstorage BKI_DEFAULT(p) BKI_ARRAY_DEFAULT(x);
/*
* This flag represents a "NOT NULL" constraint against this datatype.
@ -221,7 +221,7 @@ CATALOG(pg_type,1247,TypeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(71,TypeRelati
* a default expression for the type. Currently this is only used for
* domains.
*/
pg_node_tree typdefaultbin BKI_DEFAULT(_null_);
pg_node_tree typdefaultbin BKI_DEFAULT(_null_) BKI_ARRAY_DEFAULT(_null_);
/*
* typdefault is NULL if the type has no associated default value. If
@ -231,7 +231,7 @@ CATALOG(pg_type,1247,TypeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(71,TypeRelati
* external representation of the type's default value, which may be fed
* to the type's input converter to produce a constant.
*/
text typdefault BKI_DEFAULT(_null_);
text typdefault BKI_DEFAULT(_null_) BKI_ARRAY_DEFAULT(_null_);
/*
* Access permissions

29
src/include/catalog/reformat_dat_file.pl
Unescape View File

@ -5,9 +5,9 @@
# Perl script that reads in catalog data file(s) and writes out
# functionally equivalent file(s) in a standard format.
#
# In each entry of a reformatted file, metadata fields (if any) come
# first, with normal attributes starting on the following line, in
# the same order as the columns of the corresponding catalog.
# In each entry of a reformatted file, metadata fields (if present)
# come first, with normal attributes starting on the following line,
# in the same order as the columns of the corresponding catalog.
# Comments and blank lines are preserved.
#
# Portions Copyright (c) 1996-2018, PostgreSQL Global Development Group
@ -26,6 +26,11 @@ use FindBin;
use lib "$FindBin::RealBin/../../backend/catalog/";
use Catalog;
# Names of the metadata fields of a catalog entry. (line_number is also
# a metadata field, but we never write it out, so it's not listed here.)
my @METADATA =
('oid', 'oid_symbol', 'array_type_oid', 'descr', 'autogenerated');
my @input_files;
my $output_path = '';
my $full_tuples = 0;
@ -62,9 +67,6 @@ if ($output_path ne '' && substr($output_path, -1) ne '/')
$output_path .= '/';
}
# Metadata of a catalog entry
my @METADATA = ('oid', 'oid_symbol', 'descr');
# Read all the input files into internal data structures.
# We pass data file names as arguments and then look for matching
# headers to parse the schema from.
@ -142,6 +144,9 @@ foreach my $catname (@catnames)
if (!$full_tuples)
{
# If it's an autogenerated entry, drop it completely.
next if $values{autogenerated};
# Else, just drop any default/computed fields.
strip_default_values(\%values, $schema, $catname);
}
@ -162,10 +167,6 @@ foreach my $catname (@catnames)
print $dat " },\n";
}
# Strings -- handle accordingly or ignore. It was necessary to
# ignore bare commas during the initial data conversion. This
# should be a no-op now, but we may as well keep that behavior.
# Preserve blank lines.
elsif ($data =~ /^\s*$/)
{
@ -207,6 +208,14 @@ sub strip_default_values
{
delete $row->{pronargs} if defined $row->{proargtypes};
}
# If a pg_type entry has an auto-generated array type, then its
# typarray field is a computed value too (see GenerateArrayTypes).
if ($catname eq 'pg_type')
{
delete $row->{typarray} if defined $row->{array_type_oid};
}
return;
}

Write Preview
Loading…
Cancel
Save