postgres

Commit Graph

Author	SHA1	Message	Date
Heikki Linnakangas	87a350e1f2	Never store 0 as the nextMXact Before this commit, when multixid wraparound happens, MultiXactState->nextMXact goes to 0, which is invalid. All the readers need to deal with that possibility and skip over the 0. That's error-prone and we've missed it a few times in the past. This commit changes the responsibility so that all the writers of MultiXactState->nextMXact skip over the zero already, and readers can trust that it's never 0. We were already doing that for MultiXactState->oldestMultiXactId; none of its writers would set it to 0. ReadMultiXactIdRange() was nevertheless checking for that possibility. For clarity, remove that check. Reviewed-by: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com> Reviewed-by: Maxim Orlov <orlovmg@gmail.com> Discussion: https://www.postgresql.org/message-id/3624730d-6dae-42bf-9458-76c4c965fb27@iki.fi	5 days ago
Heikki Linnakangas	bd8d9c9bdf	Widen MultiXactOffset to 64 bits This eliminates MultiXactOffset wraparound and the 2^32 limit on the total number of multixid members. Multixids are still limited to 2^31, but this is a nice improvement because 'members' can grow much faster than the number of multixids. On such systems, you can now run longer before hitting hard limits or triggering anti-wraparound vacuums. Not having to deal with MultiXactOffset wraparound also simplifies the code and removes some gnarly corner cases. We no longer need to perform emergency anti-wraparound freezing because of running out of 'members' space, so the offset stop limit is gone. But you might still not want 'members' to consume huge amounts of disk space. For that reason, I kept the logic for lowering vacuum's multixid freezing cutoff if a large amount of 'members' space is used. The thresholds for that are roughly the same as the "safe" and "danger" thresholds used before, 2 billion transactions and 4 billion transactions. This keeps the behavior for the freeze cutoff roughly the same as before. It might make sense to make this smarter or configurable, now that the threshold is only needed to manage disk usage, but that's left for the future. Add code to pg_upgrade to convert multitransactions from the old to the new format, rewriting the pg_multixact SLRU files. Because pg_upgrade now rewrites the files, we can get rid of some hacks we had put in place to deal with old bugs and upgraded clusters. Bump catalog version for the pg_multixact/offsets format change. Author: Maxim Orlov <orlovmg@gmail.com> Reviewed-by: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com> Reviewed-by: Alexander Korotkov <aekorotkov@gmail.com> Reviewed-by: wenhui qiu <qiuwenhuifx@gmail.com> Discussion: https://www.postgresql.org/message-id/CACG%3DezaWg7_nt-8ey4aKv2w9LcuLthHknwCawmBgEeTnJrJTcw@mail.gmail.com	1 week ago
Heikki Linnakangas	636c1914b4	pg_resetwal: Reject negative and out of range arguments The strtoul() function that we used to parse many of the options accepts negative values, and silently wraps them to the equivalent unsigned values. For example, -1 becomes 0xFFFFFFFF, on platforms where unsigned long is 32 bits wide. Also, on platforms where "unsigned long" is 64 bits wide, we silently casted values larger than UINT32_MAX to the equivalent 32-bit value. Both of those behaviors seem undesirable, so tighten up the parsing to reject them. Reviewed-by: Chao Li <li.evan.chao@gmail.com> Discussion: https://www.postgresql.org/message-id/81adf5f3-36ad-4bcd-9ba5-1b95c7b7a807@iki.fi	1 week ago
Heikki Linnakangas	94939c5f3a	Fix pg_upgrade around multixid and mxoff wraparound pg_resetwal didn't accept multixid 0 or multixact offset UINT32_MAX, but they are both valid values that can appear in the control file. That caused pg_upgrade to fail if you tried to upgrade a cluster exactly at multixid or offset wraparound, because pg_upgrade calls pg_resetwal to restore multixid/offset on the new cluster to the values from the old cluster. To fix, allow those values in pg_resetwal. Fixes bugs #18863 and #18865 reported by Dmitry Kovalenko. Backpatch down to v15. Version 14 has the same bug, but the patch doesn't apply cleanly there. It could be made to work but it doesn't seem worth the effort given how rare it is to hit this problem with pg_upgrade, and how few people are upgrading to v14 anymore. Author: Maxim Orlov <orlovmg@gmail.com> Discussion: https://www.postgresql.org/message-id/CACG%3DezaApSMTjd%3DM2Sfn5Ucuggd3FG8Z8Qte8Xq9k5-%2BRQis-g@mail.gmail.com Discussion: https://www.postgresql.org/message-id/18863-72f08858855344a2@postgresql.org Discussion: https://www.postgresql.org/message-id/18865-d4c66cf35c2a67af@postgresql.org Backpatch-through: 15	1 month ago
Masahiko Sawada	30666d1857	pg_resetwal: Add --char-signedness option to change the default char signedness. With the newly added option --char-signedness, pg_resetwal updates the default char signedness flag in the controlfile. This option is primarily intended for an upcoming patch that pg_upgrade supports preserving the default char signedness during upgrades, and is not meant for manual operation. Reviewed-by: Noah Misch <noah@leadboat.com> Discussion: https://postgr.es/m/CB11ADBC-0C3F-4FE0-A678-666EE80CBB07%40amazon.com	10 months ago
Michael Paquier	ce1b0f9da0	Improve grammar of options for command arrays in TAP tests This commit rewrites a good chunk of the command arrays in TAP tests with a grammar based on the following rules: - Fat commas are used between option names and their values, making it clear to both humans and perltidy that values and names are bound together. This is particularly useful for the readability of multi-line command arrays, and there are plenty of them in the TAP tests. Most of the test code is updated to use this style. Some commands used parenthesis to show the link, or attached values and options in a single string. These are updated to use fat commas instead. - Option names are switched to use their long names, making them more self-documented. Based on a suggestion by Andrew Dunstan. - Add some trailing commas after the last item in multi-line arrays, which is a common perl style. Not all the places are taken care of, but this covers a very good chunk of them. Author: Dagfinn Ilmari Mannsåker Reviewed-by: Michael Paquier, Peter Smith, Euler Taveira Discussion: https://postgr.es/m/87jzc46d8u.fsf@wibble.ilmari.org	11 months ago
Bruce Momjian	50e6eb731d	Update copyright for 2025 Backpatch-through: 13	12 months ago
Bruce Momjian	29275b1d17	Update copyright for 2024 Reported-by: Michael Paquier Discussion: https://postgr.es/m/ZZKTDPxBBMt3C0J9@paquier.xyz Backpatch-through: 12	2 years ago
Peter Eisentraut	c538592959	Make all Perl warnings fatal There are a lot of Perl scripts in the tree, mostly code generation and TAP tests. Occasionally, these scripts produce warnings. These are probably always mistakes on the developer side (true positives). Typical examples are warnings from genbki.pl or related when you make a mess in the catalog files during development, or warnings from tests when they massage a config file that looks different on different hosts, or mistakes during merges (e.g., duplicate subroutine definitions), or just mistakes that weren't noticed because there is a lot of output in a verbose build. This changes all warnings into fatal errors, by replacing use warnings; by use warnings FATAL => 'all'; in all Perl files. Discussion: https://www.postgresql.org/message-id/flat/06f899fd-1826-05ab-42d6-adeb1fd5e200%40eisentraut.org	2 years ago
Peter Eisentraut	7b5275eec3	pg_resetwal: Add more tests and test coverage pg_resetwal had poor test coverage. There are some TAP tests, but they all run with -n, so they don't actually test the full functionality. (There is a non-dry-run call of pg_resetwal in the recovery test suite, but that is incidental.) This adds a bunch of more tests to test all the different options and scenarios. Reviewed-by: Aleksander Alekseev <aleksander@timescale.com> Discussion: https://www.postgresql.org/message-id/flat/0f3ab4a1-ae80-56e8-3426-6b4a02507687@eisentraut.org	2 years ago
Bruce Momjian	c8e1ba736b	Update copyright for 2023 Backpatch-through: 11	3 years ago
Daniel Gustafsson	549ec201d6	Replace Test::More plans with done_testing Rather than doing manual book keeping to plan the number of tests to run in each TAP suite, conclude each run with done_testing() summing up the the number of tests that ran. This removes the need for maintaning and updating the plan count at the expense of an accurate count of remaining during the test suite runtime. This patch has been discussed a number of times, often in the context of other patches which updates tests, so a larger number of discussions can be found in the archives. Reviewed-by: Julien Rouhaud <rjuju123@gmail.com> Reviewed-by: Dagfinn Ilmari Mannsåker <ilmari@ilmari.org> Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us> Reviewed-by: Michael Paquier <michael@paquier.xyz> Reviewed-by: Kyotaro Horiguchi <horikyota.ntt@gmail.com> Discussion: https://postgr.es/m/DD399313-3D56-4666-8079-88949DAC870F@yesql.se	4 years ago
Bruce Momjian	27b77ecf9f	Update copyright for 2022 Backpatch-through: 10	4 years ago
Andrew Dunstan	b3b4d8e68a	Move Perl test modules to a better namespace The five modules in our TAP test framework all had names in the top level namespace. This is unwise because, even though we're not exporting them to CPAN, the names can leak, for example if they are exported by the RPM build process. We therefore move the modules to the PostgreSQL::Test namespace. In the process PostgresNode is renamed to Cluster, and TestLib is renamed to Utils. PostgresVersion becomes simply PostgreSQL::Version, to avoid possible confusion about what it's the version of. Discussion: https://postgr.es/m/aede93a4-7d92-ef26-398f-5094944c2504@dunslane.net Reviewed by Erik Rijkers and Michael Paquier	4 years ago
Andrew Dunstan	201a76183e	Unify PostgresNode's new() and get_new_node() methods There is only one constructor now for PostgresNode, with the idiomatic name 'new'. The method is not exported by the class, and must be called as "PostgresNode->new('name',[args])". All the TAP tests that use PostgresNode are modified accordingly. Third party scripts will need adjusting, which is a fairly mechanical process (I just used a sed script).	4 years ago
Andrew Dunstan	8fa6e6919c	Add a copyright notice to perl files lacking one.	5 years ago
Tom Lane	f04d4ac919	Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com	8 years ago
Stephen Frost	da9b580d89	Refactor dir/file permissions Consolidate directory and file create permissions for tools which work with the PG data directory by adding a new module (common/file_perm.c) that contains variables (pg_file_create_mode, pg_dir_create_mode) and constants to initialize them (0600 for files and 0700 for directories). Convert mkdir() calls in the backend to MakePGDirectory() if the original call used default permissions (always the case for regular PG directories). Add tests to make sure permissions in PGDATA are set correctly by the tools which modify the PG data directory. Authors: David Steele <david@pgmasters.net>, Adam Brightwell <adam.brightwell@crunchydata.com> Reviewed-By: Michael Paquier, with discussion amongst many others. Discussion: https://postgr.es/m/ad346fe6-b23e-59f1-ecb7-0e08390ad629%40pgmasters.net	8 years ago
Peter Eisentraut	5700aa1301	pg_resetwal: Add simple test suite Some subsequent patches will add to this, but to avoid conflicts, set up the basics separately.	8 years ago

19 Commits (87a350e1f284bb99591f9185c0be0ae28899f38a)