postgres

Commit Graph

Author	SHA1	Message	Date
Robert Haas	0fbfd37cef	Allow extensions to mark an individual index as disabled. Up until now, the only way for a loadable module to disable the use of a particular index was to use build_simple_rel_hook (or, previous to yesterday's commit, get_relation_info_hook) to remove it from the index list. While that works, it has some disadvantages. First, the index becomes invisible for all purposes, and can no longer be used for optimizations such as self-join elimination or left join removal, which can severely degrade the resulting plan. Second, if the module attempts to compel the use of a certain index by removing all other indexes from the index list and disabling other scan types, but the planner is unable to use the chosen index for some reason, it will fall back to a sequential scan, because that is only disabled, whereas the other indexes are, from the planner's point of view, completely gone. While this situation ideally shouldn't occur, it's hard for a loadable module to be completely sure whether the planner will view a certain index as usable for a certain query. If it isn't, it may be better to fall back to a scan using a disabled index rather than falling back to an also-disabled sequential scan. Reviewed-by: Alexandra Wang <alexandra.wang.oss@gmail.com> Discussion: http://postgr.es/m/CA%2BTgmoYS4ZCVAF2jTce%3DbMP0Oq_db_srocR4cZyO0OBp9oUoGg%40mail.gmail.com	2 days ago
Michael Paquier	03facc1211	Switch to FATAL error for missing checkpoint record without backup_label Crash recovery started without a backup_label previously crashed with a PANIC if the checkpoint record could not be found. This commit lowers the report generated to be a FATAL instead. With recovery methods being more imaginative these days, this should provide more flexibility when handling PostgreSQL recovery processing in the event of a driver error, similarly to `15f68cebdc`. An extra benefit of this change is that it becomes possible to add a test to check that a FATAL is hit with an expected error message pattern. With the recovery code becoming more complicated over the last couple of years, I suspect that this will be benefitial to cover in the long-term. The original PANIC behavior has been introduced in the early days of crash recovery, as of `4d14fe0048` (PANIC did not exist yet, the code used STOP). Author: Nitin Jadhav <nitinjadhavpostgres@gmail.com> Discussion: https://postgr.es/m/CAMm1aWZbQ-Acp_xAxC7mX9uZZMH8+NpfepY9w=AOxbBVT9E=uA@mail.gmail.com	2 days ago
Michael Paquier	6307b096e2	Fix misuse of "volatile" in xml.c What should be used is not "volatile foo ptr" but "foo volatile ptr", The incorrect (former) style means that what the pointer variable points to is volatile. The correct (latter) style means that the pointer variable itself needs to be treated as volatile. The latter style is required to ensure a consistent treatment of these variables after a longjmp with the TRY/CATCH blocks. Some casts can be removed thanks to this change. Issue introduced by `2e94721747`, so no backpatch is required. A similar set of issues has been fixed in `93001888d8` for contrib/xml2/. Author: ChangAo Chen <cca5507@qq.com> Discussion: https://postgr.es/m/tencent_5BE8DAD985EE140ED62EA728C8D4E1311F0A@qq.com	2 days ago
Nathan Bossart	7c8280eeb5	pg_{dump,restore}: Refactor handling of conflicting options. This commit makes use of the function added by commit `b2898baaf7` for these applications' handling of conflicting options. It doesn't fix any bugs, but it does trim several lines of code. Author: Jian He <jian.universality@gmail.com> Reviewed-by: Steven Niu <niushiji@gmail.com> Reviewed-by: Zsolt Parragi <zsolt.parragi@percona.com> Discussion: https://postgr.es/m/CACJufxHDYn%2B3-2jR_kwYB0U7UrNP%2B0EPvAWzBBD5EfUzzr1uiw%40mail.gmail.com	3 days ago
Robert Haas	91f33a2ae9	Replace get_relation_info_hook with build_simple_rel_hook. For a long time, PostgreSQL has had a get_relation_info_hook which plugins can use to editorialize on the information that get_relation_info obtains from the catalogs. However, this hook is only called for baserels of type RTE_RELATION, and there is potential utility in a similar call back for other types of RTEs. This might have had utility even before commit `4020b370f2` added pgs_mask to RelOptInfo, but it certainly has utility now. So, move the callback up one level, deleting get_relation_info_hook and adding build_simple_rel_hook instead. The new callback is called just slightly later than before and with slightly different arguments, but it should be fairly straightforward to adjust existing code that currently uses get_relation_info_hook: the values previously available as relationObjectId and inhparent are now available via rte->relid and rte->inh, and calls where rte->rtekind != RTE_RELATION can be ignored if desired. Reviewed-by: Alexandra Wang <alexandra.wang.oss@gmail.com> Discussion: http://postgr.es/m/CA%2BTgmoYg8uUWyco7Pb3HYLMBRQoO6Zh9hwgm27V39Pb6Pdf%3Dug%40mail.gmail.com	3 days ago
Robert Haas	8300d3ad4a	Consider startup cost as a figure of merit for partial paths. Previously, the comments stated that there was no purpose to considering startup cost for partial paths, but this is not the case: it's perfectly reasonable to want a fast-start path for a plan that involves a LIMIT (perhaps over an aggregate, so that there is enough data being processed to justify parallel query but yet we don't want all the result rows). Accordingly, rewrite add_partial_path and add_partial_path_precheck to consider startup costs. This also fixes an independent bug in add_partial_path_precheck: commit `e222534679` failed to update it to do anything with the new disabled_nodes field. That bug fix is formally separate from the rest of this patch and could be committed separately, but I think it makes more sense to fix both issues together, because then we can (as this commit does) just make add_partial_path_precheck do the cost comparisons in the same way as compare_path_costs_fuzzily, which hopefully reduces the chances of ending up with something that's still incorrect. This patch is based on earlier work on this topic by Tomas Vondra, but I have rewritten a great deal of it. Co-authored-by: Robert Haas <rhaas@postgresql.org> Co-authored-by: Tomas Vondra <tomas@vondra.me> Discussion: http://postgr.es/m/CA+TgmobRufbUSksBoxytGJS1P+mQY4rWctCk-d0iAUO6-k9Wrg@mail.gmail.com	3 days ago
Robert Haas	ffc226ab64	Prevent restore of incremental backup from bloating VM fork. When I (rhaas) wrote the WAL summarizer code, I incorrectly believed that XLOG_SMGR_TRUNCATE truncates all forks to the same length. In fact, what other parts of the code do is compute the truncation length for the FSM and VM forks from the truncation length used for the main fork. But, because I was confused, I coded the WAL summarizer to set the limit block for the VM fork to the same value as for the main fork. (Incremental backup always copies FSM forks in full, so there is no similar issue in that case.) Doing that doesn't directly cause any data corruption, as far as I can see. However, it does create a serious risk of consuming a large amount of extra disk space, because pg_combinebackup's reconstruct.c believes that the reconstructed file should always be at least as long as the limit block value. We might want to be smarter about that at some point in the future, because it's always safe to omit all-zeroes blocks at the end of the last segment of a relation, and doing so could save disk space, but the current algorithm will rarely waste enough disk space to worry about unless we believe that a relation has been truncated to a length much longer than its actual length on disk, which is exactly what happens as a result of the problem mentioned in the previous paragraph. To fix, create a new visibilitymap helper function and use it to include the right limit block in the summary files. Incremental backups taken with existing summary files will still have this issue, but this should improve the situation going forward. Diagnosed-by: Oleg Tkachenko <oatkachenko@gmail.com> Diagnosed-by: Amul Sul <sulamul@gmail.com> Discussion: http://postgr.es/m/CAAJ_b97PqG89hvPNJ8cGwmk94gJ9KOf_pLsowUyQGZgJY32o9g@mail.gmail.com Discussion: http://postgr.es/m/6897DAF7-B699-41BF-A6FB-B818FCFFD585%40gmail.com Backpatch-through: 17	3 days ago
Amit Kapila	06d8302262	Remove trailing period from errmsg in subscriptioncmds.c. Author: Sahitya Chandra <sahityajb@gmail.com> Discussion: https://postgr.es/m/20260308142806.181309-1-sahityajb@gmail.com	3 days ago
Peter Eisentraut	2799e29fb8	Move comment back to better place Commit `f014b1b9bb` inserted some new code in between existing code and a trailing comment. Move the comment back to near the code it belongs to.	3 days ago
Fujii Masao	173aa8c5e8	doc: Document IF NOT EXISTS option for ALTER FOREIGN TABLE ADD COLUMN. Commit `2cd40adb85` added the IF NOT EXISTS option to ALTER TABLE ADD COLUMN. This also enabled IF NOT EXISTS for ALTER FOREIGN TABLE ADD COLUMN, but the ALTER FOREIGN TABLE documentation was not updated to mention it. This commit updates the documentation to describe the IF NOT EXISTS option for ALTER FOREIGN TABLE ADD COLUMN. While updating that section, also this commit clarifies that the COLUMN keyword is optional in ALTER FOREIGN TABLE ADD/DROP COLUMN. Previously, part of the documentation could be read as if COLUMN were required. This commit adds regression tests covering these ALTER FOREIGN TABLE syntaxes. Backpatch to all supported versions. Suggested-by: Fujii Masao <masao.fujii@gmail.com> Author: Chao Li <lic@highgo.com> Reviewed-by: Robert Treat <rob@xzilla.net> Reviewed-by: Fujii Masao <masao.fujii@gmail.com> Discussion: https://postgr.es/m/CAHGQGwFk=rrhrwGwPtQxBesbT4DzSZ86Q3ftcwCu3AR5bOiXLw@mail.gmail.com Backpatch-through: 14	3 days ago
Michael Paquier	4da2afd01f	Fix size underestimation of DSA pagemap for odd-sized segments When make_new_segment() creates an odd-sized segment, the pagemap was only sized based on a number of usable_pages entries, forgetting that a segment also contains metadata pages, and that the FreePageManager uses absolute page indices that cover the entire segment. This miscalculation could cause accesses to pagemap entries to be out of bounds. During subsequent reuse of the allocated segment, allocations landing on pages with indices higher than usable_pages could cause out-of-bounds pagemap reads and/or writes. On write, 'span' pointers are stored into the data area, corrupting the allocated objects. On read (aka during a dsa_free), garbage is interpreted as a span pointer, typically crashing the server in dsa_get_address(). The normal geometric path correctly sizes the pagemap for all pages in the segment. The odd-sized path needs to do the same, but it works forward from usable_pages rather than backward from total_size. This commit fixes the sizing of the odd-sized case by adding pagemap entries for the metadata pages after the initial metadata_bytes calculation, using an integer ceiling division to compute the exact number of additional entries needed in one go, avoiding any iteration in the calculation. An assertion is added in the code path for odd-sized segments, ensuring that the pagemap includes the metadata area, and that the result is appropriately sized. This problem would show up depending on the size requested for the allocation of a DSA segment. The reporter has noticed this issue when a parallel hash join makes a DSA allocation large enough to trigger the odd-sized segment path, but it could happen for anything that does a DSA allocation. A regression test is added to test_dsa, down to v17 where the test module has been introduced. This adds a set of cheap tests to check the problem, the new assertion being useful for this purpose. Sami has proposed a test that took a longer time than what I have done here; the test committed is faster and good enough to check the odd-sized allocation path. Author: Paul Bunn <paul.bunn@icloud.com> Reviewed-by: Sami Imseih <samimseih@gmail.com> Reviewed-by: Chao Li <li.evan.chao@gmail.com> Reviewed-by: Michael Paquier <michael@paquier.xyz> Discussion: https://postgr.es/m/044401dcabac$fe432490$fac96db0$@icloud.com Backpatch-through: 14	3 days ago
Michael Paquier	ccd7abaa45	Refactor tests for catalog diff comparisons in stats_import.sql The tests of stats_import.sql include a set of queries to do differential checks of the three statistics catalog relations, based on the comparison of a source relation and a target relation, used for the copy of the stats data with the restore functions: - pg_statistic - pg_stats_ext - pg_stats_ext_exprs This commit refactors the tests to reduce the bloat of such differential queries, by creating a set of objects that make the differential queries smaller: - View for a base relation type. - First function to retrieve stats data, that returns a type based on the view previously created. - Second function that checks the difference, based on two calls of the first function. This change leads to a nice reduction of stats_import.sql, with a larger effect on the output file. While on it, this adds some sanity checks for the three catalogs, to warn developers that the stats import facilities may need to be updated if any of the three catalogs change. These are rare in practice, see `918eee0c49` as one example. Another stylistic change is the use of the extended output format for the differential queries, so as we avoid long lines of output if a diff is caught. Author: Corey Huinker <corey.huinker@gmail.com> Discussion: https://postgr.es/m/CADkLM=eEhxJpSUP+eC=eMGZZsVOpnfKDvVkuCbsFg9CajYwDsA@mail.gmail.com	3 days ago
Michael Paquier	9e8193a262	Fix typo in stats_import.sql The test mentioned pg_stat_ext_exprs, but the correct catalog name is pg_stats_ext_exprs. Thinko in `ba97bf9cb7`. Discussion: https://postgr.es/m/CADkLM=eEhxJpSUP+eC=eMGZZsVOpnfKDvVkuCbsFg9CajYwDsA@mail.gmail.com	3 days ago
Álvaro Herrera	eb2c867b0a	Fix invalid boolean if-test We were testing the truth value of the array of booleans (which is always true) instead of the boolean element specific to the affected table column. This causes a binary-upgrade dump fail to omit the name of a constraint; that is, the correct constraint name is always printed, even when it's not needed. The affected case is a binary-upgrade dump of a not-null constraint in an inherited column, which must in addition have no comment. Another point is that in order for this to make a difference, the constraint must have the default name in the child table. That is, the constraint must have been created _in the parent table_ with the name that it would have in the child table, like so: CREATE TABLE parent (a int CONSTRAINT child_a_not_null NOT NULL); CREATE TABLE child () INHERITS (parent); Otherwise, the correct name must be printed by binary-upgrade pg_dump anyway, since it wouldn't match the name produced at the parent. Moreover, when it does hit, the pre-18-compatibility code (which has to work with a constraint that has no name) gets involved and uses the UPDATE on pg_constraint using the conkey instead of column name ... and so everything ends up working correctly AFAICS. I think it might cause a problem if the table and column names are overly long, but I didn't want to spend time investigating further. Still, it's wrong code, and static analyzers have twice complained about it, so fix it by adding the array index accessor that was obviously meant. Reported-by: Ranier Vilela <ranier.vf@gmail.com> Reported-by: George Tarasov <george.v.tarasov@gmail.com> Backpatch-through: 18 Discussion: https://postgr.es/m/CAEudQAo7ah=4TDheuEjtb0dsv6bHoK7uBNqv53Tsub2h-xBSJw@mail.gmail.com Discussion: https://postgr.es/m/f3029f25-acc9-4cb9-a74f-fe93bcfb3a27@gmail.com	5 days ago
Jacob Champion	e982331b52	libpq: Introduce PQAUTHDATA_OAUTH_BEARER_TOKEN_V2 For the libpq-oauth module to eventually make use of the PGoauthBearerRequest API, it needs some additional functionality: the derived Issuer ID for the authorization server needs to be provided, and error messages need to be built without relying on PGconn internals. These features seem useful for application hooks, too, so that they don't each have to reinvent the wheel. The original plan was for additions to PGoauthBearerRequest to be made without a version bump to the PGauthData type. Applications would simply check a LIBPQ_HAS_* macro at compile time to decide whether they could use the new features. That theoretically works for applications linked against libpq, since it's not safe to downgrade libpq from the version you've compiled against. We've since found that this strategy won't work for plugins, due to a complication first noticed during the libpq-oauth module split: it's normal for a plugin on disk to be newer than the libpq that's loading it, because you might have upgraded your installation while an application was running. (In other words, a plugin architecture causes the compile-time and run-time dependency arrows to point in opposite directions, so plugins won't be able to rely on the LIBPQ_HAS_* macros to determine what APIs are available to them.) Instead, extend the original PGoauthBearerRequest (now retroactively referred to as "v1" in the code) with a v2 subclass-style struct. When an application implements and accepts PQAUTHDATA_OAUTH_BEARER_TOKEN_V2, it may safely cast the base request pointer it receives in its callbacks to v2 in order to make use of the new functionality. libpq will query the application for a v2 hook first, then v1 to maintain backwards compatibility, before giving up and using the builtin flow. Reviewed-by: Chao Li <li.evan.chao@gmail.com> Reviewed-by: Zsolt Parragi <zsolt.parragi@percona.com> Discussion: https://postgr.es/m/CAOYmi%2BmrGg%2Bn_X2MOLgeWcj3v_M00gR8uz_D7mM8z%3DdX1JYVbg%40mail.gmail.com	6 days ago
Nathan Bossart	b2898baaf7	pg_dumpall: Fix handling of conflicting options. pg_dumpall is missing checks for some conflicting options, including those passed through to pg_dump. To fix, introduce a new function that checks whether mutually exclusive options are set, and use that in pg_dumpall. A similar change could likely be made for pg_dump and pg_restore, but that is left as a future exercise. This is arguably a bug fix, but since this might break existing scripts, no back-patch for now. Author: Jian He <jian.universality@gmail.com> Co-authored-by: Nathan Bossart <nathandbossart@gmail.com> Reviewed-by: Wang Peng <215722532@qq.com> Reviewed-by: Zsolt Parragi <zsolt.parragi@percona.com> Reviewed-by: Chao Li <li.evan.chao@gmail.com> Discussion: https://postgr.es/m/CACJufxFf5%3DwSv2MsuO8iZOvpLZQ1-meAMwhw7JX5gNvWo5PDug%40mail.gmail.com	6 days ago
Masahiko Sawada	50ea4e09b6	Use palloc_object() and palloc_array() in more areas of the logical replication. The idea is to encourage the use of newer routines across the tree, as these offer stronger type-safety guarantees than raw palloc(). Similar work has been done in commits `1b105f9472`, `0c3c5c3b06`, `31d3847a37`, and `4f7dacc5b8`. This commit extends those changes to more locations within src/backend/replication/logical/. Author: Peter Smith <smithpb2250@gmail.com> Reviewed-by: Chao Li <li.evan.chao@gmail.com> Reviewed-by: Hayato Kuroda <kuroda.hayato@fujitsu.com> Reviewed-by: Masahiko Sawada <sawada.mshk@gmail.com> Discussion: https://postgr.es/m/CAHut+Pv4N7Vpxo18+NAR1r9RGvR8b0BtwTkoeCE2PfFoXgmR6A@mail.gmail.com	6 days ago
Tom Lane	415100aa62	Support grouping-expression references and GROUPING() in subqueries. Until now, substitute_grouped_columns and its predecessor check_ungrouped_columns intentionally did not cope with references to GROUP BY expressions (anything more complex than a Var) within subqueries of the query having GROUP BY. Because they didn't try to match subexpressions of subqueries to the GROUP BY list, they'd drill down to raw Vars of the grouping level and then fail with "subquery uses ungrouped column from outer query". There have been remarkably few complaints about this deficiency, so nobody ever did anything about it. The reason for not wanting to deal with it is that within a subquery, Vars will have varlevelsup different from zero and will thus not be equal() to the expressions seen in the outer query. We recognized this at least as far back as `96ca8ffeb`, although I think the comment I added about it then was just documenting a pre-existing deficiency. It looks like at the time, the solutions I considered were (1) write a version of equal() that permits an offset in varlevelsup, or (2) dynamically apply IncrementVarSublevelsUp at each subexpression. (1) would require an amount of new code that seems rather out of proportion to the benefit, while (2) would add an exponential amount of cost to the matching process. But rethinking it now, what seems attractive is (3) apply IncrementVarSublevelsUp to the groupingClause list not the subexpressions, and do so only once per subquery depth level. Then we can still use plain equal() to check for matches, and we're not incurring cost proportional to some power of the subquery's complexity. This patch continues to use the old logic when the GROUP BY list is all Vars. We could discard the special comparison logic for that and always do it the more general way, but that would be a good deal slower. (Micro-benchmarking just parse analysis suggests it's about 50% slower than the Vars-only path. But we've not heard complaints about the speed of matching within the main query, so I doubt that applying the same matching logic within subqueries will be a problem.) The lack of complaints suggests strongly that this is a very minority use-case, so I don't want to make the typical case slower to fix it. While testing that, I was surprised to discover a nearby bug: GROUPING() within a subquery fails to match GROUP BY Vars that are join alias Vars. It tries to apply flatten_join_alias_vars to make such cases work, but that fails to work inside a subquery because varlevelsup is wrong. Therefore, this patch invents a new entry point flatten_join_alias_for_parser() that allows specification of a sublevels_up offset. (It seems cleaner to give the parser its own entry point rather than abuse the planner's conventions even further.) While this is pretty clearly a bug fix, I'm hesitant to take the risk of back-patching, seeing that the existing behavior has stood for so long with so few complaints. Maybe we can reconsider once this patch has baked awhile in master. Reported-by: PALAYRET Jacques <jacques.palayret@meteo.fr> Author: Tom Lane <tgl@sss.pgh.pa.us> Discussion: https://postgr.es/m/531183.1772058731@sss.pgh.pa.us	6 days ago
Jeff Davis	8185bb5347	CREATE SUBSCRIPTION ... SERVER. Allow CREATE SUBSCRIPTION to accept a foreign server using the SERVER clause instead of a raw connection string using the CONNECTION clause. * Enables a user with sufficient privileges to create a subscription using a foreign server by name without specifying the connection details. * Integrates with user mappings (and other FDW infrastructure) using the subscription owner. * Provides a layer of indirection to manage multiple subscriptions to the same remote server more easily. Also add CREATE FOREIGN DATA WRAPPER ... CONNECTION clause to specify a connection_function. To be eligible for a subscription, the foreign server's foreign data wrapper must specify a connection_function. Add connection_function support to postgres_fdw, and bump postgres_fdw version to 1.3. Bump catversion. Reviewed-by: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com> Reviewed-by: Shlok Kyal <shlok.kyal.oss@gmail.com> Reviewed-by: Masahiko Sawada <sawada.mshk@gmail.com> Reviewed-by: Amit Kapila <amit.kapila16@gmail.com> Discussion: https://postgr.es/m/61831790a0a937038f78ce09f8dd4cef7de7456a.camel@j-davis.com	6 days ago
Álvaro Herrera	868825aaeb	Don't include wait_event.h in pgstat.h wait_event.h itself includes wait_event_types.h, which is a generated file, so it's nice that we can avoid compiling >10% of the tree just because that file is regenerated. To avoid breaking too many third-party modules, we now #include utils/wait_classes.h in storage/latch.h. Then, the very common case of doing WaitLatch(..., PG_WAIT_EXTENSION) continues to work by including just storage/latch.h. (I didn't try to determine how many modules would actually break if we don't do this, but this seems a convenient and low-impact measure.) Reviewed-by: Andres Freund <andres@anarazel.de> Discussion: https://postgr.es/m/202602181214.gcmhx2vhlxzp@alvherre.pgsql	6 days ago
Peter Eisentraut	90ca7c1429	doc: Fix capitalization of Unicode Reviewed-by: Chao Li <li.evan.chao@gmail.com> Discussion: https://www.postgresql.org/message-id/flat/2a668979-ed92-49a3-abf9-a3ec2d460ec2%40eisentraut.org	6 days ago
Peter Eisentraut	16686a853f	Fix Python deprecation warning Starting with Python 3.14, contrib/unaccent/generate_unaccent_rules.py complains DeprecationWarning: codecs.open() is deprecated. Use open() instead. This makes that change. This works for all Python 3.x versions. Reviewed-by: Chao Li <li.evan.chao@gmail.com> Discussion: https://www.postgresql.org/message-id/flat/2a668979-ed92-49a3-abf9-a3ec2d460ec2%40eisentraut.org	6 days ago
Peter Eisentraut	258248d0bd	Make unconstify and unvolatize use StaticAssertVariableIsOfTypeMacro The unconstify and unvolatize macros had an almost identical assertion as was already defined in StaticAssertVariableIsOfTypeMacro, only it had a less useful error message and didn't have a sizeof fallback. Author: Jelte Fennema-Nio <postgres@jeltef.nl> Discussion: https://www.postgresql.org/message-id/flat/CAGECzQR21OnnKiZO_1rLWO0-16kg1JBxnVq-wymYW0-_1cUNtg@mail.gmail.com	6 days ago
Peter Eisentraut	e2308350c9	Use typeof everywhere instead of compiler specific spellings We define typeof ourselves as __typeof__ if it does not exist. So let's actually use that for consistency. The meson/autoconf checks for __builtin_types_compatible_p still use __typeof__ though, because there we have not redefined it. Author: Jelte Fennema-Nio <postgres@jeltef.nl> Discussion: https://www.postgresql.org/message-id/flat/CAGECzQR21OnnKiZO_1rLWO0-16kg1JBxnVq-wymYW0-_1cUNtg@mail.gmail.com	6 days ago
Peter Eisentraut	aa7c868523	Portable StaticAssertExpr Use a different way to write StaticAssertExpr() that does not require the GCC extension statement expressions. For C, we put the static_assert into a struct. This appears to be a common approach. We still need to keep the fallback implementation to support buggy MSVC < 19.33. For C++, we put it into a lambda expression. (The C approach doesn't work; it's not permitted to define a new type inside sizeof.) Reviewed-by: Jelte Fennema-Nio <postgres@jeltef.nl> Discussion: https://www.postgresql.org/message-id/flat/5fa3a9f5-eb9a-4408-9baf-403d281f8b10%40eisentraut.org	6 days ago
Fujii Masao	6eedb2a5fd	Fix publisher shutdown hang caused by logical walsender busy loop. Previously, when logical replication was running, shutting down the publisher could cause the logical walsender to enter a busy loop and prevent the publisher from completing shutdown. During shutdown, the logical walsender waits for all pending WAL to be written out. However, some WAL records could remain unflushed, causing the walsender to wait indefinitely. The issue occurred because the walsender used XLogBackgroundFlush() to flush pending WAL. This function does not guarantee that all WAL is written. For example, WAL generated by a transaction without an assigned transaction ID that aborts might not be flushed. This commit fixes the bug by making the logical walsender call XLogFlush() instead, ensuring that all pending WAL is written and preventing the busy loop during shutdown. Backpatch to all supported versions. Author: Anthonin Bonnefoy <anthonin.bonnefoy@datadoghq.com> Reviewed-by: Alexander Lakhin <exclusion@gmail.com> Reviewed-by: Fujii Masao <masao.fujii@gmail.com> Discussion: https://postgr.es/m/CAO6_Xqo3co3BuUVEVzkaBVw9LidBgeeQ_2hfxeLMQcXwovB3GQ@mail.gmail.com Backpatch-through: 14	6 days ago
Fujii Masao	2007df4333	Improve tests for recovery_target_timeline GUC. Commit `fd7d7b7191` added regression tests to verify recovery_target_timeline settings. To confirm that invalid values are rejected, those tests started the server with an invalid setting and then verified that startup failed. While functionally correct, this approach was expensive because it required setting up and starting the server for each test case. This commit updates the tests for recovery_target_timeline to use the simpler approach introduced by commit `bffd7130` for recovery_target_xid, using ALTER SYSTEM SET to verify that invalid settings are rejected. This avoids the need to set up and start the server when checking invalid recovery_target_timeline values. Author: David Steele <david@pgbackrest.org> Reviewed-by: Fujii Masao <masao.fujii@gmail.com> Discussion: https://postgr.es/m/CAHGQGwG44vZbSoBmg076G+xkR6n=Tj2=q+fVkfP7yEsyF1daFA@mail.gmail.com	6 days ago
Michael Paquier	d5ea206728	Fix inconsistency with HeapTuple freeing in extended_stats_funcs.c heap_freetuple() is a thin wrapper doing a pfree(), and the function import_pg_statistic(), introduced by `ba97bf9cb7`, had the idea to call directly pfree() rather than the "dedicated" heap tuple routine. upsert_pg_statistic_ext_data already uses heap_freetuple(). This code is harmless as-is, but let's be consistent across the board. Reported-by: Yonghao Lee <yonghao_lee@qq.com> Discussion: https://postgr.es/m/tencent_CA1315EE8FB9C62F742C71E95FAD72214205@qq.com	6 days ago
Michael Paquier	2d4ead6f4b	Fix order of columns in pg_stat_recovery recovery_last_xact_time is listed before current_chunk_start_time in the documentation, the function definition and the view definition, but their order was reversed in the code. Thinko in `01d485b142`. Mea culpa. Author: Shinya Kato <shinya11.kato@gmail.com> Discussion: https://postgr.es/m/CAOzEurQQ1naKmPJhfE5WOUQjtf5tu08Kw3QCGY5UY=7Rt9fE=w@mail.gmail.com	6 days ago
Amit Kapila	f1ddaa1535	Fix inconsistent elevel in pg_sync_replication_slots() retry logic. The commit `0d2d4a0ec3` allowed pg_sync_replication_slots() to retry sync attempts, but missed a case, when WAL prior to a slot's confirmed_flush_lsn is not yet flushed locally. By changing the elevel from ERROR to LOG, we allow the sync loop to continue. This provides the opportunity for the slot to be synchronized once the standby catches up with the necessary WAL. Author: Zhijie Hou <houzj.fnst@fujitsu.com> Reviewed-by: shveta malik <shveta.malik@gmail.com> Reviewed-by: Amit Kapila <amit.kapila16@gmail.com> Discussion: https://postgr.es/m/CAFPTHDZAA+gWDntpa5ucqKKba41=tXmoXqN3q4rpjO9cdxgQrw@mail.gmail.com	6 days ago
Michael Paquier	01d485b142	Add system view pg_stat_recovery This commit introduces pg_stat_recovery, that exposes at SQL level the state of recovery as tracked by XLogRecoveryCtlData in shared memory, maintained by the startup process. This new view includes the following fields, that are useful for monitoring purposes on a standby, once it has reached a consistent state (making the execution of the SQL function possible): - Last-successfully replayed WAL record LSN boundaries and its timeline. - Currently replaying WAL record end LSN and its timeline. - Current WAL chunk start time. - Promotion trigger state. - Timestamp of latest processed commit/abort. - Recovery pause state. Some of this data can already be recovered from different system functions, but not all of it. See pg_get_wal_replay_pause_state or pg_last_xact_replay_timestamp. This new view offers a stronger consistency guarantee, by grabbing the recovery state for all fields through one spinlock acquisition. The system view relies on a new function, called pg_stat_get_recovery(). Querying this data requires the pg_read_all_stats privilege. The view returns no rows if the node is not in recovery. This feature originates from a suggestion I have made while discussion the addition of a CONNECTING state to the WAL receiver's shared memory state, because we lacked access to some of the state data. The author has taken the time to implement it, so thanks for that. Bump catalog version. Author: Xuneng Zhou <xunengzhou@gmail.com> Discussion: https://postgr.es/m/CABPTF7W+Nody-+P9y4PNk37-QWuLpfUrEonHuEhrX+Vx9Kq+Kw@mail.gmail.com Discussion: https://postgr.es/m/aW13GJn_RfTJIFCa@paquier.xyz	6 days ago
Michael Paquier	42a12856a6	Refactor code retrieving string for RecoveryPauseState This refactoring is going to be useful in an upcoming commit, to avoid some code duplication with the function pg_get_wal_replay_pause_state(), that returns a string for the recovery pause state. Refactoring opportunity noticed while hacking on a different patch. Discussion: https://postgr.es/m/CABPTF7W+Nody-+P9y4PNk37-QWuLpfUrEonHuEhrX+Vx9Kq+Kw@mail.gmail.com	6 days ago
Tom Lane	f95d73ed43	Simplify creation of built-in functions with non-default ACLs. Up to now, to create such a function, one had to make a pg_proc.dat entry and then modify it with GRANT/REVOKE commands, which we put in system_functions.sql. That seems a little ugly though, because it violates the idea of having a single source of truth about the initial contents of pg_proc, and it results in leaving dead rows in the initial contents of pg_proc. This patch improves matters by allowing aclitemin to work during early bootstrap, before pg_authid has been loaded. On the same principle that we use for early access to pg_type details, put a table of known built-in role names into bootstrap.c, and use that in bootstrap mode. To create a built-in function with a non-default ACL, one should write the desired ACL list in its pg_proc.dat entry, using a simplified version of aclitemout's notation: omit the grantor (if it is the bootstrap superuser, which it pretty much always should be) and spell the bootstrap superuser's name as POSTGRES, similarly to the notation used elsewhere in src/include/catalog. This results in entries like proacl => '{POSTGRES=X,pg_monitor=X}' which shows that we've revoked public execute permissions and instead granted that to pg_monitor. In addition to fixing up pg_proc.dat entries, I got rid of some role grants that had been stuck into system_functions.sql, and instead put them into a new file pg_auth_members.dat; that seems like a far less random place to put the information. The correctness of the data changes can be verified by comparing the initial contents of pg_proc and pg_auth_members before and after. pg_proc should match exactly, but the OID column of pg_auth_members will probably be different because those OIDs now get assigned a little earlier in bootstrap. (I forced a catversion bump out of caution, but it wasn't really necessary.) Author: Tom Lane <tgl@sss.pgh.pa.us> Reviewed-by: Álvaro Herrera <alvherre@kurilemu.de> Discussion: https://postgr.es/m/183292bb-4891-4c96-a3ca-e78b5e0e1358@dunslane.net	6 days ago
Tom Lane	7664319ccb	Be more wary of false matches in initdb's replace_token(). Do not replace the target string unless the occurrence is surrounded by whitespace or line start/end. This avoids potential false match to a substring of a field. While we've not had trouble with that up to now, the next patch creates hazards of false matches to POSTGRES within an ACL field. There is one call site that needs adjustment, as it was presuming it could write "::1" and have that match "::1/128". For all the others, this restriction is okay and strictly safer. Author: Tom Lane <tgl@sss.pgh.pa.us> Reviewed-by: Álvaro Herrera <alvherre@kurilemu.de> Discussion: https://postgr.es/m/183292bb-4891-4c96-a3ca-e78b5e0e1358@dunslane.net	6 days ago
Melanie Plageman	34cb4254bd	Prefix PruneState->all_{visible,frozen} with set_ The PruneState had members called "all_visible" and "all_frozen" which reflect not the current state of the page but the state it could be in once pruning and freezing have been executed. These are then saved in the PruneFreezeResult so the caller can set the VM accordingly. Prefix the PruneState members as well as the corresponsding PruneFreezeResult members with "set_" to clarify that they represent the proposed state of the all-visible and all-frozen bits for a heap page in the visibility map, not the current state. Author: Melanie Plageman <melanieplageman@gmail.com> Suggested-by: Andres Freund <andres@anarazel.de> Reviewed-by: Chao Li <li.evan.chao@gmail.com> Discussion: https://postgr.es/m/bqc4kh5midfn44gnjiqez3bjqv4zogydguvdn446riw45jcf3y%404ez66il7ebvk	6 days ago
Melanie Plageman	68c2dcb913	Add PageGetPruneXid() helper This is similar to the other page accessors in bufpage.h. It improves readability and avoids long lines. Author: Melanie Plageman <melanieplageman@gmail.com> Reviewed-by: Chao Li <li.evan.chao@gmail.com> Discussion: https://postgr.es/m/BD8B69E7-26D8-4706-9164-597C6AE57812%40gmail.com	7 days ago
Melanie Plageman	59663e4207	Move commonly used context into PruneState and simplify helpers heap_page_prune_and_freeze() and many of its helpers use the heap buffer, block number, and page. Other helpers took the heap page and didn't use it. Initializing these values once during prune_freeze_setup() simplifies the helpers' interfaces and avoids any repeated calls to BufferGetBlockNumber() and BufferGetPage(). While updating PruneState, also reorganize its fields to make the layout and member documentation more consistent. Author: Melanie Plageman <melanieplageman@gmail.com> Reviewed-by: Chao Li <li.evan.chao@gmail.com> Discussion: https://postgr.es/m/BD8B69E7-26D8-4706-9164-597C6AE57812%40gmail.com	7 days ago
Tom Lane	ac0accafd6	Exit after fatal errors in client-side compression code. It looks like whoever wrote the astreamer (nee bbstreamer) code thought that pg_log_error() is equivalent to elog(ERROR), but it's not; it just prints a message. So all these places tried to continue on after a compression or decompression error return, with the inevitable result being garbage output and possibly cascading error messages. We should use pg_fatal() instead. These error conditions are probably pretty unlikely in practice, which no doubt accounts for the lack of field complaints. Author: Tom Lane <tgl@sss.pgh.pa.us> Reviewed-by: Chao Li <li.evan.chao@gmail.com> Discussion: https://postgr.es/m/1531718.1772644615@sss.pgh.pa.us Backpatch-through: 15	7 days ago
Jacob Champion	a6483f5ac9	oauth: Add TLS support for oauth_validator tests The oauth_validator tests don't currently support HTTPS, which makes testing PGOAUTHCAFILE difficult. Add a localhost certificate to src/test/ssl and make use of it in oauth_server.py. In passing, explain the hardcoded use of IPv4 in our issuer identifier, after intermittent failures on NetBSD led to commit `8d9d5843b`. (The new certificate is still set up for IPv6, to make it easier to improve that behavior in the future.) Patch by Jonathan Gonzalez V., with some additional tests and tweaks by me. Author: Jonathan Gonzalez V. <jonathan.abdiel@gmail.com> Discussion: https://postgr.es/m/8a296a2c128aba924bff0ae48af2b88bf8f9188d.camel@gmail.com	7 days ago
Jacob Champion	b8d7685835	libpq: Add PQgetThreadLock() to mirror PQregisterThreadLock() Allow libpq clients to retrieve the current pg_g_threadlock pointer with PQgetThreadLock(). Single-threaded applications could already do this in a convoluted way: pgthreadlock_t tlock; tlock = PQregisterThreadLock(NULL); PQregisterThreadLock(tlock); /* re-register the callback / / use tlock */ But a generic library can't do that without potentially breaking concurrent libpq connections. The motivation for doing this now is the libpq-oauth plugin, which currently relies on direct injection of pg_g_threadlock, and should ideally not. Reviewed-by: Zsolt Parragi <zsolt.parragi@percona.com> Discussion: https://postgr.es/m/CAOYmi%2BmEU_q9sr1PMmE-4rLwFN%3DOjyndDwFZvpsMU3RNJLrM9g%40mail.gmail.com Discussion: https://postgr.es/m/CAOYmi%2B%3DMHD%2BWKD4rsTn0v8220mYfyLGhEc5EfhmtqrAb7SmC5g%40mail.gmail.com	7 days ago
Jacob Champion	f8c0b91a60	oauth: Report cleanup errors as warnings on stderr Using conn->errorMessage for these "shouldn't-happen" cases will only work if the connection itself fails. Our SSL and password callbacks print WARNINGs when they find themselves in similar situations, so follow their lead. Reviewed-by: Zsolt Parragi <zsolt.parragi@percona.com> Discussion: https://postgr.es/m/CAOYmi%2BmEU_q9sr1PMmE-4rLwFN%3DOjyndDwFZvpsMU3RNJLrM9g%40mail.gmail.com	7 days ago
Alexander Korotkov	177037341a	Fix handling of updated tuples in the MERGE statement This branch missed the IsolationUsesXactSnapshot() check. That led to EPQ on repeatable read and serializable isolation levels. This commit fixes the issue and provides a simple isolation check for that. Backpatch through v15 where MERGE statement was introduced. Reported-by: Tender Wang <tndrwang@gmail.com> Discussion: https://postgr.es/m/CAPpHfdvzZSaNYdj5ac-tYRi6MuuZnYHiUkZ3D-AoY-ny8v%2BS%2Bw%40mail.gmail.com Author: Tender Wang <tndrwang@gmail.com> Reviewed-by: Dean Rasheed <dean.a.rasheed@gmail.com> Backpatch-through: 15	7 days ago
Fujii Masao	bffd7130e9	Improve validation of recovery_target_xid GUC values. Previously, the recovery_target_xid GUC values were not sufficiently validated. As a result, clearly invalid inputs such as the string "bogus", a decimal value like "1.1", or 0 (a transaction ID smaller than the minimum valid value of 3) were unexpectedly accepted. In these cases, the value was interpreted as transaction ID 0, which could cause recovery to behave unexpectedly. This commit improves validation of recovery_target_xid GUC so that invalid values are rejected with an error. This prevents recovery from proceeding with misconfigured recovery_target_xid settings. Also this commit updates the documentation to clarify the allowed values for recovery_target_xid GUC. Author: David Steele <david@pgbackrest.org> Reviewed-by: Hüseyin Demir <huseyin.d3r@gmail.com> Reviewed-by: Fujii Masao <masao.fujii@gmail.com> Reviewed-by: Michael Paquier <michael@paquier.xyz> Discussion: https://postgr.es/m/f14463ab-990b-4ae9-a177-998d2677aae0@pgbackrest.org	7 days ago
Fujii Masao	9b0e5bd532	doc: Clarify that COLUMN is optional in ALTER TABLE ... ADD/DROP COLUMN. In ALTER TABLE ... ADD/DROP COLUMN, the COLUMN keyword is optional. However, part of the documentation could be read as if COLUMN were required, which may mislead users about the command syntax. This commit updates the ALTER TABLE documentation to clearly state that COLUMN is optional for ADD and DROP. Also this commit adds regression tests covering ALTER TABLE ... ADD/DROP without the COLUMN keyword. Backpatch to all supported versions. Author: Chao Li <lic@highgo.com> Reviewed-by: Robert Treat <rob@xzilla.net> Reviewed-by: Fujii Masao <masao.fujii@gmail.com> Discussion: https://postgr.es/m/CAEoWx2n6ShLMOnjOtf63TjjgGbgiTVT5OMsSOFmbjGb6Xue1Bw@mail.gmail.com Backpatch-through: 14	1 week ago
Michael Paquier	5f8124a0cf	Move definition of XLogRecoveryCtlData to xlogrecovery.h XLogRecoveryCtlData is the structure that stores the shared-memory state of WAL recovery, including information such as promotion requests, the timeline ID (TLI), and the LSNs of replayed records. This refactoring is independently useful because it allows code outside of core to access the recovery state in live. It will be used by an upcoming patch that introduces a SQL function for querying this information, that can be accessed on a standby once a consistent state has been reached. This only moves code around, changing nothing functionally. Author: Xuneng Zhou <xunengzhou@gmail.com> Discussion: https://postgr.es/m/CABPTF7W+Nody-+P9y4PNk37-QWuLpfUrEonHuEhrX+Vx9Kq+Kw@mail.gmail.com	1 week ago
Michael Paquier	ea4744782b	Fix rare instability in recovery TAP test 004_timeline_switch This fixes a problem similar to `ad8c86d22c`. In this case, the test could fail under the following circumstances: - The primary is stopped with teardown_node(), meaning that it may not be able to send all its WAL records to standby_1 and standby_2. - If standby_2 receives more records than standby_1, attempting to reconnect standby_2 to the promoted standby_1 would fail because of a timeline fork. This race condition is fixed with a simple trick: instead of tearing down the primary, it is stopped cleanly so as all the WAL records of the primary are received and flushed by both standby_1 and standby_2. Once we do that, there is no need for a wait_for_catchup() before stopping the node. The test wants to check that a timeline jump can be achieved when reconnecting a standby to a promoted standby in the same cluster, hence an immediate stop of the primary is not required. This failure is harder to reach than the previous instability of 009_twophase, still the buildfarm has been able to detect this failure at least once. I have tried Alexander Lakhin's test trick with the bgwriter and very aggressive standby snapshots, but I could not reproduce it directly. It is reachable, as the buildfarm has proved. Backpatch down to all supported branches, and this problem can lead to spurious failures in the buildfarm. Discussion: https://postgr.es/m/493401a8-063f-436a-8287-a235d9e065fc@gmail.com Backpatch-through: 14	1 week ago
Michael Paquier	34dfca2934	Change default value of default_toast_compression to "lz4", take two The default value for default_toast_compression was "pglz". The main reason for this choice is that this option is always available, pglz code being embedded in Postgres. However, it is known that LZ4 is more efficient than pglz: less CPU required, more compression on average. As of this commit, the default value of default_toast_compression becomes "lz4", if available. By switching to LZ4 as the default, users should see natural speedups on TOAST data reads and/or writes. Support for LZ4 in TOAST compression was added in Postgres v14, or 5 releases ago. This should be long enough to consider this feature as stable. While at it, quotes are removed from default_toast_compression in postgresql.conf.sample. Quotes are not required in this case. The in-place value replacement done by initdb if the build supports LZ4 would not use them in the postgresql.conf file added to a freshly-initialized cluster. Note that this is a version lighter than `7c1849311e`, that included a replacement of --with-lz4 by --without-lz4 in configure builds, forcing a requirement for LZ4 in all environments. The buildfarm did not like it, at all. This commit switches default_toast_compression to lz4 as default only when --with-lz4 is defined, which should keep the buildfarm at bay while still allowing users to benefit from LZ4 compression in TOAST as long as the code is compiled with it. Author: Euler Taveira <euler@eulerto.com> Reviewed-by: Peter Eisentraut <peter@eisentraut.org> Reviewed-by: Aleksander Alekseev <aleksander@tigerdata.com> Discussion: https://posgr.es/m/435df33a-129e-4f0c-a803-f3935c5a5ecb@eisentraut.org	1 week ago
Michael Paquier	4f0b3afab4	Revert "Change default value of default_toast_compression to "lz4"" This reverts commit `7c1849311e`, due to the fact that more than 60% of the buildfarm members do not have lz4 installed. As we are in the last commit fest of the development cycle, and that it could take a couple of weeks to stabilize things, this change is reverted for now. This commit will be reworked in a lighter version, as default_toast_compression's default can be changed to "lz4" without the switch from --with-lz4 to --without-lz4. This approach will keep the buildfarm at bay, and still allow builds to take advantage of LZ4 in TOAST by default, as long as the code is compiled with LZ4 support. A harder requirement based on LZ4 should be achievable at some point, but it is going to require some work from the buildfarm owners first. Perhaps this part could be revisited at the beginning of the next development cycle. Discussion: https://postgr.es/m/CAOYmi+meTT0NbLbnVqOJD5OKwCtHL86PQ+RZZTrn6umfmHyWaw@mail.gmail.com	1 week ago
Andrew Dunstan	3c19983cc0	pg_restore: add --no-globals option to skip globals This is a followup to commit `763aaa06f0` Add non-text output formats to pg_dumpall. Add a --no-globals option to pg_restore that skips restoring global objects (roles and tablespaces) when restoring from a pg_dumpall archive. When -C/--create is not specified, databases that do not already exist on the target server are also skipped. This is useful when restoring only specific databases from a pg_dumpall archive without needing the global objects to be restored first. Author: Mahendra Singh Thalor <mahi6run@gmail.com> With small tweaks by me. Discussion: https://postgr.es/m/CAKYtNArdcc5kx1MdTtTKFNYiauo3=zCA-NB0LmBCW-RU_kSb3A@mail.gmail.com	1 week ago
Andrew Dunstan	c7572cd48d	Improve writing map.dat preamble Fix code from commit `763aaa06f0` Suggestion from Alvaro Herrera following a bug discovered by Coverity.	1 week ago

1 2 3 4 5 ...

63503 Commits (0fbfd37cefb7eb30c0fa8a158751c19ddeddf1f0) All Branches Search

63503 Commits (0fbfd37cefb7eb30c0fa8a158751c19ddeddf1f0)

All Branches