You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

Masahiko Sawada b2ae077205 Fix re-distributing previously distributed invalidation messages during logical decoding. ... Commit 4909b38af0 introduced logic to distribute invalidation messages from catalog-modifying transactions to all concurrent in-progress transactions. However, since each transaction distributes not only its original invalidation messages but also previously distributed messages to other transactions, this leads to an exponential increase in allocation request size for invalidation messages, ultimately causing memory allocation failure. This commit fixes this issue by tracking distributed invalidation messages separately per decoded transaction and not redistributing these messages to other in-progress transactions. The maximum size of distributed invalidation messages that one transaction can store is limited to MAX_DISTR_INVAL_MSG_PER_TXN (8MB). Once the size of the distributed invalidation messages exceeds this threshold, we invalidate all caches in locations where distributed invalidation messages need to be executed. Back-patch to all supported versions where we introduced the fix by commit 4909b38af0. Note that this commit adds two new fields to ReorderBufferTXN to store the distributed transactions. This change breaks ABI compatibility in back branches, affecting third-party extensions that depend on the size of the ReorderBufferTXN struct, though this scenario seems unlikely. Additionally, it adds a new flag to the txn_flags field of ReorderBufferTXN to indicate distributed invalidation message overflow. This should not affect existing implementations, as it is unlikely that third-party extensions use unused bits in the txn_flags field. Bug: #18938 #18942 Author: vignesh C <vignesh21@gmail.com> Reported-by: Duncan Sands <duncan.sands@deepbluecap.com> Reported-by: John Hutchins <john.hutchins@wicourts.gov> Reported-by: Laurence Parry <greenreaper@hotmail.com> Reported-by: Max Madden <maxmmadden@gmail.com> Reported-by: Braulio Fdo Gonzalez <brauliofg@gmail.com> Reviewed-by: Masahiko Sawada <sawada.mshk@gmail.com> Reviewed-by: Amit Kapila <amit.kapila16@gmail.com> Reviewed-by: Hayato Kuroda <kuroda.hayato@fujitsu.com> Discussion: https://postgr.es/m/680bdaf6-f7d1-4536-b580-05c2760c67c6@deepbluecap.com Discussion: https://postgr.es/m/18942-0ab1e5ae156613ad@postgresql.org Discussion: https://postgr.es/m/18938-57c9a1c463b68ce0@postgresql.org Discussion: https://postgr.es/m/CAD1FGCT2sYrP_70RTuo56QTizyc+J3wJdtn2gtO3VttQFpdMZg@mail.gmail.com Discussion: https://postgr.es/m/CANO2=B=2BT1hSYCE=nuuTnVTnjidMg0+-FfnRnqM6kd23qoygg@mail.gmail.com Backpatch-through: 13		3 months ago
..
binary.out	Message style improvements	10 years ago
catalog_change_snapshot.out	Fix assertion failures while processing NEW_CID record in logical decoding.	3 years ago
concurrent_ddl_dml.out	Improve display of query results in isolation tests.	4 years ago
concurrent_stream.out	Improve display of query results in isolation tests.	4 years ago
ddl.out	Make test_decoding ddl.out shorter	2 years ago
decoding_in_xact.out	Revert "Logical decoding of sequences"	3 years ago
decoding_into_rel.out	Revert "Logical decoding of sequences"	3 years ago
delayed_startup.out	Improve display of query results in isolation tests.	4 years ago
invalidation_distribution.out	Fix re-distributing previously distributed invalidation messages during logical decoding.	3 months ago
messages.out	Add required database and origin filtering for logical messages.	10 years ago
mxact.out	Revert "Logical decoding of sequences"	3 years ago
oldest_xmin.out	Fix xmin advancement during fast_forward decoding.	5 months ago
ondisk_startup.out	Revert "Logical decoding of sequences"	3 years ago
permissions.out	Improve several permission-related error messages.	3 years ago
prepared.out	Add skip-empty-xacts option to test_decoding for use in the regression tests.	11 years ago
replorigin.out	Fix the misuse of origin filter across multiple pg_logical_slot_get_changes() calls.	2 years ago
rewrite.out	Revert "Logical decoding of sequences"	3 years ago
skip_snapshot_restore.out	Fix possibility of logical decoding partial transaction changes.	1 year ago
slot.out	Revert "Logical decoding of sequences"	3 years ago
slot_creation_error.out	Close race condition in slot_creation_error.spec.	4 years ago
snapshot_transfer.out	Improve display of query results in isolation tests.	4 years ago
spill.out	Make contrib/test_decoding regression tests safe for CZ locale.	9 years ago
stats.out	pgstat: add/extend tests for resetting various kinds of stats.	3 years ago
stream.out	Stabilize the test added by commit 022564f60c.	11 months ago
subxact_without_top.out	Improve display of query results in isolation tests.	4 years ago
time.out	logical decoding: fix decoding of a commit's commit time.	10 years ago
toast.out	Revert "Logical decoding of sequences"	3 years ago
truncate.out	Revert "Logical decoding of sequences"	3 years ago
twophase.out	Stabilize the test added by commit 022564f60c.	11 months ago
twophase_snapshot.out	Improve display of query results in isolation tests.	4 years ago
twophase_stream.out	Add option to enable two_phase commits via pg_create_logical_replication_slot.	5 years ago
xact.out	Don't propagate a null subtransaction snapshot up to parent transaction.	9 years ago