12 Commits (5fd424c87a86dd19d6cb15a18d1662ffc7368de1)

Author	SHA1	Message	Date
Peter Geoghegan	df4f3ab517	Add show_data option to pg_get_wal_block_info. ... Allow users to opt out of returning FPI data and block data from pg_get_wal_block_info as an optimization. Testing has shown that this can make function execution over twice as fast in some cases. When pg_get_wal_block_info is called with "show_data := false", it always returns NULL values for its block_data and block_fpi_data bytea output parameters. Nothing else changes. In particular, the function will still return the usual per-block summary of block data/FPI space overhead. Use of "show_data := false" is therefore feasible with all queries that don't specifically require these raw binary strings. Follow-up to recent work in commit 122376f0. There still hasn't been a stable release with the pg_get_wal_block_info function, so no bump in the pg_walinspect extension version. Per suggestion from Melanie Plageman. Author: Peter Geoghegan <pg@bowt.ie> Discussion: https://postgr.es/m/CAAKRu_bJvbcYBRj2cN6G2xV7B7-Ja+pjTO1nEnEhRR8OXYiABA@mail.gmail.com Discussion: https://postgr.es/m/CAH2-Wzm9shOkEDM10_+qOZkRSQhKVxwBFiehH6EHWQQRd_rDPw@mail.gmail.com	2 years ago
Peter Geoghegan	122376f028	Show record information in pg_get_wal_block_info. ... Expand the output parameters in pg_walinspect's pg_get_wal_block_info function to return additional information that was previously only available from pg_walinspect's pg_get_wal_records_info function. Some of the details are attributed to individual block references, rather than aggregated into whole-record values, since the function returns one row per block reference per WAL record (unlike pg_get_wal_records_info, which always returns one row per WAL record). This structure is much easier to work with when writing queries that track how individual blocks changed over time, or when attributing costs to individual blocks (not WAL records) is useful. This is the second time that pg_get_wal_block_info has been enhanced in recent weeks. Commit 9ecb134a expanded on the original version of the function added in commit c31cf1c0 (where it first appeared under the name pg_get_wal_fpi_info). There still hasn't been a stable release since commit c31cf1c0, so no bump in the pg_walinspect extension version. Author: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com> Author: Peter Geoghegan <pg@bowt.ie> Reviewed-By: Peter Geoghegan <pg@bowt.ie> Reviewed-By: Kyotaro HORIGUCHI <horikyota.ntt@gmail.com> Discussion: https://postgr.es/m/CALj2ACVRK5=Z+2ZVsjgTTSkfEnQzCuwny7iigpG7g1btk4Ws2A@mail.gmail.com	2 years ago
Michael Paquier	4f1882b960	Improve a bit the tests of pg_walinspect ... This commit improves the tests of pg_walinspect on a few things: - Remove aggregates for queries that should fail. If the code is reworked in such a way that the behavior of these queries is changed, we would get more input from them, written this way. - Expect at least one record reported in the valid queries doing scans across ranges, rather than zero records. - Adjust a few comments, for consistency. Author: Bharath Rupireddy Discussion: https://postgr.es/m/CALj2ACVaoXW3nJD9zq8E66BEf-phgJfFcKRVJq9GXkuX0b3ULQ@mail.gmail.com	3 years ago
Michael Paquier	e643a315fc	Tweak regression test of pg_walinspect to be less collation-sensitive ... \dx was used on pg_walinspect to list its objects in 1.0 but the names of the objects in this version do not have an order that is always guaranteed depending on the collation used. Rather than append a COLLATE clause in the query of \dx, this tweaks the regression test to produce an output whose order is guaranteed. Reported-by: Andres Freund Author: Bharath Rupireddy Discussion: https://postgr.es/m/20230314215440.gma7g4keswdnldj5@awork3.anarazel.de	3 years ago
Michael Paquier	5c1b662807	Rework design of functions in pg_walinspect ... This commit reworks a bit the set-returning functions of pg_walinspect, making them more flexible regarding their end LSN: - pg_get_wal_records_info() - pg_get_wal_stats() - pg_get_wal_block_info() The end LSNs given to these functions is now handled so as a value higher than the current LSN of the cluster (insert LSN for a primary, or replay LSN for a standby) does not raise an error, giving more flexibility to monitoring queries. Instead, the functions return results up to the current LSN, as found at the beginning of each function call. As an effect of that, pg_get_wal_records_info_till_end_of_wal() and pg_get_wal_stats_till_end_of_wal() are now removed from 1.1, as the existing, equivalent functions are able to offer the same possibilities. Author: Bharath Rupireddy Discussion: https://postgr.es/m/CALj2ACU0_q-o4DSweyaW9NO1KBx-QkN6G_OzYQvpjf3CZVASkg@mail.gmail.com	3 years ago
Michael Paquier	1f282c24e4	Refactor and improve tests of pg_walinspect ... The regression tests of pg_walinspect are reworked on a few aspects: - Reorganization on the validation checks done for the start and end LSNs on the six SQL functions currently available in 1.1. - Addition of a few patterns doing bound checks for invalid start LSN, invalid end LSN, and failures in reading LSN positions, for anything that's been missing. - Use of a consistent style across the whole, limiting blank lines across the queries. - Addition of a new test script for upgrades. For the time being, this is straight-forward with a check that the upgrade from 1.0 works correctly. This will be made more complicated once the interface of this extension is reworked in 1.1 with a follow-up patch. Most of the contents of this commit are extracted from a larger patch by the same author, largely reorganized by me to minimize the differences with the upcoming work aimed to lift the bound checks on the input LSNs used by the functions of this extension. Author: Bharath Rupireddy, Michael Paquier Discussion: https://postgr.es/m/CALj2ACU0_q-o4DSweyaW9NO1KBx-QkN6G_OzYQvpjf3CZVASkg@mail.gmail.com	3 years ago
Michael Paquier	9ecb134a93	pg_walinspect: pg_get_wal_fpi_info() -> pg_get_wal_block_info() ... This commit reworks pg_get_wal_fpi_info() to become aware of all the block information that can be attached to a record rather than just its full-page writes: - Addition of the block id as assigned by XLogRegisterBuffer(), XLogRegisterBlock() or XLogRegisterBufData(). - Addition of the block data, as bytea, or NULL if none. The length of the block data can be guessed with length(), so there is no need to store its length in a separate field. - Addition of the full-page image length, as counted without a hole or even compressed. - Modification of the handling of the full-page image data. This is still a bytea, but it could become NULL if none is assigned to a block. - Addition of the full-page image flags, tracking if a page is stored with a hole, if it needs to be applied and the type of compression applied to it, as of all the BKPIMAGE_* values in xlogrecord.h. The information of each block is returned as one single record, with the record's ReadRecPtr included to be able to join the block information with the existing pg_get_wal_records_info(). Note that it is perfectly possible for a block to hold both data and full-page image. Thanks also to Kyotaro Horiguchi and Matthias van de Meent for the discussion. This commit uses some of the work proposed by Melanie, though it has been largely redesigned and rewritten by me. Bharath has helped in refining a bit the whole. Reported-by: Melanie Plageman Author: Michael Paquier, Melanie Plageman, Bharath Rupireddy Discussion: https://postgr.es/m/CAAKRu_bORebdZmcV8V4cZBzU8M_C6tDDdbiPhCZ6i-iuSXW9TA@mail.gmail.com	3 years ago
Michael Paquier	c31cf1c03d	pg_walinspect: Add pg_get_wal_fpi_info() ... This function is able to extract the full page images from a range of records, specified as of input arguments start_lsn and end_lsn. Like the other functions of this module, an error is returned if using LSNs that do not reflect real system values. All the FPIs stored in a single record are extracted. The module's version is bumped to 1.1. Author: Bharath Rupireddy Reviewed-by: Bertrand Drouvot Discussion: https://postgr.es/m/CALj2ACVCcvzd7WiWvD=6_7NBvVB_r6G0EGSxL4F8vosAi6Se4g@mail.gmail.com	3 years ago
Robert Haas	a448e49bcb	Revert 56-bit relfilenode change and follow-up commits. ... There are still some alignment-related failures in the buildfarm, which might or might not be able to be fixed quickly, but I've also just realized that it increased the size of many WAL records by 4 bytes because a block reference contains a RelFileLocator. The effect of that hasn't been studied or discussed, so revert for now.	3 years ago
Robert Haas	05d4cbf9b6	Increase width of RelFileNumbers from 32 bits to 56 bits. ... RelFileNumbers are now assigned using a separate counter, instead of being assigned from the OID counter. This counter never wraps around: if all 2^56 possible RelFileNumbers are used, an internal error occurs. As the cluster is limited to 2^64 total bytes of WAL, this limitation should not cause a problem in practice. If the counter were 64 bits wide rather than 56 bits wide, we would need to increase the width of the BufferTag, which might adversely impact buffer lookup performance. Also, this lets us use bigint for pg_class.relfilenode and other places where these values are exposed at the SQL level without worrying about overflow. This should remove the need to keep "tombstone" files around until the next checkpoint when relations are removed. We do that to keep RelFileNumbers from being recycled, but now that won't happen anyway. However, this patch doesn't actually change anything in this area; it just makes it possible for a future patch to do so. Dilip Kumar, based on an idea from Andres Freund, who also reviewed some earlier versions of the patch. Further review and some wordsmithing by me. Also reviewed at various points by Ashutosh Sharma, Vignesh C, Amul Sul, Álvaro Herrera, and Tom Lane. Discussion: http://postgr.es/m/CA+Tgmobp7+7kmi4gkq7Y+4AM9fTvL+O1oQ4-5gFTT+6Ng-dQ=g@mail.gmail.com	3 years ago
Jeff Davis	dad9ba1c82	Fix buildfarm failures in pg_walinspect tests.	3 years ago
Jeff Davis	2258e76f90	Add contrib/pg_walinspect. ... Provides similar functionality to pg_waldump, but from a SQL interface rather than a separate utility. Author: Bharath Rupireddy Reviewed-by: Greg Stark, Kyotaro Horiguchi, Andres Freund, Ashutosh Sharma, Nitin Jadhav, RKN Sai Krishna Discussion: https://postgr.es/m/CALj2ACUGUYXsEQdKhEdsBzhGEyF3xggvLdD8C0VT72TNEfOiog%40mail.gmail.com	3 years ago