20 Commits (5fd424c87a86dd19d6cb15a18d1662ffc7368de1)

Author	SHA1	Message	Date
Tom Lane	e2933a6e11	Prevent instability in contrib/pageinspect's regression test. ... pageinspect has occasionally failed on slow buildfarm members, with symptoms indicating that the expected effects of VACUUM FREEZE didn't happen. This is presumably because a background transaction such as auto-analyze was holding back global xmin. We can work around that by using a temp table in the test. Since commit a7212be8b, that will use an up-to-date cutoff xmin regardless of other processes. And pageinspect itself shouldn't really care whether the table is temp. Back-patch to v14. There would be no point in older branches without back-patching a7212be8b, which seems like more trouble than the problem is worth. Discussion: https://postgr.es/m/2892135.1668976646@sss.pgh.pa.us	3 years ago
Michael Paquier	cd4868a570	pageinspect: Fix handling of all-zero pages ... Getting from get_raw_page() an all-zero page is considered as a valid case by the buffer manager and it can happen for example when finding a corrupted page with zero_damaged_pages enabled (using zero_damaged_pages to look at corrupted pages happens), or after a crash when a relation file is extended before any WAL for its new data is generated (before a vacuum or autovacuum job comes in to do some cleanup). However, all the functions of pageinspect, as of the index AMs (except hash that has its own idea of new pages), heap, the FSM or the page header have never worked with all-zero pages, causing various crashes when going through the page internals. This commit changes all the pageinspect functions to be compliant with all-zero pages, where the choice is made to return NULL or no rows for SRFs when finding a new page. get_raw_page() still works the same way, returning a batch of zeros in the bytea of the page retrieved. A hard error could be used but NULL, while more invasive, is useful when scanning relation files in full to get a batch of results for a single relation in one query. Tests are added for all the code paths impacted. Reported-by: Daria Lepikhova Author: Michael Paquier Discussion: https://postgr.es/m/561e187b-3549-c8d5-03f5-525c14e65bd0@postgrespro.ru Backpatch-through: 10	3 years ago
Michael Paquier	076f4d9539	pageinspect: Fix handling of page sizes and AM types ... This commit fixes a set of issues related to the use of the SQL functions in this module when the caller is able to pass down raw page data as input argument: - The page size check was fuzzy in a couple of places, sometimes looking after only a sub-range, but what we are looking for is an exact match on BLCKSZ. After considering a few options here, I have settled down to do a generalization of get_page_from_raw(). Most of the SQL functions already used that, and this is not strictly required if not accessing an 8-byte-wide value from a raw page, but this feels safer in the long run for alignment-picky environment, particularly if a code path begins to access such values. This also reduces the number of strings that need to be translated. - The BRIN function brin_page_items() uses a Relation but it did not check the access method of the opened index, potentially leading to crashes. All the other functions in need of a Relation already did that. - Some code paths could fail on elog(), but we should to use ereport() for failures that can be triggered by the user. Tests are added to stress all the cases that are fixed as of this commit, with some junk raw pages (\set VERBOSITY ensures that this works across all page sizes) and unexpected index types when functions open relations. Author: Michael Paquier, Justin Prysby Discussion: https://postgr.es/m/20220218030020.GA1137@telsasoft.com Backpatch-through: 10	4 years ago
Tom Lane	c2dc1a7976	Disable vacuum page skipping in selected test cases. ... By default VACUUM will skip pages that it can't immediately get exclusive access to, which means that even activities as harmless and unpredictable as checkpoint buffer writes might prevent a page from being processed. Ordinarily this is no big deal, but we have a small number of test cases that examine the results of VACUUM's processing and therefore will fail if the page of interest is skipped. This seems to be the explanation for some rare buildfarm failures. To fix, add the DISABLE_PAGE_SKIPPING option to the VACUUM commands in tests where this could be an issue. In passing, remove a duplicated query in pageinspect/sql/page.sql. Back-patch as necessary (some of these cases are as old as v10). Discussion: https://postgr.es/m/413923.1611006484@sss.pgh.pa.us	5 years ago
Peter Eisentraut	f18aa1b203	pageinspect: Change block number arguments to bigint ... Block numbers are 32-bit unsigned integers. Therefore, the smallest SQL integer type that they can fit in is bigint. However, in the pageinspect module, most input and output parameters dealing with block numbers were declared as int. The behavior with block numbers larger than a signed 32-bit integer was therefore dubious. Change these arguments to type bigint and add some more explicit error checking on the block range. (Other contrib modules appear to do this correctly already.) Since we are changing argument types of existing functions, in order to not misbehave if the binary is updated before the extension is updated, we need to create new C symbols for the entry points, similar to how it's done in other extensions as well. Reported-by: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com> Reviewed-by: Alvaro Herrera <alvherre@alvh.no-ip.org> Reviewed-by: Michael Paquier <michael@paquier.xyz> Discussion: https://www.postgresql.org/message-id/flat/d8f6bdd536df403b9b33816e9f7e0b9d@G08CNEXMBPEKD05.g08.fujitsu.local	5 years ago
Michael Paquier	58b4cb30a5	Redesign pageinspect function printing infomask bits ... After more discussion, the new function added by ddbd5d8 could have been designed in a better way. Based on an idea from Álvaro, instead of returning one column which includes both the raw and combined flags, use two columns, with one for the raw flags and one for the combined flags. This also takes care of some issues with HEAP_LOCKED_UPGRADED and HEAP_XMAX_IS_LOCKED_ONLY which are not really combined flags as they depend on conditions defined by other raw bits, as mentioned by Amit. While on it, fix an extra issue with combined flags. A combined flag was returned if at least one of its bits was set, but all its bits need to be set to include it in the result. Author: Michael Paquier Reviewed-by: Álvaro Herrera, Amit Kapila Discussion: https://postgr.es/m/20190913114950.GA3824@alvherre.pgsql	6 years ago
Michael Paquier	ddbd5d8731	Add to pageinspect function to make t_infomask/t_infomask2 human-readable ... Flags of t_infomask and t_infomask2 for each tuple are already included in the information returned by heap_page_items as integers, and we lacked a way to make that information human-readable. Per discussion, the function includes an option which controls if combined flags should be decomposed or not. The default is false, to not decompose combined flags. The module is bumped to version 1.8. Author: Craig Ringer, Sawada Masahiko Reviewed-by: Peter Geoghegan, Robert Haas, Álvaro Herrera, Moon Insung, Amit Kapila, Michael Paquier, Tomas Vondra Discussion: https://postgr.es/m/CAMsr+YEY7jeaXOb+oX+RhDyOFuTMdmHjGsBxL=igCm03J0go9Q@mail.gmail.com	6 years ago
Amit Kapila	7db0cde6b5	Revert "Avoid the creation of the free space map for small heap relations". ... This feature was using a process local map to track the first few blocks in the relation. The map was reset each time we get the block with enough freespace. It was discussed that it would be better to track this map on a per-relation basis in relcache and then invalidate the same whenever vacuum frees up some space in the page or when FSM is created. The new design would be better both in terms of API design and performance. List of commits reverted, in reverse chronological order: 06c8a5090e Improve code comments in b0eaa4c51b. 13e8643bfc During pg_upgrade, conditionally skip transfer of FSMs. 6f918159a9 Add more tests for FSM. 9c32e4c350 Clear the local map when not used. 29d108cdec Update the documentation for FSM behavior.. 08ecdfe7e5 Make FSM test portable. b0eaa4c51b Avoid creation of the free space map for small heap relations. Discussion: https://postgr.es/m/20190416180452.3pm6uegx54iitbt5@alap3.anarazel.de	6 years ago
Amit Kapila	08ecdfe7e5	Make FSM test portable. ... In b0eaa4c51b, we allow FSM to be created only after 4 pages. One of the tests check the FSM contents and to do that it populates many tuples in the relation. The FSM contents depend on the availability of freespace in the page and it could vary because of the alignment of tuples. This commit removes the dependency on FSM contents. Author: Amit Kapila Discussion: https://postgr.es/m/CAA4eK1KADF6K1bagr0--mGv3dMcZ%3DH_Z-Qtvdfbp5PjaC6PJJA%40mail.gmail.com	7 years ago
Amit Kapila	b0eaa4c51b	Avoid creation of the free space map for small heap relations, take 2. ... Previously, all heaps had FSMs. For very small tables, this means that the FSM took up more space than the heap did. This is wasteful, so now we refrain from creating the FSM for heaps with 4 pages or fewer. If the last known target block has insufficient space, we still try to insert into some other page before giving up and extending the relation, since doing otherwise leads to table bloat. Testing showed that trying every page penalized performance slightly, so we compromise and try every other page. This way, we visit at most two pages. Any pages with wasted free space become visible at next relation extension, so we still control table bloat. As a bonus, directly attempting one or two pages can even be faster than consulting the FSM would have been. Once the FSM is created for a heap we don't remove it even if somebody deletes all the rows from the corresponding relation. We don't think it is a useful optimization as it is quite likely that relation will again grow to the same size. Author: John Naylor, Amit Kapila Reviewed-by: Amit Kapila Tested-by: Mithun C Y Discussion: https://www.postgresql.org/message-id/CAJVSVGWvB13PzpbLEecFuGFc5V2fsO736BsdTakPiPAcdMM5tQ@mail.gmail.com	7 years ago
Amit Kapila	a23676503b	Revert "Avoid creation of the free space map for small heap relations." ... This reverts commit ac88d2962a.	7 years ago
Amit Kapila	ac88d2962a	Avoid creation of the free space map for small heap relations. ... Previously, all heaps had FSMs. For very small tables, this means that the FSM took up more space than the heap did. This is wasteful, so now we refrain from creating the FSM for heaps with 4 pages or fewer. If the last known target block has insufficient space, we still try to insert into some other page before giving up and extending the relation, since doing otherwise leads to table bloat. Testing showed that trying every page penalized performance slightly, so we compromise and try every other page. This way, we visit at most two pages. Any pages with wasted free space become visible at next relation extension, so we still control table bloat. As a bonus, directly attempting one or two pages can even be faster than consulting the FSM would have been. Once the FSM is created for a heap we don't remove it even if somebody deletes all the rows from the corresponding relation. We don't think it is a useful optimization as it is quite likely that relation will again grow to the same size. Author: John Naylor with design inputs and some code contribution by Amit Kapila Reviewed-by: Amit Kapila Tested-by: Mithun C Y Discussion: https://www.postgresql.org/message-id/CAJVSVGWvB13PzpbLEecFuGFc5V2fsO736BsdTakPiPAcdMM5tQ@mail.gmail.com	7 years ago
Alvaro Herrera	bef5fcc36b	pgstatindex, pageinspect: handle partitioned indexes ... Commit 8b08f7d482 failed to update these modules to at least give non-broken error messages for partitioned indexes. Add appropriate error support to them. Peter G. was complaining about a problem of unfriendly error messages; while we haven't fixed that yet, subsequent discussion let to discovery of these unhandled cases. Author: Michaël Paquier Reported-by: Peter Geoghegan Discussion: https://postgr.es/m/CAH2-WzkOKptQiE51Bh4_xeEHhaBwHkZkGtKizrFMgEkfUuRRQg@mail.gmail.com	7 years ago
Tom Lane	18869e202b	Fix new test case to not be endian-dependent. ... Per buildfarm. Discussion: https://postgr.es/m/ec295792-a69f-350f-6287-25a20e8f31d5@gmail.com	8 years ago
Tom Lane	39cfe86195	Fix incorrect computations of length of null bitmap in pageinspect. ... Instead of using our standard macro for this calculation, this code did it itself ... and got it wrong, leading to incorrect display of the null bitmap in some cases. Noted and fixed by Maksim Milyutin. In passing, remove a uselessly duplicative error check. Errors were introduced in commit d6061f83a; back-patch to 9.6 where that came in. Maksim Milyutin, reviewed by Andrey Borodin Discussion: https://postgr.es/m/ec295792-a69f-350f-6287-25a20e8f31d5@gmail.com	8 years ago
Peter Eisentraut	fef2bcdcba	pageinspect: Add page_checksum function ... Author: Tomas Vondra <tomas.vondra@2ndquadrant.com> Reviewed-by: Ashutosh Sharma <ashu.coek88@gmail.com>	9 years ago
Peter Eisentraut	a02731cb10	pageinspect: Add test for page_header function	9 years ago
Stephen Frost	c08d82f38e	Add relkind checks to certain contrib modules ... The contrib extensions pageinspect, pg_visibility and pgstattuple only work against regular relations which have storage. They don't work against foreign tables, partitioned (parent) tables, views, et al. Add checks to the user-callable functions to return a useful error message to the user if they mistakenly pass an invalid relation to a function which doesn't accept that kind of relation. In passing, improve some of the existing checks to use ereport() instead of elog(), add a function to consolidate common checks where appropriate, and add some regression tests. Author: Amit Langote, with various changes by me Reviewed by: Michael Paquier and Corey Huinker Discussion: https://postgr.es/m/ab91fd9d-4751-ee77-c87b-4dd704c1e59c@lab.ntt.co.jp	9 years ago
Peter Eisentraut	00a86856c1	pageinspect: Make page test more portable ... Choose test data that makes the output independent of endianness.	9 years ago
Peter Eisentraut	adfb81d9e1	pageinspect: Add tests	9 years ago