11 Commits (e35add48ccc2e5aa94de360f1a43c6c150bda54a)

Author	SHA1	Message	Date
Peter Eisentraut	95b856de23	Remove incidental md5() function use from test ... To allow test to pass in OpenSSL FIPS mode, similar to 657f5f223e, for a new test that has been added since. Reviewed-by: Tomas Vondra <tomas@vondra.me> Discussion: https://www.postgresql.org/message-id/86763810-70a1-4872-8ba7-1676f788e5a2@eisentraut.org	2 years ago
Tomas Vondra	0bd4b0689b	Stabilize test of BRIN parallel create ... As explained in 4d916dd876, the test instability is caused by delayed cleanup of deleted rows. This commit removes the DELETE, stabilizing the test without accidentally disabling parallel builds. The intent of the delete however was to produce empty ranges, and test that the parallel index build populates those correctly. But there's another way to create empty ranges - partial indexes, which does not rely on cleanup of deleted rows. Idea to use partial indexes by Matthias van de Meent, patch by me. Discussion: https://postgr.es/m/95d9cd43-5a92-407c-b7e4-54cd303630fe%40enterprisedb.com	2 years ago
Tomas Vondra	a89cd7bfcd	Revert "Stabilize test of BRIN parallel create" ... This reverts commit 4d916dd876. The goal of that commit was to stabilize a test of parallel BRIN build, but using a TEMPORARY table disables parallel index builds on that table, making the test useless. Discussion: https://postgr.es/m/95d9cd43-5a92-407c-b7e4-54cd303630fe%40enterprisedb.com	2 years ago
Tomas Vondra	4d916dd876	Stabilize test of BRIN parallel create ... The test for parallel create of BRIN indexes added by commit 8225c2fd40 happens to be unstable - a background transaction (e.g. auto-analyze) may hold back global xmin for the initial VACUUM / CREATE INDEX. If the cleanup happens before the next CREATE INDEX, the indexes will not be exactly the same. This is the same issue as e2933a6e11, so fix it the same way by making the table TEMPORARY, which uses an up-to-date cutoff xmin that is not held back by other processes. Reported by Alexander Lakhin, who also suggested the fix. Author: Alexander Lakhin Discussion: https://postgr.es/m/b58901cd-a7cc-29c6-e2b1-e3d7317c3c69@gmail.com Discussion: https://postgr.es/m/2892135.1668976646@sss.pgh.pa.us	2 years ago
Tomas Vondra	8225c2fd40	Add regression test for BRIN parallel builds ... Adds a regression test for parallel CREATE INDEX for BRIN indexes, to improve coverage for BRIN code, particularly code to allow parallel index builds introduced by b437571714. The test is added to pageinspect, as that allows comparing the index to one built without parallelism. Another option would be to just build the index with parallelism and then check it produces correct results. But checking the index is exactly as if built without parallelism makes these query checks unnecessary. Discussion: https://postgr.es/m/1df00a66-db5a-4e66-809a-99b386a06d86%40enterprisedb.com	2 years ago
Tomas Vondra	dae761a87e	Add empty BRIN ranges during CREATE INDEX ... When building BRIN indexes, the brinbuildCallback only advances to the next page range when seeing a tuple that doesn't belong to the current one. This means that the index may end up missing ranges at the end of the table, if those pages do not contain any indexable tuples. We tend not to have completely empty pages at the end of a relation, but this also applies to partial indexes, where the tuples may simply not match the index predicate. This results in inefficient scans using the affected BRIN index - without the summaries, the page ranges have to be read and processed, which consumes I/O and possibly also CPU time. The existing code already added empty ranges for earlier parts of the table, this commit makes sure we add them for the ranges at the end of the table too. Patch by Matthias van de Meent, with review/improvements by me. Author: Matthias van de Meent Reviewed-by: Tomas Vondra Discussion: https://postgr.es/m/CAEze2WiMsPZg%3DxkvSF_jt4%3D69k6K7gz5B8V2wY3gCGZ%2B1BzCbQ%40mail.gmail.com	2 years ago
Michael Paquier	be753639d3	pageinspect: Fix crash with gist_page_items() ... Attempting to use this function with a raw page not coming from a GiST index would cause a crash, as it was missing the same sanity checks as gist_page_items_bytea(). This slightly refactors the code so as all the basic validation checks for GiST pages are done in a single routine, in the same fashion as the pageinspect functions for hash and BRIN. This fixes an issue similar to 076f4d9. A test is added to stress for this case. While on it, I have added a similar test for brin_page_items() with a combination make of a valid GiST index and a raw btree page. This one was already protected, but it was not tested. Reported-by: Egor Chindyaskin Author: Dmitry Koval Discussion: https://postgr.es/m/17815-fc4a2d3b74705703@postgresql.org Backpatch-through: 14	3 years ago
Michael Paquier	cd4868a570	pageinspect: Fix handling of all-zero pages ... Getting from get_raw_page() an all-zero page is considered as a valid case by the buffer manager and it can happen for example when finding a corrupted page with zero_damaged_pages enabled (using zero_damaged_pages to look at corrupted pages happens), or after a crash when a relation file is extended before any WAL for its new data is generated (before a vacuum or autovacuum job comes in to do some cleanup). However, all the functions of pageinspect, as of the index AMs (except hash that has its own idea of new pages), heap, the FSM or the page header have never worked with all-zero pages, causing various crashes when going through the page internals. This commit changes all the pageinspect functions to be compliant with all-zero pages, where the choice is made to return NULL or no rows for SRFs when finding a new page. get_raw_page() still works the same way, returning a batch of zeros in the bytea of the page retrieved. A hard error could be used but NULL, while more invasive, is useful when scanning relation files in full to get a batch of results for a single relation in one query. Tests are added for all the code paths impacted. Reported-by: Daria Lepikhova Author: Michael Paquier Discussion: https://postgr.es/m/561e187b-3549-c8d5-03f5-525c14e65bd0@postgrespro.ru Backpatch-through: 10	4 years ago
Michael Paquier	291e517a4d	pageinspect: Add more sanity checks to prevent out-of-bound reads ... A couple of code paths use the special area on the page passed by the function caller, expecting to find some data in it. However, feeding an incorrect page can lead to out-of-bound reads when trying to access the page special area (like a heap page that has no special area, leading PageGetSpecialPointer() to grab a pointer outside the allocated page). The functions used for hash and btree indexes have some protection already against that, while some other functions using a relation OID as argument would make sure that the access method involved is correct, but functions taking in input a raw page without knowing the relation the page is attached to would run into problems. This commit improves the set of checks used in the code paths of BRIN, btree (including one check if a leaf page is found with a non-zero level), GIN and GiST to verify that the page given in input has a special area size that fits with each access method, which is done though PageGetSpecialSize(), becore calling PageGetSpecialPointer(). The scope of the checks done is limited to work with pages that one would pass after getting a block with get_raw_page(), as it is possible to craft byteas that could bypass existing code paths. Having too many checks would also impact the usability of pageinspect, as the existing code is very useful to look at the content details in a corrupted page, so the focus is really to avoid out-of-bound reads as this is never a good thing even with functions whose execution is limited to superusers. The safest approach could be to rework the functions so as these fetch a block using a relation OID and a block number, but there are also cases where using a raw page is useful. Tests are added to cover all the code paths that needed such checks, and an error message for hash indexes is reworded to fit better with what this commit adds. Reported-By: Alexander Lakhin Author: Julien Rouhaud, Michael Paquier Discussion: https://postgr.es/m/16527-ef7606186f0610a1@postgresql.org Discussion: https://postgr.es/m/561e187b-3549-c8d5-03f5-525c14e65bd0@postgrespro.ru Backpatch-through: 10	4 years ago
Michael Paquier	076f4d9539	pageinspect: Fix handling of page sizes and AM types ... This commit fixes a set of issues related to the use of the SQL functions in this module when the caller is able to pass down raw page data as input argument: - The page size check was fuzzy in a couple of places, sometimes looking after only a sub-range, but what we are looking for is an exact match on BLCKSZ. After considering a few options here, I have settled down to do a generalization of get_page_from_raw(). Most of the SQL functions already used that, and this is not strictly required if not accessing an 8-byte-wide value from a raw page, but this feels safer in the long run for alignment-picky environment, particularly if a code path begins to access such values. This also reduces the number of strings that need to be translated. - The BRIN function brin_page_items() uses a Relation but it did not check the access method of the opened index, potentially leading to crashes. All the other functions in need of a Relation already did that. - Some code paths could fail on elog(), but we should to use ereport() for failures that can be triggered by the user. Tests are added to stress all the cases that are fixed as of this commit, with some junk raw pages (\set VERBOSITY ensures that this works across all page sizes) and unexpected index types when functions open relations. Author: Michael Paquier, Justin Prysby Discussion: https://postgr.es/m/20220218030020.GA1137@telsasoft.com Backpatch-through: 10	4 years ago
Peter Eisentraut	adfb81d9e1	pageinspect: Add tests	9 years ago