24 Commits (7df2c1f8daeb361133ac8bdeaf59ceb0484e315a)

Author	SHA1	Message	Date
Robert Haas	3452dc5240	Push tuple limits through Gather and Gather Merge. ... If we only need, say, 10 tuples in total, then we certainly don't need more than 10 tuples from any single process. Pushing down the limit lets workers exit early when possible. For Gather Merge, there is an additional benefit: a Sort immediately below the Gather Merge can be done as a bounded sort if there is an applicable limit. Robert Haas and Tom Lane Discussion: http://postgr.es/m/CA+TgmoYa3QKKrLj5rX7UvGqhH73G1Li4B-EKxrmASaca2tFu9Q@mail.gmail.com	8 years ago
Andres Freund	d36f7efb39	Add minimal regression test for blessed record type transfer. ... Test that blessed records can be transferred through a TupleQueue and correctly decoded by another backend. While touching the file, make sure that force_parallel_mode settings only cover relevant tests. Author: Thomas Munro, editorialized by Andres Freund Reviewed-By: Andres Freund Discussion: https://postgr.es/m/20170823054644.efuzftxjpfi6wwqs%40alap3.anarazel.de	8 years ago
Tom Lane	a20aac890a	Temporarily revert test case from a2b70c89ca. ... That code patch was good as far as it went, but the associated test case has exposed fundamental brain damage in the parallel scan mechanism, which is going to take nontrivial work to correct. In the interests of getting the buildfarm back to green so that unrelated work can proceed, let's temporarily remove the test case.	8 years ago
Tom Lane	a2b70c89ca	Fix ExecReScanGatherMerge. ... Not surprisingly, since it'd never ever been tested, ExecReScanGatherMerge didn't work. Fix it, and add a regression test case to exercise it. Amit Kapila Discussion: https://postgr.es/m/CAA4eK1JkByysFJNh9M349u_nNjqETuEnY_y1VUc_kJiU0bxtaQ@mail.gmail.com	8 years ago
Andres Freund	d2bc501573	Expand coverage of parallel gather merge a bit. ... Previously paths reaching heap_compare_slots weren't covered. Author: Rushabh Lathia Reviewed-By: Andres Freund Discussion: https://postgr.es/m/CAGPqQf3C+3PBujb+7m=ceWeii4-vBY=XS99LjzrpkpefvzJbFg@mail.gmail.com https://postgr.es/m/27200.1502482851@sss.pgh.pa.us Backpatch: 10, where gather merge was introduced	8 years ago
Tom Lane	39151781c8	Fix testing of parallel-safety of SubPlans. ... is_parallel_safe() supposed that the only relevant property of a SubPlan was the parallel safety of the referenced subplan tree. This is wrong: the testexpr or args subtrees might contain parallel-unsafe stuff, as demonstrated by the test case added here. However, just recursing into the subtrees fails in a different way: we'll typically find PARAM_EXEC Params representing the subplan's output columns in the testexpr. The previous coding supposed that any Param must be treated as parallel-restricted, so that a naive attempt at fixing this disabled parallel pushdown of SubPlans altogether. We must instead determine, for any visited Param, whether it is one that would be computed by a surrounding SubPlan node; if so, it's safe to push down along with the SubPlan node. We might later be able to extend this logic to cope with Params used for correlated subplans and other cases; but that's a task for v11 or beyond. Tom Lane and Amit Kapila Discussion: https://postgr.es/m/7064.1492022469@sss.pgh.pa.us	9 years ago
Andres Freund	d611517fc4	Fix parallel bitmapscan tests on builds without USE_PREFETCH. ... This was broken in 5a5931533e.	9 years ago
Andres Freund	b2ff37d43c	Add minimal test for EXPLAIN ANALYZE of parallel query. ... This displays the number of workers launched, thus the test is dependant on configuration to some degree. We'll see whether that turns out ot be a problem. Author: Rafia Sabih Discussion: https://postgr.es/m/20170331185540.zmsue4ndvqtnayqw@alap3.anarazel.de	9 years ago
Andres Freund	5a5931533e	Increase parallel bitmap scan test coverage. ... Author: Dilip Kumar Discussion: https://postgr.es/m/20170331184603.qcp7t4md5bzxbx32@alap3.anarazel.de	9 years ago
Robert Haas	355d3993c5	Add a Gather Merge executor node. ... Like Gather, we spawn multiple workers and run the same plan in each one; however, Gather Merge is used when each worker produces the same output ordering and we want to preserve that output ordering while merging together the streams of tuples from various workers. (In a way, Gather Merge is like a hybrid of Gather and MergeAppend.) This works out to a win if it saves us from having to perform an expensive Sort. In cases where only a small amount of data would need to be sorted, it may actually be faster to use a regular Gather node and then sort the results afterward, because Gather Merge sometimes needs to wait synchronously for tuples whereas a pure Gather generally doesn't. But if this avoids an expensive sort then it's a win. Rushabh Lathia, reviewed and tested by Amit Kapila, Thomas Munro, and Neha Sharma, and reviewed and revised by me. Discussion: http://postgr.es/m/CAGPqQf09oPX-cQRpBKS0Gq49Z+m6KBxgxd_p9gX8CKk_d75HoQ@mail.gmail.com	9 years ago
Robert Haas	f35742ccb7	Support parallel bitmap heap scans. ... The index is scanned by a single process, but then all cooperating processes can iterate jointly over the resulting set of heap blocks. In the future, we might also want to support using a parallel bitmap index scan to set up for a parallel bitmap heap scan, but that's a job for another day. Dilip Kumar, with some corrections and cosmetic changes by me. The larger patch set of which this is a part has been reviewed and tested by (at least) Andres Freund, Amit Khandekar, Tushar Ahuja, Rafia Sabih, Haribabu Kommi, Thomas Munro, and me. Discussion: http://postgr.es/m/CAFiTN-uc4=0WxRGfCzs-xfkMYcSEWUC-Fon6thkJGjkh9i=13A@mail.gmail.com	9 years ago
Robert Haas	3bc7dafa9b	Consider parallel merge joins. ... Commit 45be99f8cd took the position that performing a merge join in parallel was not likely to work out well, but this conclusion was greeted with skepticism even at the time. Whether it was true then or not, it's clearly not true any more now that we have parallel index scan. Dilip Kumar, reviewed by Amit Kapila and by me. Discussion: http://postgr.es/m/CAFiTN-v3=cM6nyFwFGp0fmvY4=kk79Hq9Fgu0u8CSJ-EEq1Tiw@mail.gmail.com	9 years ago
Robert Haas	0414b26bac	Add optimizer and executor support for parallel index-only scans. ... Commit 5262f7a4fc added similar support for parallel index scans; this extends that work to index-only scans. As with parallel index scans, this requires support from the index AM, so currently parallel index-only scans will only be possible for btree indexes. Rafia Sabih, reviewed and tested by Rahila Syed, Tushar Ahuja, and Amit Kapila Discussion: http://postgr.es/m/CAOGQiiPEAs4C=TBp0XShxBvnWXuzGL2u++Hm1=qnCpd6_Mf8Fw@mail.gmail.com	9 years ago
Robert Haas	5262f7a4fc	Add optimizer and executor support for parallel index scans. ... In combination with 569174f1be, which taught the btree AM how to perform parallel index scans, this allows parallel index scan plans on btree indexes. This infrastructure should be general enough to support parallel index scans for other index AMs as well, if someone updates them to support parallel scans. Amit Kapila, reviewed and tested by Anastasia Lubennikova, Tushar Ahuja, and Haribabu Kommi, and me.	9 years ago
Robert Haas	51ee6f3160	Replace min_parallel_relation_size with two new GUCs. ... When min_parallel_relation_size was added, the only supported type of parallel scan was a parallel sequential scan, but there are pending patches for parallel index scan, parallel index-only scan, and parallel bitmap heap scan. Those patches introduce two new types of complications: first, what's relevant is not really the total size of the relation but the portion of it that we will scan; and second, index pages and heap pages shouldn't necessarily be treated in exactly the same way. Typically, the number of index pages will be quite small, but that doesn't necessarily mean that a parallel index scan can't pay off. Therefore, we introduce min_parallel_table_scan_size, which works out a degree of parallelism for scans based on the number of table pages that will be scanned (and which is therefore equivalent to min_parallel_relation_size for parallel sequential scans) and also min_parallel_index_scan_size which can be used to work out a degree of parallelism based on the number of index pages that will be scanned. Amit Kapila and Robert Haas Discussion: http://postgr.es/m/CAA4eK1KowGSYYVpd2qPpaPPA5R90r++QwDFbrRECTE9H_HvpOg@mail.gmail.com Discussion: http://postgr.es/m/CAA4eK1+TnM4pXQbvn7OXqam+k_HZqb0ROZUMxOiL6DWJYCyYow@mail.gmail.com	9 years ago
Robert Haas	5e6d8d2bbb	Allow parallel workers to execute subplans. ... This doesn't do anything to make Param nodes anything other than parallel-restricted, so this only helps with uncorrelated subplans, and it's not necessarily very cheap because each worker will run the subplan separately (just as a Hash Join will build a separate copy of the hash table in each participating process), but it's a first step toward supporting cases that are more likely to help in practice, and is occasionally useful on its own. Amit Kapila, reviewed and tested by Rafia Sabih, Dilip Kumar, and me. Discussion: http://postgr.es/m/CAA4eK1+e8Z45D2n+rnDMDYsVEb5iW7jqaCH_tvPMYau=1Rru9w@mail.gmail.com	9 years ago
Peter Eisentraut	f9472d7256	Run select_parallel test by itself ... Remove the plpgsql wrapping that hides the context. So now the test will fail if the work doesn't actually happen in a parallel worker. Run the test in its own test group to ensure it won't run out of resources for that.	9 years ago
Tom Lane	598aa194af	Still another try at fixing scanjoin_target insertion into parallel plans. ... The previous code neglected the fact that the scanjoin_target might carry sortgroupref labelings that we need to absorb. Instead, do create_projection_path() unconditionally, and tweak the path's cost estimate after the fact. (I'm now convinced that we ought to refactor the way we account for sometimes not needing a separate projection step, but right now is not the time for that sort of cleanup.) Problem identified by Amit Kapila, patch by me.	10 years ago
Robert Haas	54f5c5150f	Try again to fix the way the scanjoin_target is used with partial paths. ... Commit 04ae11f62e removed some broken code to apply the scan/join target to partial paths, but its theory that this processing step is totally unnecessary turns out to be wrong. Put similar code back again, but this time, check for parallel-safety and avoid in-place modifications to paths that may already have been used as part of some other path. (This is not an entirely elegant solution to this problem; it might be better, for example, to postpone generate_gather_paths for the topmost scan/join rel until after the scan/join target has been applied. But this is not the time for such redesign work.) Amit Kapila and Robert Haas	10 years ago
Robert Haas	292794f82b	Remove PID from 'parallel worker' context message. ... Discussion: <bfd204ab-ab1a-792a-b345-0274a09a4b5f@2ndquadrant.com>	10 years ago
Robert Haas	103512cee9	Attempt to fix broken regression test. ... In commit 8c1d9d56e9, I attempted to add a regression test that would fail if the target list was pushed into a parallel worker, but due to brain fade on my part, it just randomly fails whether anything bad or not, because the error check inside the parallel_restricted() function tests whether there is *any process in the system* that is not connected to a client, not whether the process running the query is not connected to a client. A little experimentation has left me pessimistic about the prospects of doing better here in a short amount of time, so let's just fall back to checking that the plan is as we expect and leave the execution-time check for another day.	10 years ago
Robert Haas	8c1d9d56e9	Add regression test for 04ae11f62e. ... The code in this area needs further revision, and it would be best not to re-break the things we've already fixed. Per a gripe from Tom Lane.	10 years ago
Robert Haas	12f862099d	Fix regression test for force_parallel_mode=on. ... Commit 14a254fb52 managed not to exercise the code it was intended to test, and the comment explaining why no "parallel worker" line showed up in the context wasn't right. Amit Kapila, tweaked by me per Amit's analysis.	10 years ago
Noah Misch	14a254fb52	Test parallel query essentials in "make check". ... Clément Prévost and Peter Eisentraut	10 years ago