open-dsi
/
postgres
mirror of https://github.com/postgres/postgres
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
38611 Commits
38 Branches
662 Tags
732 MiB
Tag: Branch: Tree: 4b8e24b9ad
Branches Tags
${ item.name }
Create tag ${ searchTerm }
Create branch ${ searchTerm }
from '4b8e24b9ad'
${ noResults }
postgres/src/backend/optimizer/plan/setrefs.c

2398 lines
70 KiB
Raw Normal View History Unescape

Postgres95 1.01 Distribution - Virgin Sources
30 years ago
/*-------------------------------------------------------------------------
*
Change my-function-name-- to my_function_name, and optimizer renames.
27 years ago
* setrefs.c
Further planner/optimizer cleanups. Move all set_tlist_references and fix_opids processing to a single recursive pass over the plan tree executed at the very tail end of planning, rather than haphazardly here and there at different places. Now that tlist Vars do not get modified until the very end, it's possible to get rid of the klugy var_equal and match_varid partial-matching routines, and just use plain equal() throughout the optimizer. This is a step towards allowing merge and hash joins to be done on expressions instead of only Vars ...
26 years ago
* Post-processing of a completed plan tree: fix references to subplan
Fix the plan-invalidation mechanism to treat regclass constants that refer to a relation as a reason to invalidate a plan when the relation changes. This handles scenarios such as dropping/recreating a sequence that is referenced by nextval('seq') in a cached plan. Rather than teach plancache.c all about digging through plan trees to find regclass Consts, we charge the planner's setrefs.c with making a list of the relation OIDs on which each plan depends. That way the list can be built cheaply during a plan tree traversal that has to happen anyway. Per bug #3662 and subsequent discussion.
18 years ago
* vars, compute regproc values for operators, etc
Postgres95 1.01 Distribution - Virgin Sources
30 years ago
*
Update copyright for 2015 Backpatch certain files through 9.0
11 years ago
* Portions Copyright (c) 1996-2015, PostgreSQL Global Development Group
Add: * Portions Copyright (c) 1996-2000, PostgreSQL, Inc to all files copyright Regents of Berkeley. Man, that's a lot of files.
26 years ago
* Portions Copyright (c) 1994, Regents of the University of California
Postgres95 1.01 Distribution - Virgin Sources
30 years ago
*
*
* IDENTIFICATION
Remove cvs keywords from all files.
15 years ago
* src/backend/optimizer/plan/setrefs.c
Postgres95 1.01 Distribution - Virgin Sources
30 years ago
*
*-------------------------------------------------------------------------
*/
#include "postgres.h"
Improve the plan cache invalidation mechanism to make it invalidate plans when user-defined functions used in a plan are modified. Also invalidate plans when schemas, operators, or operator classes are modified; but for these cases we just invalidate everything rather than tracking exact dependencies, since these types of objects seldom change in a production database. Tom Lane; loosely based on a patch by Martin Pihlak.
17 years ago
#include "access/transam.h"
Fix the plan-invalidation mechanism to treat regclass constants that refer to a relation as a reason to invalidate a plan when the relation changes. This handles scenarios such as dropping/recreating a sequence that is referenced by nextval('seq') in a cached plan. Rather than teach plancache.c all about digging through plan trees to find regclass Consts, we charge the planner's setrefs.c with making a list of the relation OIDs on which each plan depends. That way the list can be built cheaply during a plan tree traversal that has to happen anyway. Per bug #3662 and subsequent discussion.
18 years ago
#include "catalog/pg_type.h"
Fix oversight in planning of GROUP queries: when an expression is used as both a GROUP BY item and an output expression, the top-level Group node should just copy up the evaluated expression value from its input, rather than re-evaluating the expression. Aside from any performance benefit this might offer, this avoids a crash when there is a sub-SELECT in said expression.
25 years ago
#include "nodes/makefuncs.h"
Move exprType(), exprTypmod(), expression_tree_walker(), and related routines into nodes/nodeFuncs, so as to reduce wanton cross-subsystem #includes inside the backend. There's probably more that should be done along this line, but this is a start anyway.
17 years ago
#include "nodes/nodeFuncs.h"
Rearrange planner to save the whole PlannerInfo (subroot) for a subquery. Formerly, set_subquery_pathlist and other creators of plans for subqueries saved only the rangetable and rowMarks lists from the lower-level PlannerInfo. But there's no reason not to remember the whole PlannerInfo, and indeed this turns out to simplify matters in a number of places. The immediate reason for doing this was so that the subroot will still be accessible when we're trying to extract column statistics out of an already-planned subquery. But now that I've done it, it seems like a good code-beautification effort in its own right. I also chose to get rid of the transient subrtable and subrowmark fields in SubqueryScan nodes, in favor of having setrefs.c look up the subquery's RelOptInfo. That required changing all the APIs in setrefs.c to pass PlannerInfo not PlannerGlobal, which was a large but quite mechanical transformation. One side-effect not foreseen at the beginning is that this finally broke inheritance_planner's assumption that replanning the same subquery RTE N times would necessarily give interchangeable results each time. That assumption was always pretty risky, but now we really have to make a separate RTE for each instance so that there's a place to carry the separate subroots.
14 years ago
#include "optimizer/pathnode.h"
Postgres95 1.01 Distribution - Virgin Sources
30 years ago
#include "optimizer/planmain.h"
Better fix for permissions tests in excluded subqueries. This reverts the code changes in 50c137487c96e629e0e5372bb3d1b5f1a2f71a88, which turned out to induce crashes and not completely fix the problem anyway. That commit only considered single subqueries that were excluded by constraint-exclusion logic, but actually the problem also exists for subqueries that are appendrel members (ie part of a UNION ALL list). In such cases we can't add a dummy subpath to the appendrel's AppendPath list without defeating the logic that recognizes when an appendrel is completely excluded. Instead, fix the problem by having setrefs.c scan the rangetable an extra time looking for subqueries that didn't get into the plan tree. (This approach depends on the 9.2 change that made set_subquery_pathlist generate dummy paths for excluded single subqueries, so that the exclusion behavior is the same for single subqueries and appendrel members.) Note: it turns out that the appendrel form of the missed-permissions-checks bug exists as far back as 8.4. However, since the practical effect of that bug seems pretty minimal, consensus is to not attempt to fix it in the back branches, at least not yet. Possibly we could back-port this patch once it's gotten a reasonable amount of testing in HEAD. For the moment I'm just going to revert the previous patch in 9.2.
13 years ago
#include "optimizer/planner.h"
Postgres95 1.01 Distribution - Virgin Sources
30 years ago
#include "optimizer/tlist.h"
Restructure SELECT INTO's parsetree representation into CreateTableAsStmt. Making this operation look like a utility statement seems generally a good idea, and particularly so in light of the desire to provide command triggers for utility statements. The original choice of representing it as SELECT with an IntoClause appendage had metastasized into rather a lot of places, unfortunately, so that this patch is a great deal more complicated than one might at first expect. In particular, keeping EXPLAIN working for SELECT INTO and CREATE TABLE AS subcommands required restructuring some EXPLAIN-related APIs. Add-on code that calls ExplainOnePlan or ExplainOneUtility, or uses ExplainOneQuery_hook, will need adjustment. Also, the cases PREPARE ... SELECT INTO and CREATE RULE ... SELECT INTO, which formerly were accepted though undocumented, are no longer accepted. The PREPARE case can be replaced with use of CREATE TABLE AS EXECUTE. The CREATE RULE case doesn't seem to have much real-world use (since the rule would work only once before failing with "table already exists"), so we'll not bother with that one. Both SELECT INTO and CREATE TABLE AS still return a command tag of "SELECT nnnn". There was some discussion of returning "CREATE TABLE nnnn", but for the moment backwards compatibility wins the day. Andres Freund and Tom Lane
14 years ago
#include "tcop/utility.h"
Support expressions of the form 'scalar op ANY (array)' and 'scalar op ALL (array)', where the operator is applied between the lefthand scalar and each element of the array. The operator must yield boolean; the result of the construct is the OR or AND of the per-element results, respectively. Original coding by Joe Conway, after an idea of Peter's. Rewritten by Tom to keep the implementation strictly separate from subqueries.
23 years ago
#include "utils/lsyscache.h"
Improve the plan cache invalidation mechanism to make it invalidate plans when user-defined functions used in a plan are modified. Also invalidate plans when schemas, operators, or operator classes are modified; but for these cases we just invalidate everything rather than tracking exact dependencies, since these types of objects seldom change in a production database. Tom Lane; loosely based on a patch by Martin Pihlak.
17 years ago
#include "utils/syscache.h"
Restructure representation of join alias variables. An explicit JOIN now has an RTE of its own, and references to its outputs now are Vars referencing the JOIN RTE, rather than CASE-expressions. This allows reverse-listing in ruleutils.c to use the correct alias easily, rather than painfully reverse-engineering the alias namespace as it used to do. Also, nested FULL JOINs work correctly, because the result of the inner joins are simple Vars that the planner can cope with. This fixes a bug reported a couple times now, notably by Tatsuo on 18-Nov-01. The alias Vars are expanded into COALESCE expressions where needed at the very end of planning, rather than during parsing. Also, beginnings of support for showing plan qualifier expressions in EXPLAIN. There are probably still cases that need work. initdb forced due to change of stored-rule representation.
24 years ago
Postgres95 1.01 Distribution - Virgin Sources
30 years ago
Revise searching of subplan target lists to use something more efficient than tlist_member calls. Building a large join tlist is still O(N^2), but with a much smaller constant factor than before.
21 years ago
typedef struct
{
Index varno; /* RT index of Var */
AttrNumber varattno; /* attr number of Var */
AttrNumber resno; /* TLE position of Var */
} tlist_vinfo;
typedef struct
{
List *tlist; /* underlying target list */
int num_vars; /* number of plain Var tlist entries */
Add a concept of "placeholder" variables to the planner. These are variables that represent some expression that we desire to compute below the top level of the plan, and then let that value "bubble up" as though it were a plain Var (ie, a column value). The immediate application is to allow sub-selects to be flattened even when they are below an outer join and have non-nullable output expressions. Formerly we couldn't flatten because such an expression wouldn't properly go to NULL when evaluated above the outer join. Now, we wrap it in a PlaceHolderVar and arrange for the actual evaluation to occur below the outer join. When the resulting Var bubbles up through the join, it will be set to NULL if necessary, yielding the correct results. This fixes a planner limitation that's existed since 7.1. In future we might want to use this mechanism to re-introduce some form of Hellerstein's "expensive functions" optimization, ie place the evaluation of an expensive function at the most suitable point in the plan tree.
17 years ago
bool has_ph_vars; /* are there PlaceHolderVar entries? */
bool has_non_vars; /* are there other entries? */
Some more FLEXIBLE_ARRAY_MEMBER fixes.
11 years ago
tlist_vinfo vars[FLEXIBLE_ARRAY_MEMBER]; /* has num_vars entries */
} indexed_tlist;
Revise searching of subplan target lists to use something more efficient than tlist_member calls. Building a large join tlist is still O(N^2), but with a much smaller constant factor than before.
21 years ago
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
typedef struct
{
Rearrange planner to save the whole PlannerInfo (subroot) for a subquery. Formerly, set_subquery_pathlist and other creators of plans for subqueries saved only the rangetable and rowMarks lists from the lower-level PlannerInfo. But there's no reason not to remember the whole PlannerInfo, and indeed this turns out to simplify matters in a number of places. The immediate reason for doing this was so that the subroot will still be accessible when we're trying to extract column statistics out of an already-planned subquery. But now that I've done it, it seems like a good code-beautification effort in its own right. I also chose to get rid of the transient subrtable and subrowmark fields in SubqueryScan nodes, in favor of having setrefs.c look up the subquery's RelOptInfo. That required changing all the APIs in setrefs.c to pass PlannerInfo not PlannerGlobal, which was a large but quite mechanical transformation. One side-effect not foreseen at the beginning is that this finally broke inheritance_planner's assumption that replanning the same subquery RTE N times would necessarily give interchangeable results each time. That assumption was always pretty risky, but now we really have to make a separate RTE for each instance so that there's a place to carry the separate subroots.
14 years ago
PlannerInfo *root;
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
int rtoffset;
Re-run pgindent with updated list of typedefs. (Updated README should avoid this problem in the future.)
18 years ago
} fix_scan_expr_context;
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
Ye-old pgindent run. Same 4-space tabs.
26 years ago
typedef struct
{
Rearrange planner to save the whole PlannerInfo (subroot) for a subquery. Formerly, set_subquery_pathlist and other creators of plans for subqueries saved only the rangetable and rowMarks lists from the lower-level PlannerInfo. But there's no reason not to remember the whole PlannerInfo, and indeed this turns out to simplify matters in a number of places. The immediate reason for doing this was so that the subroot will still be accessible when we're trying to extract column statistics out of an already-planned subquery. But now that I've done it, it seems like a good code-beautification effort in its own right. I also chose to get rid of the transient subrtable and subrowmark fields in SubqueryScan nodes, in favor of having setrefs.c look up the subquery's RelOptInfo. That required changing all the APIs in setrefs.c to pass PlannerInfo not PlannerGlobal, which was a large but quite mechanical transformation. One side-effect not foreseen at the beginning is that this finally broke inheritance_planner's assumption that replanning the same subquery RTE N times would necessarily give interchangeable results each time. That assumption was always pretty risky, but now we really have to make a separate RTE for each instance so that there's a place to carry the separate subroots.
14 years ago
PlannerInfo *root;
Revise searching of subplan target lists to use something more efficient than tlist_member calls. Building a large join tlist is still O(N^2), but with a much smaller constant factor than before.
21 years ago
indexed_tlist *outer_itlist;
indexed_tlist *inner_itlist;
Further planner/optimizer cleanups. Move all set_tlist_references and fix_opids processing to a single recursive pass over the plan tree executed at the very tail end of planning, rather than haphazardly here and there at different places. Now that tlist Vars do not get modified until the very end, it's possible to get rid of the klugy var_equal and match_varid partial-matching routines, and just use plain equal() throughout the optimizer. This is a step towards allowing merge and hash joins to be done on expressions instead of only Vars ...
26 years ago
Index acceptable_rel;
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
int rtoffset;
Re-run pgindent with updated list of typedefs. (Updated README should avoid this problem in the future.)
18 years ago
} fix_join_expr_context;
Clean up routines in setrefs.c by replacing individual tree walking logic with expression_tree_walker/mutator calls.
26 years ago
Ye-old pgindent run. Same 4-space tabs.
26 years ago
typedef struct
{
Rearrange planner to save the whole PlannerInfo (subroot) for a subquery. Formerly, set_subquery_pathlist and other creators of plans for subqueries saved only the rangetable and rowMarks lists from the lower-level PlannerInfo. But there's no reason not to remember the whole PlannerInfo, and indeed this turns out to simplify matters in a number of places. The immediate reason for doing this was so that the subroot will still be accessible when we're trying to extract column statistics out of an already-planned subquery. But now that I've done it, it seems like a good code-beautification effort in its own right. I also chose to get rid of the transient subrtable and subrowmark fields in SubqueryScan nodes, in favor of having setrefs.c look up the subquery's RelOptInfo. That required changing all the APIs in setrefs.c to pass PlannerInfo not PlannerGlobal, which was a large but quite mechanical transformation. One side-effect not foreseen at the beginning is that this finally broke inheritance_planner's assumption that replanning the same subquery RTE N times would necessarily give interchangeable results each time. That assumption was always pretty risky, but now we really have to make a separate RTE for each instance so that there's a place to carry the separate subroots.
14 years ago
PlannerInfo *root;
Revise searching of subplan target lists to use something more efficient than tlist_member calls. Building a large join tlist is still O(N^2), but with a much smaller constant factor than before.
21 years ago
indexed_tlist *subplan_itlist;
Rearrange the implementation of index-only scans. This commit changes index-only scans so that data is read directly from the index tuple without first generating a faux heap tuple. The only immediate benefit is that indexes on system columns (such as OID) can be used in index-only scans, but this is necessary infrastructure if we are ever to support index-only scans on expression indexes. The executor is now ready for that, though the planner still needs substantial work to recognize the possibility. To do this, Vars in index-only plan nodes have to refer to index columns not heap columns. I introduced a new special varno, INDEX_VAR, to mark such Vars to avoid confusion. (In passing, this commit renames the two existing special varnos to OUTER_VAR and INNER_VAR.) This allows ruleutils.c to handle them with logic similar to what we use for subplan reference Vars. Since index-only scans are now fundamentally different from regular indexscans so far as their expression subtrees are concerned, I also chose to change them to have their own plan node type (and hence, their own executor source file).
14 years ago
Index newvarno;
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
int rtoffset;
Re-run pgindent with updated list of typedefs. (Updated README should avoid this problem in the future.)
18 years ago
} fix_upper_expr_context;
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
Fix the code that adds regclass constants to a plan's list of relation OIDs that it depends on for replan-forcing purposes. We need to consider plain OID constants too, because eval_const_expressions folds a RelabelType atop a Const to just a Const. This change could result in OID values that aren't really for tables getting added to the dependency list, but the worst-case consequence would be occasional useless replans. Per report from Gabriele Messineo.
17 years ago
/*
* Check if a Const node is a regclass value. We accept plain OID too,
* since a regclass Const will get folded to that type if it's an argument
* to oideq or similar operators. (This might result in some extraneous
* values in a plan's list of relation dependencies, but the worst result
* would be occasional useless replans.)
*/
#define ISREGCLASSCONST(con) \
(((con)->consttype == REGCLASSOID || (con)->consttype == OIDOID) && \
!(con)->constisnull)
Rearrange planner to save the whole PlannerInfo (subroot) for a subquery. Formerly, set_subquery_pathlist and other creators of plans for subqueries saved only the rangetable and rowMarks lists from the lower-level PlannerInfo. But there's no reason not to remember the whole PlannerInfo, and indeed this turns out to simplify matters in a number of places. The immediate reason for doing this was so that the subroot will still be accessible when we're trying to extract column statistics out of an already-planned subquery. But now that I've done it, it seems like a good code-beautification effort in its own right. I also chose to get rid of the transient subrtable and subrowmark fields in SubqueryScan nodes, in favor of having setrefs.c look up the subquery's RelOptInfo. That required changing all the APIs in setrefs.c to pass PlannerInfo not PlannerGlobal, which was a large but quite mechanical transformation. One side-effect not foreseen at the beginning is that this finally broke inheritance_planner's assumption that replanning the same subquery RTE N times would necessarily give interchangeable results each time. That assumption was always pretty risky, but now we really have to make a separate RTE for each instance so that there's a place to carry the separate subroots.
14 years ago
#define fix_scan_list(root, lst, rtoffset) \
((List *) fix_scan_expr(root, (Node *) (lst), rtoffset))
Clean up routines in setrefs.c by replacing individual tree walking logic with expression_tree_walker/mutator calls.
26 years ago
Better fix for permissions tests in excluded subqueries. This reverts the code changes in 50c137487c96e629e0e5372bb3d1b5f1a2f71a88, which turned out to induce crashes and not completely fix the problem anyway. That commit only considered single subqueries that were excluded by constraint-exclusion logic, but actually the problem also exists for subqueries that are appendrel members (ie part of a UNION ALL list). In such cases we can't add a dummy subpath to the appendrel's AppendPath list without defeating the logic that recognizes when an appendrel is completely excluded. Instead, fix the problem by having setrefs.c scan the rangetable an extra time looking for subqueries that didn't get into the plan tree. (This approach depends on the 9.2 change that made set_subquery_pathlist generate dummy paths for excluded single subqueries, so that the exclusion behavior is the same for single subqueries and appendrel members.) Note: it turns out that the appendrel form of the missed-permissions-checks bug exists as far back as 8.4. However, since the practical effect of that bug seems pretty minimal, consensus is to not attempt to fix it in the back branches, at least not yet. Possibly we could back-port this patch once it's gotten a reasonable amount of testing in HEAD. For the moment I'm just going to revert the previous patch in 9.2.
13 years ago
static void add_rtes_to_flat_rtable(PlannerInfo *root, bool recursing);
static void flatten_unplanned_rtes(PlannerGlobal *glob, RangeTblEntry *rte);
static bool flatten_rtes_walker(Node *node, PlannerGlobal *glob);
static void add_rte_to_flat_rtable(PlannerGlobal *glob, RangeTblEntry *rte);
Rearrange planner to save the whole PlannerInfo (subroot) for a subquery. Formerly, set_subquery_pathlist and other creators of plans for subqueries saved only the rangetable and rowMarks lists from the lower-level PlannerInfo. But there's no reason not to remember the whole PlannerInfo, and indeed this turns out to simplify matters in a number of places. The immediate reason for doing this was so that the subroot will still be accessible when we're trying to extract column statistics out of an already-planned subquery. But now that I've done it, it seems like a good code-beautification effort in its own right. I also chose to get rid of the transient subrtable and subrowmark fields in SubqueryScan nodes, in favor of having setrefs.c look up the subquery's RelOptInfo. That required changing all the APIs in setrefs.c to pass PlannerInfo not PlannerGlobal, which was a large but quite mechanical transformation. One side-effect not foreseen at the beginning is that this finally broke inheritance_planner's assumption that replanning the same subquery RTE N times would necessarily give interchangeable results each time. That assumption was always pretty risky, but now we really have to make a separate RTE for each instance so that there's a place to carry the separate subroots.
14 years ago
static Plan *set_plan_refs(PlannerInfo *root, Plan *plan, int rtoffset);
Rearrange the implementation of index-only scans. This commit changes index-only scans so that data is read directly from the index tuple without first generating a faux heap tuple. The only immediate benefit is that indexes on system columns (such as OID) can be used in index-only scans, but this is necessary infrastructure if we are ever to support index-only scans on expression indexes. The executor is now ready for that, though the planner still needs substantial work to recognize the possibility. To do this, Vars in index-only plan nodes have to refer to index columns not heap columns. I introduced a new special varno, INDEX_VAR, to mark such Vars to avoid confusion. (In passing, this commit renames the two existing special varnos to OUTER_VAR and INNER_VAR.) This allows ruleutils.c to handle them with logic similar to what we use for subplan reference Vars. Since index-only scans are now fundamentally different from regular indexscans so far as their expression subtrees are concerned, I also chose to change them to have their own plan node type (and hence, their own executor source file).
14 years ago
static Plan *set_indexonlyscan_references(PlannerInfo *root,
IndexOnlyScan *plan,
int rtoffset);
Rearrange planner to save the whole PlannerInfo (subroot) for a subquery. Formerly, set_subquery_pathlist and other creators of plans for subqueries saved only the rangetable and rowMarks lists from the lower-level PlannerInfo. But there's no reason not to remember the whole PlannerInfo, and indeed this turns out to simplify matters in a number of places. The immediate reason for doing this was so that the subroot will still be accessible when we're trying to extract column statistics out of an already-planned subquery. But now that I've done it, it seems like a good code-beautification effort in its own right. I also chose to get rid of the transient subrtable and subrowmark fields in SubqueryScan nodes, in favor of having setrefs.c look up the subquery's RelOptInfo. That required changing all the APIs in setrefs.c to pass PlannerInfo not PlannerGlobal, which was a large but quite mechanical transformation. One side-effect not foreseen at the beginning is that this finally broke inheritance_planner's assumption that replanning the same subquery RTE N times would necessarily give interchangeable results each time. That assumption was always pretty risky, but now we really have to make a separate RTE for each instance so that there's a place to carry the separate subroots.
14 years ago
static Plan *set_subqueryscan_references(PlannerInfo *root,
pgindent run for 8.3.
18 years ago
SubqueryScan *plan,
int rtoffset);
Teach the planner to remove SubqueryScan nodes from the plan if they aren't doing anything useful (ie, neither selection nor projection). Also, extend to SubqueryScan the hacks already in place to avoid unnecessary ExecProject calls when the result would just be the same tuple the subquery already delivered. This saves some overhead in UNION and other set operations, as well as avoiding overhead for unflatten-able subqueries. Per example from Sokolov Yura.
21 years ago
static bool trivial_subqueryscan(SubqueryScan *plan);
Code review for foreign/custom join pushdown patch. Commit e7cb7ee14555cc9c5773e2c102efd6371f6f2005 included some design decisions that seem pretty questionable to me, and there was quite a lot of stuff not to like about the documentation and comments. Clean up as follows: * Consider foreign joins only between foreign tables on the same server, rather than between any two foreign tables with the same underlying FDW handler function. In most if not all cases, the FDW would simply have had to apply the same-server restriction itself (far more expensively, both for lack of caching and because it would be repeated for each combination of input sub-joins), or else risk nasty bugs. Anyone who's really intent on doing something outside this restriction can always use the set_join_pathlist_hook. * Rename fdw_ps_tlist/custom_ps_tlist to fdw_scan_tlist/custom_scan_tlist to better reflect what they're for, and allow these custom scan tlists to be used even for base relations. * Change make_foreignscan() API to include passing the fdw_scan_tlist value, since the FDW is required to set that. Backwards compatibility doesn't seem like an adequate reason to expect FDWs to set it in some ad-hoc extra step, and anyway existing FDWs can just pass NIL. * Change the API of path-generating subroutines of add_paths_to_joinrel, and in particular that of GetForeignJoinPaths and set_join_pathlist_hook, so that various less-used parameters are passed in a struct rather than as separate parameter-list entries. The objective here is to reduce the probability that future additions to those parameter lists will result in source-level API breaks for users of these hooks. It's possible that this is even a small win for the core code, since most CPU architectures can't pass more than half a dozen parameters efficiently anyway. I kept root, joinrel, outerrel, innerrel, and jointype as separate parameters to reduce code churn in joinpath.c --- in particular, putting jointype into the struct would have been problematic because of the subroutines' habit of changing their local copies of that variable. * Avoid ad-hocery in ExecAssignScanProjectionInfo. It was probably all right for it to know about IndexOnlyScan, but if the list is to grow we should refactor the knowledge out to the callers. * Restore nodeForeignscan.c's previous use of the relcache to avoid extra GetFdwRoutine lookups for base-relation scans. * Lots of cleanup of documentation and missed comments. Re-order some code additions into more logical places.
10 years ago
static void set_foreignscan_references(PlannerInfo *root,
ForeignScan *fscan,
int rtoffset);
static void set_customscan_references(PlannerInfo *root,
CustomScan *cscan,
int rtoffset);
Rearrange CustomScan API. Make it work more like FDW plans do: instead of assuming that there are expressions in a CustomScan plan node that the core code doesn't know about, insist that all subexpressions that need planner attention be in a "custom_exprs" list in the Plan representation. (Of course, the custom plugin can break the list apart again at executor initialization.) This lets us revert the parts of the patch that exposed setrefs.c and subselect.c processing to the outside world. Also revert the GetSpecialCustomVar stuff in ruleutils.c; that concept may work in future, but it's far from fully baked right now.
11 years ago
static Node *fix_scan_expr(PlannerInfo *root, Node *node, int rtoffset);
Re-run pgindent with updated list of typedefs. (Updated README should avoid this problem in the future.)
18 years ago
static Node *fix_scan_expr_mutator(Node *node, fix_scan_expr_context *context);
Change fix_scan_expr() to avoid copying the input node tree in the common case where rtoffset == 0. In that case there is no need to change Var nodes, and since filling in unset opfuncid fields is always safe, scribbling on the input tree to that extent is not objectionable. This brings the cost of this operation back down to what it was in 8.2 for simple queries. Per investigation of performance gripe from Guillaume Smet.
18 years ago
static bool fix_scan_expr_walker(Node *node, fix_scan_expr_context *context);
Rearrange planner to save the whole PlannerInfo (subroot) for a subquery. Formerly, set_subquery_pathlist and other creators of plans for subqueries saved only the rangetable and rowMarks lists from the lower-level PlannerInfo. But there's no reason not to remember the whole PlannerInfo, and indeed this turns out to simplify matters in a number of places. The immediate reason for doing this was so that the subroot will still be accessible when we're trying to extract column statistics out of an already-planned subquery. But now that I've done it, it seems like a good code-beautification effort in its own right. I also chose to get rid of the transient subrtable and subrowmark fields in SubqueryScan nodes, in favor of having setrefs.c look up the subquery's RelOptInfo. That required changing all the APIs in setrefs.c to pass PlannerInfo not PlannerGlobal, which was a large but quite mechanical transformation. One side-effect not foreseen at the beginning is that this finally broke inheritance_planner's assumption that replanning the same subquery RTE N times would necessarily give interchangeable results each time. That assumption was always pretty risky, but now we really have to make a separate RTE for each instance so that there's a place to carry the separate subroots.
14 years ago
static void set_join_references(PlannerInfo *root, Join *join, int rtoffset);
static void set_upper_references(PlannerInfo *root, Plan *plan, int rtoffset);
Now that plans have flat rangetable lists, it's a lot easier to get EXPLAIN to drill down into subplan targetlists to print the referent expression for an OUTER or INNER var in an upper plan node. Hence, make it do that always, and banish the old hack of showing "?columnN?" when things got too complicated. Along the way, fix an EXPLAIN bug I introduced by suppressing subqueries from execution-time range tables: get_name_for_var_field() assumed it could look at rte->subquery to find out the real type of a RECORD var. That doesn't work anymore, but instead we can look at the input plan of the SubqueryScan plan node.
19 years ago
static void set_dummy_tlist_references(Plan *plan, int rtoffset);
Revise searching of subplan target lists to use something more efficient than tlist_member calls. Building a large join tlist is still O(N^2), but with a much smaller constant factor than before.
21 years ago
static indexed_tlist *build_tlist_index(List *tlist);
static Var *search_indexed_tlist_for_var(Var *var,
Standard pgindent run for 8.1.
20 years ago
indexed_tlist *itlist,
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
Index newvarno,
int rtoffset);
Revise searching of subplan target lists to use something more efficient than tlist_member calls. Building a large join tlist is still O(N^2), but with a much smaller constant factor than before.
21 years ago
static Var *search_indexed_tlist_for_non_var(Node *node,
Standard pgindent run for 8.1.
20 years ago
indexed_tlist *itlist,
Index newvarno);
While doing the final setrefs.c pass over a plan tree, try to match up non-Var sort/group expressions using ressortgroupref labels instead of depending entirely on equal()-ity of the upper node's tlist expressions to the lower node's. This avoids emitting the wrong outputs in cases where there are textually identical volatile sort/group expressions, as for example select distinct random(),random() from generate_series(1,10); Per report from Andrew Gierth. Backpatch to 8.4. Arguably this is wrong all the way back, but the only known case where there's an observable problem is when using hash aggregation to implement DISTINCT, which is new as of 8.4. So for the moment I'll refrain from backpatching further.
16 years ago
static Var *search_indexed_tlist_for_sortgroupref(Node *node,
Index sortgroupref,
indexed_tlist *itlist,
Index newvarno);
Rearrange planner to save the whole PlannerInfo (subroot) for a subquery. Formerly, set_subquery_pathlist and other creators of plans for subqueries saved only the rangetable and rowMarks lists from the lower-level PlannerInfo. But there's no reason not to remember the whole PlannerInfo, and indeed this turns out to simplify matters in a number of places. The immediate reason for doing this was so that the subroot will still be accessible when we're trying to extract column statistics out of an already-planned subquery. But now that I've done it, it seems like a good code-beautification effort in its own right. I also chose to get rid of the transient subrtable and subrowmark fields in SubqueryScan nodes, in favor of having setrefs.c look up the subquery's RelOptInfo. That required changing all the APIs in setrefs.c to pass PlannerInfo not PlannerGlobal, which was a large but quite mechanical transformation. One side-effect not foreseen at the beginning is that this finally broke inheritance_planner's assumption that replanning the same subquery RTE N times would necessarily give interchangeable results each time. That assumption was always pretty risky, but now we really have to make a separate RTE for each instance so that there's a place to carry the separate subroots.
14 years ago
static List *fix_join_expr(PlannerInfo *root,
pgindent run for 8.3.
18 years ago
List *clauses,
indexed_tlist *outer_itlist,
indexed_tlist *inner_itlist,
Index acceptable_rel, int rtoffset);
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
static Node *fix_join_expr_mutator(Node *node,
Re-run pgindent with updated list of typedefs. (Updated README should avoid this problem in the future.)
18 years ago
fix_join_expr_context *context);
Rearrange planner to save the whole PlannerInfo (subroot) for a subquery. Formerly, set_subquery_pathlist and other creators of plans for subqueries saved only the rangetable and rowMarks lists from the lower-level PlannerInfo. But there's no reason not to remember the whole PlannerInfo, and indeed this turns out to simplify matters in a number of places. The immediate reason for doing this was so that the subroot will still be accessible when we're trying to extract column statistics out of an already-planned subquery. But now that I've done it, it seems like a good code-beautification effort in its own right. I also chose to get rid of the transient subrtable and subrowmark fields in SubqueryScan nodes, in favor of having setrefs.c look up the subquery's RelOptInfo. That required changing all the APIs in setrefs.c to pass PlannerInfo not PlannerGlobal, which was a large but quite mechanical transformation. One side-effect not foreseen at the beginning is that this finally broke inheritance_planner's assumption that replanning the same subquery RTE N times would necessarily give interchangeable results each time. That assumption was always pretty risky, but now we really have to make a separate RTE for each instance so that there's a place to carry the separate subroots.
14 years ago
static Node *fix_upper_expr(PlannerInfo *root,
pgindent run for 8.3.
18 years ago
Node *node,
indexed_tlist *subplan_itlist,
Rearrange the implementation of index-only scans. This commit changes index-only scans so that data is read directly from the index tuple without first generating a faux heap tuple. The only immediate benefit is that indexes on system columns (such as OID) can be used in index-only scans, but this is necessary infrastructure if we are ever to support index-only scans on expression indexes. The executor is now ready for that, though the planner still needs substantial work to recognize the possibility. To do this, Vars in index-only plan nodes have to refer to index columns not heap columns. I introduced a new special varno, INDEX_VAR, to mark such Vars to avoid confusion. (In passing, this commit renames the two existing special varnos to OUTER_VAR and INNER_VAR.) This allows ruleutils.c to handle them with logic similar to what we use for subplan reference Vars. Since index-only scans are now fundamentally different from regular indexscans so far as their expression subtrees are concerned, I also chose to change them to have their own plan node type (and hence, their own executor source file).
14 years ago
Index newvarno,
pgindent run for 8.3.
18 years ago
int rtoffset);
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
static Node *fix_upper_expr_mutator(Node *node,
Re-run pgindent with updated list of typedefs. (Updated README should avoid this problem in the future.)
18 years ago
fix_upper_expr_context *context);
Fix planner's handling of RETURNING lists in writable CTEs. setrefs.c failed to do "rtoffset" adjustment of Vars in RETURNING lists, which meant they were left with the wrong varnos when the RETURNING list was in a subquery. That was never possible before writable CTEs, of course, but now it's broken. The executor fails to notice any problem because ExecEvalVar just references the ecxt_scantuple for any normal varno; but EXPLAIN breaks when the varno is wrong, as illustrated in a recent complaint from Bartosz Dmytrak. Since the eventual rtoffset of the subquery is not known at the time we are preparing its plan node, the previous scheme of executing set_returning_clause_references() at that time cannot handle this adjustment. Fortunately, it turns out that we don't really need to do it that way, because all the needed information is available during normal setrefs.c execution; we just have to dig it out of the ModifyTable node. So, do that, and get rid of the kluge of early setrefs processing of RETURNING lists. (This is a little bit of a cheat in the case of inherited UPDATE/DELETE, because we are not passing a "root" struct that corresponds exactly to what the subplan was built with. But that doesn't matter, and anyway this is less ugly than early setrefs processing was.) Back-patch to 9.1, where the problem became possible to hit.
14 years ago
static List *set_returning_clause_references(PlannerInfo *root,
List *rlist,
Plan *topplan,
Index resultRelation,
int rtoffset);
Phase 2 of read-only-plans project: restructure expression-tree nodes so that all executable expression nodes inherit from a common supertype Expr. This is somewhat of an exercise in code purity rather than any real functional advance, but getting rid of the extra Oper or Func node formerly used in each operator or function call should provide at least a little space and speed improvement. initdb forced by changes in stored-rules representation.
23 years ago
static bool fix_opfuncids_walker(Node *node, void *context);
Improve the plan cache invalidation mechanism to make it invalidate plans when user-defined functions used in a plan are modified. Also invalidate plans when schemas, operators, or operator classes are modified; but for these cases we just invalidate everything rather than tracking exact dependencies, since these types of objects seldom change in a production database. Tom Lane; loosely based on a patch by Martin Pihlak.
17 years ago
static bool extract_query_dependencies_walker(Node *node,
Rearrange planner to save the whole PlannerInfo (subroot) for a subquery. Formerly, set_subquery_pathlist and other creators of plans for subqueries saved only the rangetable and rowMarks lists from the lower-level PlannerInfo. But there's no reason not to remember the whole PlannerInfo, and indeed this turns out to simplify matters in a number of places. The immediate reason for doing this was so that the subroot will still be accessible when we're trying to extract column statistics out of an already-planned subquery. But now that I've done it, it seems like a good code-beautification effort in its own right. I also chose to get rid of the transient subrtable and subrowmark fields in SubqueryScan nodes, in favor of having setrefs.c look up the subquery's RelOptInfo. That required changing all the APIs in setrefs.c to pass PlannerInfo not PlannerGlobal, which was a large but quite mechanical transformation. One side-effect not foreseen at the beginning is that this finally broke inheritance_planner's assumption that replanning the same subquery RTE N times would necessarily give interchangeable results each time. That assumption was always pretty risky, but now we really have to make a separate RTE for each instance so that there's a place to carry the separate subroots.
14 years ago
PlannerInfo *context);
Support expressions of the form 'scalar op ANY (array)' and 'scalar op ALL (array)', where the operator is applied between the lefthand scalar and each element of the array. The operator must yield boolean; the result of the construct is the OR or AND of the per-element results, respectively. Original coding by Joe Conway, after an idea of Peter's. Rewritten by Tom to keep the implementation strictly separate from subqueries.
23 years ago
Postgres95 1.01 Distribution - Virgin Sources
30 years ago
/*****************************************************************************
Massive commit to run PGINDENT on all *.c and *.h files.
28 years ago
*
* SUBPLAN REFERENCES
*
Postgres95 1.01 Distribution - Virgin Sources
30 years ago
*****************************************************************************/
Massive commit to run PGINDENT on all *.c and *.h files.
28 years ago
/*
Further planner/optimizer cleanups. Move all set_tlist_references and fix_opids processing to a single recursive pass over the plan tree executed at the very tail end of planning, rather than haphazardly here and there at different places. Now that tlist Vars do not get modified until the very end, it's possible to get rid of the klugy var_equal and match_varid partial-matching routines, and just use plain equal() throughout the optimizer. This is a step towards allowing merge and hash joins to be done on expressions instead of only Vars ...
26 years ago
* set_plan_references
Massive commit to run PGINDENT on all *.c and *.h files.
28 years ago
*
pgindent run for 9.4 This includes removing tabs after periods in C comments, which was applied to back branches, so this change should not effect backpatching.
12 years ago
* This is the final processing pass of the planner/optimizer. The plan
Teach the planner to remove SubqueryScan nodes from the plan if they aren't doing anything useful (ie, neither selection nor projection). Also, extend to SubqueryScan the hacks already in place to avoid unnecessary ExecProject calls when the result would just be the same tuple the subquery already delivered. This saves some overhead in UNION and other set operations, as well as avoiding overhead for unflatten-able subqueries. Per example from Sokolov Yura.
21 years ago
* tree is complete; we just have to adjust some representational details
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
* for the convenience of the executor:
*
* 1. We flatten the various subquery rangetables into a single list, and
* zero out RangeTblEntry fields that are not useful to the executor.
*
* 2. We adjust Vars in scan nodes to be consistent with the flat rangetable.
*
* 3. We adjust Vars in upper plan nodes to refer to the outputs of their
* subplans.
*
Implement UPDATE tab SET (col1,col2,...) = (SELECT ...), ... This SQL-standard feature allows a sub-SELECT yielding multiple columns (but only one row) to be used to compute the new values of several columns to be updated. While the same results can be had with an independent sub-SELECT per column, such a workaround can require a great deal of duplicated computation. The standard actually says that the source for a multi-column assignment could be any row-valued expression. The implementation used here is tightly tied to our existing sub-SELECT support and can't handle other cases; the Bison grammar would have some issues with them too. However, I don't feel too bad about this since other cases can be converted into sub-SELECTs. For instance, "SET (a,b,c) = row_valued_function(x)" could be written "SET (a,b,c) = (SELECT * FROM row_valued_function(x))".
11 years ago
* 4. PARAM_MULTIEXPR Params are replaced by regular PARAM_EXEC Params,
* now that we have finished planning all MULTIEXPR subplans.
*
* 5. We compute regproc OIDs for operators (ie, we look up the function
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
* that implements each op).
Massive commit to run PGINDENT on all *.c and *.h files.
28 years ago
*
Implement UPDATE tab SET (col1,col2,...) = (SELECT ...), ... This SQL-standard feature allows a sub-SELECT yielding multiple columns (but only one row) to be used to compute the new values of several columns to be updated. While the same results can be had with an independent sub-SELECT per column, such a workaround can require a great deal of duplicated computation. The standard actually says that the source for a multi-column assignment could be any row-valued expression. The implementation used here is tightly tied to our existing sub-SELECT support and can't handle other cases; the Bison grammar would have some issues with them too. However, I don't feel too bad about this since other cases can be converted into sub-SELECTs. For instance, "SET (a,b,c) = row_valued_function(x)" could be written "SET (a,b,c) = (SELECT * FROM row_valued_function(x))".
11 years ago
* 6. We create lists of specific objects that the plan depends on.
Fix the plan-invalidation mechanism to treat regclass constants that refer to a relation as a reason to invalidate a plan when the relation changes. This handles scenarios such as dropping/recreating a sequence that is referenced by nextval('seq') in a cached plan. Rather than teach plancache.c all about digging through plan trees to find regclass Consts, we charge the planner's setrefs.c with making a list of the relation OIDs on which each plan depends. That way the list can be built cheaply during a plan tree traversal that has to happen anyway. Per bug #3662 and subsequent discussion.
18 years ago
* This will be used by plancache.c to drive invalidation of cached plans.
Improve the plan cache invalidation mechanism to make it invalidate plans when user-defined functions used in a plan are modified. Also invalidate plans when schemas, operators, or operator classes are modified; but for these cases we just invalidate everything rather than tracking exact dependencies, since these types of objects seldom change in a production database. Tom Lane; loosely based on a patch by Martin Pihlak.
17 years ago
* Relation dependencies are represented by OIDs, and everything else by
* PlanInvalItems (this distinction is motivated by the shared-inval APIs).
* Currently, relations and user-defined functions are the only types of
* objects that are explicitly tracked this way.
Fix the plan-invalidation mechanism to treat regclass constants that refer to a relation as a reason to invalidate a plan when the relation changes. This handles scenarios such as dropping/recreating a sequence that is referenced by nextval('seq') in a cached plan. Rather than teach plancache.c all about digging through plan trees to find regclass Consts, we charge the planner's setrefs.c with making a list of the relation OIDs on which each plan depends. That way the list can be built cheaply during a plan tree traversal that has to happen anyway. Per bug #3662 and subsequent discussion.
18 years ago
*
Teach the planner to remove SubqueryScan nodes from the plan if they aren't doing anything useful (ie, neither selection nor projection). Also, extend to SubqueryScan the hacks already in place to avoid unnecessary ExecProject calls when the result would just be the same tuple the subquery already delivered. This saves some overhead in UNION and other set operations, as well as avoiding overhead for unflatten-able subqueries. Per example from Sokolov Yura.
21 years ago
* We also perform one final optimization step, which is to delete
* SubqueryScan plan nodes that aren't doing anything useful (ie, have
* no qual and a no-op targetlist). The reason for doing this last is that
* it can't readily be done before set_plan_references, because it would
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
* break set_upper_references: the Vars in the subquery's top tlist
* wouldn't match up with the Vars in the outer plan tree. The SubqueryScan
Teach the planner to remove SubqueryScan nodes from the plan if they aren't doing anything useful (ie, neither selection nor projection). Also, extend to SubqueryScan the hacks already in place to avoid unnecessary ExecProject calls when the result would just be the same tuple the subquery already delivered. This saves some overhead in UNION and other set operations, as well as avoiding overhead for unflatten-able subqueries. Per example from Sokolov Yura.
21 years ago
* serves a necessary function as a buffer between outer query and subquery
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
* variable numbering ... but after we've flattened the rangetable this is
Fix the plan-invalidation mechanism to treat regclass constants that refer to a relation as a reason to invalidate a plan when the relation changes. This handles scenarios such as dropping/recreating a sequence that is referenced by nextval('seq') in a cached plan. Rather than teach plancache.c all about digging through plan trees to find regclass Consts, we charge the planner's setrefs.c with making a list of the relation OIDs on which each plan depends. That way the list can be built cheaply during a plan tree traversal that has to happen anyway. Per bug #3662 and subsequent discussion.
18 years ago
* no longer a problem, since then there's only one rtindex namespace.
Teach the planner to remove SubqueryScan nodes from the plan if they aren't doing anything useful (ie, neither selection nor projection). Also, extend to SubqueryScan the hacks already in place to avoid unnecessary ExecProject calls when the result would just be the same tuple the subquery already delivered. This saves some overhead in UNION and other set operations, as well as avoiding overhead for unflatten-able subqueries. Per example from Sokolov Yura.
21 years ago
*
* set_plan_references recursively traverses the whole plan tree.
*
* The return value is normally the same Plan node passed in, but can be
* different when the passed-in Plan is a SubqueryScan we decide isn't needed.
*
Rearrange planner to save the whole PlannerInfo (subroot) for a subquery. Formerly, set_subquery_pathlist and other creators of plans for subqueries saved only the rangetable and rowMarks lists from the lower-level PlannerInfo. But there's no reason not to remember the whole PlannerInfo, and indeed this turns out to simplify matters in a number of places. The immediate reason for doing this was so that the subroot will still be accessible when we're trying to extract column statistics out of an already-planned subquery. But now that I've done it, it seems like a good code-beautification effort in its own right. I also chose to get rid of the transient subrtable and subrowmark fields in SubqueryScan nodes, in favor of having setrefs.c look up the subquery's RelOptInfo. That required changing all the APIs in setrefs.c to pass PlannerInfo not PlannerGlobal, which was a large but quite mechanical transformation. One side-effect not foreseen at the beginning is that this finally broke inheritance_planner's assumption that replanning the same subquery RTE N times would necessarily give interchangeable results each time. That assumption was always pretty risky, but now we really have to make a separate RTE for each instance so that there's a place to carry the separate subroots.
14 years ago
* The flattened rangetable entries are appended to root->glob->finalrtable.
* Also, rowmarks entries are appended to root->glob->finalrowmarks, and the
* RT indexes of ModifyTable result relations to root->glob->resultRelations.
* Plan dependencies are appended to root->glob->relationOids (for relations)
* and root->glob->invalItems (for everything else).
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
*
* Notice that we modify Plan nodes in-place, but use expression_tree_mutator
pgindent run for 9.4 This includes removing tabs after periods in C comments, which was applied to back branches, so this change should not effect backpatching.
12 years ago
* to process targetlist and qual expressions. We can assume that the Plan
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
* nodes were just built by the planner and are not multiply referenced, but
* it's not so safe to assume that for expression tree nodes.
Postgres95 1.01 Distribution - Virgin Sources
30 years ago
*/
Teach the planner to remove SubqueryScan nodes from the plan if they aren't doing anything useful (ie, neither selection nor projection). Also, extend to SubqueryScan the hacks already in place to avoid unnecessary ExecProject calls when the result would just be the same tuple the subquery already delivered. This saves some overhead in UNION and other set operations, as well as avoiding overhead for unflatten-able subqueries. Per example from Sokolov Yura.
21 years ago
Plan *
Rearrange planner to save the whole PlannerInfo (subroot) for a subquery. Formerly, set_subquery_pathlist and other creators of plans for subqueries saved only the rangetable and rowMarks lists from the lower-level PlannerInfo. But there's no reason not to remember the whole PlannerInfo, and indeed this turns out to simplify matters in a number of places. The immediate reason for doing this was so that the subroot will still be accessible when we're trying to extract column statistics out of an already-planned subquery. But now that I've done it, it seems like a good code-beautification effort in its own right. I also chose to get rid of the transient subrtable and subrowmark fields in SubqueryScan nodes, in favor of having setrefs.c look up the subquery's RelOptInfo. That required changing all the APIs in setrefs.c to pass PlannerInfo not PlannerGlobal, which was a large but quite mechanical transformation. One side-effect not foreseen at the beginning is that this finally broke inheritance_planner's assumption that replanning the same subquery RTE N times would necessarily give interchangeable results each time. That assumption was always pretty risky, but now we really have to make a separate RTE for each instance so that there's a place to carry the separate subroots.
14 years ago
set_plan_references(PlannerInfo *root, Plan *plan)
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
{
Rearrange planner to save the whole PlannerInfo (subroot) for a subquery. Formerly, set_subquery_pathlist and other creators of plans for subqueries saved only the rangetable and rowMarks lists from the lower-level PlannerInfo. But there's no reason not to remember the whole PlannerInfo, and indeed this turns out to simplify matters in a number of places. The immediate reason for doing this was so that the subroot will still be accessible when we're trying to extract column statistics out of an already-planned subquery. But now that I've done it, it seems like a good code-beautification effort in its own right. I also chose to get rid of the transient subrtable and subrowmark fields in SubqueryScan nodes, in favor of having setrefs.c look up the subquery's RelOptInfo. That required changing all the APIs in setrefs.c to pass PlannerInfo not PlannerGlobal, which was a large but quite mechanical transformation. One side-effect not foreseen at the beginning is that this finally broke inheritance_planner's assumption that replanning the same subquery RTE N times would necessarily give interchangeable results each time. That assumption was always pretty risky, but now we really have to make a separate RTE for each instance so that there's a place to carry the separate subroots.
14 years ago
PlannerGlobal *glob = root->glob;
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
int rtoffset = list_length(glob->finalrtable);
ListCell *lc;
/*
Better fix for permissions tests in excluded subqueries. This reverts the code changes in 50c137487c96e629e0e5372bb3d1b5f1a2f71a88, which turned out to induce crashes and not completely fix the problem anyway. That commit only considered single subqueries that were excluded by constraint-exclusion logic, but actually the problem also exists for subqueries that are appendrel members (ie part of a UNION ALL list). In such cases we can't add a dummy subpath to the appendrel's AppendPath list without defeating the logic that recognizes when an appendrel is completely excluded. Instead, fix the problem by having setrefs.c scan the rangetable an extra time looking for subqueries that didn't get into the plan tree. (This approach depends on the 9.2 change that made set_subquery_pathlist generate dummy paths for excluded single subqueries, so that the exclusion behavior is the same for single subqueries and appendrel members.) Note: it turns out that the appendrel form of the missed-permissions-checks bug exists as far back as 8.4. However, since the practical effect of that bug seems pretty minimal, consensus is to not attempt to fix it in the back branches, at least not yet. Possibly we could back-port this patch once it's gotten a reasonable amount of testing in HEAD. For the moment I'm just going to revert the previous patch in 9.2.
13 years ago
* Add all the query's RTEs to the flattened rangetable. The live ones
* will have their rangetable indexes increased by rtoffset. (Additional
* RTEs, not referenced by the Plan tree, might get added after those.)
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
*/
Better fix for permissions tests in excluded subqueries. This reverts the code changes in 50c137487c96e629e0e5372bb3d1b5f1a2f71a88, which turned out to induce crashes and not completely fix the problem anyway. That commit only considered single subqueries that were excluded by constraint-exclusion logic, but actually the problem also exists for subqueries that are appendrel members (ie part of a UNION ALL list). In such cases we can't add a dummy subpath to the appendrel's AppendPath list without defeating the logic that recognizes when an appendrel is completely excluded. Instead, fix the problem by having setrefs.c scan the rangetable an extra time looking for subqueries that didn't get into the plan tree. (This approach depends on the 9.2 change that made set_subquery_pathlist generate dummy paths for excluded single subqueries, so that the exclusion behavior is the same for single subqueries and appendrel members.) Note: it turns out that the appendrel form of the missed-permissions-checks bug exists as far back as 8.4. However, since the practical effect of that bug seems pretty minimal, consensus is to not attempt to fix it in the back branches, at least not yet. Possibly we could back-port this patch once it's gotten a reasonable amount of testing in HEAD. For the moment I'm just going to revert the previous patch in 9.2.
13 years ago
add_rtes_to_flat_rtable(root, false);
Rearrange the implementation of index-only scans. This commit changes index-only scans so that data is read directly from the index tuple without first generating a faux heap tuple. The only immediate benefit is that indexes on system columns (such as OID) can be used in index-only scans, but this is necessary infrastructure if we are ever to support index-only scans on expression indexes. The executor is now ready for that, though the planner still needs substantial work to recognize the possibility. To do this, Vars in index-only plan nodes have to refer to index columns not heap columns. I introduced a new special varno, INDEX_VAR, to mark such Vars to avoid confusion. (In passing, this commit renames the two existing special varnos to OUTER_VAR and INNER_VAR.) This allows ruleutils.c to handle them with logic similar to what we use for subplan reference Vars. Since index-only scans are now fundamentally different from regular indexscans so far as their expression subtrees are concerned, I also chose to change them to have their own plan node type (and hence, their own executor source file).
14 years ago
Move the handling of SELECT FOR UPDATE locking and rechecking out of execMain.c and into a new plan node type LockRows. Like the recent change to put table updating into a ModifyTable plan node, this increases planning flexibility by allowing the operations to occur below the top level of the plan tree. It's necessary in any case to restore the previous behavior of having FOR UPDATE locking occur before ModifyTable does. This partially refactors EvalPlanQual to allow multiple rows-under-test to be inserted into the EPQ machinery before starting an EPQ test query. That isn't sufficient to fix EPQ's general bogosity in the face of plans that return multiple rows per test row, though. Since this patch is mostly about getting some plan node infrastructure in place and not about fixing ten-year-old bugs, I will leave EPQ improvements for another day. Another behavioral change that we could now think about is doing FOR UPDATE before LIMIT, but that too seems like it should be treated as a followon patch.
16 years ago
/*
Re-implement EvalPlanQual processing to improve its performance and eliminate a lot of strange behaviors that occurred in join cases. We now identify the "current" row for every joined relation in UPDATE, DELETE, and SELECT FOR UPDATE/SHARE queries. If an EvalPlanQual recheck is necessary, we jam the appropriate row into each scan node in the rechecking plan, forcing it to emit only that one row. The former behavior could rescan the whole of each joined relation for each recheck, which was terrible for performance, and what's much worse could result in duplicated output tuples. Also, the original implementation of EvalPlanQual could not re-use the recheck execution tree --- it had to go through a full executor init and shutdown for every row to be tested. To avoid this overhead, I've associated a special runtime Param with each LockRows or ModifyTable plan node, and arranged to make every scan node below such a node depend on that Param. Thus, by signaling a change in that Param, the EPQ machinery can just rescan the already-built test plan. This patch also adds a prohibition on set-returning functions in the targetlist of SELECT FOR UPDATE/SHARE. This is needed to avoid the duplicate-output-tuple problem. It seems fairly reasonable since the other restrictions on SELECT FOR UPDATE are meant to ensure that there is a unique correspondence between source tuples and result tuples, which an output SRF destroys as much as anything else does.
16 years ago
* Adjust RT indexes of PlanRowMarks and add to final rowmarks list
Move the handling of SELECT FOR UPDATE locking and rechecking out of execMain.c and into a new plan node type LockRows. Like the recent change to put table updating into a ModifyTable plan node, this increases planning flexibility by allowing the operations to occur below the top level of the plan tree. It's necessary in any case to restore the previous behavior of having FOR UPDATE locking occur before ModifyTable does. This partially refactors EvalPlanQual to allow multiple rows-under-test to be inserted into the EPQ machinery before starting an EPQ test query. That isn't sufficient to fix EPQ's general bogosity in the face of plans that return multiple rows per test row, though. Since this patch is mostly about getting some plan node infrastructure in place and not about fixing ten-year-old bugs, I will leave EPQ improvements for another day. Another behavioral change that we could now think about is doing FOR UPDATE before LIMIT, but that too seems like it should be treated as a followon patch.
16 years ago
*/
Rearrange planner to save the whole PlannerInfo (subroot) for a subquery. Formerly, set_subquery_pathlist and other creators of plans for subqueries saved only the rangetable and rowMarks lists from the lower-level PlannerInfo. But there's no reason not to remember the whole PlannerInfo, and indeed this turns out to simplify matters in a number of places. The immediate reason for doing this was so that the subroot will still be accessible when we're trying to extract column statistics out of an already-planned subquery. But now that I've done it, it seems like a good code-beautification effort in its own right. I also chose to get rid of the transient subrtable and subrowmark fields in SubqueryScan nodes, in favor of having setrefs.c look up the subquery's RelOptInfo. That required changing all the APIs in setrefs.c to pass PlannerInfo not PlannerGlobal, which was a large but quite mechanical transformation. One side-effect not foreseen at the beginning is that this finally broke inheritance_planner's assumption that replanning the same subquery RTE N times would necessarily give interchangeable results each time. That assumption was always pretty risky, but now we really have to make a separate RTE for each instance so that there's a place to carry the separate subroots.
14 years ago
foreach(lc, root->rowMarks)
Move the handling of SELECT FOR UPDATE locking and rechecking out of execMain.c and into a new plan node type LockRows. Like the recent change to put table updating into a ModifyTable plan node, this increases planning flexibility by allowing the operations to occur below the top level of the plan tree. It's necessary in any case to restore the previous behavior of having FOR UPDATE locking occur before ModifyTable does. This partially refactors EvalPlanQual to allow multiple rows-under-test to be inserted into the EPQ machinery before starting an EPQ test query. That isn't sufficient to fix EPQ's general bogosity in the face of plans that return multiple rows per test row, though. Since this patch is mostly about getting some plan node infrastructure in place and not about fixing ten-year-old bugs, I will leave EPQ improvements for another day. Another behavioral change that we could now think about is doing FOR UPDATE before LIMIT, but that too seems like it should be treated as a followon patch.
16 years ago
{
Re-implement EvalPlanQual processing to improve its performance and eliminate a lot of strange behaviors that occurred in join cases. We now identify the "current" row for every joined relation in UPDATE, DELETE, and SELECT FOR UPDATE/SHARE queries. If an EvalPlanQual recheck is necessary, we jam the appropriate row into each scan node in the rechecking plan, forcing it to emit only that one row. The former behavior could rescan the whole of each joined relation for each recheck, which was terrible for performance, and what's much worse could result in duplicated output tuples. Also, the original implementation of EvalPlanQual could not re-use the recheck execution tree --- it had to go through a full executor init and shutdown for every row to be tested. To avoid this overhead, I've associated a special runtime Param with each LockRows or ModifyTable plan node, and arranged to make every scan node below such a node depend on that Param. Thus, by signaling a change in that Param, the EPQ machinery can just rescan the already-built test plan. This patch also adds a prohibition on set-returning functions in the targetlist of SELECT FOR UPDATE/SHARE. This is needed to avoid the duplicate-output-tuple problem. It seems fairly reasonable since the other restrictions on SELECT FOR UPDATE are meant to ensure that there is a unique correspondence between source tuples and result tuples, which an output SRF destroys as much as anything else does.
16 years ago
PlanRowMark *rc = (PlanRowMark *) lfirst(lc);
PlanRowMark *newrc;
Move the handling of SELECT FOR UPDATE locking and rechecking out of execMain.c and into a new plan node type LockRows. Like the recent change to put table updating into a ModifyTable plan node, this increases planning flexibility by allowing the operations to occur below the top level of the plan tree. It's necessary in any case to restore the previous behavior of having FOR UPDATE locking occur before ModifyTable does. This partially refactors EvalPlanQual to allow multiple rows-under-test to be inserted into the EPQ machinery before starting an EPQ test query. That isn't sufficient to fix EPQ's general bogosity in the face of plans that return multiple rows per test row, though. Since this patch is mostly about getting some plan node infrastructure in place and not about fixing ten-year-old bugs, I will leave EPQ improvements for another day. Another behavioral change that we could now think about is doing FOR UPDATE before LIMIT, but that too seems like it should be treated as a followon patch.
16 years ago
Re-implement EvalPlanQual processing to improve its performance and eliminate a lot of strange behaviors that occurred in join cases. We now identify the "current" row for every joined relation in UPDATE, DELETE, and SELECT FOR UPDATE/SHARE queries. If an EvalPlanQual recheck is necessary, we jam the appropriate row into each scan node in the rechecking plan, forcing it to emit only that one row. The former behavior could rescan the whole of each joined relation for each recheck, which was terrible for performance, and what's much worse could result in duplicated output tuples. Also, the original implementation of EvalPlanQual could not re-use the recheck execution tree --- it had to go through a full executor init and shutdown for every row to be tested. To avoid this overhead, I've associated a special runtime Param with each LockRows or ModifyTable plan node, and arranged to make every scan node below such a node depend on that Param. Thus, by signaling a change in that Param, the EPQ machinery can just rescan the already-built test plan. This patch also adds a prohibition on set-returning functions in the targetlist of SELECT FOR UPDATE/SHARE. This is needed to avoid the duplicate-output-tuple problem. It seems fairly reasonable since the other restrictions on SELECT FOR UPDATE are meant to ensure that there is a unique correspondence between source tuples and result tuples, which an output SRF destroys as much as anything else does.
16 years ago
Assert(IsA(rc, PlanRowMark));
/* flat copy is enough since all fields are scalars */
newrc = (PlanRowMark *) palloc(sizeof(PlanRowMark));
memcpy(newrc, rc, sizeof(PlanRowMark));
Move the handling of SELECT FOR UPDATE locking and rechecking out of execMain.c and into a new plan node type LockRows. Like the recent change to put table updating into a ModifyTable plan node, this increases planning flexibility by allowing the operations to occur below the top level of the plan tree. It's necessary in any case to restore the previous behavior of having FOR UPDATE locking occur before ModifyTable does. This partially refactors EvalPlanQual to allow multiple rows-under-test to be inserted into the EPQ machinery before starting an EPQ test query. That isn't sufficient to fix EPQ's general bogosity in the face of plans that return multiple rows per test row, though. Since this patch is mostly about getting some plan node infrastructure in place and not about fixing ten-year-old bugs, I will leave EPQ improvements for another day. Another behavioral change that we could now think about is doing FOR UPDATE before LIMIT, but that too seems like it should be treated as a followon patch.
16 years ago
Fix improper matching of resjunk column names for FOR UPDATE in subselect. Flattening of subquery range tables during setrefs.c could lead to the rangetable indexes in PlanRowMark nodes not matching up with the column names previously assigned to the corresponding resjunk ctid (resp. tableoid or wholerow) columns. Typical symptom would be either a "cannot extract system attribute from virtual tuple" error or an Assert failure. This wasn't a problem before 9.0 because we didn't support FOR UPDATE below the top query level, and so the final flattening could never renumber an RTE that was relevant to FOR UPDATE. Fix by using a plan-tree-wide unique number for each PlanRowMark to label the associated resjunk columns, so that the number need not change during flattening. Per report from David Johnston (though I'm darned if I can see how this got past initial testing of the relevant code). Back-patch to 9.0.
15 years ago
/* adjust indexes ... but *not* the rowmarkId */
Move the handling of SELECT FOR UPDATE locking and rechecking out of execMain.c and into a new plan node type LockRows. Like the recent change to put table updating into a ModifyTable plan node, this increases planning flexibility by allowing the operations to occur below the top level of the plan tree. It's necessary in any case to restore the previous behavior of having FOR UPDATE locking occur before ModifyTable does. This partially refactors EvalPlanQual to allow multiple rows-under-test to be inserted into the EPQ machinery before starting an EPQ test query. That isn't sufficient to fix EPQ's general bogosity in the face of plans that return multiple rows per test row, though. Since this patch is mostly about getting some plan node infrastructure in place and not about fixing ten-year-old bugs, I will leave EPQ improvements for another day. Another behavioral change that we could now think about is doing FOR UPDATE before LIMIT, but that too seems like it should be treated as a followon patch.
16 years ago
newrc->rti += rtoffset;
newrc->prti += rtoffset;
glob->finalrowmarks = lappend(glob->finalrowmarks, newrc);
}
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
/* Now fix the Plan tree */
Rearrange planner to save the whole PlannerInfo (subroot) for a subquery. Formerly, set_subquery_pathlist and other creators of plans for subqueries saved only the rangetable and rowMarks lists from the lower-level PlannerInfo. But there's no reason not to remember the whole PlannerInfo, and indeed this turns out to simplify matters in a number of places. The immediate reason for doing this was so that the subroot will still be accessible when we're trying to extract column statistics out of an already-planned subquery. But now that I've done it, it seems like a good code-beautification effort in its own right. I also chose to get rid of the transient subrtable and subrowmark fields in SubqueryScan nodes, in favor of having setrefs.c look up the subquery's RelOptInfo. That required changing all the APIs in setrefs.c to pass PlannerInfo not PlannerGlobal, which was a large but quite mechanical transformation. One side-effect not foreseen at the beginning is that this finally broke inheritance_planner's assumption that replanning the same subquery RTE N times would necessarily give interchangeable results each time. That assumption was always pretty risky, but now we really have to make a separate RTE for each instance so that there's a place to carry the separate subroots.
14 years ago
return set_plan_refs(root, plan, rtoffset);
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
}
Better fix for permissions tests in excluded subqueries. This reverts the code changes in 50c137487c96e629e0e5372bb3d1b5f1a2f71a88, which turned out to induce crashes and not completely fix the problem anyway. That commit only considered single subqueries that were excluded by constraint-exclusion logic, but actually the problem also exists for subqueries that are appendrel members (ie part of a UNION ALL list). In such cases we can't add a dummy subpath to the appendrel's AppendPath list without defeating the logic that recognizes when an appendrel is completely excluded. Instead, fix the problem by having setrefs.c scan the rangetable an extra time looking for subqueries that didn't get into the plan tree. (This approach depends on the 9.2 change that made set_subquery_pathlist generate dummy paths for excluded single subqueries, so that the exclusion behavior is the same for single subqueries and appendrel members.) Note: it turns out that the appendrel form of the missed-permissions-checks bug exists as far back as 8.4. However, since the practical effect of that bug seems pretty minimal, consensus is to not attempt to fix it in the back branches, at least not yet. Possibly we could back-port this patch once it's gotten a reasonable amount of testing in HEAD. For the moment I'm just going to revert the previous patch in 9.2.
13 years ago
/*
* Extract RangeTblEntries from the plan's rangetable, and add to flat rtable
*
* This can recurse into subquery plans; "recursing" is true if so.
*/
static void
add_rtes_to_flat_rtable(PlannerInfo *root, bool recursing)
{
PlannerGlobal *glob = root->glob;
Index rti;
ListCell *lc;
/*
* Add the query's own RTEs to the flattened rangetable.
*
* At top level, we must add all RTEs so that their indexes in the
* flattened rangetable match up with their original indexes. When
* recursing, we only care about extracting relation RTEs.
*/
foreach(lc, root->parse->rtable)
{
RangeTblEntry *rte = (RangeTblEntry *) lfirst(lc);
if (!recursing || rte->rtekind == RTE_RELATION)
add_rte_to_flat_rtable(glob, rte);
}
/*
* If there are any dead subqueries, they are not referenced in the Plan
* tree, so we must add RTEs contained in them to the flattened rtable
pgindent run for 9.4 This includes removing tabs after periods in C comments, which was applied to back branches, so this change should not effect backpatching.
12 years ago
* separately. (If we failed to do this, the executor would not perform
Better fix for permissions tests in excluded subqueries. This reverts the code changes in 50c137487c96e629e0e5372bb3d1b5f1a2f71a88, which turned out to induce crashes and not completely fix the problem anyway. That commit only considered single subqueries that were excluded by constraint-exclusion logic, but actually the problem also exists for subqueries that are appendrel members (ie part of a UNION ALL list). In such cases we can't add a dummy subpath to the appendrel's AppendPath list without defeating the logic that recognizes when an appendrel is completely excluded. Instead, fix the problem by having setrefs.c scan the rangetable an extra time looking for subqueries that didn't get into the plan tree. (This approach depends on the 9.2 change that made set_subquery_pathlist generate dummy paths for excluded single subqueries, so that the exclusion behavior is the same for single subqueries and appendrel members.) Note: it turns out that the appendrel form of the missed-permissions-checks bug exists as far back as 8.4. However, since the practical effect of that bug seems pretty minimal, consensus is to not attempt to fix it in the back branches, at least not yet. Possibly we could back-port this patch once it's gotten a reasonable amount of testing in HEAD. For the moment I'm just going to revert the previous patch in 9.2.
13 years ago
* expected permission checks for tables mentioned in such subqueries.)
*
* Note: this pass over the rangetable can't be combined with the previous
* one, because that would mess up the numbering of the live RTEs in the
* flattened rangetable.
*/
rti = 1;
foreach(lc, root->parse->rtable)
{
RangeTblEntry *rte = (RangeTblEntry *) lfirst(lc);
/*
* We should ignore inheritance-parent RTEs: their contents have been
* pulled up into our rangetable already. Also ignore any subquery
* RTEs without matching RelOptInfos, as they likewise have been
* pulled up.
*/
Fix inherited UPDATE/DELETE with UNION ALL subqueries. Fix an oversight in commit b3aaf9081a1a95c245fd605dcf02c91b3a5c3a29: we do indeed need to process the planner's append_rel_list when copying RTE subqueries, because if any of them were flattenable UNION ALL subqueries, the append_rel_list shows which subquery RTEs were pulled up out of which other ones. Without this, UNION ALL subqueries aren't correctly inserted into the update plans for inheritance child tables after the first one, typically resulting in no update happening for those child table(s). Per report from Victor Yegorov. Experimentation with this case also exposed a fault in commit a7b965382cf0cb30aeacb112572718045e6d4be7: if an inherited UPDATE/DELETE was proven totally dummy by constraint exclusion, we might arrive at add_rtes_to_flat_rtable with root->simple_rel_array being NULL. This should be interpreted as not having any RelOptInfos. I chose to code the guard as a check against simple_rel_array_size, so as to also provide some protection against indexing off the end of the array. Back-patch to 9.2 where the faulty code was added.
12 years ago
if (rte->rtekind == RTE_SUBQUERY && !rte->inh &&
rti < root->simple_rel_array_size)
Better fix for permissions tests in excluded subqueries. This reverts the code changes in 50c137487c96e629e0e5372bb3d1b5f1a2f71a88, which turned out to induce crashes and not completely fix the problem anyway. That commit only considered single subqueries that were excluded by constraint-exclusion logic, but actually the problem also exists for subqueries that are appendrel members (ie part of a UNION ALL list). In such cases we can't add a dummy subpath to the appendrel's AppendPath list without defeating the logic that recognizes when an appendrel is completely excluded. Instead, fix the problem by having setrefs.c scan the rangetable an extra time looking for subqueries that didn't get into the plan tree. (This approach depends on the 9.2 change that made set_subquery_pathlist generate dummy paths for excluded single subqueries, so that the exclusion behavior is the same for single subqueries and appendrel members.) Note: it turns out that the appendrel form of the missed-permissions-checks bug exists as far back as 8.4. However, since the practical effect of that bug seems pretty minimal, consensus is to not attempt to fix it in the back branches, at least not yet. Possibly we could back-port this patch once it's gotten a reasonable amount of testing in HEAD. For the moment I'm just going to revert the previous patch in 9.2.
13 years ago
{
RelOptInfo *rel = root->simple_rel_array[rti];
if (rel != NULL)
{
Assert(rel->relid == rti); /* sanity check on array */
/*
* The subquery might never have been planned at all, if it
* was excluded on the basis of self-contradictory constraints
pgindent run for 9.4 This includes removing tabs after periods in C comments, which was applied to back branches, so this change should not effect backpatching.
12 years ago
* in our query level. In this case apply
Better fix for permissions tests in excluded subqueries. This reverts the code changes in 50c137487c96e629e0e5372bb3d1b5f1a2f71a88, which turned out to induce crashes and not completely fix the problem anyway. That commit only considered single subqueries that were excluded by constraint-exclusion logic, but actually the problem also exists for subqueries that are appendrel members (ie part of a UNION ALL list). In such cases we can't add a dummy subpath to the appendrel's AppendPath list without defeating the logic that recognizes when an appendrel is completely excluded. Instead, fix the problem by having setrefs.c scan the rangetable an extra time looking for subqueries that didn't get into the plan tree. (This approach depends on the 9.2 change that made set_subquery_pathlist generate dummy paths for excluded single subqueries, so that the exclusion behavior is the same for single subqueries and appendrel members.) Note: it turns out that the appendrel form of the missed-permissions-checks bug exists as far back as 8.4. However, since the practical effect of that bug seems pretty minimal, consensus is to not attempt to fix it in the back branches, at least not yet. Possibly we could back-port this patch once it's gotten a reasonable amount of testing in HEAD. For the moment I'm just going to revert the previous patch in 9.2.
13 years ago
* flatten_unplanned_rtes.
*
* If it was planned but the plan is dummy, we assume that it
* has been omitted from our plan tree (see
* set_subquery_pathlist), and recurse to pull up its RTEs.
*
* Otherwise, it should be represented by a SubqueryScan node
* somewhere in our plan tree, and we'll pull up its RTEs when
* we process that plan node.
*
* However, if we're recursing, then we should pull up RTEs
* whether the subplan is dummy or not, because we've found
* that some upper query level is treating this one as dummy,
* and so we won't scan this level's plan tree at all.
*/
if (rel->subplan == NULL)
flatten_unplanned_rtes(glob, rte);
else if (recursing || is_dummy_plan(rel->subplan))
{
Assert(rel->subroot != NULL);
add_rtes_to_flat_rtable(rel->subroot, true);
}
}
}
rti++;
}
}
/*
* Extract RangeTblEntries from a subquery that was never planned at all
*/
static void
flatten_unplanned_rtes(PlannerGlobal *glob, RangeTblEntry *rte)
{
/* Use query_tree_walker to find all RTEs in the parse tree */
(void) query_tree_walker(rte->subquery,
flatten_rtes_walker,
(void *) glob,
QTW_EXAMINE_RTES);
}
static bool
flatten_rtes_walker(Node *node, PlannerGlobal *glob)
{
if (node == NULL)
return false;
if (IsA(node, RangeTblEntry))
{
RangeTblEntry *rte = (RangeTblEntry *) node;
/* As above, we need only save relation RTEs */
if (rte->rtekind == RTE_RELATION)
add_rte_to_flat_rtable(glob, rte);
return false;
}
if (IsA(node, Query))
{
/* Recurse into subselects */
return query_tree_walker((Query *) node,
flatten_rtes_walker,
(void *) glob,
QTW_EXAMINE_RTES);
}
return expression_tree_walker(node, flatten_rtes_walker,
(void *) glob);
}
/*
* Add (a copy of) the given RTE to the final rangetable
*
* In the flat rangetable, we zero out substructure pointers that are not
* needed by the executor; this reduces the storage space and copying cost
Represent columns requiring insert and update privileges indentently. Previously, relation range table entries used a single Bitmapset field representing which columns required either UPDATE or INSERT privileges, despite the fact that INSERT and UPDATE privileges are separately cataloged, and may be independently held. As statements so far required either insert or update privileges but never both, that was sufficient. The required permission could be inferred from the top level statement run. The upcoming INSERT ... ON CONFLICT UPDATE feature needs to independently check for both privileges in one statement though, so that is not sufficient anymore. Bumps catversion as stored rules change. Author: Peter Geoghegan Reviewed-By: Andres Freund
11 years ago
* for cached plans. We keep only the alias and eref Alias fields, which are
* needed by EXPLAIN, and the selectedCols, insertedCols and updatedCols
* bitmaps, which are needed for executor-startup permissions checking and for
Better fix for permissions tests in excluded subqueries. This reverts the code changes in 50c137487c96e629e0e5372bb3d1b5f1a2f71a88, which turned out to induce crashes and not completely fix the problem anyway. That commit only considered single subqueries that were excluded by constraint-exclusion logic, but actually the problem also exists for subqueries that are appendrel members (ie part of a UNION ALL list). In such cases we can't add a dummy subpath to the appendrel's AppendPath list without defeating the logic that recognizes when an appendrel is completely excluded. Instead, fix the problem by having setrefs.c scan the rangetable an extra time looking for subqueries that didn't get into the plan tree. (This approach depends on the 9.2 change that made set_subquery_pathlist generate dummy paths for excluded single subqueries, so that the exclusion behavior is the same for single subqueries and appendrel members.) Note: it turns out that the appendrel form of the missed-permissions-checks bug exists as far back as 8.4. However, since the practical effect of that bug seems pretty minimal, consensus is to not attempt to fix it in the back branches, at least not yet. Possibly we could back-port this patch once it's gotten a reasonable amount of testing in HEAD. For the moment I'm just going to revert the previous patch in 9.2.
13 years ago
* trigger event checking.
*/
static void
add_rte_to_flat_rtable(PlannerGlobal *glob, RangeTblEntry *rte)
{
RangeTblEntry *newrte;
/* flat copy to duplicate all the scalar fields */
newrte = (RangeTblEntry *) palloc(sizeof(RangeTblEntry));
memcpy(newrte, rte, sizeof(RangeTblEntry));
/* zap unneeded sub-structure */
newrte->subquery = NULL;
newrte->joinaliasvars = NIL;
Support multi-argument UNNEST(), and TABLE() syntax for multiple functions. This patch adds the ability to write TABLE( function1(), function2(), ...) as a single FROM-clause entry. The result is the concatenation of the first row from each function, followed by the second row from each function, etc; with NULLs inserted if any function produces fewer rows than others. This is believed to be a much more useful behavior than what Postgres currently does with multiple SRFs in a SELECT list. This syntax also provides a reasonable way to combine use of column definition lists with WITH ORDINALITY: put the column definition list inside TABLE(), where it's clear that it doesn't control the ordinality column as well. Also implement SQL-compliant multiple-argument UNNEST(), by turning UNNEST(a,b,c) into TABLE(unnest(a), unnest(b), unnest(c)). The SQL standard specifies TABLE() with only a single function, not multiple functions, and it seems to require an implicit UNNEST() which is not what this patch does. There may be something wrong with that reading of the spec, though, because if it's right then the spec's TABLE() is just a pointless alternative spelling of UNNEST(). After further review of that, we might choose to adopt a different syntax for what this patch does, but in any case this functionality seems clearly worthwhile. Andrew Gierth, reviewed by Zoltán Böszörményi and Heikki Linnakangas, and significantly revised by me
12 years ago
newrte->functions = NIL;
Better fix for permissions tests in excluded subqueries. This reverts the code changes in 50c137487c96e629e0e5372bb3d1b5f1a2f71a88, which turned out to induce crashes and not completely fix the problem anyway. That commit only considered single subqueries that were excluded by constraint-exclusion logic, but actually the problem also exists for subqueries that are appendrel members (ie part of a UNION ALL list). In such cases we can't add a dummy subpath to the appendrel's AppendPath list without defeating the logic that recognizes when an appendrel is completely excluded. Instead, fix the problem by having setrefs.c scan the rangetable an extra time looking for subqueries that didn't get into the plan tree. (This approach depends on the 9.2 change that made set_subquery_pathlist generate dummy paths for excluded single subqueries, so that the exclusion behavior is the same for single subqueries and appendrel members.) Note: it turns out that the appendrel form of the missed-permissions-checks bug exists as far back as 8.4. However, since the practical effect of that bug seems pretty minimal, consensus is to not attempt to fix it in the back branches, at least not yet. Possibly we could back-port this patch once it's gotten a reasonable amount of testing in HEAD. For the moment I'm just going to revert the previous patch in 9.2.
13 years ago
newrte->values_lists = NIL;
newrte->values_collations = NIL;
newrte->ctecoltypes = NIL;
newrte->ctecoltypmods = NIL;
newrte->ctecolcollations = NIL;
glob->finalrtable = lappend(glob->finalrtable, newrte);
/*
* Check for RT index overflow; it's very unlikely, but if it did happen,
* the executor would get confused by varnos that match the special varno
* values.
*/
if (IS_SPECIAL_VARNO(list_length(glob->finalrtable)))
ereport(ERROR,
(errcode(ERRCODE_PROGRAM_LIMIT_EXCEEDED),
errmsg("too many range table entries")));
/*
* If it's a plain relation RTE, add the table to relationOids.
*
* We do this even though the RTE might be unreferenced in the plan tree;
* this would correspond to cases such as views that were expanded, child
* tables that were eliminated by constraint exclusion, etc. Schema
* invalidation on such a rel must still force rebuilding of the plan.
*
* Note we don't bother to avoid making duplicate list entries. We could,
* but it would probably cost more cycles than it would save.
*/
if (newrte->rtekind == RTE_RELATION)
glob->relationOids = lappend_oid(glob->relationOids, newrte->relid);
}
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
/*
* set_plan_refs: recurse through the Plan nodes of a single subquery level
*/
static Plan *
Rearrange planner to save the whole PlannerInfo (subroot) for a subquery. Formerly, set_subquery_pathlist and other creators of plans for subqueries saved only the rangetable and rowMarks lists from the lower-level PlannerInfo. But there's no reason not to remember the whole PlannerInfo, and indeed this turns out to simplify matters in a number of places. The immediate reason for doing this was so that the subroot will still be accessible when we're trying to extract column statistics out of an already-planned subquery. But now that I've done it, it seems like a good code-beautification effort in its own right. I also chose to get rid of the transient subrtable and subrowmark fields in SubqueryScan nodes, in favor of having setrefs.c look up the subquery's RelOptInfo. That required changing all the APIs in setrefs.c to pass PlannerInfo not PlannerGlobal, which was a large but quite mechanical transformation. One side-effect not foreseen at the beginning is that this finally broke inheritance_planner's assumption that replanning the same subquery RTE N times would necessarily give interchangeable results each time. That assumption was always pretty risky, but now we really have to make a separate RTE for each instance so that there's a place to carry the separate subroots.
14 years ago
set_plan_refs(PlannerInfo *root, Plan *plan, int rtoffset)
Postgres95 1.01 Distribution - Virgin Sources
30 years ago
{
Pgindent run for 8.0.
21 years ago
ListCell *l;
Further planner/optimizer cleanups. Move all set_tlist_references and fix_opids processing to a single recursive pass over the plan tree executed at the very tail end of planning, rather than haphazardly here and there at different places. Now that tlist Vars do not get modified until the very end, it's possible to get rid of the klugy var_equal and match_varid partial-matching routines, and just use plain equal() throughout the optimizer. This is a step towards allowing merge and hash joins to be done on expressions instead of only Vars ...
26 years ago
Massive commit to run PGINDENT on all *.c and *.h files.
28 years ago
if (plan == NULL)
Teach the planner to remove SubqueryScan nodes from the plan if they aren't doing anything useful (ie, neither selection nor projection). Also, extend to SubqueryScan the hacks already in place to avoid unnecessary ExecProject calls when the result would just be the same tuple the subquery already delivered. This saves some overhead in UNION and other set operations, as well as avoiding overhead for unflatten-able subqueries. Per example from Sokolov Yura.
21 years ago
return NULL;
Massive commit to run PGINDENT on all *.c and *.h files.
28 years ago
Further planner/optimizer cleanups. Move all set_tlist_references and fix_opids processing to a single recursive pass over the plan tree executed at the very tail end of planning, rather than haphazardly here and there at different places. Now that tlist Vars do not get modified until the very end, it's possible to get rid of the klugy var_equal and match_varid partial-matching routines, and just use plain equal() throughout the optimizer. This is a step towards allowing merge and hash joins to be done on expressions instead of only Vars ...
26 years ago
/*
* Plan-type-specific fixes
*/
switch (nodeTag(plan))
{
case T_SeqScan:
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
{
pgindent run for 8.3.
18 years ago
SeqScan *splan = (SeqScan *) plan;
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
splan->scanrelid += rtoffset;
splan->plan.targetlist =
Rearrange planner to save the whole PlannerInfo (subroot) for a subquery. Formerly, set_subquery_pathlist and other creators of plans for subqueries saved only the rangetable and rowMarks lists from the lower-level PlannerInfo. But there's no reason not to remember the whole PlannerInfo, and indeed this turns out to simplify matters in a number of places. The immediate reason for doing this was so that the subroot will still be accessible when we're trying to extract column statistics out of an already-planned subquery. But now that I've done it, it seems like a good code-beautification effort in its own right. I also chose to get rid of the transient subrtable and subrowmark fields in SubqueryScan nodes, in favor of having setrefs.c look up the subquery's RelOptInfo. That required changing all the APIs in setrefs.c to pass PlannerInfo not PlannerGlobal, which was a large but quite mechanical transformation. One side-effect not foreseen at the beginning is that this finally broke inheritance_planner's assumption that replanning the same subquery RTE N times would necessarily give interchangeable results each time. That assumption was always pretty risky, but now we really have to make a separate RTE for each instance so that there's a place to carry the separate subroots.
14 years ago
fix_scan_list(root, splan->plan.targetlist, rtoffset);
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
splan->plan.qual =
Rearrange planner to save the whole PlannerInfo (subroot) for a subquery. Formerly, set_subquery_pathlist and other creators of plans for subqueries saved only the rangetable and rowMarks lists from the lower-level PlannerInfo. But there's no reason not to remember the whole PlannerInfo, and indeed this turns out to simplify matters in a number of places. The immediate reason for doing this was so that the subroot will still be accessible when we're trying to extract column statistics out of an already-planned subquery. But now that I've done it, it seems like a good code-beautification effort in its own right. I also chose to get rid of the transient subrtable and subrowmark fields in SubqueryScan nodes, in favor of having setrefs.c look up the subquery's RelOptInfo. That required changing all the APIs in setrefs.c to pass PlannerInfo not PlannerGlobal, which was a large but quite mechanical transformation. One side-effect not foreseen at the beginning is that this finally broke inheritance_planner's assumption that replanning the same subquery RTE N times would necessarily give interchangeable results each time. That assumption was always pretty risky, but now we really have to make a separate RTE for each instance so that there's a place to carry the separate subroots.
14 years ago
fix_scan_list(root, splan->plan.qual, rtoffset);
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
}
Further planner/optimizer cleanups. Move all set_tlist_references and fix_opids processing to a single recursive pass over the plan tree executed at the very tail end of planning, rather than haphazardly here and there at different places. Now that tlist Vars do not get modified until the very end, it's possible to get rid of the klugy var_equal and match_varid partial-matching routines, and just use plain equal() throughout the optimizer. This is a step towards allowing merge and hash joins to be done on expressions instead of only Vars ...
26 years ago
break;
TABLESAMPLE, SQL Standard and extensible Add a TABLESAMPLE clause to SELECT statements that allows user to specify random BERNOULLI sampling or block level SYSTEM sampling. Implementation allows for extensible sampling functions to be written, using a standard API. Basic version follows SQLStandard exactly. Usable concrete use cases for the sampling API follow in later commits. Petr Jelinek Reviewed by Michael Paquier and Simon Riggs
10 years ago
case T_SampleScan:
{
pgindent run for 9.5
10 years ago
SampleScan *splan = (SampleScan *) plan;
TABLESAMPLE, SQL Standard and extensible Add a TABLESAMPLE clause to SELECT statements that allows user to specify random BERNOULLI sampling or block level SYSTEM sampling. Implementation allows for extensible sampling functions to be written, using a standard API. Basic version follows SQLStandard exactly. Usable concrete use cases for the sampling API follow in later commits. Petr Jelinek Reviewed by Michael Paquier and Simon Riggs
10 years ago
splan->scanrelid += rtoffset;
splan->plan.targetlist =
fix_scan_list(root, splan->plan.targetlist, rtoffset);
splan->plan.qual =
fix_scan_list(root, splan->plan.qual, rtoffset);
}
break;
Further planner/optimizer cleanups. Move all set_tlist_references and fix_opids processing to a single recursive pass over the plan tree executed at the very tail end of planning, rather than haphazardly here and there at different places. Now that tlist Vars do not get modified until the very end, it's possible to get rid of the klugy var_equal and match_varid partial-matching routines, and just use plain equal() throughout the optimizer. This is a step towards allowing merge and hash joins to be done on expressions instead of only Vars ...
26 years ago
case T_IndexScan:
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
{
pgindent run for 8.3.
18 years ago
IndexScan *splan = (IndexScan *) plan;
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
splan->scan.scanrelid += rtoffset;
splan->scan.plan.targetlist =
Rearrange planner to save the whole PlannerInfo (subroot) for a subquery. Formerly, set_subquery_pathlist and other creators of plans for subqueries saved only the rangetable and rowMarks lists from the lower-level PlannerInfo. But there's no reason not to remember the whole PlannerInfo, and indeed this turns out to simplify matters in a number of places. The immediate reason for doing this was so that the subroot will still be accessible when we're trying to extract column statistics out of an already-planned subquery. But now that I've done it, it seems like a good code-beautification effort in its own right. I also chose to get rid of the transient subrtable and subrowmark fields in SubqueryScan nodes, in favor of having setrefs.c look up the subquery's RelOptInfo. That required changing all the APIs in setrefs.c to pass PlannerInfo not PlannerGlobal, which was a large but quite mechanical transformation. One side-effect not foreseen at the beginning is that this finally broke inheritance_planner's assumption that replanning the same subquery RTE N times would necessarily give interchangeable results each time. That assumption was always pretty risky, but now we really have to make a separate RTE for each instance so that there's a place to carry the separate subroots.
14 years ago
fix_scan_list(root, splan->scan.plan.targetlist, rtoffset);
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
splan->scan.plan.qual =
Rearrange planner to save the whole PlannerInfo (subroot) for a subquery. Formerly, set_subquery_pathlist and other creators of plans for subqueries saved only the rangetable and rowMarks lists from the lower-level PlannerInfo. But there's no reason not to remember the whole PlannerInfo, and indeed this turns out to simplify matters in a number of places. The immediate reason for doing this was so that the subroot will still be accessible when we're trying to extract column statistics out of an already-planned subquery. But now that I've done it, it seems like a good code-beautification effort in its own right. I also chose to get rid of the transient subrtable and subrowmark fields in SubqueryScan nodes, in favor of having setrefs.c look up the subquery's RelOptInfo. That required changing all the APIs in setrefs.c to pass PlannerInfo not PlannerGlobal, which was a large but quite mechanical transformation. One side-effect not foreseen at the beginning is that this finally broke inheritance_planner's assumption that replanning the same subquery RTE N times would necessarily give interchangeable results each time. That assumption was always pretty risky, but now we really have to make a separate RTE for each instance so that there's a place to carry the separate subroots.
14 years ago
fix_scan_list(root, splan->scan.plan.qual, rtoffset);
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
splan->indexqual =
Rearrange planner to save the whole PlannerInfo (subroot) for a subquery. Formerly, set_subquery_pathlist and other creators of plans for subqueries saved only the rangetable and rowMarks lists from the lower-level PlannerInfo. But there's no reason not to remember the whole PlannerInfo, and indeed this turns out to simplify matters in a number of places. The immediate reason for doing this was so that the subroot will still be accessible when we're trying to extract column statistics out of an already-planned subquery. But now that I've done it, it seems like a good code-beautification effort in its own right. I also chose to get rid of the transient subrtable and subrowmark fields in SubqueryScan nodes, in favor of having setrefs.c look up the subquery's RelOptInfo. That required changing all the APIs in setrefs.c to pass PlannerInfo not PlannerGlobal, which was a large but quite mechanical transformation. One side-effect not foreseen at the beginning is that this finally broke inheritance_planner's assumption that replanning the same subquery RTE N times would necessarily give interchangeable results each time. That assumption was always pretty risky, but now we really have to make a separate RTE for each instance so that there's a place to carry the separate subroots.
14 years ago
fix_scan_list(root, splan->indexqual, rtoffset);
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
splan->indexqualorig =
Rearrange planner to save the whole PlannerInfo (subroot) for a subquery. Formerly, set_subquery_pathlist and other creators of plans for subqueries saved only the rangetable and rowMarks lists from the lower-level PlannerInfo. But there's no reason not to remember the whole PlannerInfo, and indeed this turns out to simplify matters in a number of places. The immediate reason for doing this was so that the subroot will still be accessible when we're trying to extract column statistics out of an already-planned subquery. But now that I've done it, it seems like a good code-beautification effort in its own right. I also chose to get rid of the transient subrtable and subrowmark fields in SubqueryScan nodes, in favor of having setrefs.c look up the subquery's RelOptInfo. That required changing all the APIs in setrefs.c to pass PlannerInfo not PlannerGlobal, which was a large but quite mechanical transformation. One side-effect not foreseen at the beginning is that this finally broke inheritance_planner's assumption that replanning the same subquery RTE N times would necessarily give interchangeable results each time. That assumption was always pretty risky, but now we really have to make a separate RTE for each instance so that there's a place to carry the separate subroots.
14 years ago
fix_scan_list(root, splan->indexqualorig, rtoffset);
Create core infrastructure for KNNGIST. This is a heavily revised version of builtin_knngist_core-0.9. The ordering operators are no longer mixed in with actual quals, which would have confused not only humans but significant parts of the planner. Instead, ordering operators are carried separately throughout planning and execution. Since the API for ambeginscan and amrescan functions had to be changed anyway, this commit takes the opportunity to rationalize that a bit. RelationGetIndexScan no longer forces a premature index_rescan call; instead, callers of index_beginscan must call index_rescan too. Aside from making the AM-side initialization logic a bit less peculiar, this has the advantage that we do not make a useless extra am_rescan call when there are runtime key values. AMs formerly could not assume that the key values passed to amrescan were actually valid; now they can. Teodor Sigaev and Tom Lane
15 years ago
splan->indexorderby =
Rearrange planner to save the whole PlannerInfo (subroot) for a subquery. Formerly, set_subquery_pathlist and other creators of plans for subqueries saved only the rangetable and rowMarks lists from the lower-level PlannerInfo. But there's no reason not to remember the whole PlannerInfo, and indeed this turns out to simplify matters in a number of places. The immediate reason for doing this was so that the subroot will still be accessible when we're trying to extract column statistics out of an already-planned subquery. But now that I've done it, it seems like a good code-beautification effort in its own right. I also chose to get rid of the transient subrtable and subrowmark fields in SubqueryScan nodes, in favor of having setrefs.c look up the subquery's RelOptInfo. That required changing all the APIs in setrefs.c to pass PlannerInfo not PlannerGlobal, which was a large but quite mechanical transformation. One side-effect not foreseen at the beginning is that this finally broke inheritance_planner's assumption that replanning the same subquery RTE N times would necessarily give interchangeable results each time. That assumption was always pretty risky, but now we really have to make a separate RTE for each instance so that there's a place to carry the separate subroots.
14 years ago
fix_scan_list(root, splan->indexorderby, rtoffset);
Create core infrastructure for KNNGIST. This is a heavily revised version of builtin_knngist_core-0.9. The ordering operators are no longer mixed in with actual quals, which would have confused not only humans but significant parts of the planner. Instead, ordering operators are carried separately throughout planning and execution. Since the API for ambeginscan and amrescan functions had to be changed anyway, this commit takes the opportunity to rationalize that a bit. RelationGetIndexScan no longer forces a premature index_rescan call; instead, callers of index_beginscan must call index_rescan too. Aside from making the AM-side initialization logic a bit less peculiar, this has the advantage that we do not make a useless extra am_rescan call when there are runtime key values. AMs formerly could not assume that the key values passed to amrescan were actually valid; now they can. Teodor Sigaev and Tom Lane
15 years ago
splan->indexorderbyorig =
Rearrange planner to save the whole PlannerInfo (subroot) for a subquery. Formerly, set_subquery_pathlist and other creators of plans for subqueries saved only the rangetable and rowMarks lists from the lower-level PlannerInfo. But there's no reason not to remember the whole PlannerInfo, and indeed this turns out to simplify matters in a number of places. The immediate reason for doing this was so that the subroot will still be accessible when we're trying to extract column statistics out of an already-planned subquery. But now that I've done it, it seems like a good code-beautification effort in its own right. I also chose to get rid of the transient subrtable and subrowmark fields in SubqueryScan nodes, in favor of having setrefs.c look up the subquery's RelOptInfo. That required changing all the APIs in setrefs.c to pass PlannerInfo not PlannerGlobal, which was a large but quite mechanical transformation. One side-effect not foreseen at the beginning is that this finally broke inheritance_planner's assumption that replanning the same subquery RTE N times would necessarily give interchangeable results each time. That assumption was always pretty risky, but now we really have to make a separate RTE for each instance so that there's a place to carry the separate subroots.
14 years ago
fix_scan_list(root, splan->indexorderbyorig, rtoffset);
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
}
Repair bug reported by ldm@apartia.com: Append nodes, which don't actually use their targetlist, are given a targetlist that is just a pointer to the first appended plan's targetlist. This is OK, but what is not OK is that any sub-select expressions in said tlist were being entered in the subPlan lists of both the Append and the first appended plan. That led to two startup and two shutdown calls for the same plan node at exec time, which led to crashes. Fix is to not generate a list of subPlans for an Append node. Same problem and fix apply to other node types that don't have a real, functioning targetlist: Material, Sort, Unique, Hash.
26 years ago
break;
Rearrange the implementation of index-only scans. This commit changes index-only scans so that data is read directly from the index tuple without first generating a faux heap tuple. The only immediate benefit is that indexes on system columns (such as OID) can be used in index-only scans, but this is necessary infrastructure if we are ever to support index-only scans on expression indexes. The executor is now ready for that, though the planner still needs substantial work to recognize the possibility. To do this, Vars in index-only plan nodes have to refer to index columns not heap columns. I introduced a new special varno, INDEX_VAR, to mark such Vars to avoid confusion. (In passing, this commit renames the two existing special varnos to OUTER_VAR and INNER_VAR.) This allows ruleutils.c to handle them with logic similar to what we use for subplan reference Vars. Since index-only scans are now fundamentally different from regular indexscans so far as their expression subtrees are concerned, I also chose to change them to have their own plan node type (and hence, their own executor source file).
14 years ago
case T_IndexOnlyScan:
{
Run pgindent on 9.2 source tree in preparation for first 9.3 commit-fest.
13 years ago
IndexOnlyScan *splan = (IndexOnlyScan *) plan;
Rearrange the implementation of index-only scans. This commit changes index-only scans so that data is read directly from the index tuple without first generating a faux heap tuple. The only immediate benefit is that indexes on system columns (such as OID) can be used in index-only scans, but this is necessary infrastructure if we are ever to support index-only scans on expression indexes. The executor is now ready for that, though the planner still needs substantial work to recognize the possibility. To do this, Vars in index-only plan nodes have to refer to index columns not heap columns. I introduced a new special varno, INDEX_VAR, to mark such Vars to avoid confusion. (In passing, this commit renames the two existing special varnos to OUTER_VAR and INNER_VAR.) This allows ruleutils.c to handle them with logic similar to what we use for subplan reference Vars. Since index-only scans are now fundamentally different from regular indexscans so far as their expression subtrees are concerned, I also chose to change them to have their own plan node type (and hence, their own executor source file).
14 years ago
return set_indexonlyscan_references(root, splan, rtoffset);
}
break;
Create executor and planner-backend support for decoupled heap and index scans, using in-memory tuple ID bitmaps as the intermediary. The planner frontend (path creation and cost estimation) is not there yet, so none of this code can be executed. I have tested it using some hacked planner code that is far too ugly to see the light of day, however. Committing now so that the bulk of the infrastructure changes go in before the tree drifts under me.
21 years ago
case T_BitmapIndexScan:
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
{
BitmapIndexScan *splan = (BitmapIndexScan *) plan;
splan->scan.scanrelid += rtoffset;
/* no need to fix targetlist and qual */
Assert(splan->scan.plan.targetlist == NIL);
Assert(splan->scan.plan.qual == NIL);
splan->indexqual =
Rearrange planner to save the whole PlannerInfo (subroot) for a subquery. Formerly, set_subquery_pathlist and other creators of plans for subqueries saved only the rangetable and rowMarks lists from the lower-level PlannerInfo. But there's no reason not to remember the whole PlannerInfo, and indeed this turns out to simplify matters in a number of places. The immediate reason for doing this was so that the subroot will still be accessible when we're trying to extract column statistics out of an already-planned subquery. But now that I've done it, it seems like a good code-beautification effort in its own right. I also chose to get rid of the transient subrtable and subrowmark fields in SubqueryScan nodes, in favor of having setrefs.c look up the subquery's RelOptInfo. That required changing all the APIs in setrefs.c to pass PlannerInfo not PlannerGlobal, which was a large but quite mechanical transformation. One side-effect not foreseen at the beginning is that this finally broke inheritance_planner's assumption that replanning the same subquery RTE N times would necessarily give interchangeable results each time. That assumption was always pretty risky, but now we really have to make a separate RTE for each instance so that there's a place to carry the separate subroots.
14 years ago
fix_scan_list(root, splan->indexqual, rtoffset);
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
splan->indexqualorig =
Rearrange planner to save the whole PlannerInfo (subroot) for a subquery. Formerly, set_subquery_pathlist and other creators of plans for subqueries saved only the rangetable and rowMarks lists from the lower-level PlannerInfo. But there's no reason not to remember the whole PlannerInfo, and indeed this turns out to simplify matters in a number of places. The immediate reason for doing this was so that the subroot will still be accessible when we're trying to extract column statistics out of an already-planned subquery. But now that I've done it, it seems like a good code-beautification effort in its own right. I also chose to get rid of the transient subrtable and subrowmark fields in SubqueryScan nodes, in favor of having setrefs.c look up the subquery's RelOptInfo. That required changing all the APIs in setrefs.c to pass PlannerInfo not PlannerGlobal, which was a large but quite mechanical transformation. One side-effect not foreseen at the beginning is that this finally broke inheritance_planner's assumption that replanning the same subquery RTE N times would necessarily give interchangeable results each time. That assumption was always pretty risky, but now we really have to make a separate RTE for each instance so that there's a place to carry the separate subroots.
14 years ago
fix_scan_list(root, splan->indexqualorig, rtoffset);
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
}
Create executor and planner-backend support for decoupled heap and index scans, using in-memory tuple ID bitmaps as the intermediary. The planner frontend (path creation and cost estimation) is not there yet, so none of this code can be executed. I have tested it using some hacked planner code that is far too ugly to see the light of day, however. Committing now so that the bulk of the infrastructure changes go in before the tree drifts under me.
21 years ago
break;
case T_BitmapHeapScan:
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
{
BitmapHeapScan *splan = (BitmapHeapScan *) plan;
splan->scan.scanrelid += rtoffset;
splan->scan.plan.targetlist =
Rearrange planner to save the whole PlannerInfo (subroot) for a subquery. Formerly, set_subquery_pathlist and other creators of plans for subqueries saved only the rangetable and rowMarks lists from the lower-level PlannerInfo. But there's no reason not to remember the whole PlannerInfo, and indeed this turns out to simplify matters in a number of places. The immediate reason for doing this was so that the subroot will still be accessible when we're trying to extract column statistics out of an already-planned subquery. But now that I've done it, it seems like a good code-beautification effort in its own right. I also chose to get rid of the transient subrtable and subrowmark fields in SubqueryScan nodes, in favor of having setrefs.c look up the subquery's RelOptInfo. That required changing all the APIs in setrefs.c to pass PlannerInfo not PlannerGlobal, which was a large but quite mechanical transformation. One side-effect not foreseen at the beginning is that this finally broke inheritance_planner's assumption that replanning the same subquery RTE N times would necessarily give interchangeable results each time. That assumption was always pretty risky, but now we really have to make a separate RTE for each instance so that there's a place to carry the separate subroots.
14 years ago
fix_scan_list(root, splan->scan.plan.targetlist, rtoffset);
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
splan->scan.plan.qual =
Rearrange planner to save the whole PlannerInfo (subroot) for a subquery. Formerly, set_subquery_pathlist and other creators of plans for subqueries saved only the rangetable and rowMarks lists from the lower-level PlannerInfo. But there's no reason not to remember the whole PlannerInfo, and indeed this turns out to simplify matters in a number of places. The immediate reason for doing this was so that the subroot will still be accessible when we're trying to extract column statistics out of an already-planned subquery. But now that I've done it, it seems like a good code-beautification effort in its own right. I also chose to get rid of the transient subrtable and subrowmark fields in SubqueryScan nodes, in favor of having setrefs.c look up the subquery's RelOptInfo. That required changing all the APIs in setrefs.c to pass PlannerInfo not PlannerGlobal, which was a large but quite mechanical transformation. One side-effect not foreseen at the beginning is that this finally broke inheritance_planner's assumption that replanning the same subquery RTE N times would necessarily give interchangeable results each time. That assumption was always pretty risky, but now we really have to make a separate RTE for each instance so that there's a place to carry the separate subroots.
14 years ago
fix_scan_list(root, splan->scan.plan.qual, rtoffset);
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
splan->bitmapqualorig =
Rearrange planner to save the whole PlannerInfo (subroot) for a subquery. Formerly, set_subquery_pathlist and other creators of plans for subqueries saved only the rangetable and rowMarks lists from the lower-level PlannerInfo. But there's no reason not to remember the whole PlannerInfo, and indeed this turns out to simplify matters in a number of places. The immediate reason for doing this was so that the subroot will still be accessible when we're trying to extract column statistics out of an already-planned subquery. But now that I've done it, it seems like a good code-beautification effort in its own right. I also chose to get rid of the transient subrtable and subrowmark fields in SubqueryScan nodes, in favor of having setrefs.c look up the subquery's RelOptInfo. That required changing all the APIs in setrefs.c to pass PlannerInfo not PlannerGlobal, which was a large but quite mechanical transformation. One side-effect not foreseen at the beginning is that this finally broke inheritance_planner's assumption that replanning the same subquery RTE N times would necessarily give interchangeable results each time. That assumption was always pretty risky, but now we really have to make a separate RTE for each instance so that there's a place to carry the separate subroots.
14 years ago
fix_scan_list(root, splan->bitmapqualorig, rtoffset);
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
}
Create executor and planner-backend support for decoupled heap and index scans, using in-memory tuple ID bitmaps as the intermediary. The planner frontend (path creation and cost estimation) is not there yet, so none of this code can be executed. I have tested it using some hacked planner code that is far too ugly to see the light of day, however. Committing now so that the bulk of the infrastructure changes go in before the tree drifts under me.
21 years ago
break;
Repair bug reported by ldm@apartia.com: Append nodes, which don't actually use their targetlist, are given a targetlist that is just a pointer to the first appended plan's targetlist. This is OK, but what is not OK is that any sub-select expressions in said tlist were being entered in the subPlan lists of both the Append and the first appended plan. That led to two startup and two shutdown calls for the same plan node at exec time, which led to crashes. Fix is to not generate a list of subPlans for an Append node. Same problem and fix apply to other node types that don't have a real, functioning targetlist: Material, Sort, Unique, Hash.
26 years ago
case T_TidScan:
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
{
pgindent run for 8.3.
18 years ago
TidScan *splan = (TidScan *) plan;
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
splan->scan.scanrelid += rtoffset;
splan->scan.plan.targetlist =
Rearrange planner to save the whole PlannerInfo (subroot) for a subquery. Formerly, set_subquery_pathlist and other creators of plans for subqueries saved only the rangetable and rowMarks lists from the lower-level PlannerInfo. But there's no reason not to remember the whole PlannerInfo, and indeed this turns out to simplify matters in a number of places. The immediate reason for doing this was so that the subroot will still be accessible when we're trying to extract column statistics out of an already-planned subquery. But now that I've done it, it seems like a good code-beautification effort in its own right. I also chose to get rid of the transient subrtable and subrowmark fields in SubqueryScan nodes, in favor of having setrefs.c look up the subquery's RelOptInfo. That required changing all the APIs in setrefs.c to pass PlannerInfo not PlannerGlobal, which was a large but quite mechanical transformation. One side-effect not foreseen at the beginning is that this finally broke inheritance_planner's assumption that replanning the same subquery RTE N times would necessarily give interchangeable results each time. That assumption was always pretty risky, but now we really have to make a separate RTE for each instance so that there's a place to carry the separate subroots.
14 years ago
fix_scan_list(root, splan->scan.plan.targetlist, rtoffset);
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
splan->scan.plan.qual =
Rearrange planner to save the whole PlannerInfo (subroot) for a subquery. Formerly, set_subquery_pathlist and other creators of plans for subqueries saved only the rangetable and rowMarks lists from the lower-level PlannerInfo. But there's no reason not to remember the whole PlannerInfo, and indeed this turns out to simplify matters in a number of places. The immediate reason for doing this was so that the subroot will still be accessible when we're trying to extract column statistics out of an already-planned subquery. But now that I've done it, it seems like a good code-beautification effort in its own right. I also chose to get rid of the transient subrtable and subrowmark fields in SubqueryScan nodes, in favor of having setrefs.c look up the subquery's RelOptInfo. That required changing all the APIs in setrefs.c to pass PlannerInfo not PlannerGlobal, which was a large but quite mechanical transformation. One side-effect not foreseen at the beginning is that this finally broke inheritance_planner's assumption that replanning the same subquery RTE N times would necessarily give interchangeable results each time. That assumption was always pretty risky, but now we really have to make a separate RTE for each instance so that there's a place to carry the separate subroots.
14 years ago
fix_scan_list(root, splan->scan.plan.qual, rtoffset);
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
splan->tidquals =
Rearrange planner to save the whole PlannerInfo (subroot) for a subquery. Formerly, set_subquery_pathlist and other creators of plans for subqueries saved only the rangetable and rowMarks lists from the lower-level PlannerInfo. But there's no reason not to remember the whole PlannerInfo, and indeed this turns out to simplify matters in a number of places. The immediate reason for doing this was so that the subroot will still be accessible when we're trying to extract column statistics out of an already-planned subquery. But now that I've done it, it seems like a good code-beautification effort in its own right. I also chose to get rid of the transient subrtable and subrowmark fields in SubqueryScan nodes, in favor of having setrefs.c look up the subquery's RelOptInfo. That required changing all the APIs in setrefs.c to pass PlannerInfo not PlannerGlobal, which was a large but quite mechanical transformation. One side-effect not foreseen at the beginning is that this finally broke inheritance_planner's assumption that replanning the same subquery RTE N times would necessarily give interchangeable results each time. That assumption was always pretty risky, but now we really have to make a separate RTE for each instance so that there's a place to carry the separate subroots.
14 years ago
fix_scan_list(root, splan->tidquals, rtoffset);
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
}
Further planner/optimizer cleanups. Move all set_tlist_references and fix_opids processing to a single recursive pass over the plan tree executed at the very tail end of planning, rather than haphazardly here and there at different places. Now that tlist Vars do not get modified until the very end, it's possible to get rid of the klugy var_equal and match_varid partial-matching routines, and just use plain equal() throughout the optimizer. This is a step towards allowing merge and hash joins to be done on expressions instead of only Vars ...
26 years ago
break;
Subselects in FROM clause, per ISO syntax: FROM (SELECT ...) [AS] alias. (Don't forget that an alias is required.) Views reimplemented as expanding to subselect-in-FROM. Grouping, aggregates, DISTINCT in views actually work now (he says optimistically). No UNION support in subselects/views yet, but I have some ideas about that. Rule-related permissions checking moved out of rewriter and into executor. INITDB REQUIRED!
25 years ago
case T_SubqueryScan:
Teach the planner to remove SubqueryScan nodes from the plan if they aren't doing anything useful (ie, neither selection nor projection). Also, extend to SubqueryScan the hacks already in place to avoid unnecessary ExecProject calls when the result would just be the same tuple the subquery already delivered. This saves some overhead in UNION and other set operations, as well as avoiding overhead for unflatten-able subqueries. Per example from Sokolov Yura.
21 years ago
/* Needs special treatment, see comments below */
Rearrange planner to save the whole PlannerInfo (subroot) for a subquery. Formerly, set_subquery_pathlist and other creators of plans for subqueries saved only the rangetable and rowMarks lists from the lower-level PlannerInfo. But there's no reason not to remember the whole PlannerInfo, and indeed this turns out to simplify matters in a number of places. The immediate reason for doing this was so that the subroot will still be accessible when we're trying to extract column statistics out of an already-planned subquery. But now that I've done it, it seems like a good code-beautification effort in its own right. I also chose to get rid of the transient subrtable and subrowmark fields in SubqueryScan nodes, in favor of having setrefs.c look up the subquery's RelOptInfo. That required changing all the APIs in setrefs.c to pass PlannerInfo not PlannerGlobal, which was a large but quite mechanical transformation. One side-effect not foreseen at the beginning is that this finally broke inheritance_planner's assumption that replanning the same subquery RTE N times would necessarily give interchangeable results each time. That assumption was always pretty risky, but now we really have to make a separate RTE for each instance so that there's a place to carry the separate subroots.
14 years ago
return set_subqueryscan_references(root,
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
(SubqueryScan *) plan,
rtoffset);
First pass at set-returning-functions in FROM, by Joe Conway with some kibitzing from Tom Lane. Not everything works yet, and there's no documentation or regression test, but let's commit this so Joe doesn't need to cope with tracking changes in so many files ...
24 years ago
case T_FunctionScan:
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
{
FunctionScan *splan = (FunctionScan *) plan;
splan->scan.scanrelid += rtoffset;
splan->scan.plan.targetlist =
Rearrange planner to save the whole PlannerInfo (subroot) for a subquery. Formerly, set_subquery_pathlist and other creators of plans for subqueries saved only the rangetable and rowMarks lists from the lower-level PlannerInfo. But there's no reason not to remember the whole PlannerInfo, and indeed this turns out to simplify matters in a number of places. The immediate reason for doing this was so that the subroot will still be accessible when we're trying to extract column statistics out of an already-planned subquery. But now that I've done it, it seems like a good code-beautification effort in its own right. I also chose to get rid of the transient subrtable and subrowmark fields in SubqueryScan nodes, in favor of having setrefs.c look up the subquery's RelOptInfo. That required changing all the APIs in setrefs.c to pass PlannerInfo not PlannerGlobal, which was a large but quite mechanical transformation. One side-effect not foreseen at the beginning is that this finally broke inheritance_planner's assumption that replanning the same subquery RTE N times would necessarily give interchangeable results each time. That assumption was always pretty risky, but now we really have to make a separate RTE for each instance so that there's a place to carry the separate subroots.
14 years ago
fix_scan_list(root, splan->scan.plan.targetlist, rtoffset);
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
splan->scan.plan.qual =
Rearrange planner to save the whole PlannerInfo (subroot) for a subquery. Formerly, set_subquery_pathlist and other creators of plans for subqueries saved only the rangetable and rowMarks lists from the lower-level PlannerInfo. But there's no reason not to remember the whole PlannerInfo, and indeed this turns out to simplify matters in a number of places. The immediate reason for doing this was so that the subroot will still be accessible when we're trying to extract column statistics out of an already-planned subquery. But now that I've done it, it seems like a good code-beautification effort in its own right. I also chose to get rid of the transient subrtable and subrowmark fields in SubqueryScan nodes, in favor of having setrefs.c look up the subquery's RelOptInfo. That required changing all the APIs in setrefs.c to pass PlannerInfo not PlannerGlobal, which was a large but quite mechanical transformation. One side-effect not foreseen at the beginning is that this finally broke inheritance_planner's assumption that replanning the same subquery RTE N times would necessarily give interchangeable results each time. That assumption was always pretty risky, but now we really have to make a separate RTE for each instance so that there's a place to carry the separate subroots.
14 years ago
fix_scan_list(root, splan->scan.plan.qual, rtoffset);
Support multi-argument UNNEST(), and TABLE() syntax for multiple functions. This patch adds the ability to write TABLE( function1(), function2(), ...) as a single FROM-clause entry. The result is the concatenation of the first row from each function, followed by the second row from each function, etc; with NULLs inserted if any function produces fewer rows than others. This is believed to be a much more useful behavior than what Postgres currently does with multiple SRFs in a SELECT list. This syntax also provides a reasonable way to combine use of column definition lists with WITH ORDINALITY: put the column definition list inside TABLE(), where it's clear that it doesn't control the ordinality column as well. Also implement SQL-compliant multiple-argument UNNEST(), by turning UNNEST(a,b,c) into TABLE(unnest(a), unnest(b), unnest(c)). The SQL standard specifies TABLE() with only a single function, not multiple functions, and it seems to require an implicit UNNEST() which is not what this patch does. There may be something wrong with that reading of the spec, though, because if it's right then the spec's TABLE() is just a pointless alternative spelling of UNNEST(). After further review of that, we might choose to adopt a different syntax for what this patch does, but in any case this functionality seems clearly worthwhile. Andrew Gierth, reviewed by Zoltán Böszörményi and Heikki Linnakangas, and significantly revised by me
12 years ago
splan->functions =
fix_scan_list(root, splan->functions, rtoffset);
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
}
First pass at set-returning-functions in FROM, by Joe Conway with some kibitzing from Tom Lane. Not everything works yet, and there's no documentation or regression test, but let's commit this so Joe doesn't need to cope with tracking changes in so many files ...
24 years ago
break;
Add support for multi-row VALUES clauses as part of INSERT statements (e.g. "INSERT ... VALUES (...), (...), ...") and elsewhere as allowed by the spec. (e.g. similar to a FROM clause subselect). initdb required. Joe Conway and Tom Lane.
19 years ago
case T_ValuesScan:
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
{
ValuesScan *splan = (ValuesScan *) plan;
splan->scan.scanrelid += rtoffset;
splan->scan.plan.targetlist =
Rearrange planner to save the whole PlannerInfo (subroot) for a subquery. Formerly, set_subquery_pathlist and other creators of plans for subqueries saved only the rangetable and rowMarks lists from the lower-level PlannerInfo. But there's no reason not to remember the whole PlannerInfo, and indeed this turns out to simplify matters in a number of places. The immediate reason for doing this was so that the subroot will still be accessible when we're trying to extract column statistics out of an already-planned subquery. But now that I've done it, it seems like a good code-beautification effort in its own right. I also chose to get rid of the transient subrtable and subrowmark fields in SubqueryScan nodes, in favor of having setrefs.c look up the subquery's RelOptInfo. That required changing all the APIs in setrefs.c to pass PlannerInfo not PlannerGlobal, which was a large but quite mechanical transformation. One side-effect not foreseen at the beginning is that this finally broke inheritance_planner's assumption that replanning the same subquery RTE N times would necessarily give interchangeable results each time. That assumption was always pretty risky, but now we really have to make a separate RTE for each instance so that there's a place to carry the separate subroots.
14 years ago
fix_scan_list(root, splan->scan.plan.targetlist, rtoffset);
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
splan->scan.plan.qual =
Rearrange planner to save the whole PlannerInfo (subroot) for a subquery. Formerly, set_subquery_pathlist and other creators of plans for subqueries saved only the rangetable and rowMarks lists from the lower-level PlannerInfo. But there's no reason not to remember the whole PlannerInfo, and indeed this turns out to simplify matters in a number of places. The immediate reason for doing this was so that the subroot will still be accessible when we're trying to extract column statistics out of an already-planned subquery. But now that I've done it, it seems like a good code-beautification effort in its own right. I also chose to get rid of the transient subrtable and subrowmark fields in SubqueryScan nodes, in favor of having setrefs.c look up the subquery's RelOptInfo. That required changing all the APIs in setrefs.c to pass PlannerInfo not PlannerGlobal, which was a large but quite mechanical transformation. One side-effect not foreseen at the beginning is that this finally broke inheritance_planner's assumption that replanning the same subquery RTE N times would necessarily give interchangeable results each time. That assumption was always pretty risky, but now we really have to make a separate RTE for each instance so that there's a place to carry the separate subroots.
14 years ago
fix_scan_list(root, splan->scan.plan.qual, rtoffset);
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
splan->values_lists =
Rearrange planner to save the whole PlannerInfo (subroot) for a subquery. Formerly, set_subquery_pathlist and other creators of plans for subqueries saved only the rangetable and rowMarks lists from the lower-level PlannerInfo. But there's no reason not to remember the whole PlannerInfo, and indeed this turns out to simplify matters in a number of places. The immediate reason for doing this was so that the subroot will still be accessible when we're trying to extract column statistics out of an already-planned subquery. But now that I've done it, it seems like a good code-beautification effort in its own right. I also chose to get rid of the transient subrtable and subrowmark fields in SubqueryScan nodes, in favor of having setrefs.c look up the subquery's RelOptInfo. That required changing all the APIs in setrefs.c to pass PlannerInfo not PlannerGlobal, which was a large but quite mechanical transformation. One side-effect not foreseen at the beginning is that this finally broke inheritance_planner's assumption that replanning the same subquery RTE N times would necessarily give interchangeable results each time. That assumption was always pretty risky, but now we really have to make a separate RTE for each instance so that there's a place to carry the separate subroots.
14 years ago
fix_scan_list(root, splan->values_lists, rtoffset);
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
}
Add support for multi-row VALUES clauses as part of INSERT statements (e.g. "INSERT ... VALUES (...), (...), ...") and elsewhere as allowed by the spec. (e.g. similar to a FROM clause subselect). initdb required. Joe Conway and Tom Lane.
19 years ago
break;
Implement SQL-standard WITH clauses, including WITH RECURSIVE. There are some unimplemented aspects: recursive queries must use UNION ALL (should allow UNION too), and we don't have SEARCH or CYCLE clauses. These might or might not get done for 8.4, but even without them it's a pretty useful feature. There are also a couple of small loose ends and definitional quibbles, which I'll send a memo about to pgsql-hackers shortly. But let's land the patch now so we can get on with other development. Yoshiyuki Asaba, with lots of help from Tatsuo Ishii and Tom Lane
17 years ago
case T_CteScan:
{
8.4 pgindent run, with new combined Linux/FreeBSD/MinGW typedef list provided by Andrew.
16 years ago
CteScan *splan = (CteScan *) plan;
Implement SQL-standard WITH clauses, including WITH RECURSIVE. There are some unimplemented aspects: recursive queries must use UNION ALL (should allow UNION too), and we don't have SEARCH or CYCLE clauses. These might or might not get done for 8.4, but even without them it's a pretty useful feature. There are also a couple of small loose ends and definitional quibbles, which I'll send a memo about to pgsql-hackers shortly. But let's land the patch now so we can get on with other development. Yoshiyuki Asaba, with lots of help from Tatsuo Ishii and Tom Lane
17 years ago
splan->scan.scanrelid += rtoffset;
splan->scan.plan.targetlist =
Rearrange planner to save the whole PlannerInfo (subroot) for a subquery. Formerly, set_subquery_pathlist and other creators of plans for subqueries saved only the rangetable and rowMarks lists from the lower-level PlannerInfo. But there's no reason not to remember the whole PlannerInfo, and indeed this turns out to simplify matters in a number of places. The immediate reason for doing this was so that the subroot will still be accessible when we're trying to extract column statistics out of an already-planned subquery. But now that I've done it, it seems like a good code-beautification effort in its own right. I also chose to get rid of the transient subrtable and subrowmark fields in SubqueryScan nodes, in favor of having setrefs.c look up the subquery's RelOptInfo. That required changing all the APIs in setrefs.c to pass PlannerInfo not PlannerGlobal, which was a large but quite mechanical transformation. One side-effect not foreseen at the beginning is that this finally broke inheritance_planner's assumption that replanning the same subquery RTE N times would necessarily give interchangeable results each time. That assumption was always pretty risky, but now we really have to make a separate RTE for each instance so that there's a place to carry the separate subroots.
14 years ago
fix_scan_list(root, splan->scan.plan.targetlist, rtoffset);
Implement SQL-standard WITH clauses, including WITH RECURSIVE. There are some unimplemented aspects: recursive queries must use UNION ALL (should allow UNION too), and we don't have SEARCH or CYCLE clauses. These might or might not get done for 8.4, but even without them it's a pretty useful feature. There are also a couple of small loose ends and definitional quibbles, which I'll send a memo about to pgsql-hackers shortly. But let's land the patch now so we can get on with other development. Yoshiyuki Asaba, with lots of help from Tatsuo Ishii and Tom Lane
17 years ago
splan->scan.plan.qual =
Rearrange planner to save the whole PlannerInfo (subroot) for a subquery. Formerly, set_subquery_pathlist and other creators of plans for subqueries saved only the rangetable and rowMarks lists from the lower-level PlannerInfo. But there's no reason not to remember the whole PlannerInfo, and indeed this turns out to simplify matters in a number of places. The immediate reason for doing this was so that the subroot will still be accessible when we're trying to extract column statistics out of an already-planned subquery. But now that I've done it, it seems like a good code-beautification effort in its own right. I also chose to get rid of the transient subrtable and subrowmark fields in SubqueryScan nodes, in favor of having setrefs.c look up the subquery's RelOptInfo. That required changing all the APIs in setrefs.c to pass PlannerInfo not PlannerGlobal, which was a large but quite mechanical transformation. One side-effect not foreseen at the beginning is that this finally broke inheritance_planner's assumption that replanning the same subquery RTE N times would necessarily give interchangeable results each time. That assumption was always pretty risky, but now we really have to make a separate RTE for each instance so that there's a place to carry the separate subroots.
14 years ago
fix_scan_list(root, splan->scan.plan.qual, rtoffset);
Implement SQL-standard WITH clauses, including WITH RECURSIVE. There are some unimplemented aspects: recursive queries must use UNION ALL (should allow UNION too), and we don't have SEARCH or CYCLE clauses. These might or might not get done for 8.4, but even without them it's a pretty useful feature. There are also a couple of small loose ends and definitional quibbles, which I'll send a memo about to pgsql-hackers shortly. But let's land the patch now so we can get on with other development. Yoshiyuki Asaba, with lots of help from Tatsuo Ishii and Tom Lane
17 years ago
}
break;
case T_WorkTableScan:
{
WorkTableScan *splan = (WorkTableScan *) plan;
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
Implement SQL-standard WITH clauses, including WITH RECURSIVE. There are some unimplemented aspects: recursive queries must use UNION ALL (should allow UNION too), and we don't have SEARCH or CYCLE clauses. These might or might not get done for 8.4, but even without them it's a pretty useful feature. There are also a couple of small loose ends and definitional quibbles, which I'll send a memo about to pgsql-hackers shortly. But let's land the patch now so we can get on with other development. Yoshiyuki Asaba, with lots of help from Tatsuo Ishii and Tom Lane
17 years ago
splan->scan.scanrelid += rtoffset;
splan->scan.plan.targetlist =
Rearrange planner to save the whole PlannerInfo (subroot) for a subquery. Formerly, set_subquery_pathlist and other creators of plans for subqueries saved only the rangetable and rowMarks lists from the lower-level PlannerInfo. But there's no reason not to remember the whole PlannerInfo, and indeed this turns out to simplify matters in a number of places. The immediate reason for doing this was so that the subroot will still be accessible when we're trying to extract column statistics out of an already-planned subquery. But now that I've done it, it seems like a good code-beautification effort in its own right. I also chose to get rid of the transient subrtable and subrowmark fields in SubqueryScan nodes, in favor of having setrefs.c look up the subquery's RelOptInfo. That required changing all the APIs in setrefs.c to pass PlannerInfo not PlannerGlobal, which was a large but quite mechanical transformation. One side-effect not foreseen at the beginning is that this finally broke inheritance_planner's assumption that replanning the same subquery RTE N times would necessarily give interchangeable results each time. That assumption was always pretty risky, but now we really have to make a separate RTE for each instance so that there's a place to carry the separate subroots.
14 years ago
fix_scan_list(root, splan->scan.plan.targetlist, rtoffset);
Implement SQL-standard WITH clauses, including WITH RECURSIVE. There are some unimplemented aspects: recursive queries must use UNION ALL (should allow UNION too), and we don't have SEARCH or CYCLE clauses. These might or might not get done for 8.4, but even without them it's a pretty useful feature. There are also a couple of small loose ends and definitional quibbles, which I'll send a memo about to pgsql-hackers shortly. But let's land the patch now so we can get on with other development. Yoshiyuki Asaba, with lots of help from Tatsuo Ishii and Tom Lane
17 years ago
splan->scan.plan.qual =
Rearrange planner to save the whole PlannerInfo (subroot) for a subquery. Formerly, set_subquery_pathlist and other creators of plans for subqueries saved only the rangetable and rowMarks lists from the lower-level PlannerInfo. But there's no reason not to remember the whole PlannerInfo, and indeed this turns out to simplify matters in a number of places. The immediate reason for doing this was so that the subroot will still be accessible when we're trying to extract column statistics out of an already-planned subquery. But now that I've done it, it seems like a good code-beautification effort in its own right. I also chose to get rid of the transient subrtable and subrowmark fields in SubqueryScan nodes, in favor of having setrefs.c look up the subquery's RelOptInfo. That required changing all the APIs in setrefs.c to pass PlannerInfo not PlannerGlobal, which was a large but quite mechanical transformation. One side-effect not foreseen at the beginning is that this finally broke inheritance_planner's assumption that replanning the same subquery RTE N times would necessarily give interchangeable results each time. That assumption was always pretty risky, but now we really have to make a separate RTE for each instance so that there's a place to carry the separate subroots.
14 years ago
fix_scan_list(root, splan->scan.plan.qual, rtoffset);
Implement SQL-standard WITH clauses, including WITH RECURSIVE. There are some unimplemented aspects: recursive queries must use UNION ALL (should allow UNION too), and we don't have SEARCH or CYCLE clauses. These might or might not get done for 8.4, but even without them it's a pretty useful feature. There are also a couple of small loose ends and definitional quibbles, which I'll send a memo about to pgsql-hackers shortly. But let's land the patch now so we can get on with other development. Yoshiyuki Asaba, with lots of help from Tatsuo Ishii and Tom Lane
17 years ago
}
break;
Implement an API to let foreign-data wrappers actually be functional. This commit provides the core code and documentation needed. A contrib module test case will follow shortly. Shigeru Hanada, Jan Urbanski, Heikki Linnakangas
15 years ago
case T_ForeignScan:
Allow FDWs and custom scan providers to replace joins with scans. Foreign data wrappers can use this capability for so-called "join pushdown"; that is, instead of executing two separate foreign scans and then joining the results locally, they can generate a path which performs the join on the remote server and then is scanned locally. This commit does not extend postgres_fdw to take advantage of this capability; it just provides the infrastructure. Custom scan providers can use this in a similar way. Previously, it was only possible for a custom scan provider to scan a single relation. Now, it can scan an entire join tree, provided of course that it knows how to produce the same results that the join would have produced if executed normally. KaiGai Kohei, reviewed by Shigeru Hanada, Ashutosh Bapat, and me.
11 years ago
set_foreignscan_references(root, (ForeignScan *) plan, rtoffset);
Implement an API to let foreign-data wrappers actually be functional. This commit provides the core code and documentation needed. A contrib module test case will follow shortly. Shigeru Hanada, Jan Urbanski, Heikki Linnakangas
15 years ago
break;
Introduce custom path and scan providers. This allows extension modules to define their own methods for scanning a relation, and get the core code to use them. It's unclear as yet how much use this capability will find, but we won't find out if we never commit it. KaiGai Kohei, reviewed at various times and in various levels of detail by Shigeru Hanada, Tom Lane, Andres Freund, Álvaro Herrera, and myself.
11 years ago
case T_CustomScan:
Allow FDWs and custom scan providers to replace joins with scans. Foreign data wrappers can use this capability for so-called "join pushdown"; that is, instead of executing two separate foreign scans and then joining the results locally, they can generate a path which performs the join on the remote server and then is scanned locally. This commit does not extend postgres_fdw to take advantage of this capability; it just provides the infrastructure. Custom scan providers can use this in a similar way. Previously, it was only possible for a custom scan provider to scan a single relation. Now, it can scan an entire join tree, provided of course that it knows how to produce the same results that the join would have produced if executed normally. KaiGai Kohei, reviewed by Shigeru Hanada, Ashutosh Bapat, and me.
11 years ago
set_customscan_references(root, (CustomScan *) plan, rtoffset);
Introduce custom path and scan providers. This allows extension modules to define their own methods for scanning a relation, and get the core code to use them. It's unclear as yet how much use this capability will find, but we won't find out if we never commit it. KaiGai Kohei, reviewed at various times and in various levels of detail by Shigeru Hanada, Tom Lane, Andres Freund, Álvaro Herrera, and myself.
11 years ago
break;
Further planner/optimizer cleanups. Move all set_tlist_references and fix_opids processing to a single recursive pass over the plan tree executed at the very tail end of planning, rather than haphazardly here and there at different places. Now that tlist Vars do not get modified until the very end, it's possible to get rid of the klugy var_equal and match_varid partial-matching routines, and just use plain equal() throughout the optimizer. This is a step towards allowing merge and hash joins to be done on expressions instead of only Vars ...
26 years ago
case T_NestLoop:
case T_MergeJoin:
case T_HashJoin:
Rearrange planner to save the whole PlannerInfo (subroot) for a subquery. Formerly, set_subquery_pathlist and other creators of plans for subqueries saved only the rangetable and rowMarks lists from the lower-level PlannerInfo. But there's no reason not to remember the whole PlannerInfo, and indeed this turns out to simplify matters in a number of places. The immediate reason for doing this was so that the subroot will still be accessible when we're trying to extract column statistics out of an already-planned subquery. But now that I've done it, it seems like a good code-beautification effort in its own right. I also chose to get rid of the transient subrtable and subrowmark fields in SubqueryScan nodes, in favor of having setrefs.c look up the subquery's RelOptInfo. That required changing all the APIs in setrefs.c to pass PlannerInfo not PlannerGlobal, which was a large but quite mechanical transformation. One side-effect not foreseen at the beginning is that this finally broke inheritance_planner's assumption that replanning the same subquery RTE N times would necessarily give interchangeable results each time. That assumption was always pretty risky, but now we really have to make a separate RTE for each instance so that there's a place to carry the separate subroots.
14 years ago
set_join_references(root, (Join *) plan, rtoffset);
Further planner/optimizer cleanups. Move all set_tlist_references and fix_opids processing to a single recursive pass over the plan tree executed at the very tail end of planning, rather than haphazardly here and there at different places. Now that tlist Vars do not get modified until the very end, it's possible to get rid of the klugy var_equal and match_varid partial-matching routines, and just use plain equal() throughout the optimizer. This is a step towards allowing merge and hash joins to be done on expressions instead of only Vars ...
26 years ago
break;
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
Now that switch_outer processing no longer relies on being run after join_references(), it's practical to consolidate all join_references() processing into the set_plan_references traversal in setrefs.c. This seems considerably cleaner than the old way where we did it for join quals in createplan.c and for targetlists in setrefs.c.
23 years ago
case T_Hash:
Further planner/optimizer cleanups. Move all set_tlist_references and fix_opids processing to a single recursive pass over the plan tree executed at the very tail end of planning, rather than haphazardly here and there at different places. Now that tlist Vars do not get modified until the very end, it's possible to get rid of the klugy var_equal and match_varid partial-matching routines, and just use plain equal() throughout the optimizer. This is a step towards allowing merge and hash joins to be done on expressions instead of only Vars ...
26 years ago
case T_Material:
case T_Sort:
case T_Unique:
Reimplementation of UNION/INTERSECT/EXCEPT. INTERSECT/EXCEPT now meet the SQL92 semantics, including support for ALL option. All three can be used in subqueries and views. DISTINCT and ORDER BY work now in views, too. This rewrite fixes many problems with cross-datatype UNIONs and INSERT/SELECT where the SELECT yields different datatypes than the INSERT needs. I did that by making UNION subqueries and SELECT in INSERT be treated like subselects-in-FROM, thereby allowing an extra level of targetlist where the datatype conversions can be inserted safely. INITDB NEEDED!
25 years ago
case T_SetOp:
Ye-old pgindent run. Same 4-space tabs.
26 years ago
/*
* These plan types don't actually bother to evaluate their
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
* targetlists, because they just return their unmodified input
pgindent run for 9.4 This includes removing tabs after periods in C comments, which was applied to back branches, so this change should not effect backpatching.
12 years ago
* tuples. Even though the targetlist won't be used by the
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
* executor, we fix it up for possible use by EXPLAIN (not to
* mention ease of debugging --- wrong varnos are very confusing).
*/
Now that plans have flat rangetable lists, it's a lot easier to get EXPLAIN to drill down into subplan targetlists to print the referent expression for an OUTER or INNER var in an upper plan node. Hence, make it do that always, and banish the old hack of showing "?columnN?" when things got too complicated. Along the way, fix an EXPLAIN bug I introduced by suppressing subqueries from execution-time range tables: get_name_for_var_field() assumed it could look at rte->subquery to find out the real type of a RECORD var. That doesn't work anymore, but instead we can look at the input plan of the SubqueryScan plan node.
19 years ago
set_dummy_tlist_references(plan, rtoffset);
pgindent run for 8.3.
18 years ago
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
/*
Standard pgindent run for 8.1.
20 years ago
* Since these plan types don't check quals either, we should not
* find any qual expression attached to them.
Further planner/optimizer cleanups. Move all set_tlist_references and fix_opids processing to a single recursive pass over the plan tree executed at the very tail end of planning, rather than haphazardly here and there at different places. Now that tlist Vars do not get modified until the very end, it's possible to get rid of the klugy var_equal and match_varid partial-matching routines, and just use plain equal() throughout the optimizer. This is a step towards allowing merge and hash joins to be done on expressions instead of only Vars ...
26 years ago
*/
Teach the planner to remove SubqueryScan nodes from the plan if they aren't doing anything useful (ie, neither selection nor projection). Also, extend to SubqueryScan the hacks already in place to avoid unnecessary ExecProject calls when the result would just be the same tuple the subquery already delivered. This saves some overhead in UNION and other set operations, as well as avoiding overhead for unflatten-able subqueries. Per example from Sokolov Yura.
21 years ago
Assert(plan->qual == NIL);
Further planner/optimizer cleanups. Move all set_tlist_references and fix_opids processing to a single recursive pass over the plan tree executed at the very tail end of planning, rather than haphazardly here and there at different places. Now that tlist Vars do not get modified until the very end, it's possible to get rid of the klugy var_equal and match_varid partial-matching routines, and just use plain equal() throughout the optimizer. This is a step towards allowing merge and hash joins to be done on expressions instead of only Vars ...
26 years ago
break;
Move the handling of SELECT FOR UPDATE locking and rechecking out of execMain.c and into a new plan node type LockRows. Like the recent change to put table updating into a ModifyTable plan node, this increases planning flexibility by allowing the operations to occur below the top level of the plan tree. It's necessary in any case to restore the previous behavior of having FOR UPDATE locking occur before ModifyTable does. This partially refactors EvalPlanQual to allow multiple rows-under-test to be inserted into the EPQ machinery before starting an EPQ test query. That isn't sufficient to fix EPQ's general bogosity in the face of plans that return multiple rows per test row, though. Since this patch is mostly about getting some plan node infrastructure in place and not about fixing ten-year-old bugs, I will leave EPQ improvements for another day. Another behavioral change that we could now think about is doing FOR UPDATE before LIMIT, but that too seems like it should be treated as a followon patch.
16 years ago
case T_LockRows:
{
LockRows *splan = (LockRows *) plan;
/*
* Like the plan types above, LockRows doesn't evaluate its
pgindent run for 9.4 This includes removing tabs after periods in C comments, which was applied to back branches, so this change should not effect backpatching.
12 years ago
* tlist or quals. But we have to fix up the RT indexes in
pgindent run for 9.0
16 years ago
* its rowmarks.
Move the handling of SELECT FOR UPDATE locking and rechecking out of execMain.c and into a new plan node type LockRows. Like the recent change to put table updating into a ModifyTable plan node, this increases planning flexibility by allowing the operations to occur below the top level of the plan tree. It's necessary in any case to restore the previous behavior of having FOR UPDATE locking occur before ModifyTable does. This partially refactors EvalPlanQual to allow multiple rows-under-test to be inserted into the EPQ machinery before starting an EPQ test query. That isn't sufficient to fix EPQ's general bogosity in the face of plans that return multiple rows per test row, though. Since this patch is mostly about getting some plan node infrastructure in place and not about fixing ten-year-old bugs, I will leave EPQ improvements for another day. Another behavioral change that we could now think about is doing FOR UPDATE before LIMIT, but that too seems like it should be treated as a followon patch.
16 years ago
*/
set_dummy_tlist_references(plan, rtoffset);
Assert(splan->plan.qual == NIL);
foreach(l, splan->rowMarks)
{
Re-implement EvalPlanQual processing to improve its performance and eliminate a lot of strange behaviors that occurred in join cases. We now identify the "current" row for every joined relation in UPDATE, DELETE, and SELECT FOR UPDATE/SHARE queries. If an EvalPlanQual recheck is necessary, we jam the appropriate row into each scan node in the rechecking plan, forcing it to emit only that one row. The former behavior could rescan the whole of each joined relation for each recheck, which was terrible for performance, and what's much worse could result in duplicated output tuples. Also, the original implementation of EvalPlanQual could not re-use the recheck execution tree --- it had to go through a full executor init and shutdown for every row to be tested. To avoid this overhead, I've associated a special runtime Param with each LockRows or ModifyTable plan node, and arranged to make every scan node below such a node depend on that Param. Thus, by signaling a change in that Param, the EPQ machinery can just rescan the already-built test plan. This patch also adds a prohibition on set-returning functions in the targetlist of SELECT FOR UPDATE/SHARE. This is needed to avoid the duplicate-output-tuple problem. It seems fairly reasonable since the other restrictions on SELECT FOR UPDATE are meant to ensure that there is a unique correspondence between source tuples and result tuples, which an output SRF destroys as much as anything else does.
16 years ago
PlanRowMark *rc = (PlanRowMark *) lfirst(l);
Move the handling of SELECT FOR UPDATE locking and rechecking out of execMain.c and into a new plan node type LockRows. Like the recent change to put table updating into a ModifyTable plan node, this increases planning flexibility by allowing the operations to occur below the top level of the plan tree. It's necessary in any case to restore the previous behavior of having FOR UPDATE locking occur before ModifyTable does. This partially refactors EvalPlanQual to allow multiple rows-under-test to be inserted into the EPQ machinery before starting an EPQ test query. That isn't sufficient to fix EPQ's general bogosity in the face of plans that return multiple rows per test row, though. Since this patch is mostly about getting some plan node infrastructure in place and not about fixing ten-year-old bugs, I will leave EPQ improvements for another day. Another behavioral change that we could now think about is doing FOR UPDATE before LIMIT, but that too seems like it should be treated as a followon patch.
16 years ago
rc->rti += rtoffset;
rc->prti += rtoffset;
}
}
break;
Fix oversights in processing of LIMIT expressions during planning.
22 years ago
case T_Limit:
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
{
pgindent run for 8.3.
18 years ago
Limit *splan = (Limit *) plan;
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
/*
* Like the plan types above, Limit doesn't evaluate its tlist
* or quals. It does have live expressions for limit/offset,
pgindent run for 8.3.
18 years ago
* however; and those cannot contain subplan variable refs, so
* fix_scan_expr works for them.
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
*/
Now that plans have flat rangetable lists, it's a lot easier to get EXPLAIN to drill down into subplan targetlists to print the referent expression for an OUTER or INNER var in an upper plan node. Hence, make it do that always, and banish the old hack of showing "?columnN?" when things got too complicated. Along the way, fix an EXPLAIN bug I introduced by suppressing subqueries from execution-time range tables: get_name_for_var_field() assumed it could look at rte->subquery to find out the real type of a RECORD var. That doesn't work anymore, but instead we can look at the input plan of the SubqueryScan plan node.
19 years ago
set_dummy_tlist_references(plan, rtoffset);
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
Assert(splan->plan.qual == NIL);
Now that plans have flat rangetable lists, it's a lot easier to get EXPLAIN to drill down into subplan targetlists to print the referent expression for an OUTER or INNER var in an upper plan node. Hence, make it do that always, and banish the old hack of showing "?columnN?" when things got too complicated. Along the way, fix an EXPLAIN bug I introduced by suppressing subqueries from execution-time range tables: get_name_for_var_field() assumed it could look at rte->subquery to find out the real type of a RECORD var. That doesn't work anymore, but instead we can look at the input plan of the SubqueryScan plan node.
19 years ago
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
splan->limitOffset =
Rearrange planner to save the whole PlannerInfo (subroot) for a subquery. Formerly, set_subquery_pathlist and other creators of plans for subqueries saved only the rangetable and rowMarks lists from the lower-level PlannerInfo. But there's no reason not to remember the whole PlannerInfo, and indeed this turns out to simplify matters in a number of places. The immediate reason for doing this was so that the subroot will still be accessible when we're trying to extract column statistics out of an already-planned subquery. But now that I've done it, it seems like a good code-beautification effort in its own right. I also chose to get rid of the transient subrtable and subrowmark fields in SubqueryScan nodes, in favor of having setrefs.c look up the subquery's RelOptInfo. That required changing all the APIs in setrefs.c to pass PlannerInfo not PlannerGlobal, which was a large but quite mechanical transformation. One side-effect not foreseen at the beginning is that this finally broke inheritance_planner's assumption that replanning the same subquery RTE N times would necessarily give interchangeable results each time. That assumption was always pretty risky, but now we really have to make a separate RTE for each instance so that there's a place to carry the separate subroots.
14 years ago
fix_scan_expr(root, splan->limitOffset, rtoffset);
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
splan->limitCount =
Rearrange planner to save the whole PlannerInfo (subroot) for a subquery. Formerly, set_subquery_pathlist and other creators of plans for subqueries saved only the rangetable and rowMarks lists from the lower-level PlannerInfo. But there's no reason not to remember the whole PlannerInfo, and indeed this turns out to simplify matters in a number of places. The immediate reason for doing this was so that the subroot will still be accessible when we're trying to extract column statistics out of an already-planned subquery. But now that I've done it, it seems like a good code-beautification effort in its own right. I also chose to get rid of the transient subrtable and subrowmark fields in SubqueryScan nodes, in favor of having setrefs.c look up the subquery's RelOptInfo. That required changing all the APIs in setrefs.c to pass PlannerInfo not PlannerGlobal, which was a large but quite mechanical transformation. One side-effect not foreseen at the beginning is that this finally broke inheritance_planner's assumption that replanning the same subquery RTE N times would necessarily give interchangeable results each time. That assumption was always pretty risky, but now we really have to make a separate RTE for each instance so that there's a place to carry the separate subroots.
14 years ago
fix_scan_expr(root, splan->limitCount, rtoffset);
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
}
Fix oversights in processing of LIMIT expressions during planning.
22 years ago
break;
Further planner/optimizer cleanups. Move all set_tlist_references and fix_opids processing to a single recursive pass over the plan tree executed at the very tail end of planning, rather than haphazardly here and there at different places. Now that tlist Vars do not get modified until the very end, it's possible to get rid of the klugy var_equal and match_varid partial-matching routines, and just use plain equal() throughout the optimizer. This is a step towards allowing merge and hash joins to be done on expressions instead of only Vars ...
26 years ago
case T_Agg:
Support GROUPING SETS, CUBE and ROLLUP. This SQL standard functionality allows to aggregate data by different GROUP BY clauses at once. Each grouping set returns rows with columns grouped by in other sets set to NULL. This could previously be achieved by doing each grouping as a separate query, conjoined by UNION ALLs. Besides being considerably more concise, grouping sets will in many cases be faster, requiring only one scan over the underlying data. The current implementation of grouping sets only supports using sorting for input. Individual sets that share a sort order are computed in one pass. If there are sets that don't share a sort order, additional sort & aggregation steps are performed. These additional passes are sourced by the previous sort step; thus avoiding repeated scans of the source data. The code is structured in a way that adding support for purely using hash aggregation or a mix of hashing and sorting is possible. Sorting was chosen to be supported first, as it is the most generic method of implementation. Instead of, as in an earlier versions of the patch, representing the chain of sort and aggregation steps as full blown planner and executor nodes, all but the first sort are performed inside the aggregation node itself. This avoids the need to do some unusual gymnastics to handle having to return aggregated and non-aggregated tuples from underlying nodes, as well as having to shut down underlying nodes early to limit memory usage. The optimizer still builds Sort/Agg node to describe each phase, but they're not part of the plan tree, but instead additional data for the aggregation node. They're a convenient and preexisting way to describe aggregation and sorting. The first (and possibly only) sort step is still performed as a separate execution step. That retains similarity with existing group by plans, makes rescans fairly simple, avoids very deep plans (leading to slow explains) and easily allows to avoid the sorting step if the underlying data is sorted by other means. A somewhat ugly side of this patch is having to deal with a grammar ambiguity between the new CUBE keyword and the cube extension/functions named cube (and rollup). To avoid breaking existing deployments of the cube extension it has not been renamed, neither has cube been made a reserved keyword. Instead precedence hacking is used to make GROUP BY cube(..) refer to the CUBE grouping sets feature, and not the function cube(). To actually group by a function cube(), unlikely as that might be, the function name has to be quoted. Needs a catversion bump because stored rules may change. Author: Andrew Gierth and Atri Sharma, with contributions from Andres Freund Reviewed-By: Andres Freund, Noah Misch, Tom Lane, Svenne Krap, Tomas Vondra, Erik Rijkers, Marti Raudsepp, Pavel Stehule Discussion: CAOeZVidmVRe2jU6aMk_5qkxnB7dfmPROzM7Ur8JPW5j8Y5X-Lw@mail.gmail.com
10 years ago
set_upper_references(root, plan, rtoffset);
break;
Further planner/optimizer cleanups. Move all set_tlist_references and fix_opids processing to a single recursive pass over the plan tree executed at the very tail end of planning, rather than haphazardly here and there at different places. Now that tlist Vars do not get modified until the very end, it's possible to get rid of the klugy var_equal and match_varid partial-matching routines, and just use plain equal() throughout the optimizer. This is a step towards allowing merge and hash joins to be done on expressions instead of only Vars ...
26 years ago
case T_Group:
Rearrange planner to save the whole PlannerInfo (subroot) for a subquery. Formerly, set_subquery_pathlist and other creators of plans for subqueries saved only the rangetable and rowMarks lists from the lower-level PlannerInfo. But there's no reason not to remember the whole PlannerInfo, and indeed this turns out to simplify matters in a number of places. The immediate reason for doing this was so that the subroot will still be accessible when we're trying to extract column statistics out of an already-planned subquery. But now that I've done it, it seems like a good code-beautification effort in its own right. I also chose to get rid of the transient subrtable and subrowmark fields in SubqueryScan nodes, in favor of having setrefs.c look up the subquery's RelOptInfo. That required changing all the APIs in setrefs.c to pass PlannerInfo not PlannerGlobal, which was a large but quite mechanical transformation. One side-effect not foreseen at the beginning is that this finally broke inheritance_planner's assumption that replanning the same subquery RTE N times would necessarily give interchangeable results each time. That assumption was always pretty risky, but now we really have to make a separate RTE for each instance so that there's a place to carry the separate subroots.
14 years ago
set_upper_references(root, plan, rtoffset);
Further planner/optimizer cleanups. Move all set_tlist_references and fix_opids processing to a single recursive pass over the plan tree executed at the very tail end of planning, rather than haphazardly here and there at different places. Now that tlist Vars do not get modified until the very end, it's possible to get rid of the klugy var_equal and match_varid partial-matching routines, and just use plain equal() throughout the optimizer. This is a step towards allowing merge and hash joins to be done on expressions instead of only Vars ...
26 years ago
break;
Extend the set of frame options supported for window functions. This patch allows the frame to start from CURRENT ROW (in either RANGE or ROWS mode), and it also adds support for ROWS n PRECEDING and ROWS n FOLLOWING start and end points. (RANGE value PRECEDING/FOLLOWING isn't there yet --- the grammar works, but that's all.) Hitoshi Harada, reviewed by Pavel Stehule
16 years ago
case T_WindowAgg:
{
pgindent run for 9.0
16 years ago
WindowAgg *wplan = (WindowAgg *) plan;
Extend the set of frame options supported for window functions. This patch allows the frame to start from CURRENT ROW (in either RANGE or ROWS mode), and it also adds support for ROWS n PRECEDING and ROWS n FOLLOWING start and end points. (RANGE value PRECEDING/FOLLOWING isn't there yet --- the grammar works, but that's all.) Hitoshi Harada, reviewed by Pavel Stehule
16 years ago
Rearrange planner to save the whole PlannerInfo (subroot) for a subquery. Formerly, set_subquery_pathlist and other creators of plans for subqueries saved only the rangetable and rowMarks lists from the lower-level PlannerInfo. But there's no reason not to remember the whole PlannerInfo, and indeed this turns out to simplify matters in a number of places. The immediate reason for doing this was so that the subroot will still be accessible when we're trying to extract column statistics out of an already-planned subquery. But now that I've done it, it seems like a good code-beautification effort in its own right. I also chose to get rid of the transient subrtable and subrowmark fields in SubqueryScan nodes, in favor of having setrefs.c look up the subquery's RelOptInfo. That required changing all the APIs in setrefs.c to pass PlannerInfo not PlannerGlobal, which was a large but quite mechanical transformation. One side-effect not foreseen at the beginning is that this finally broke inheritance_planner's assumption that replanning the same subquery RTE N times would necessarily give interchangeable results each time. That assumption was always pretty risky, but now we really have to make a separate RTE for each instance so that there's a place to carry the separate subroots.
14 years ago
set_upper_references(root, plan, rtoffset);
Extend the set of frame options supported for window functions. This patch allows the frame to start from CURRENT ROW (in either RANGE or ROWS mode), and it also adds support for ROWS n PRECEDING and ROWS n FOLLOWING start and end points. (RANGE value PRECEDING/FOLLOWING isn't there yet --- the grammar works, but that's all.) Hitoshi Harada, reviewed by Pavel Stehule
16 years ago
/*
* Like Limit node limit/offset expressions, WindowAgg has
* frame offset expressions, which cannot contain subplan
* variable refs, so fix_scan_expr works for them.
*/
wplan->startOffset =
Rearrange planner to save the whole PlannerInfo (subroot) for a subquery. Formerly, set_subquery_pathlist and other creators of plans for subqueries saved only the rangetable and rowMarks lists from the lower-level PlannerInfo. But there's no reason not to remember the whole PlannerInfo, and indeed this turns out to simplify matters in a number of places. The immediate reason for doing this was so that the subroot will still be accessible when we're trying to extract column statistics out of an already-planned subquery. But now that I've done it, it seems like a good code-beautification effort in its own right. I also chose to get rid of the transient subrtable and subrowmark fields in SubqueryScan nodes, in favor of having setrefs.c look up the subquery's RelOptInfo. That required changing all the APIs in setrefs.c to pass PlannerInfo not PlannerGlobal, which was a large but quite mechanical transformation. One side-effect not foreseen at the beginning is that this finally broke inheritance_planner's assumption that replanning the same subquery RTE N times would necessarily give interchangeable results each time. That assumption was always pretty risky, but now we really have to make a separate RTE for each instance so that there's a place to carry the separate subroots.
14 years ago
fix_scan_expr(root, wplan->startOffset, rtoffset);
Extend the set of frame options supported for window functions. This patch allows the frame to start from CURRENT ROW (in either RANGE or ROWS mode), and it also adds support for ROWS n PRECEDING and ROWS n FOLLOWING start and end points. (RANGE value PRECEDING/FOLLOWING isn't there yet --- the grammar works, but that's all.) Hitoshi Harada, reviewed by Pavel Stehule
16 years ago
wplan->endOffset =
Rearrange planner to save the whole PlannerInfo (subroot) for a subquery. Formerly, set_subquery_pathlist and other creators of plans for subqueries saved only the rangetable and rowMarks lists from the lower-level PlannerInfo. But there's no reason not to remember the whole PlannerInfo, and indeed this turns out to simplify matters in a number of places. The immediate reason for doing this was so that the subroot will still be accessible when we're trying to extract column statistics out of an already-planned subquery. But now that I've done it, it seems like a good code-beautification effort in its own right. I also chose to get rid of the transient subrtable and subrowmark fields in SubqueryScan nodes, in favor of having setrefs.c look up the subquery's RelOptInfo. That required changing all the APIs in setrefs.c to pass PlannerInfo not PlannerGlobal, which was a large but quite mechanical transformation. One side-effect not foreseen at the beginning is that this finally broke inheritance_planner's assumption that replanning the same subquery RTE N times would necessarily give interchangeable results each time. That assumption was always pretty risky, but now we really have to make a separate RTE for each instance so that there's a place to carry the separate subroots.
14 years ago
fix_scan_expr(root, wplan->endOffset, rtoffset);
Extend the set of frame options supported for window functions. This patch allows the frame to start from CURRENT ROW (in either RANGE or ROWS mode), and it also adds support for ROWS n PRECEDING and ROWS n FOLLOWING start and end points. (RANGE value PRECEDING/FOLLOWING isn't there yet --- the grammar works, but that's all.) Hitoshi Harada, reviewed by Pavel Stehule
16 years ago
}
break;
Further planner/optimizer cleanups. Move all set_tlist_references and fix_opids processing to a single recursive pass over the plan tree executed at the very tail end of planning, rather than haphazardly here and there at different places. Now that tlist Vars do not get modified until the very end, it's possible to get rid of the klugy var_equal and match_varid partial-matching routines, and just use plain equal() throughout the optimizer. This is a step towards allowing merge and hash joins to be done on expressions instead of only Vars ...
26 years ago
case T_Result:
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
{
pgindent run for 8.3.
18 years ago
Result *splan = (Result *) plan;
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
/*
* Result may or may not have a subplan; if not, it's more
* like a scan node than an upper node.
*/
if (splan->plan.lefttree != NULL)
Rearrange planner to save the whole PlannerInfo (subroot) for a subquery. Formerly, set_subquery_pathlist and other creators of plans for subqueries saved only the rangetable and rowMarks lists from the lower-level PlannerInfo. But there's no reason not to remember the whole PlannerInfo, and indeed this turns out to simplify matters in a number of places. The immediate reason for doing this was so that the subroot will still be accessible when we're trying to extract column statistics out of an already-planned subquery. But now that I've done it, it seems like a good code-beautification effort in its own right. I also chose to get rid of the transient subrtable and subrowmark fields in SubqueryScan nodes, in favor of having setrefs.c look up the subquery's RelOptInfo. That required changing all the APIs in setrefs.c to pass PlannerInfo not PlannerGlobal, which was a large but quite mechanical transformation. One side-effect not foreseen at the beginning is that this finally broke inheritance_planner's assumption that replanning the same subquery RTE N times would necessarily give interchangeable results each time. That assumption was always pretty risky, but now we really have to make a separate RTE for each instance so that there's a place to carry the separate subroots.
14 years ago
set_upper_references(root, plan, rtoffset);
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
else
{
splan->plan.targetlist =
Rearrange planner to save the whole PlannerInfo (subroot) for a subquery. Formerly, set_subquery_pathlist and other creators of plans for subqueries saved only the rangetable and rowMarks lists from the lower-level PlannerInfo. But there's no reason not to remember the whole PlannerInfo, and indeed this turns out to simplify matters in a number of places. The immediate reason for doing this was so that the subroot will still be accessible when we're trying to extract column statistics out of an already-planned subquery. But now that I've done it, it seems like a good code-beautification effort in its own right. I also chose to get rid of the transient subrtable and subrowmark fields in SubqueryScan nodes, in favor of having setrefs.c look up the subquery's RelOptInfo. That required changing all the APIs in setrefs.c to pass PlannerInfo not PlannerGlobal, which was a large but quite mechanical transformation. One side-effect not foreseen at the beginning is that this finally broke inheritance_planner's assumption that replanning the same subquery RTE N times would necessarily give interchangeable results each time. That assumption was always pretty risky, but now we really have to make a separate RTE for each instance so that there's a place to carry the separate subroots.
14 years ago
fix_scan_list(root, splan->plan.targetlist, rtoffset);
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
splan->plan.qual =
Rearrange planner to save the whole PlannerInfo (subroot) for a subquery. Formerly, set_subquery_pathlist and other creators of plans for subqueries saved only the rangetable and rowMarks lists from the lower-level PlannerInfo. But there's no reason not to remember the whole PlannerInfo, and indeed this turns out to simplify matters in a number of places. The immediate reason for doing this was so that the subroot will still be accessible when we're trying to extract column statistics out of an already-planned subquery. But now that I've done it, it seems like a good code-beautification effort in its own right. I also chose to get rid of the transient subrtable and subrowmark fields in SubqueryScan nodes, in favor of having setrefs.c look up the subquery's RelOptInfo. That required changing all the APIs in setrefs.c to pass PlannerInfo not PlannerGlobal, which was a large but quite mechanical transformation. One side-effect not foreseen at the beginning is that this finally broke inheritance_planner's assumption that replanning the same subquery RTE N times would necessarily give interchangeable results each time. That assumption was always pretty risky, but now we really have to make a separate RTE for each instance so that there's a place to carry the separate subroots.
14 years ago
fix_scan_list(root, splan->plan.qual, rtoffset);
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
}
/* resconstantqual can't contain any subplan variable refs */
splan->resconstantqual =
Rearrange planner to save the whole PlannerInfo (subroot) for a subquery. Formerly, set_subquery_pathlist and other creators of plans for subqueries saved only the rangetable and rowMarks lists from the lower-level PlannerInfo. But there's no reason not to remember the whole PlannerInfo, and indeed this turns out to simplify matters in a number of places. The immediate reason for doing this was so that the subroot will still be accessible when we're trying to extract column statistics out of an already-planned subquery. But now that I've done it, it seems like a good code-beautification effort in its own right. I also chose to get rid of the transient subrtable and subrowmark fields in SubqueryScan nodes, in favor of having setrefs.c look up the subquery's RelOptInfo. That required changing all the APIs in setrefs.c to pass PlannerInfo not PlannerGlobal, which was a large but quite mechanical transformation. One side-effect not foreseen at the beginning is that this finally broke inheritance_planner's assumption that replanning the same subquery RTE N times would necessarily give interchangeable results each time. That assumption was always pretty risky, but now we really have to make a separate RTE for each instance so that there's a place to carry the separate subroots.
14 years ago
fix_scan_expr(root, splan->resconstantqual, rtoffset);
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
}
Further planner/optimizer cleanups. Move all set_tlist_references and fix_opids processing to a single recursive pass over the plan tree executed at the very tail end of planning, rather than haphazardly here and there at different places. Now that tlist Vars do not get modified until the very end, it's possible to get rid of the klugy var_equal and match_varid partial-matching routines, and just use plain equal() throughout the optimizer. This is a step towards allowing merge and hash joins to be done on expressions instead of only Vars ...
26 years ago
break;
Split the processing of INSERT/UPDATE/DELETE operations out of execMain.c. They are now handled by a new plan node type called ModifyTable, which is placed at the top of the plan tree. In itself this change doesn't do much, except perhaps make the handling of RETURNING lists and inherited UPDATEs a tad less klugy. But it is necessary preparation for the intended extension of allowing RETURNING queries inside WITH. Marko Tiikkaja
16 years ago
case T_ModifyTable:
{
ModifyTable *splan = (ModifyTable *) plan;
Fix planner's handling of RETURNING lists in writable CTEs. setrefs.c failed to do "rtoffset" adjustment of Vars in RETURNING lists, which meant they were left with the wrong varnos when the RETURNING list was in a subquery. That was never possible before writable CTEs, of course, but now it's broken. The executor fails to notice any problem because ExecEvalVar just references the ecxt_scantuple for any normal varno; but EXPLAIN breaks when the varno is wrong, as illustrated in a recent complaint from Bartosz Dmytrak. Since the eventual rtoffset of the subquery is not known at the time we are preparing its plan node, the previous scheme of executing set_returning_clause_references() at that time cannot handle this adjustment. Fortunately, it turns out that we don't really need to do it that way, because all the needed information is available during normal setrefs.c execution; we just have to dig it out of the ModifyTable node. So, do that, and get rid of the kluge of early setrefs processing of RETURNING lists. (This is a little bit of a cheat in the case of inherited UPDATE/DELETE, because we are not passing a "root" struct that corresponds exactly to what the subplan was built with. But that doesn't matter, and anyway this is less ugly than early setrefs processing was.) Back-patch to 9.1, where the problem became possible to hit.
14 years ago
Assert(splan->plan.targetlist == NIL);
Split the processing of INSERT/UPDATE/DELETE operations out of execMain.c. They are now handled by a new plan node type called ModifyTable, which is placed at the top of the plan tree. In itself this change doesn't do much, except perhaps make the handling of RETURNING lists and inherited UPDATEs a tad less klugy. But it is necessary preparation for the intended extension of allowing RETURNING queries inside WITH. Marko Tiikkaja
16 years ago
Assert(splan->plan.qual == NIL);
Process withCheckOption exprs in setrefs.c While withCheckOption exprs had been handled in many cases by happenstance, they need to be handled during set_plan_references and more specifically down in set_plan_refs for ModifyTable plan nodes. This is to ensure that the opfuncid's are set for operators referenced in the withCheckOption exprs. Identified as an issue by Thom Brown Patch by Dean Rasheed Back-patch to 9.4, where withCheckOption was introduced.
11 years ago
splan->withCheckOptionLists =
fix_scan_list(root, splan->withCheckOptionLists, rtoffset);
Fix planner's handling of RETURNING lists in writable CTEs. setrefs.c failed to do "rtoffset" adjustment of Vars in RETURNING lists, which meant they were left with the wrong varnos when the RETURNING list was in a subquery. That was never possible before writable CTEs, of course, but now it's broken. The executor fails to notice any problem because ExecEvalVar just references the ecxt_scantuple for any normal varno; but EXPLAIN breaks when the varno is wrong, as illustrated in a recent complaint from Bartosz Dmytrak. Since the eventual rtoffset of the subquery is not known at the time we are preparing its plan node, the previous scheme of executing set_returning_clause_references() at that time cannot handle this adjustment. Fortunately, it turns out that we don't really need to do it that way, because all the needed information is available during normal setrefs.c execution; we just have to dig it out of the ModifyTable node. So, do that, and get rid of the kluge of early setrefs processing of RETURNING lists. (This is a little bit of a cheat in the case of inherited UPDATE/DELETE, because we are not passing a "root" struct that corresponds exactly to what the subplan was built with. But that doesn't matter, and anyway this is less ugly than early setrefs processing was.) Back-patch to 9.1, where the problem became possible to hit.
14 years ago
if (splan->returningLists)
{
List *newRL = NIL;
ListCell *lcrl,
*lcrr,
*lcp;
/*
* Pass each per-subplan returningList through
* set_returning_clause_references().
*/
Assert(list_length(splan->returningLists) == list_length(splan->resultRelations));
Assert(list_length(splan->returningLists) == list_length(splan->plans));
forthree(lcrl, splan->returningLists,
lcrr, splan->resultRelations,
lcp, splan->plans)
{
Run pgindent on 9.2 source tree in preparation for first 9.3 commit-fest.
13 years ago
List *rlist = (List *) lfirst(lcrl);
Index resultrel = lfirst_int(lcrr);
Plan *subplan = (Plan *) lfirst(lcp);
Fix planner's handling of RETURNING lists in writable CTEs. setrefs.c failed to do "rtoffset" adjustment of Vars in RETURNING lists, which meant they were left with the wrong varnos when the RETURNING list was in a subquery. That was never possible before writable CTEs, of course, but now it's broken. The executor fails to notice any problem because ExecEvalVar just references the ecxt_scantuple for any normal varno; but EXPLAIN breaks when the varno is wrong, as illustrated in a recent complaint from Bartosz Dmytrak. Since the eventual rtoffset of the subquery is not known at the time we are preparing its plan node, the previous scheme of executing set_returning_clause_references() at that time cannot handle this adjustment. Fortunately, it turns out that we don't really need to do it that way, because all the needed information is available during normal setrefs.c execution; we just have to dig it out of the ModifyTable node. So, do that, and get rid of the kluge of early setrefs processing of RETURNING lists. (This is a little bit of a cheat in the case of inherited UPDATE/DELETE, because we are not passing a "root" struct that corresponds exactly to what the subplan was built with. But that doesn't matter, and anyway this is less ugly than early setrefs processing was.) Back-patch to 9.1, where the problem became possible to hit.
14 years ago
rlist = set_returning_clause_references(root,
rlist,
subplan,
resultrel,
rtoffset);
newRL = lappend(newRL, rlist);
}
splan->returningLists = newRL;
/*
* Set up the visible plan targetlist as being the same as
* the first RETURNING list. This is for the use of
* EXPLAIN; the executor won't pay any attention to the
pgindent run for 9.4 This includes removing tabs after periods in C comments, which was applied to back branches, so this change should not effect backpatching.
12 years ago
* targetlist. We postpone this step until here so that
Fix planner's handling of RETURNING lists in writable CTEs. setrefs.c failed to do "rtoffset" adjustment of Vars in RETURNING lists, which meant they were left with the wrong varnos when the RETURNING list was in a subquery. That was never possible before writable CTEs, of course, but now it's broken. The executor fails to notice any problem because ExecEvalVar just references the ecxt_scantuple for any normal varno; but EXPLAIN breaks when the varno is wrong, as illustrated in a recent complaint from Bartosz Dmytrak. Since the eventual rtoffset of the subquery is not known at the time we are preparing its plan node, the previous scheme of executing set_returning_clause_references() at that time cannot handle this adjustment. Fortunately, it turns out that we don't really need to do it that way, because all the needed information is available during normal setrefs.c execution; we just have to dig it out of the ModifyTable node. So, do that, and get rid of the kluge of early setrefs processing of RETURNING lists. (This is a little bit of a cheat in the case of inherited UPDATE/DELETE, because we are not passing a "root" struct that corresponds exactly to what the subplan was built with. But that doesn't matter, and anyway this is less ugly than early setrefs processing was.) Back-patch to 9.1, where the problem became possible to hit.
14 years ago
* we don't have to do set_returning_clause_references()
* twice on identical targetlists.
*/
splan->plan.targetlist = copyObject(linitial(newRL));
}
Add support for INSERT ... ON CONFLICT DO NOTHING/UPDATE. The newly added ON CONFLICT clause allows to specify an alternative to raising a unique or exclusion constraint violation error when inserting. ON CONFLICT refers to constraints that can either be specified using a inference clause (by specifying the columns of a unique constraint) or by naming a unique or exclusion constraint. DO NOTHING avoids the constraint violation, without touching the pre-existing row. DO UPDATE SET ... [WHERE ...] updates the pre-existing tuple, and has access to both the tuple proposed for insertion and the existing tuple; the optional WHERE clause can be used to prevent an update from being executed. The UPDATE SET and WHERE clauses have access to the tuple proposed for insertion using the "magic" EXCLUDED alias, and to the pre-existing tuple using the table name or its alias. This feature is often referred to as upsert. This is implemented using a new infrastructure called "speculative insertion". It is an optimistic variant of regular insertion that first does a pre-check for existing tuples and then attempts an insert. If a violating tuple was inserted concurrently, the speculatively inserted tuple is deleted and a new attempt is made. If the pre-check finds a matching tuple the alternative DO NOTHING or DO UPDATE action is taken. If the insertion succeeds without detecting a conflict, the tuple is deemed inserted. To handle the possible ambiguity between the excluded alias and a table named excluded, and for convenience with long relation names, INSERT INTO now can alias its target table. Bumps catversion as stored rules change. Author: Peter Geoghegan, with significant contributions from Heikki Linnakangas and Andres Freund. Testing infrastructure by Jeff Janes. Reviewed-By: Heikki Linnakangas, Andres Freund, Robert Haas, Simon Riggs, Dean Rasheed, Stephen Frost and many others.
11 years ago
/*
* We treat ModifyTable with ON CONFLICT as a form of 'pseudo
Fix ON CONFLICT bugs that manifest when used in rules. Specifically the tlist and rti of the pseudo "excluded" relation weren't properly treated by expression_tree_walker, which lead to errors when excluded was referenced inside a rule because the varnos where not properly adjusted. Similar omissions in OffsetVarNodes and expression_tree_mutator had less impact, but should obviously be fixed nonetheless. A couple tests of for ON CONFLICT UPDATE into INSERT rule bearing relations have been added. In passing I updated a couple comments.
10 years ago
* join', where the inner side is the EXCLUDED tuple.
* Therefore use fix_join_expr to setup the relevant variables
* to INNER_VAR. We explicitly don't create any OUTER_VARs as
Add support for INSERT ... ON CONFLICT DO NOTHING/UPDATE. The newly added ON CONFLICT clause allows to specify an alternative to raising a unique or exclusion constraint violation error when inserting. ON CONFLICT refers to constraints that can either be specified using a inference clause (by specifying the columns of a unique constraint) or by naming a unique or exclusion constraint. DO NOTHING avoids the constraint violation, without touching the pre-existing row. DO UPDATE SET ... [WHERE ...] updates the pre-existing tuple, and has access to both the tuple proposed for insertion and the existing tuple; the optional WHERE clause can be used to prevent an update from being executed. The UPDATE SET and WHERE clauses have access to the tuple proposed for insertion using the "magic" EXCLUDED alias, and to the pre-existing tuple using the table name or its alias. This feature is often referred to as upsert. This is implemented using a new infrastructure called "speculative insertion". It is an optimistic variant of regular insertion that first does a pre-check for existing tuples and then attempts an insert. If a violating tuple was inserted concurrently, the speculatively inserted tuple is deleted and a new attempt is made. If the pre-check finds a matching tuple the alternative DO NOTHING or DO UPDATE action is taken. If the insertion succeeds without detecting a conflict, the tuple is deemed inserted. To handle the possible ambiguity between the excluded alias and a table named excluded, and for convenience with long relation names, INSERT INTO now can alias its target table. Bumps catversion as stored rules change. Author: Peter Geoghegan, with significant contributions from Heikki Linnakangas and Andres Freund. Testing infrastructure by Jeff Janes. Reviewed-By: Heikki Linnakangas, Andres Freund, Robert Haas, Simon Riggs, Dean Rasheed, Stephen Frost and many others.
11 years ago
* those are already used by RETURNING and it seems better to
* be non-conflicting.
*/
if (splan->onConflictSet)
{
indexed_tlist *itlist;
itlist = build_tlist_index(splan->exclRelTlist);
splan->onConflictSet =
fix_join_expr(root, splan->onConflictSet,
NULL, itlist,
linitial_int(splan->resultRelations),
rtoffset);
splan->onConflictWhere = (Node *)
fix_join_expr(root, (List *) splan->onConflictWhere,
NULL, itlist,
linitial_int(splan->resultRelations),
rtoffset);
Fix ON CONFLICT bugs that manifest when used in rules. Specifically the tlist and rti of the pseudo "excluded" relation weren't properly treated by expression_tree_walker, which lead to errors when excluded was referenced inside a rule because the varnos where not properly adjusted. Similar omissions in OffsetVarNodes and expression_tree_mutator had less impact, but should obviously be fixed nonetheless. A couple tests of for ON CONFLICT UPDATE into INSERT rule bearing relations have been added. In passing I updated a couple comments.
10 years ago
splan->exclRelTlist =
fix_scan_list(root, splan->exclRelTlist, rtoffset);
Add support for INSERT ... ON CONFLICT DO NOTHING/UPDATE. The newly added ON CONFLICT clause allows to specify an alternative to raising a unique or exclusion constraint violation error when inserting. ON CONFLICT refers to constraints that can either be specified using a inference clause (by specifying the columns of a unique constraint) or by naming a unique or exclusion constraint. DO NOTHING avoids the constraint violation, without touching the pre-existing row. DO UPDATE SET ... [WHERE ...] updates the pre-existing tuple, and has access to both the tuple proposed for insertion and the existing tuple; the optional WHERE clause can be used to prevent an update from being executed. The UPDATE SET and WHERE clauses have access to the tuple proposed for insertion using the "magic" EXCLUDED alias, and to the pre-existing tuple using the table name or its alias. This feature is often referred to as upsert. This is implemented using a new infrastructure called "speculative insertion". It is an optimistic variant of regular insertion that first does a pre-check for existing tuples and then attempts an insert. If a violating tuple was inserted concurrently, the speculatively inserted tuple is deleted and a new attempt is made. If the pre-check finds a matching tuple the alternative DO NOTHING or DO UPDATE action is taken. If the insertion succeeds without detecting a conflict, the tuple is deemed inserted. To handle the possible ambiguity between the excluded alias and a table named excluded, and for convenience with long relation names, INSERT INTO now can alias its target table. Bumps catversion as stored rules change. Author: Peter Geoghegan, with significant contributions from Heikki Linnakangas and Andres Freund. Testing infrastructure by Jeff Janes. Reviewed-By: Heikki Linnakangas, Andres Freund, Robert Haas, Simon Riggs, Dean Rasheed, Stephen Frost and many others.
11 years ago
}
Fix EXPLAIN output for cases where parent table is excluded by constraints. The previous coding in EXPLAIN always labeled a ModifyTable node with the name of the target table affected by its first child plan. When originally written, this was necessarily the parent table of the inheritance tree, so everything was unconfusing. But when we added NO INHERIT constraints, it became possible for the parent table to be deleted from the plan by constraint exclusion while still leaving child tables present. This led to the ModifyTable plan node being labeled with the first surviving child, which was deemed confusing. Fix it by retaining the parent table's RT index in a new field in ModifyTable. Etsuro Fujita, reviewed by Ashutosh Bapat and myself
11 years ago
splan->nominalRelation += rtoffset;
Add support for INSERT ... ON CONFLICT DO NOTHING/UPDATE. The newly added ON CONFLICT clause allows to specify an alternative to raising a unique or exclusion constraint violation error when inserting. ON CONFLICT refers to constraints that can either be specified using a inference clause (by specifying the columns of a unique constraint) or by naming a unique or exclusion constraint. DO NOTHING avoids the constraint violation, without touching the pre-existing row. DO UPDATE SET ... [WHERE ...] updates the pre-existing tuple, and has access to both the tuple proposed for insertion and the existing tuple; the optional WHERE clause can be used to prevent an update from being executed. The UPDATE SET and WHERE clauses have access to the tuple proposed for insertion using the "magic" EXCLUDED alias, and to the pre-existing tuple using the table name or its alias. This feature is often referred to as upsert. This is implemented using a new infrastructure called "speculative insertion". It is an optimistic variant of regular insertion that first does a pre-check for existing tuples and then attempts an insert. If a violating tuple was inserted concurrently, the speculatively inserted tuple is deleted and a new attempt is made. If the pre-check finds a matching tuple the alternative DO NOTHING or DO UPDATE action is taken. If the insertion succeeds without detecting a conflict, the tuple is deemed inserted. To handle the possible ambiguity between the excluded alias and a table named excluded, and for convenience with long relation names, INSERT INTO now can alias its target table. Bumps catversion as stored rules change. Author: Peter Geoghegan, with significant contributions from Heikki Linnakangas and Andres Freund. Testing infrastructure by Jeff Janes. Reviewed-By: Heikki Linnakangas, Andres Freund, Robert Haas, Simon Riggs, Dean Rasheed, Stephen Frost and many others.
11 years ago
splan->exclRelRTI += rtoffset;
Fix EXPLAIN output for cases where parent table is excluded by constraints. The previous coding in EXPLAIN always labeled a ModifyTable node with the name of the target table affected by its first child plan. When originally written, this was necessarily the parent table of the inheritance tree, so everything was unconfusing. But when we added NO INHERIT constraints, it became possible for the parent table to be deleted from the plan by constraint exclusion while still leaving child tables present. This led to the ModifyTable plan node being labeled with the first surviving child, which was deemed confusing. Fix it by retaining the parent table's RT index in a new field in ModifyTable. Etsuro Fujita, reviewed by Ashutosh Bapat and myself
11 years ago
Split the processing of INSERT/UPDATE/DELETE operations out of execMain.c. They are now handled by a new plan node type called ModifyTable, which is placed at the top of the plan tree. In itself this change doesn't do much, except perhaps make the handling of RETURNING lists and inherited UPDATEs a tad less klugy. But it is necessary preparation for the intended extension of allowing RETURNING queries inside WITH. Marko Tiikkaja
16 years ago
foreach(l, splan->resultRelations)
{
lfirst_int(l) += rtoffset;
}
Re-implement EvalPlanQual processing to improve its performance and eliminate a lot of strange behaviors that occurred in join cases. We now identify the "current" row for every joined relation in UPDATE, DELETE, and SELECT FOR UPDATE/SHARE queries. If an EvalPlanQual recheck is necessary, we jam the appropriate row into each scan node in the rechecking plan, forcing it to emit only that one row. The former behavior could rescan the whole of each joined relation for each recheck, which was terrible for performance, and what's much worse could result in duplicated output tuples. Also, the original implementation of EvalPlanQual could not re-use the recheck execution tree --- it had to go through a full executor init and shutdown for every row to be tested. To avoid this overhead, I've associated a special runtime Param with each LockRows or ModifyTable plan node, and arranged to make every scan node below such a node depend on that Param. Thus, by signaling a change in that Param, the EPQ machinery can just rescan the already-built test plan. This patch also adds a prohibition on set-returning functions in the targetlist of SELECT FOR UPDATE/SHARE. This is needed to avoid the duplicate-output-tuple problem. It seems fairly reasonable since the other restrictions on SELECT FOR UPDATE are meant to ensure that there is a unique correspondence between source tuples and result tuples, which an output SRF destroys as much as anything else does.
16 years ago
foreach(l, splan->rowMarks)
{
PlanRowMark *rc = (PlanRowMark *) lfirst(l);
rc->rti += rtoffset;
rc->prti += rtoffset;
}
Split the processing of INSERT/UPDATE/DELETE operations out of execMain.c. They are now handled by a new plan node type called ModifyTable, which is placed at the top of the plan tree. In itself this change doesn't do much, except perhaps make the handling of RETURNING lists and inherited UPDATEs a tad less klugy. But it is necessary preparation for the intended extension of allowing RETURNING queries inside WITH. Marko Tiikkaja
16 years ago
foreach(l, splan->plans)
{
Rearrange planner to save the whole PlannerInfo (subroot) for a subquery. Formerly, set_subquery_pathlist and other creators of plans for subqueries saved only the rangetable and rowMarks lists from the lower-level PlannerInfo. But there's no reason not to remember the whole PlannerInfo, and indeed this turns out to simplify matters in a number of places. The immediate reason for doing this was so that the subroot will still be accessible when we're trying to extract column statistics out of an already-planned subquery. But now that I've done it, it seems like a good code-beautification effort in its own right. I also chose to get rid of the transient subrtable and subrowmark fields in SubqueryScan nodes, in favor of having setrefs.c look up the subquery's RelOptInfo. That required changing all the APIs in setrefs.c to pass PlannerInfo not PlannerGlobal, which was a large but quite mechanical transformation. One side-effect not foreseen at the beginning is that this finally broke inheritance_planner's assumption that replanning the same subquery RTE N times would necessarily give interchangeable results each time. That assumption was always pretty risky, but now we really have to make a separate RTE for each instance so that there's a place to carry the separate subroots.
14 years ago
lfirst(l) = set_plan_refs(root,
Split the processing of INSERT/UPDATE/DELETE operations out of execMain.c. They are now handled by a new plan node type called ModifyTable, which is placed at the top of the plan tree. In itself this change doesn't do much, except perhaps make the handling of RETURNING lists and inherited UPDATEs a tad less klugy. But it is necessary preparation for the intended extension of allowing RETURNING queries inside WITH. Marko Tiikkaja
16 years ago
(Plan *) lfirst(l),
rtoffset);
}
Support data-modifying commands (INSERT/UPDATE/DELETE) in WITH. This patch implements data-modifying WITH queries according to the semantics that the updates all happen with the same command counter value, and in an unspecified order. Therefore one WITH clause can't see the effects of another, nor can the outer query see the effects other than through the RETURNING values. And attempts to do conflicting updates will have unpredictable results. We'll need to document all that. This commit just fixes the code; documentation updates are waiting on author. Marko Tiikkaja and Hitoshi Harada
15 years ago
/*
* Append this ModifyTable node's final result relation RT
* index(es) to the global list for the plan, and set its
* resultRelIndex to reflect their starting position in the
* global list.
*/
Rearrange planner to save the whole PlannerInfo (subroot) for a subquery. Formerly, set_subquery_pathlist and other creators of plans for subqueries saved only the rangetable and rowMarks lists from the lower-level PlannerInfo. But there's no reason not to remember the whole PlannerInfo, and indeed this turns out to simplify matters in a number of places. The immediate reason for doing this was so that the subroot will still be accessible when we're trying to extract column statistics out of an already-planned subquery. But now that I've done it, it seems like a good code-beautification effort in its own right. I also chose to get rid of the transient subrtable and subrowmark fields in SubqueryScan nodes, in favor of having setrefs.c look up the subquery's RelOptInfo. That required changing all the APIs in setrefs.c to pass PlannerInfo not PlannerGlobal, which was a large but quite mechanical transformation. One side-effect not foreseen at the beginning is that this finally broke inheritance_planner's assumption that replanning the same subquery RTE N times would necessarily give interchangeable results each time. That assumption was always pretty risky, but now we really have to make a separate RTE for each instance so that there's a place to carry the separate subroots.
14 years ago
splan->resultRelIndex = list_length(root->glob->resultRelations);
root->glob->resultRelations =
list_concat(root->glob->resultRelations,
Support data-modifying commands (INSERT/UPDATE/DELETE) in WITH. This patch implements data-modifying WITH queries according to the semantics that the updates all happen with the same command counter value, and in an unspecified order. Therefore one WITH clause can't see the effects of another, nor can the outer query see the effects other than through the RETURNING values. And attempts to do conflicting updates will have unpredictable results. We'll need to document all that. This commit just fixes the code; documentation updates are waiting on author. Marko Tiikkaja and Hitoshi Harada
15 years ago
list_copy(splan->resultRelations));
Split the processing of INSERT/UPDATE/DELETE operations out of execMain.c. They are now handled by a new plan node type called ModifyTable, which is placed at the top of the plan tree. In itself this change doesn't do much, except perhaps make the handling of RETURNING lists and inherited UPDATEs a tad less klugy. But it is necessary preparation for the intended extension of allowing RETURNING queries inside WITH. Marko Tiikkaja
16 years ago
}
break;
Further planner/optimizer cleanups. Move all set_tlist_references and fix_opids processing to a single recursive pass over the plan tree executed at the very tail end of planning, rather than haphazardly here and there at different places. Now that tlist Vars do not get modified until the very end, it's possible to get rid of the klugy var_equal and match_varid partial-matching routines, and just use plain equal() throughout the optimizer. This is a step towards allowing merge and hash joins to be done on expressions instead of only Vars ...
26 years ago
case T_Append:
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
{
pgindent run for 8.3.
18 years ago
Append *splan = (Append *) plan;
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
/*
* Append, like Sort et al, doesn't actually evaluate its
* targetlist or check quals.
*/
Now that plans have flat rangetable lists, it's a lot easier to get EXPLAIN to drill down into subplan targetlists to print the referent expression for an OUTER or INNER var in an upper plan node. Hence, make it do that always, and banish the old hack of showing "?columnN?" when things got too complicated. Along the way, fix an EXPLAIN bug I introduced by suppressing subqueries from execution-time range tables: get_name_for_var_field() assumed it could look at rte->subquery to find out the real type of a RECORD var. That doesn't work anymore, but instead we can look at the input plan of the SubqueryScan plan node.
19 years ago
set_dummy_tlist_references(plan, rtoffset);
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
Assert(splan->plan.qual == NIL);
foreach(l, splan->appendplans)
{
Rearrange planner to save the whole PlannerInfo (subroot) for a subquery. Formerly, set_subquery_pathlist and other creators of plans for subqueries saved only the rangetable and rowMarks lists from the lower-level PlannerInfo. But there's no reason not to remember the whole PlannerInfo, and indeed this turns out to simplify matters in a number of places. The immediate reason for doing this was so that the subroot will still be accessible when we're trying to extract column statistics out of an already-planned subquery. But now that I've done it, it seems like a good code-beautification effort in its own right. I also chose to get rid of the transient subrtable and subrowmark fields in SubqueryScan nodes, in favor of having setrefs.c look up the subquery's RelOptInfo. That required changing all the APIs in setrefs.c to pass PlannerInfo not PlannerGlobal, which was a large but quite mechanical transformation. One side-effect not foreseen at the beginning is that this finally broke inheritance_planner's assumption that replanning the same subquery RTE N times would necessarily give interchangeable results each time. That assumption was always pretty risky, but now we really have to make a separate RTE for each instance so that there's a place to carry the separate subroots.
14 years ago
lfirst(l) = set_plan_refs(root,
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
(Plan *) lfirst(l),
rtoffset);
}
}
Further planner/optimizer cleanups. Move all set_tlist_references and fix_opids processing to a single recursive pass over the plan tree executed at the very tail end of planning, rather than haphazardly here and there at different places. Now that tlist Vars do not get modified until the very end, it's possible to get rid of the klugy var_equal and match_varid partial-matching routines, and just use plain equal() throughout the optimizer. This is a step towards allowing merge and hash joins to be done on expressions instead of only Vars ...
26 years ago
break;
Support MergeAppend plans, to allow sorted output from append relations. This patch eliminates the former need to sort the output of an Append scan when an ordered scan of an inheritance tree is wanted. This should be particularly useful for fast-start cases such as queries with LIMIT. Original patch by Greg Stark, with further hacking by Hans-Jurgen Schonig, Robert Haas, and Tom Lane.
15 years ago
case T_MergeAppend:
{
MergeAppend *splan = (MergeAppend *) plan;
/*
* MergeAppend, like Sort et al, doesn't actually evaluate its
* targetlist or check quals.
*/
set_dummy_tlist_references(plan, rtoffset);
Assert(splan->plan.qual == NIL);
foreach(l, splan->mergeplans)
{
Rearrange planner to save the whole PlannerInfo (subroot) for a subquery. Formerly, set_subquery_pathlist and other creators of plans for subqueries saved only the rangetable and rowMarks lists from the lower-level PlannerInfo. But there's no reason not to remember the whole PlannerInfo, and indeed this turns out to simplify matters in a number of places. The immediate reason for doing this was so that the subroot will still be accessible when we're trying to extract column statistics out of an already-planned subquery. But now that I've done it, it seems like a good code-beautification effort in its own right. I also chose to get rid of the transient subrtable and subrowmark fields in SubqueryScan nodes, in favor of having setrefs.c look up the subquery's RelOptInfo. That required changing all the APIs in setrefs.c to pass PlannerInfo not PlannerGlobal, which was a large but quite mechanical transformation. One side-effect not foreseen at the beginning is that this finally broke inheritance_planner's assumption that replanning the same subquery RTE N times would necessarily give interchangeable results each time. That assumption was always pretty risky, but now we really have to make a separate RTE for each instance so that there's a place to carry the separate subroots.
14 years ago
lfirst(l) = set_plan_refs(root,
Support MergeAppend plans, to allow sorted output from append relations. This patch eliminates the former need to sort the output of an Append scan when an ordered scan of an inheritance tree is wanted. This should be particularly useful for fast-start cases such as queries with LIMIT. Original patch by Greg Stark, with further hacking by Hans-Jurgen Schonig, Robert Haas, and Tom Lane.
15 years ago
(Plan *) lfirst(l),
rtoffset);
}
}
break;
Implement SQL-standard WITH clauses, including WITH RECURSIVE. There are some unimplemented aspects: recursive queries must use UNION ALL (should allow UNION too), and we don't have SEARCH or CYCLE clauses. These might or might not get done for 8.4, but even without them it's a pretty useful feature. There are also a couple of small loose ends and definitional quibbles, which I'll send a memo about to pgsql-hackers shortly. But let's land the patch now so we can get on with other development. Yoshiyuki Asaba, with lots of help from Tatsuo Ishii and Tom Lane
17 years ago
case T_RecursiveUnion:
/* This doesn't evaluate targetlist or check quals either */
set_dummy_tlist_references(plan, rtoffset);
Assert(plan->qual == NIL);
break;
Create executor and planner-backend support for decoupled heap and index scans, using in-memory tuple ID bitmaps as the intermediary. The planner frontend (path creation and cost estimation) is not there yet, so none of this code can be executed. I have tested it using some hacked planner code that is far too ugly to see the light of day, however. Committing now so that the bulk of the infrastructure changes go in before the tree drifts under me.
21 years ago
case T_BitmapAnd:
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
{
pgindent run for 8.3.
18 years ago
BitmapAnd *splan = (BitmapAnd *) plan;
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
/* BitmapAnd works like Append, but has no tlist */
Assert(splan->plan.targetlist == NIL);
Assert(splan->plan.qual == NIL);
foreach(l, splan->bitmapplans)
{
Rearrange planner to save the whole PlannerInfo (subroot) for a subquery. Formerly, set_subquery_pathlist and other creators of plans for subqueries saved only the rangetable and rowMarks lists from the lower-level PlannerInfo. But there's no reason not to remember the whole PlannerInfo, and indeed this turns out to simplify matters in a number of places. The immediate reason for doing this was so that the subroot will still be accessible when we're trying to extract column statistics out of an already-planned subquery. But now that I've done it, it seems like a good code-beautification effort in its own right. I also chose to get rid of the transient subrtable and subrowmark fields in SubqueryScan nodes, in favor of having setrefs.c look up the subquery's RelOptInfo. That required changing all the APIs in setrefs.c to pass PlannerInfo not PlannerGlobal, which was a large but quite mechanical transformation. One side-effect not foreseen at the beginning is that this finally broke inheritance_planner's assumption that replanning the same subquery RTE N times would necessarily give interchangeable results each time. That assumption was always pretty risky, but now we really have to make a separate RTE for each instance so that there's a place to carry the separate subroots.
14 years ago
lfirst(l) = set_plan_refs(root,
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
(Plan *) lfirst(l),
rtoffset);
}
}
Create executor and planner-backend support for decoupled heap and index scans, using in-memory tuple ID bitmaps as the intermediary. The planner frontend (path creation and cost estimation) is not there yet, so none of this code can be executed. I have tested it using some hacked planner code that is far too ugly to see the light of day, however. Committing now so that the bulk of the infrastructure changes go in before the tree drifts under me.
21 years ago
break;
case T_BitmapOr:
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
{
pgindent run for 8.3.
18 years ago
BitmapOr *splan = (BitmapOr *) plan;
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
/* BitmapOr works like Append, but has no tlist */
Assert(splan->plan.targetlist == NIL);
Assert(splan->plan.qual == NIL);
foreach(l, splan->bitmapplans)
{
Rearrange planner to save the whole PlannerInfo (subroot) for a subquery. Formerly, set_subquery_pathlist and other creators of plans for subqueries saved only the rangetable and rowMarks lists from the lower-level PlannerInfo. But there's no reason not to remember the whole PlannerInfo, and indeed this turns out to simplify matters in a number of places. The immediate reason for doing this was so that the subroot will still be accessible when we're trying to extract column statistics out of an already-planned subquery. But now that I've done it, it seems like a good code-beautification effort in its own right. I also chose to get rid of the transient subrtable and subrowmark fields in SubqueryScan nodes, in favor of having setrefs.c look up the subquery's RelOptInfo. That required changing all the APIs in setrefs.c to pass PlannerInfo not PlannerGlobal, which was a large but quite mechanical transformation. One side-effect not foreseen at the beginning is that this finally broke inheritance_planner's assumption that replanning the same subquery RTE N times would necessarily give interchangeable results each time. That assumption was always pretty risky, but now we really have to make a separate RTE for each instance so that there's a place to carry the separate subroots.
14 years ago
lfirst(l) = set_plan_refs(root,
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
(Plan *) lfirst(l),
rtoffset);
}
}
Create executor and planner-backend support for decoupled heap and index scans, using in-memory tuple ID bitmaps as the intermediary. The planner frontend (path creation and cost estimation) is not there yet, so none of this code can be executed. I have tested it using some hacked planner code that is far too ugly to see the light of day, however. Committing now so that the bulk of the infrastructure changes go in before the tree drifts under me.
21 years ago
break;
Further planner/optimizer cleanups. Move all set_tlist_references and fix_opids processing to a single recursive pass over the plan tree executed at the very tail end of planning, rather than haphazardly here and there at different places. Now that tlist Vars do not get modified until the very end, it's possible to get rid of the klugy var_equal and match_varid partial-matching routines, and just use plain equal() throughout the optimizer. This is a step towards allowing merge and hash joins to be done on expressions instead of only Vars ...
26 years ago
default:
Error message editing in backend/optimizer, backend/rewrite.
22 years ago
elog(ERROR, "unrecognized node type: %d",
(int) nodeTag(plan));
Further planner/optimizer cleanups. Move all set_tlist_references and fix_opids processing to a single recursive pass over the plan tree executed at the very tail end of planning, rather than haphazardly here and there at different places. Now that tlist Vars do not get modified until the very end, it's possible to get rid of the klugy var_equal and match_varid partial-matching routines, and just use plain equal() throughout the optimizer. This is a step towards allowing merge and hash joins to be done on expressions instead of only Vars ...
26 years ago
break;
}
Postgres95 1.01 Distribution - Virgin Sources
30 years ago
Further planner/optimizer cleanups. Move all set_tlist_references and fix_opids processing to a single recursive pass over the plan tree executed at the very tail end of planning, rather than haphazardly here and there at different places. Now that tlist Vars do not get modified until the very end, it's possible to get rid of the klugy var_equal and match_varid partial-matching routines, and just use plain equal() throughout the optimizer. This is a step towards allowing merge and hash joins to be done on expressions instead of only Vars ...
26 years ago
/*
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
* Now recurse into child plans, if any
Further planner/optimizer cleanups. Move all set_tlist_references and fix_opids processing to a single recursive pass over the plan tree executed at the very tail end of planning, rather than haphazardly here and there at different places. Now that tlist Vars do not get modified until the very end, it's possible to get rid of the klugy var_equal and match_varid partial-matching routines, and just use plain equal() throughout the optimizer. This is a step towards allowing merge and hash joins to be done on expressions instead of only Vars ...
26 years ago
*
Phase 1 of read-only-plans project: cause executor state nodes to point to plan nodes, not vice-versa. All executor state nodes now inherit from struct PlanState. Copying of plan trees has been simplified by not storing a list of SubPlans in Plan nodes (eliminating duplicate links). The executor still needs such a list, but it can build it during ExecutorStart since it has to scan the plan tree anyway. No initdb forced since no stored-on-disk structures changed, but you will need a full recompile because of node-numbering changes.
23 years ago
* NOTE: it is essential that we recurse into child plans AFTER we set
Further planner/optimizer cleanups. Move all set_tlist_references and fix_opids processing to a single recursive pass over the plan tree executed at the very tail end of planning, rather than haphazardly here and there at different places. Now that tlist Vars do not get modified until the very end, it's possible to get rid of the klugy var_equal and match_varid partial-matching routines, and just use plain equal() throughout the optimizer. This is a step towards allowing merge and hash joins to be done on expressions instead of only Vars ...
26 years ago
* subplan references in this plan's tlist and quals. If we did the
* reference-adjustments bottom-up, then we would fail to match this
Standard pgindent run for 8.1.
20 years ago
* plan's var nodes against the already-modified nodes of the children.
Further planner/optimizer cleanups. Move all set_tlist_references and fix_opids processing to a single recursive pass over the plan tree executed at the very tail end of planning, rather than haphazardly here and there at different places. Now that tlist Vars do not get modified until the very end, it's possible to get rid of the klugy var_equal and match_varid partial-matching routines, and just use plain equal() throughout the optimizer. This is a step towards allowing merge and hash joins to be done on expressions instead of only Vars ...
26 years ago
*/
Rearrange planner to save the whole PlannerInfo (subroot) for a subquery. Formerly, set_subquery_pathlist and other creators of plans for subqueries saved only the rangetable and rowMarks lists from the lower-level PlannerInfo. But there's no reason not to remember the whole PlannerInfo, and indeed this turns out to simplify matters in a number of places. The immediate reason for doing this was so that the subroot will still be accessible when we're trying to extract column statistics out of an already-planned subquery. But now that I've done it, it seems like a good code-beautification effort in its own right. I also chose to get rid of the transient subrtable and subrowmark fields in SubqueryScan nodes, in favor of having setrefs.c look up the subquery's RelOptInfo. That required changing all the APIs in setrefs.c to pass PlannerInfo not PlannerGlobal, which was a large but quite mechanical transformation. One side-effect not foreseen at the beginning is that this finally broke inheritance_planner's assumption that replanning the same subquery RTE N times would necessarily give interchangeable results each time. That assumption was always pretty risky, but now we really have to make a separate RTE for each instance so that there's a place to carry the separate subroots.
14 years ago
plan->lefttree = set_plan_refs(root, plan->lefttree, rtoffset);
plan->righttree = set_plan_refs(root, plan->righttree, rtoffset);
Teach the planner to remove SubqueryScan nodes from the plan if they aren't doing anything useful (ie, neither selection nor projection). Also, extend to SubqueryScan the hacks already in place to avoid unnecessary ExecProject calls when the result would just be the same tuple the subquery already delivered. This saves some overhead in UNION and other set operations, as well as avoiding overhead for unflatten-able subqueries. Per example from Sokolov Yura.
21 years ago
return plan;
}
Rearrange the implementation of index-only scans. This commit changes index-only scans so that data is read directly from the index tuple without first generating a faux heap tuple. The only immediate benefit is that indexes on system columns (such as OID) can be used in index-only scans, but this is necessary infrastructure if we are ever to support index-only scans on expression indexes. The executor is now ready for that, though the planner still needs substantial work to recognize the possibility. To do this, Vars in index-only plan nodes have to refer to index columns not heap columns. I introduced a new special varno, INDEX_VAR, to mark such Vars to avoid confusion. (In passing, this commit renames the two existing special varnos to OUTER_VAR and INNER_VAR.) This allows ruleutils.c to handle them with logic similar to what we use for subplan reference Vars. Since index-only scans are now fundamentally different from regular indexscans so far as their expression subtrees are concerned, I also chose to change them to have their own plan node type (and hence, their own executor source file).
14 years ago
/*
* set_indexonlyscan_references
* Do set_plan_references processing on an IndexOnlyScan
*
* This is unlike the handling of a plain IndexScan because we have to
* convert Vars referencing the heap into Vars referencing the index.
* We can use the fix_upper_expr machinery for that, by working from a
* targetlist describing the index columns.
*/
static Plan *
set_indexonlyscan_references(PlannerInfo *root,
IndexOnlyScan *plan,
int rtoffset)
{
indexed_tlist *index_itlist;
index_itlist = build_tlist_index(plan->indextlist);
plan->scan.scanrelid += rtoffset;
plan->scan.plan.targetlist = (List *)
fix_upper_expr(root,
(Node *) plan->scan.plan.targetlist,
index_itlist,
INDEX_VAR,
rtoffset);
plan->scan.plan.qual = (List *)
fix_upper_expr(root,
(Node *) plan->scan.plan.qual,
index_itlist,
INDEX_VAR,
rtoffset);
/* indexqual is already transformed to reference index columns */
plan->indexqual = fix_scan_list(root, plan->indexqual, rtoffset);
/* indexorderby is already transformed to reference index columns */
plan->indexorderby = fix_scan_list(root, plan->indexorderby, rtoffset);
/* indextlist must NOT be transformed to reference index columns */
plan->indextlist = fix_scan_list(root, plan->indextlist, rtoffset);
pfree(index_itlist);
return (Plan *) plan;
}
Teach the planner to remove SubqueryScan nodes from the plan if they aren't doing anything useful (ie, neither selection nor projection). Also, extend to SubqueryScan the hacks already in place to avoid unnecessary ExecProject calls when the result would just be the same tuple the subquery already delivered. This saves some overhead in UNION and other set operations, as well as avoiding overhead for unflatten-able subqueries. Per example from Sokolov Yura.
21 years ago
/*
* set_subqueryscan_references
* Do set_plan_references processing on a SubqueryScan
*
* We try to strip out the SubqueryScan entirely; if we can't, we have
* to do the normal processing on it.
*/
static Plan *
Rearrange planner to save the whole PlannerInfo (subroot) for a subquery. Formerly, set_subquery_pathlist and other creators of plans for subqueries saved only the rangetable and rowMarks lists from the lower-level PlannerInfo. But there's no reason not to remember the whole PlannerInfo, and indeed this turns out to simplify matters in a number of places. The immediate reason for doing this was so that the subroot will still be accessible when we're trying to extract column statistics out of an already-planned subquery. But now that I've done it, it seems like a good code-beautification effort in its own right. I also chose to get rid of the transient subrtable and subrowmark fields in SubqueryScan nodes, in favor of having setrefs.c look up the subquery's RelOptInfo. That required changing all the APIs in setrefs.c to pass PlannerInfo not PlannerGlobal, which was a large but quite mechanical transformation. One side-effect not foreseen at the beginning is that this finally broke inheritance_planner's assumption that replanning the same subquery RTE N times would necessarily give interchangeable results each time. That assumption was always pretty risky, but now we really have to make a separate RTE for each instance so that there's a place to carry the separate subroots.
14 years ago
set_subqueryscan_references(PlannerInfo *root,
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
SubqueryScan *plan,
int rtoffset)
Teach the planner to remove SubqueryScan nodes from the plan if they aren't doing anything useful (ie, neither selection nor projection). Also, extend to SubqueryScan the hacks already in place to avoid unnecessary ExecProject calls when the result would just be the same tuple the subquery already delivered. This saves some overhead in UNION and other set operations, as well as avoiding overhead for unflatten-able subqueries. Per example from Sokolov Yura.
21 years ago
{
Rearrange planner to save the whole PlannerInfo (subroot) for a subquery. Formerly, set_subquery_pathlist and other creators of plans for subqueries saved only the rangetable and rowMarks lists from the lower-level PlannerInfo. But there's no reason not to remember the whole PlannerInfo, and indeed this turns out to simplify matters in a number of places. The immediate reason for doing this was so that the subroot will still be accessible when we're trying to extract column statistics out of an already-planned subquery. But now that I've done it, it seems like a good code-beautification effort in its own right. I also chose to get rid of the transient subrtable and subrowmark fields in SubqueryScan nodes, in favor of having setrefs.c look up the subquery's RelOptInfo. That required changing all the APIs in setrefs.c to pass PlannerInfo not PlannerGlobal, which was a large but quite mechanical transformation. One side-effect not foreseen at the beginning is that this finally broke inheritance_planner's assumption that replanning the same subquery RTE N times would necessarily give interchangeable results each time. That assumption was always pretty risky, but now we really have to make a separate RTE for each instance so that there's a place to carry the separate subroots.
14 years ago
RelOptInfo *rel;
Teach the planner to remove SubqueryScan nodes from the plan if they aren't doing anything useful (ie, neither selection nor projection). Also, extend to SubqueryScan the hacks already in place to avoid unnecessary ExecProject calls when the result would just be the same tuple the subquery already delivered. This saves some overhead in UNION and other set operations, as well as avoiding overhead for unflatten-able subqueries. Per example from Sokolov Yura.
21 years ago
Plan *result;
Rearrange planner to save the whole PlannerInfo (subroot) for a subquery. Formerly, set_subquery_pathlist and other creators of plans for subqueries saved only the rangetable and rowMarks lists from the lower-level PlannerInfo. But there's no reason not to remember the whole PlannerInfo, and indeed this turns out to simplify matters in a number of places. The immediate reason for doing this was so that the subroot will still be accessible when we're trying to extract column statistics out of an already-planned subquery. But now that I've done it, it seems like a good code-beautification effort in its own right. I also chose to get rid of the transient subrtable and subrowmark fields in SubqueryScan nodes, in favor of having setrefs.c look up the subquery's RelOptInfo. That required changing all the APIs in setrefs.c to pass PlannerInfo not PlannerGlobal, which was a large but quite mechanical transformation. One side-effect not foreseen at the beginning is that this finally broke inheritance_planner's assumption that replanning the same subquery RTE N times would necessarily give interchangeable results each time. That assumption was always pretty risky, but now we really have to make a separate RTE for each instance so that there's a place to carry the separate subroots.
14 years ago
/* Need to look up the subquery's RelOptInfo, since we need its subroot */
rel = find_base_rel(root, plan->scan.scanrelid);
Assert(rel->subplan == plan->subplan);
Teach the planner to remove SubqueryScan nodes from the plan if they aren't doing anything useful (ie, neither selection nor projection). Also, extend to SubqueryScan the hacks already in place to avoid unnecessary ExecProject calls when the result would just be the same tuple the subquery already delivered. This saves some overhead in UNION and other set operations, as well as avoiding overhead for unflatten-able subqueries. Per example from Sokolov Yura.
21 years ago
Rearrange planner to save the whole PlannerInfo (subroot) for a subquery. Formerly, set_subquery_pathlist and other creators of plans for subqueries saved only the rangetable and rowMarks lists from the lower-level PlannerInfo. But there's no reason not to remember the whole PlannerInfo, and indeed this turns out to simplify matters in a number of places. The immediate reason for doing this was so that the subroot will still be accessible when we're trying to extract column statistics out of an already-planned subquery. But now that I've done it, it seems like a good code-beautification effort in its own right. I also chose to get rid of the transient subrtable and subrowmark fields in SubqueryScan nodes, in favor of having setrefs.c look up the subquery's RelOptInfo. That required changing all the APIs in setrefs.c to pass PlannerInfo not PlannerGlobal, which was a large but quite mechanical transformation. One side-effect not foreseen at the beginning is that this finally broke inheritance_planner's assumption that replanning the same subquery RTE N times would necessarily give interchangeable results each time. That assumption was always pretty risky, but now we really have to make a separate RTE for each instance so that there's a place to carry the separate subroots.
14 years ago
/* Recursively process the subplan */
plan->subplan = set_plan_references(rel->subroot, plan->subplan);
Teach the planner to remove SubqueryScan nodes from the plan if they aren't doing anything useful (ie, neither selection nor projection). Also, extend to SubqueryScan the hacks already in place to avoid unnecessary ExecProject calls when the result would just be the same tuple the subquery already delivered. This saves some overhead in UNION and other set operations, as well as avoiding overhead for unflatten-able subqueries. Per example from Sokolov Yura.
21 years ago
if (trivial_subqueryscan(plan))
{
/*
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
* We can omit the SubqueryScan node and just pull up the subplan.
Teach the planner to remove SubqueryScan nodes from the plan if they aren't doing anything useful (ie, neither selection nor projection). Also, extend to SubqueryScan the hacks already in place to avoid unnecessary ExecProject calls when the result would just be the same tuple the subquery already delivered. This saves some overhead in UNION and other set operations, as well as avoiding overhead for unflatten-able subqueries. Per example from Sokolov Yura.
21 years ago
*/
Fix the recently-added code that eliminates unnecessary SubqueryScan nodes from a finished plan tree. We have to copy the output column names (resname fields) from the SubqueryScan down to its child plan node; else, if this is the topmost level of the plan, the wrong column names will be delivered to the client. Per bug #2017 reported by Jolly Chen.
20 years ago
ListCell *lp,
*lc;
Teach the planner to remove SubqueryScan nodes from the plan if they aren't doing anything useful (ie, neither selection nor projection). Also, extend to SubqueryScan the hacks already in place to avoid unnecessary ExecProject calls when the result would just be the same tuple the subquery already delivered. This saves some overhead in UNION and other set operations, as well as avoiding overhead for unflatten-able subqueries. Per example from Sokolov Yura.
21 years ago
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
result = plan->subplan;
Teach the planner to remove SubqueryScan nodes from the plan if they aren't doing anything useful (ie, neither selection nor projection). Also, extend to SubqueryScan the hacks already in place to avoid unnecessary ExecProject calls when the result would just be the same tuple the subquery already delivered. This saves some overhead in UNION and other set operations, as well as avoiding overhead for unflatten-able subqueries. Per example from Sokolov Yura.
21 years ago
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
/* We have to be sure we don't lose any initplans */
Teach the planner to remove SubqueryScan nodes from the plan if they aren't doing anything useful (ie, neither selection nor projection). Also, extend to SubqueryScan the hacks already in place to avoid unnecessary ExecProject calls when the result would just be the same tuple the subquery already delivered. This saves some overhead in UNION and other set operations, as well as avoiding overhead for unflatten-able subqueries. Per example from Sokolov Yura.
21 years ago
result->initPlan = list_concat(plan->scan.plan.initPlan,
result->initPlan);
Fix the recently-added code that eliminates unnecessary SubqueryScan nodes from a finished plan tree. We have to copy the output column names (resname fields) from the SubqueryScan down to its child plan node; else, if this is the topmost level of the plan, the wrong column names will be delivered to the client. Per bug #2017 reported by Jolly Chen.
20 years ago
/*
Thinking further, it seems we had better also copy down resorigtbl/resorigcol to ensure that SubqueryScan elimination doesn't change the behavior of reporting of original column sources.
20 years ago
* We also have to transfer the SubqueryScan's result-column names
Fix the recently-added code that eliminates unnecessary SubqueryScan nodes from a finished plan tree. We have to copy the output column names (resname fields) from the SubqueryScan down to its child plan node; else, if this is the topmost level of the plan, the wrong column names will be delivered to the client. Per bug #2017 reported by Jolly Chen.
20 years ago
* into the subplan, else columns sent to client will be improperly
Thinking further, it seems we had better also copy down resorigtbl/resorigcol to ensure that SubqueryScan elimination doesn't change the behavior of reporting of original column sources.
20 years ago
* labeled if this is the topmost plan level. Copy the "source
* column" information too.
Fix the recently-added code that eliminates unnecessary SubqueryScan nodes from a finished plan tree. We have to copy the output column names (resname fields) from the SubqueryScan down to its child plan node; else, if this is the topmost level of the plan, the wrong column names will be delivered to the client. Per bug #2017 reported by Jolly Chen.
20 years ago
*/
forboth(lp, plan->scan.plan.targetlist, lc, result->targetlist)
{
TargetEntry *ptle = (TargetEntry *) lfirst(lp);
TargetEntry *ctle = (TargetEntry *) lfirst(lc);
ctle->resname = ptle->resname;
Thinking further, it seems we had better also copy down resorigtbl/resorigcol to ensure that SubqueryScan elimination doesn't change the behavior of reporting of original column sources.
20 years ago
ctle->resorigtbl = ptle->resorigtbl;
ctle->resorigcol = ptle->resorigcol;
Fix the recently-added code that eliminates unnecessary SubqueryScan nodes from a finished plan tree. We have to copy the output column names (resname fields) from the SubqueryScan down to its child plan node; else, if this is the topmost level of the plan, the wrong column names will be delivered to the client. Per bug #2017 reported by Jolly Chen.
20 years ago
}
Teach the planner to remove SubqueryScan nodes from the plan if they aren't doing anything useful (ie, neither selection nor projection). Also, extend to SubqueryScan the hacks already in place to avoid unnecessary ExecProject calls when the result would just be the same tuple the subquery already delivered. This saves some overhead in UNION and other set operations, as well as avoiding overhead for unflatten-able subqueries. Per example from Sokolov Yura.
21 years ago
}
else
{
/*
pgindent run for 9.4 This includes removing tabs after periods in C comments, which was applied to back branches, so this change should not effect backpatching.
12 years ago
* Keep the SubqueryScan node. We have to do the processing that
Standard pgindent run for 8.1.
20 years ago
* set_plan_references would otherwise have done on it. Notice we do
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
* not do set_upper_references() here, because a SubqueryScan will
Standard pgindent run for 8.1.
20 years ago
* always have been created with correct references to its subplan's
* outputs to begin with.
Teach the planner to remove SubqueryScan nodes from the plan if they aren't doing anything useful (ie, neither selection nor projection). Also, extend to SubqueryScan the hacks already in place to avoid unnecessary ExecProject calls when the result would just be the same tuple the subquery already delivered. This saves some overhead in UNION and other set operations, as well as avoiding overhead for unflatten-able subqueries. Per example from Sokolov Yura.
21 years ago
*/
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
plan->scan.scanrelid += rtoffset;
plan->scan.plan.targetlist =
Rearrange planner to save the whole PlannerInfo (subroot) for a subquery. Formerly, set_subquery_pathlist and other creators of plans for subqueries saved only the rangetable and rowMarks lists from the lower-level PlannerInfo. But there's no reason not to remember the whole PlannerInfo, and indeed this turns out to simplify matters in a number of places. The immediate reason for doing this was so that the subroot will still be accessible when we're trying to extract column statistics out of an already-planned subquery. But now that I've done it, it seems like a good code-beautification effort in its own right. I also chose to get rid of the transient subrtable and subrowmark fields in SubqueryScan nodes, in favor of having setrefs.c look up the subquery's RelOptInfo. That required changing all the APIs in setrefs.c to pass PlannerInfo not PlannerGlobal, which was a large but quite mechanical transformation. One side-effect not foreseen at the beginning is that this finally broke inheritance_planner's assumption that replanning the same subquery RTE N times would necessarily give interchangeable results each time. That assumption was always pretty risky, but now we really have to make a separate RTE for each instance so that there's a place to carry the separate subroots.
14 years ago
fix_scan_list(root, plan->scan.plan.targetlist, rtoffset);
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
plan->scan.plan.qual =
Rearrange planner to save the whole PlannerInfo (subroot) for a subquery. Formerly, set_subquery_pathlist and other creators of plans for subqueries saved only the rangetable and rowMarks lists from the lower-level PlannerInfo. But there's no reason not to remember the whole PlannerInfo, and indeed this turns out to simplify matters in a number of places. The immediate reason for doing this was so that the subroot will still be accessible when we're trying to extract column statistics out of an already-planned subquery. But now that I've done it, it seems like a good code-beautification effort in its own right. I also chose to get rid of the transient subrtable and subrowmark fields in SubqueryScan nodes, in favor of having setrefs.c look up the subquery's RelOptInfo. That required changing all the APIs in setrefs.c to pass PlannerInfo not PlannerGlobal, which was a large but quite mechanical transformation. One side-effect not foreseen at the beginning is that this finally broke inheritance_planner's assumption that replanning the same subquery RTE N times would necessarily give interchangeable results each time. That assumption was always pretty risky, but now we really have to make a separate RTE for each instance so that there's a place to carry the separate subroots.
14 years ago
fix_scan_list(root, plan->scan.plan.qual, rtoffset);
Teach the planner to remove SubqueryScan nodes from the plan if they aren't doing anything useful (ie, neither selection nor projection). Also, extend to SubqueryScan the hacks already in place to avoid unnecessary ExecProject calls when the result would just be the same tuple the subquery already delivered. This saves some overhead in UNION and other set operations, as well as avoiding overhead for unflatten-able subqueries. Per example from Sokolov Yura.
21 years ago
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
result = (Plan *) plan;
Teach the planner to remove SubqueryScan nodes from the plan if they aren't doing anything useful (ie, neither selection nor projection). Also, extend to SubqueryScan the hacks already in place to avoid unnecessary ExecProject calls when the result would just be the same tuple the subquery already delivered. This saves some overhead in UNION and other set operations, as well as avoiding overhead for unflatten-able subqueries. Per example from Sokolov Yura.
21 years ago
}
return result;
}
/*
* trivial_subqueryscan
* Detect whether a SubqueryScan can be deleted from the plan tree.
*
* We can delete it if it has no qual to check and the targetlist just
* regurgitates the output of the child plan.
*/
static bool
trivial_subqueryscan(SubqueryScan *plan)
{
int attrno;
ListCell *lp,
*lc;
if (plan->scan.plan.qual != NIL)
return false;
Back out prior patch and instead just suppress SubqueryScan elimination when there are extra resjunk columns in the child node. I found some additional cases involving Append nodes that weren't handled by the prior patch, and it's not clear how to fix them in the same way without breaking inheritance cases. So the prudent path seems to be to narrow the scope of the optimization.
20 years ago
if (list_length(plan->scan.plan.targetlist) !=
list_length(plan->subplan->targetlist))
return false; /* tlists not same length */
Teach the planner to remove SubqueryScan nodes from the plan if they aren't doing anything useful (ie, neither selection nor projection). Also, extend to SubqueryScan the hacks already in place to avoid unnecessary ExecProject calls when the result would just be the same tuple the subquery already delivered. This saves some overhead in UNION and other set operations, as well as avoiding overhead for unflatten-able subqueries. Per example from Sokolov Yura.
21 years ago
attrno = 1;
forboth(lp, plan->scan.plan.targetlist, lc, plan->subplan->targetlist)
{
TargetEntry *ptle = (TargetEntry *) lfirst(lp);
TargetEntry *ctle = (TargetEntry *) lfirst(lc);
if (ptle->resjunk != ctle->resjunk)
return false; /* tlist doesn't match junk status */
Tweak trivial_subqueryscan() to consider a SubqueryScan's targetlist trivial if it contains either Vars referencing the corresponding subplan columns, or Consts equaling the corresponding subplan columns. This lets the planner eliminate the SubqueryScan in some cases generated by generate_setop_tlist().
19 years ago
/*
pgindent run for 8.2.
19 years ago
* We accept either a Var referencing the corresponding element of the
* subplan tlist, or a Const equaling the subplan element. See
* generate_setop_tlist() for motivation.
Tweak trivial_subqueryscan() to consider a SubqueryScan's targetlist trivial if it contains either Vars referencing the corresponding subplan columns, or Consts equaling the corresponding subplan columns. This lets the planner eliminate the SubqueryScan in some cases generated by generate_setop_tlist().
19 years ago
*/
if (ptle->expr && IsA(ptle->expr, Var))
{
pgindent run for 8.2.
19 years ago
Var *var = (Var *) ptle->expr;
Tweak trivial_subqueryscan() to consider a SubqueryScan's targetlist trivial if it contains either Vars referencing the corresponding subplan columns, or Consts equaling the corresponding subplan columns. This lets the planner eliminate the SubqueryScan in some cases generated by generate_setop_tlist().
19 years ago
Assert(var->varno == plan->scan.scanrelid);
Assert(var->varlevelsup == 0);
if (var->varattno != attrno)
return false; /* out of order */
}
else if (ptle->expr && IsA(ptle->expr, Const))
{
if (!equal(ptle->expr, ctle->expr))
return false;
}
else
return false;
Teach the planner to remove SubqueryScan nodes from the plan if they aren't doing anything useful (ie, neither selection nor projection). Also, extend to SubqueryScan the hacks already in place to avoid unnecessary ExecProject calls when the result would just be the same tuple the subquery already delivered. This saves some overhead in UNION and other set operations, as well as avoiding overhead for unflatten-able subqueries. Per example from Sokolov Yura.
21 years ago
attrno++;
}
return true;
}
Code review for foreign/custom join pushdown patch. Commit e7cb7ee14555cc9c5773e2c102efd6371f6f2005 included some design decisions that seem pretty questionable to me, and there was quite a lot of stuff not to like about the documentation and comments. Clean up as follows: * Consider foreign joins only between foreign tables on the same server, rather than between any two foreign tables with the same underlying FDW handler function. In most if not all cases, the FDW would simply have had to apply the same-server restriction itself (far more expensively, both for lack of caching and because it would be repeated for each combination of input sub-joins), or else risk nasty bugs. Anyone who's really intent on doing something outside this restriction can always use the set_join_pathlist_hook. * Rename fdw_ps_tlist/custom_ps_tlist to fdw_scan_tlist/custom_scan_tlist to better reflect what they're for, and allow these custom scan tlists to be used even for base relations. * Change make_foreignscan() API to include passing the fdw_scan_tlist value, since the FDW is required to set that. Backwards compatibility doesn't seem like an adequate reason to expect FDWs to set it in some ad-hoc extra step, and anyway existing FDWs can just pass NIL. * Change the API of path-generating subroutines of add_paths_to_joinrel, and in particular that of GetForeignJoinPaths and set_join_pathlist_hook, so that various less-used parameters are passed in a struct rather than as separate parameter-list entries. The objective here is to reduce the probability that future additions to those parameter lists will result in source-level API breaks for users of these hooks. It's possible that this is even a small win for the core code, since most CPU architectures can't pass more than half a dozen parameters efficiently anyway. I kept root, joinrel, outerrel, innerrel, and jointype as separate parameters to reduce code churn in joinpath.c --- in particular, putting jointype into the struct would have been problematic because of the subroutines' habit of changing their local copies of that variable. * Avoid ad-hocery in ExecAssignScanProjectionInfo. It was probably all right for it to know about IndexOnlyScan, but if the list is to grow we should refactor the knowledge out to the callers. * Restore nodeForeignscan.c's previous use of the relcache to avoid extra GetFdwRoutine lookups for base-relation scans. * Lots of cleanup of documentation and missed comments. Re-order some code additions into more logical places.
10 years ago
/*
* set_foreignscan_references
* Do set_plan_references processing on a ForeignScan
*/
static void
set_foreignscan_references(PlannerInfo *root,
ForeignScan *fscan,
int rtoffset)
{
/* Adjust scanrelid if it's valid */
if (fscan->scan.scanrelid > 0)
fscan->scan.scanrelid += rtoffset;
if (fscan->fdw_scan_tlist != NIL || fscan->scan.scanrelid == 0)
{
/* Adjust tlist, qual, fdw_exprs to reference custom scan tuple */
indexed_tlist *itlist = build_tlist_index(fscan->fdw_scan_tlist);
fscan->scan.plan.targetlist = (List *)
fix_upper_expr(root,
(Node *) fscan->scan.plan.targetlist,
itlist,
INDEX_VAR,
rtoffset);
fscan->scan.plan.qual = (List *)
fix_upper_expr(root,
(Node *) fscan->scan.plan.qual,
itlist,
INDEX_VAR,
rtoffset);
fscan->fdw_exprs = (List *)
fix_upper_expr(root,
(Node *) fscan->fdw_exprs,
itlist,
INDEX_VAR,
rtoffset);
pfree(itlist);
/* fdw_scan_tlist itself just needs fix_scan_list() adjustments */
fscan->fdw_scan_tlist =
fix_scan_list(root, fscan->fdw_scan_tlist, rtoffset);
}
else
{
/* Adjust tlist, qual, fdw_exprs in the standard way */
fscan->scan.plan.targetlist =
fix_scan_list(root, fscan->scan.plan.targetlist, rtoffset);
fscan->scan.plan.qual =
fix_scan_list(root, fscan->scan.plan.qual, rtoffset);
fscan->fdw_exprs =
fix_scan_list(root, fscan->fdw_exprs, rtoffset);
}
/* Adjust fs_relids if needed */
if (rtoffset > 0)
{
Bitmapset *tempset = NULL;
int x = -1;
while ((x = bms_next_member(fscan->fs_relids, x)) >= 0)
tempset = bms_add_member(tempset, x + rtoffset);
fscan->fs_relids = tempset;
}
}
/*
* set_customscan_references
* Do set_plan_references processing on a CustomScan
*/
static void
set_customscan_references(PlannerInfo *root,
CustomScan *cscan,
int rtoffset)
{
/* Adjust scanrelid if it's valid */
if (cscan->scan.scanrelid > 0)
cscan->scan.scanrelid += rtoffset;
if (cscan->custom_scan_tlist != NIL || cscan->scan.scanrelid == 0)
{
/* Adjust tlist, qual, custom_exprs to reference custom scan tuple */
indexed_tlist *itlist = build_tlist_index(cscan->custom_scan_tlist);
cscan->scan.plan.targetlist = (List *)
fix_upper_expr(root,
(Node *) cscan->scan.plan.targetlist,
itlist,
INDEX_VAR,
rtoffset);
cscan->scan.plan.qual = (List *)
fix_upper_expr(root,
(Node *) cscan->scan.plan.qual,
itlist,
INDEX_VAR,
rtoffset);
cscan->custom_exprs = (List *)
fix_upper_expr(root,
(Node *) cscan->custom_exprs,
itlist,
INDEX_VAR,
rtoffset);
pfree(itlist);
/* custom_scan_tlist itself just needs fix_scan_list() adjustments */
cscan->custom_scan_tlist =
fix_scan_list(root, cscan->custom_scan_tlist, rtoffset);
}
else
{
/* Adjust tlist, qual, custom_exprs in the standard way */
cscan->scan.plan.targetlist =
fix_scan_list(root, cscan->scan.plan.targetlist, rtoffset);
cscan->scan.plan.qual =
fix_scan_list(root, cscan->scan.plan.qual, rtoffset);
cscan->custom_exprs =
fix_scan_list(root, cscan->custom_exprs, rtoffset);
}
/* Adjust custom_relids if needed */
if (rtoffset > 0)
{
Bitmapset *tempset = NULL;
int x = -1;
while ((x = bms_next_member(cscan->custom_relids, x)) >= 0)
tempset = bms_add_member(tempset, x + rtoffset);
cscan->custom_relids = tempset;
}
}
Marginal performance hack: use a dedicated routine instead of copyObject to copy nodes that are known to be Vars during plan reference adjustment. Saves useless memzero operation as well as the big switch in copyObject.
19 years ago
/*
* copyVar
* Copy a Var node.
*
* fix_scan_expr and friends do this enough times that it's worth having
* a bespoke routine instead of using the generic copyObject() function.
*/
static inline Var *
copyVar(Var *var)
{
Var *newvar = (Var *) palloc(sizeof(Var));
*newvar = *var;
return newvar;
}
Improve the plan cache invalidation mechanism to make it invalidate plans when user-defined functions used in a plan are modified. Also invalidate plans when schemas, operators, or operator classes are modified; but for these cases we just invalidate everything rather than tracking exact dependencies, since these types of objects seldom change in a production database. Tom Lane; loosely based on a patch by Martin Pihlak.
17 years ago
/*
* fix_expr_common
* Do generic set_plan_references processing on an expression node
*
* This is code that is common to all variants of expression-fixing.
* We must look up operator opcode info for OpExpr and related nodes,
Rearrange planner to save the whole PlannerInfo (subroot) for a subquery. Formerly, set_subquery_pathlist and other creators of plans for subqueries saved only the rangetable and rowMarks lists from the lower-level PlannerInfo. But there's no reason not to remember the whole PlannerInfo, and indeed this turns out to simplify matters in a number of places. The immediate reason for doing this was so that the subroot will still be accessible when we're trying to extract column statistics out of an already-planned subquery. But now that I've done it, it seems like a good code-beautification effort in its own right. I also chose to get rid of the transient subrtable and subrowmark fields in SubqueryScan nodes, in favor of having setrefs.c look up the subquery's RelOptInfo. That required changing all the APIs in setrefs.c to pass PlannerInfo not PlannerGlobal, which was a large but quite mechanical transformation. One side-effect not foreseen at the beginning is that this finally broke inheritance_planner's assumption that replanning the same subquery RTE N times would necessarily give interchangeable results each time. That assumption was always pretty risky, but now we really have to make a separate RTE for each instance so that there's a place to carry the separate subroots.
14 years ago
* add OIDs from regclass Const nodes into root->glob->relationOids, and
* add catalog TIDs for user-defined functions into root->glob->invalItems.
Support GROUPING SETS, CUBE and ROLLUP. This SQL standard functionality allows to aggregate data by different GROUP BY clauses at once. Each grouping set returns rows with columns grouped by in other sets set to NULL. This could previously be achieved by doing each grouping as a separate query, conjoined by UNION ALLs. Besides being considerably more concise, grouping sets will in many cases be faster, requiring only one scan over the underlying data. The current implementation of grouping sets only supports using sorting for input. Individual sets that share a sort order are computed in one pass. If there are sets that don't share a sort order, additional sort & aggregation steps are performed. These additional passes are sourced by the previous sort step; thus avoiding repeated scans of the source data. The code is structured in a way that adding support for purely using hash aggregation or a mix of hashing and sorting is possible. Sorting was chosen to be supported first, as it is the most generic method of implementation. Instead of, as in an earlier versions of the patch, representing the chain of sort and aggregation steps as full blown planner and executor nodes, all but the first sort are performed inside the aggregation node itself. This avoids the need to do some unusual gymnastics to handle having to return aggregated and non-aggregated tuples from underlying nodes, as well as having to shut down underlying nodes early to limit memory usage. The optimizer still builds Sort/Agg node to describe each phase, but they're not part of the plan tree, but instead additional data for the aggregation node. They're a convenient and preexisting way to describe aggregation and sorting. The first (and possibly only) sort step is still performed as a separate execution step. That retains similarity with existing group by plans, makes rescans fairly simple, avoids very deep plans (leading to slow explains) and easily allows to avoid the sorting step if the underlying data is sorted by other means. A somewhat ugly side of this patch is having to deal with a grammar ambiguity between the new CUBE keyword and the cube extension/functions named cube (and rollup). To avoid breaking existing deployments of the cube extension it has not been renamed, neither has cube been made a reserved keyword. Instead precedence hacking is used to make GROUP BY cube(..) refer to the CUBE grouping sets feature, and not the function cube(). To actually group by a function cube(), unlikely as that might be, the function name has to be quoted. Needs a catversion bump because stored rules may change. Author: Andrew Gierth and Atri Sharma, with contributions from Andres Freund Reviewed-By: Andres Freund, Noah Misch, Tom Lane, Svenne Krap, Tomas Vondra, Erik Rijkers, Marti Raudsepp, Pavel Stehule Discussion: CAOeZVidmVRe2jU6aMk_5qkxnB7dfmPROzM7Ur8JPW5j8Y5X-Lw@mail.gmail.com
10 years ago
* We also fill in column index lists for GROUPING() expressions.
Improve the plan cache invalidation mechanism to make it invalidate plans when user-defined functions used in a plan are modified. Also invalidate plans when schemas, operators, or operator classes are modified; but for these cases we just invalidate everything rather than tracking exact dependencies, since these types of objects seldom change in a production database. Tom Lane; loosely based on a patch by Martin Pihlak.
17 years ago
*
* We assume it's okay to update opcode info in-place. So this could possibly
* scribble on the planner's input data structures, but it's OK.
*/
Initial code review for CustomScan patch. Get rid of the pernicious entanglement between planner and executor headers introduced by commit 0b03e5951bf0a1a8868db13f02049cf686a82165. Also, rearrange the CustomFoo struct/typedef definitions so that all the typedef names are seen as used by the compiler. Without this pgindent will mess things up a bit, which is not so important perhaps, but it also removes a bizarre discrepancy between the declaration arrangement used for CustomExecMethods and that used for CustomScanMethods and CustomPathMethods. Clean up the commentary around ExecSupportsMarkRestore to reflect the rather large change in its API. Const-ify register_custom_path_provider's argument. This necessitates casting away const in the function, but that seems better than forcing callers of the function to do so (or else not const-ify their method pointer structs, which was sort of the whole point). De-export fix_expr_common. I don't like the exporting of fix_scan_expr or replace_nestloop_params either, but this one surely has got little excuse.
11 years ago
static void
Rearrange planner to save the whole PlannerInfo (subroot) for a subquery. Formerly, set_subquery_pathlist and other creators of plans for subqueries saved only the rangetable and rowMarks lists from the lower-level PlannerInfo. But there's no reason not to remember the whole PlannerInfo, and indeed this turns out to simplify matters in a number of places. The immediate reason for doing this was so that the subroot will still be accessible when we're trying to extract column statistics out of an already-planned subquery. But now that I've done it, it seems like a good code-beautification effort in its own right. I also chose to get rid of the transient subrtable and subrowmark fields in SubqueryScan nodes, in favor of having setrefs.c look up the subquery's RelOptInfo. That required changing all the APIs in setrefs.c to pass PlannerInfo not PlannerGlobal, which was a large but quite mechanical transformation. One side-effect not foreseen at the beginning is that this finally broke inheritance_planner's assumption that replanning the same subquery RTE N times would necessarily give interchangeable results each time. That assumption was always pretty risky, but now we really have to make a separate RTE for each instance so that there's a place to carry the separate subroots.
14 years ago
fix_expr_common(PlannerInfo *root, Node *node)
Improve the plan cache invalidation mechanism to make it invalidate plans when user-defined functions used in a plan are modified. Also invalidate plans when schemas, operators, or operator classes are modified; but for these cases we just invalidate everything rather than tracking exact dependencies, since these types of objects seldom change in a production database. Tom Lane; loosely based on a patch by Martin Pihlak.
17 years ago
{
/* We assume callers won't call us on a NULL pointer */
if (IsA(node, Aggref))
{
Rearrange planner to save the whole PlannerInfo (subroot) for a subquery. Formerly, set_subquery_pathlist and other creators of plans for subqueries saved only the rangetable and rowMarks lists from the lower-level PlannerInfo. But there's no reason not to remember the whole PlannerInfo, and indeed this turns out to simplify matters in a number of places. The immediate reason for doing this was so that the subroot will still be accessible when we're trying to extract column statistics out of an already-planned subquery. But now that I've done it, it seems like a good code-beautification effort in its own right. I also chose to get rid of the transient subrtable and subrowmark fields in SubqueryScan nodes, in favor of having setrefs.c look up the subquery's RelOptInfo. That required changing all the APIs in setrefs.c to pass PlannerInfo not PlannerGlobal, which was a large but quite mechanical transformation. One side-effect not foreseen at the beginning is that this finally broke inheritance_planner's assumption that replanning the same subquery RTE N times would necessarily give interchangeable results each time. That assumption was always pretty risky, but now we really have to make a separate RTE for each instance so that there's a place to carry the separate subroots.
14 years ago
record_plan_function_dependency(root,
Improve the plan cache invalidation mechanism to make it invalidate plans when user-defined functions used in a plan are modified. Also invalidate plans when schemas, operators, or operator classes are modified; but for these cases we just invalidate everything rather than tracking exact dependencies, since these types of objects seldom change in a production database. Tom Lane; loosely based on a patch by Martin Pihlak.
17 years ago
((Aggref *) node)->aggfnoid);
}
Support window functions a la SQL:2008. Hitoshi Harada, with some kibitzing from Heikki and Tom.
17 years ago
else if (IsA(node, WindowFunc))
{
Rearrange planner to save the whole PlannerInfo (subroot) for a subquery. Formerly, set_subquery_pathlist and other creators of plans for subqueries saved only the rangetable and rowMarks lists from the lower-level PlannerInfo. But there's no reason not to remember the whole PlannerInfo, and indeed this turns out to simplify matters in a number of places. The immediate reason for doing this was so that the subroot will still be accessible when we're trying to extract column statistics out of an already-planned subquery. But now that I've done it, it seems like a good code-beautification effort in its own right. I also chose to get rid of the transient subrtable and subrowmark fields in SubqueryScan nodes, in favor of having setrefs.c look up the subquery's RelOptInfo. That required changing all the APIs in setrefs.c to pass PlannerInfo not PlannerGlobal, which was a large but quite mechanical transformation. One side-effect not foreseen at the beginning is that this finally broke inheritance_planner's assumption that replanning the same subquery RTE N times would necessarily give interchangeable results each time. That assumption was always pretty risky, but now we really have to make a separate RTE for each instance so that there's a place to carry the separate subroots.
14 years ago
record_plan_function_dependency(root,
Support window functions a la SQL:2008. Hitoshi Harada, with some kibitzing from Heikki and Tom.
17 years ago
((WindowFunc *) node)->winfnoid);
}
Improve the plan cache invalidation mechanism to make it invalidate plans when user-defined functions used in a plan are modified. Also invalidate plans when schemas, operators, or operator classes are modified; but for these cases we just invalidate everything rather than tracking exact dependencies, since these types of objects seldom change in a production database. Tom Lane; loosely based on a patch by Martin Pihlak.
17 years ago
else if (IsA(node, FuncExpr))
{
Rearrange planner to save the whole PlannerInfo (subroot) for a subquery. Formerly, set_subquery_pathlist and other creators of plans for subqueries saved only the rangetable and rowMarks lists from the lower-level PlannerInfo. But there's no reason not to remember the whole PlannerInfo, and indeed this turns out to simplify matters in a number of places. The immediate reason for doing this was so that the subroot will still be accessible when we're trying to extract column statistics out of an already-planned subquery. But now that I've done it, it seems like a good code-beautification effort in its own right. I also chose to get rid of the transient subrtable and subrowmark fields in SubqueryScan nodes, in favor of having setrefs.c look up the subquery's RelOptInfo. That required changing all the APIs in setrefs.c to pass PlannerInfo not PlannerGlobal, which was a large but quite mechanical transformation. One side-effect not foreseen at the beginning is that this finally broke inheritance_planner's assumption that replanning the same subquery RTE N times would necessarily give interchangeable results each time. That assumption was always pretty risky, but now we really have to make a separate RTE for each instance so that there's a place to carry the separate subroots.
14 years ago
record_plan_function_dependency(root,
Improve the plan cache invalidation mechanism to make it invalidate plans when user-defined functions used in a plan are modified. Also invalidate plans when schemas, operators, or operator classes are modified; but for these cases we just invalidate everything rather than tracking exact dependencies, since these types of objects seldom change in a production database. Tom Lane; loosely based on a patch by Martin Pihlak.
17 years ago
((FuncExpr *) node)->funcid);
}
else if (IsA(node, OpExpr))
{
set_opfuncid((OpExpr *) node);
Rearrange planner to save the whole PlannerInfo (subroot) for a subquery. Formerly, set_subquery_pathlist and other creators of plans for subqueries saved only the rangetable and rowMarks lists from the lower-level PlannerInfo. But there's no reason not to remember the whole PlannerInfo, and indeed this turns out to simplify matters in a number of places. The immediate reason for doing this was so that the subroot will still be accessible when we're trying to extract column statistics out of an already-planned subquery. But now that I've done it, it seems like a good code-beautification effort in its own right. I also chose to get rid of the transient subrtable and subrowmark fields in SubqueryScan nodes, in favor of having setrefs.c look up the subquery's RelOptInfo. That required changing all the APIs in setrefs.c to pass PlannerInfo not PlannerGlobal, which was a large but quite mechanical transformation. One side-effect not foreseen at the beginning is that this finally broke inheritance_planner's assumption that replanning the same subquery RTE N times would necessarily give interchangeable results each time. That assumption was always pretty risky, but now we really have to make a separate RTE for each instance so that there's a place to carry the separate subroots.
14 years ago
record_plan_function_dependency(root,
Improve the plan cache invalidation mechanism to make it invalidate plans when user-defined functions used in a plan are modified. Also invalidate plans when schemas, operators, or operator classes are modified; but for these cases we just invalidate everything rather than tracking exact dependencies, since these types of objects seldom change in a production database. Tom Lane; loosely based on a patch by Martin Pihlak.
17 years ago
((OpExpr *) node)->opfuncid);
}
else if (IsA(node, DistinctExpr))
{
set_opfuncid((OpExpr *) node); /* rely on struct equivalence */
Rearrange planner to save the whole PlannerInfo (subroot) for a subquery. Formerly, set_subquery_pathlist and other creators of plans for subqueries saved only the rangetable and rowMarks lists from the lower-level PlannerInfo. But there's no reason not to remember the whole PlannerInfo, and indeed this turns out to simplify matters in a number of places. The immediate reason for doing this was so that the subroot will still be accessible when we're trying to extract column statistics out of an already-planned subquery. But now that I've done it, it seems like a good code-beautification effort in its own right. I also chose to get rid of the transient subrtable and subrowmark fields in SubqueryScan nodes, in favor of having setrefs.c look up the subquery's RelOptInfo. That required changing all the APIs in setrefs.c to pass PlannerInfo not PlannerGlobal, which was a large but quite mechanical transformation. One side-effect not foreseen at the beginning is that this finally broke inheritance_planner's assumption that replanning the same subquery RTE N times would necessarily give interchangeable results each time. That assumption was always pretty risky, but now we really have to make a separate RTE for each instance so that there's a place to carry the separate subroots.
14 years ago
record_plan_function_dependency(root,
Improve the plan cache invalidation mechanism to make it invalidate plans when user-defined functions used in a plan are modified. Also invalidate plans when schemas, operators, or operator classes are modified; but for these cases we just invalidate everything rather than tracking exact dependencies, since these types of objects seldom change in a production database. Tom Lane; loosely based on a patch by Martin Pihlak.
17 years ago
((DistinctExpr *) node)->opfuncid);
}
else if (IsA(node, NullIfExpr))
{
set_opfuncid((OpExpr *) node); /* rely on struct equivalence */
Rearrange planner to save the whole PlannerInfo (subroot) for a subquery. Formerly, set_subquery_pathlist and other creators of plans for subqueries saved only the rangetable and rowMarks lists from the lower-level PlannerInfo. But there's no reason not to remember the whole PlannerInfo, and indeed this turns out to simplify matters in a number of places. The immediate reason for doing this was so that the subroot will still be accessible when we're trying to extract column statistics out of an already-planned subquery. But now that I've done it, it seems like a good code-beautification effort in its own right. I also chose to get rid of the transient subrtable and subrowmark fields in SubqueryScan nodes, in favor of having setrefs.c look up the subquery's RelOptInfo. That required changing all the APIs in setrefs.c to pass PlannerInfo not PlannerGlobal, which was a large but quite mechanical transformation. One side-effect not foreseen at the beginning is that this finally broke inheritance_planner's assumption that replanning the same subquery RTE N times would necessarily give interchangeable results each time. That assumption was always pretty risky, but now we really have to make a separate RTE for each instance so that there's a place to carry the separate subroots.
14 years ago
record_plan_function_dependency(root,
Improve the plan cache invalidation mechanism to make it invalidate plans when user-defined functions used in a plan are modified. Also invalidate plans when schemas, operators, or operator classes are modified; but for these cases we just invalidate everything rather than tracking exact dependencies, since these types of objects seldom change in a production database. Tom Lane; loosely based on a patch by Martin Pihlak.
17 years ago
((NullIfExpr *) node)->opfuncid);
}
else if (IsA(node, ScalarArrayOpExpr))
{
set_sa_opfuncid((ScalarArrayOpExpr *) node);
Rearrange planner to save the whole PlannerInfo (subroot) for a subquery. Formerly, set_subquery_pathlist and other creators of plans for subqueries saved only the rangetable and rowMarks lists from the lower-level PlannerInfo. But there's no reason not to remember the whole PlannerInfo, and indeed this turns out to simplify matters in a number of places. The immediate reason for doing this was so that the subroot will still be accessible when we're trying to extract column statistics out of an already-planned subquery. But now that I've done it, it seems like a good code-beautification effort in its own right. I also chose to get rid of the transient subrtable and subrowmark fields in SubqueryScan nodes, in favor of having setrefs.c look up the subquery's RelOptInfo. That required changing all the APIs in setrefs.c to pass PlannerInfo not PlannerGlobal, which was a large but quite mechanical transformation. One side-effect not foreseen at the beginning is that this finally broke inheritance_planner's assumption that replanning the same subquery RTE N times would necessarily give interchangeable results each time. That assumption was always pretty risky, but now we really have to make a separate RTE for each instance so that there's a place to carry the separate subroots.
14 years ago
record_plan_function_dependency(root,
8.4 pgindent run, with new combined Linux/FreeBSD/MinGW typedef list provided by Andrew.
16 years ago
((ScalarArrayOpExpr *) node)->opfuncid);
Improve the plan cache invalidation mechanism to make it invalidate plans when user-defined functions used in a plan are modified. Also invalidate plans when schemas, operators, or operator classes are modified; but for these cases we just invalidate everything rather than tracking exact dependencies, since these types of objects seldom change in a production database. Tom Lane; loosely based on a patch by Martin Pihlak.
17 years ago
}
else if (IsA(node, ArrayCoerceExpr))
{
if (OidIsValid(((ArrayCoerceExpr *) node)->elemfuncid))
Rearrange planner to save the whole PlannerInfo (subroot) for a subquery. Formerly, set_subquery_pathlist and other creators of plans for subqueries saved only the rangetable and rowMarks lists from the lower-level PlannerInfo. But there's no reason not to remember the whole PlannerInfo, and indeed this turns out to simplify matters in a number of places. The immediate reason for doing this was so that the subroot will still be accessible when we're trying to extract column statistics out of an already-planned subquery. But now that I've done it, it seems like a good code-beautification effort in its own right. I also chose to get rid of the transient subrtable and subrowmark fields in SubqueryScan nodes, in favor of having setrefs.c look up the subquery's RelOptInfo. That required changing all the APIs in setrefs.c to pass PlannerInfo not PlannerGlobal, which was a large but quite mechanical transformation. One side-effect not foreseen at the beginning is that this finally broke inheritance_planner's assumption that replanning the same subquery RTE N times would necessarily give interchangeable results each time. That assumption was always pretty risky, but now we really have to make a separate RTE for each instance so that there's a place to carry the separate subroots.
14 years ago
record_plan_function_dependency(root,
8.4 pgindent run, with new combined Linux/FreeBSD/MinGW typedef list provided by Andrew.
16 years ago
((ArrayCoerceExpr *) node)->elemfuncid);
Improve the plan cache invalidation mechanism to make it invalidate plans when user-defined functions used in a plan are modified. Also invalidate plans when schemas, operators, or operator classes are modified; but for these cases we just invalidate everything rather than tracking exact dependencies, since these types of objects seldom change in a production database. Tom Lane; loosely based on a patch by Martin Pihlak.
17 years ago
}
else if (IsA(node, Const))
{
Const *con = (Const *) node;
/* Check for regclass reference */
if (ISREGCLASSCONST(con))
Rearrange planner to save the whole PlannerInfo (subroot) for a subquery. Formerly, set_subquery_pathlist and other creators of plans for subqueries saved only the rangetable and rowMarks lists from the lower-level PlannerInfo. But there's no reason not to remember the whole PlannerInfo, and indeed this turns out to simplify matters in a number of places. The immediate reason for doing this was so that the subroot will still be accessible when we're trying to extract column statistics out of an already-planned subquery. But now that I've done it, it seems like a good code-beautification effort in its own right. I also chose to get rid of the transient subrtable and subrowmark fields in SubqueryScan nodes, in favor of having setrefs.c look up the subquery's RelOptInfo. That required changing all the APIs in setrefs.c to pass PlannerInfo not PlannerGlobal, which was a large but quite mechanical transformation. One side-effect not foreseen at the beginning is that this finally broke inheritance_planner's assumption that replanning the same subquery RTE N times would necessarily give interchangeable results each time. That assumption was always pretty risky, but now we really have to make a separate RTE for each instance so that there's a place to carry the separate subroots.
14 years ago
root->glob->relationOids =
lappend_oid(root->glob->relationOids,
Improve the plan cache invalidation mechanism to make it invalidate plans when user-defined functions used in a plan are modified. Also invalidate plans when schemas, operators, or operator classes are modified; but for these cases we just invalidate everything rather than tracking exact dependencies, since these types of objects seldom change in a production database. Tom Lane; loosely based on a patch by Martin Pihlak.
17 years ago
DatumGetObjectId(con->constvalue));
}
Support GROUPING SETS, CUBE and ROLLUP. This SQL standard functionality allows to aggregate data by different GROUP BY clauses at once. Each grouping set returns rows with columns grouped by in other sets set to NULL. This could previously be achieved by doing each grouping as a separate query, conjoined by UNION ALLs. Besides being considerably more concise, grouping sets will in many cases be faster, requiring only one scan over the underlying data. The current implementation of grouping sets only supports using sorting for input. Individual sets that share a sort order are computed in one pass. If there are sets that don't share a sort order, additional sort & aggregation steps are performed. These additional passes are sourced by the previous sort step; thus avoiding repeated scans of the source data. The code is structured in a way that adding support for purely using hash aggregation or a mix of hashing and sorting is possible. Sorting was chosen to be supported first, as it is the most generic method of implementation. Instead of, as in an earlier versions of the patch, representing the chain of sort and aggregation steps as full blown planner and executor nodes, all but the first sort are performed inside the aggregation node itself. This avoids the need to do some unusual gymnastics to handle having to return aggregated and non-aggregated tuples from underlying nodes, as well as having to shut down underlying nodes early to limit memory usage. The optimizer still builds Sort/Agg node to describe each phase, but they're not part of the plan tree, but instead additional data for the aggregation node. They're a convenient and preexisting way to describe aggregation and sorting. The first (and possibly only) sort step is still performed as a separate execution step. That retains similarity with existing group by plans, makes rescans fairly simple, avoids very deep plans (leading to slow explains) and easily allows to avoid the sorting step if the underlying data is sorted by other means. A somewhat ugly side of this patch is having to deal with a grammar ambiguity between the new CUBE keyword and the cube extension/functions named cube (and rollup). To avoid breaking existing deployments of the cube extension it has not been renamed, neither has cube been made a reserved keyword. Instead precedence hacking is used to make GROUP BY cube(..) refer to the CUBE grouping sets feature, and not the function cube(). To actually group by a function cube(), unlikely as that might be, the function name has to be quoted. Needs a catversion bump because stored rules may change. Author: Andrew Gierth and Atri Sharma, with contributions from Andres Freund Reviewed-By: Andres Freund, Noah Misch, Tom Lane, Svenne Krap, Tomas Vondra, Erik Rijkers, Marti Raudsepp, Pavel Stehule Discussion: CAOeZVidmVRe2jU6aMk_5qkxnB7dfmPROzM7Ur8JPW5j8Y5X-Lw@mail.gmail.com
10 years ago
else if (IsA(node, GroupingFunc))
{
GroupingFunc *g = (GroupingFunc *) node;
AttrNumber *grouping_map = root->grouping_map;
/* If there are no grouping sets, we don't need this. */
Assert(grouping_map || g->cols == NIL);
if (grouping_map)
{
ListCell *lc;
List *cols = NIL;
foreach(lc, g->refs)
{
cols = lappend_int(cols, grouping_map[lfirst_int(lc)]);
}
Assert(!g->cols || equal(cols, g->cols));
if (!g->cols)
g->cols = cols;
}
}
Improve the plan cache invalidation mechanism to make it invalidate plans when user-defined functions used in a plan are modified. Also invalidate plans when schemas, operators, or operator classes are modified; but for these cases we just invalidate everything rather than tracking exact dependencies, since these types of objects seldom change in a production database. Tom Lane; loosely based on a patch by Martin Pihlak.
17 years ago
}
Implement UPDATE tab SET (col1,col2,...) = (SELECT ...), ... This SQL-standard feature allows a sub-SELECT yielding multiple columns (but only one row) to be used to compute the new values of several columns to be updated. While the same results can be had with an independent sub-SELECT per column, such a workaround can require a great deal of duplicated computation. The standard actually says that the source for a multi-column assignment could be any row-valued expression. The implementation used here is tightly tied to our existing sub-SELECT support and can't handle other cases; the Bison grammar would have some issues with them too. However, I don't feel too bad about this since other cases can be converted into sub-SELECTs. For instance, "SET (a,b,c) = row_valued_function(x)" could be written "SET (a,b,c) = (SELECT * FROM row_valued_function(x))".
11 years ago
/*
* fix_param_node
* Do set_plan_references processing on a Param
*
* If it's a PARAM_MULTIEXPR, replace it with the appropriate Param from
* root->multiexpr_params; otherwise no change is needed.
* Just for paranoia's sake, we make a copy of the node in either case.
*/
static Node *
fix_param_node(PlannerInfo *root, Param *p)
{
if (p->paramkind == PARAM_MULTIEXPR)
{
int subqueryid = p->paramid >> 16;
int colno = p->paramid & 0xFFFF;
List *params;
if (subqueryid <= 0 ||
subqueryid > list_length(root->multiexpr_params))
elog(ERROR, "unexpected PARAM_MULTIEXPR ID: %d", p->paramid);
params = (List *) list_nth(root->multiexpr_params, subqueryid - 1);
if (colno <= 0 || colno > list_length(params))
elog(ERROR, "unexpected PARAM_MULTIEXPR ID: %d", p->paramid);
return copyObject(list_nth(params, colno - 1));
}
return copyObject(p);
}
Teach the planner to remove SubqueryScan nodes from the plan if they aren't doing anything useful (ie, neither selection nor projection). Also, extend to SubqueryScan the hacks already in place to avoid unnecessary ExecProject calls when the result would just be the same tuple the subquery already delivered. This saves some overhead in UNION and other set operations, as well as avoiding overhead for unflatten-able subqueries. Per example from Sokolov Yura.
21 years ago
/*
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
* fix_scan_expr
* Do set_plan_references processing on a scan-level expression
Teach the planner to remove SubqueryScan nodes from the plan if they aren't doing anything useful (ie, neither selection nor projection). Also, extend to SubqueryScan the hacks already in place to avoid unnecessary ExecProject calls when the result would just be the same tuple the subquery already delivered. This saves some overhead in UNION and other set operations, as well as avoiding overhead for unflatten-able subqueries. Per example from Sokolov Yura.
21 years ago
*
Fix the plan-invalidation mechanism to treat regclass constants that refer to a relation as a reason to invalidate a plan when the relation changes. This handles scenarios such as dropping/recreating a sequence that is referenced by nextval('seq') in a cached plan. Rather than teach plancache.c all about digging through plan trees to find regclass Consts, we charge the planner's setrefs.c with making a list of the relation OIDs on which each plan depends. That way the list can be built cheaply during a plan tree traversal that has to happen anyway. Per bug #3662 and subsequent discussion.
18 years ago
* This consists of incrementing all Vars' varnos by rtoffset,
Implement UPDATE tab SET (col1,col2,...) = (SELECT ...), ... This SQL-standard feature allows a sub-SELECT yielding multiple columns (but only one row) to be used to compute the new values of several columns to be updated. While the same results can be had with an independent sub-SELECT per column, such a workaround can require a great deal of duplicated computation. The standard actually says that the source for a multi-column assignment could be any row-valued expression. The implementation used here is tightly tied to our existing sub-SELECT support and can't handle other cases; the Bison grammar would have some issues with them too. However, I don't feel too bad about this since other cases can be converted into sub-SELECTs. For instance, "SET (a,b,c) = row_valued_function(x)" could be written "SET (a,b,c) = (SELECT * FROM row_valued_function(x))".
11 years ago
* replacing PARAM_MULTIEXPR Params, expanding PlaceHolderVars,
Fix the plan-invalidation mechanism to treat regclass constants that refer to a relation as a reason to invalidate a plan when the relation changes. This handles scenarios such as dropping/recreating a sequence that is referenced by nextval('seq') in a cached plan. Rather than teach plancache.c all about digging through plan trees to find regclass Consts, we charge the planner's setrefs.c with making a list of the relation OIDs on which each plan depends. That way the list can be built cheaply during a plan tree traversal that has to happen anyway. Per bug #3662 and subsequent discussion.
18 years ago
* looking up operator opcode info for OpExpr and related nodes,
Rearrange planner to save the whole PlannerInfo (subroot) for a subquery. Formerly, set_subquery_pathlist and other creators of plans for subqueries saved only the rangetable and rowMarks lists from the lower-level PlannerInfo. But there's no reason not to remember the whole PlannerInfo, and indeed this turns out to simplify matters in a number of places. The immediate reason for doing this was so that the subroot will still be accessible when we're trying to extract column statistics out of an already-planned subquery. But now that I've done it, it seems like a good code-beautification effort in its own right. I also chose to get rid of the transient subrtable and subrowmark fields in SubqueryScan nodes, in favor of having setrefs.c look up the subquery's RelOptInfo. That required changing all the APIs in setrefs.c to pass PlannerInfo not PlannerGlobal, which was a large but quite mechanical transformation. One side-effect not foreseen at the beginning is that this finally broke inheritance_planner's assumption that replanning the same subquery RTE N times would necessarily give interchangeable results each time. That assumption was always pretty risky, but now we really have to make a separate RTE for each instance so that there's a place to carry the separate subroots.
14 years ago
* and adding OIDs from regclass Const nodes into root->glob->relationOids.
Teach the planner to remove SubqueryScan nodes from the plan if they aren't doing anything useful (ie, neither selection nor projection). Also, extend to SubqueryScan the hacks already in place to avoid unnecessary ExecProject calls when the result would just be the same tuple the subquery already delivered. This saves some overhead in UNION and other set operations, as well as avoiding overhead for unflatten-able subqueries. Per example from Sokolov Yura.
21 years ago
*/
Rearrange CustomScan API. Make it work more like FDW plans do: instead of assuming that there are expressions in a CustomScan plan node that the core code doesn't know about, insist that all subexpressions that need planner attention be in a "custom_exprs" list in the Plan representation. (Of course, the custom plugin can break the list apart again at executor initialization.) This lets us revert the parts of the patch that exposed setrefs.c and subselect.c processing to the outside world. Also revert the GetSpecialCustomVar stuff in ruleutils.c; that concept may work in future, but it's far from fully baked right now.
11 years ago
static Node *
Rearrange planner to save the whole PlannerInfo (subroot) for a subquery. Formerly, set_subquery_pathlist and other creators of plans for subqueries saved only the rangetable and rowMarks lists from the lower-level PlannerInfo. But there's no reason not to remember the whole PlannerInfo, and indeed this turns out to simplify matters in a number of places. The immediate reason for doing this was so that the subroot will still be accessible when we're trying to extract column statistics out of an already-planned subquery. But now that I've done it, it seems like a good code-beautification effort in its own right. I also chose to get rid of the transient subrtable and subrowmark fields in SubqueryScan nodes, in favor of having setrefs.c look up the subquery's RelOptInfo. That required changing all the APIs in setrefs.c to pass PlannerInfo not PlannerGlobal, which was a large but quite mechanical transformation. One side-effect not foreseen at the beginning is that this finally broke inheritance_planner's assumption that replanning the same subquery RTE N times would necessarily give interchangeable results each time. That assumption was always pretty risky, but now we really have to make a separate RTE for each instance so that there's a place to carry the separate subroots.
14 years ago
fix_scan_expr(PlannerInfo *root, Node *node, int rtoffset)
Teach the planner to remove SubqueryScan nodes from the plan if they aren't doing anything useful (ie, neither selection nor projection). Also, extend to SubqueryScan the hacks already in place to avoid unnecessary ExecProject calls when the result would just be the same tuple the subquery already delivered. This saves some overhead in UNION and other set operations, as well as avoiding overhead for unflatten-able subqueries. Per example from Sokolov Yura.
21 years ago
{
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
fix_scan_expr_context context;
Rearrange planner to save the whole PlannerInfo (subroot) for a subquery. Formerly, set_subquery_pathlist and other creators of plans for subqueries saved only the rangetable and rowMarks lists from the lower-level PlannerInfo. But there's no reason not to remember the whole PlannerInfo, and indeed this turns out to simplify matters in a number of places. The immediate reason for doing this was so that the subroot will still be accessible when we're trying to extract column statistics out of an already-planned subquery. But now that I've done it, it seems like a good code-beautification effort in its own right. I also chose to get rid of the transient subrtable and subrowmark fields in SubqueryScan nodes, in favor of having setrefs.c look up the subquery's RelOptInfo. That required changing all the APIs in setrefs.c to pass PlannerInfo not PlannerGlobal, which was a large but quite mechanical transformation. One side-effect not foreseen at the beginning is that this finally broke inheritance_planner's assumption that replanning the same subquery RTE N times would necessarily give interchangeable results each time. That assumption was always pretty risky, but now we really have to make a separate RTE for each instance so that there's a place to carry the separate subroots.
14 years ago
context.root = root;
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
context.rtoffset = rtoffset;
Change fix_scan_expr() to avoid copying the input node tree in the common case where rtoffset == 0. In that case there is no need to change Var nodes, and since filling in unset opfuncid fields is always safe, scribbling on the input tree to that extent is not objectionable. This brings the cost of this operation back down to what it was in 8.2 for simple queries. Per investigation of performance gripe from Guillaume Smet.
18 years ago
Implement UPDATE tab SET (col1,col2,...) = (SELECT ...), ... This SQL-standard feature allows a sub-SELECT yielding multiple columns (but only one row) to be used to compute the new values of several columns to be updated. While the same results can be had with an independent sub-SELECT per column, such a workaround can require a great deal of duplicated computation. The standard actually says that the source for a multi-column assignment could be any row-valued expression. The implementation used here is tightly tied to our existing sub-SELECT support and can't handle other cases; the Bison grammar would have some issues with them too. However, I don't feel too bad about this since other cases can be converted into sub-SELECTs. For instance, "SET (a,b,c) = row_valued_function(x)" could be written "SET (a,b,c) = (SELECT * FROM row_valued_function(x))".
11 years ago
if (rtoffset != 0 ||
root->multiexpr_params != NIL ||
root->glob->lastPHId != 0)
Change fix_scan_expr() to avoid copying the input node tree in the common case where rtoffset == 0. In that case there is no need to change Var nodes, and since filling in unset opfuncid fields is always safe, scribbling on the input tree to that extent is not objectionable. This brings the cost of this operation back down to what it was in 8.2 for simple queries. Per investigation of performance gripe from Guillaume Smet.
18 years ago
{
return fix_scan_expr_mutator(node, &context);
}
else
{
/*
Add a concept of "placeholder" variables to the planner. These are variables that represent some expression that we desire to compute below the top level of the plan, and then let that value "bubble up" as though it were a plain Var (ie, a column value). The immediate application is to allow sub-selects to be flattened even when they are below an outer join and have non-nullable output expressions. Formerly we couldn't flatten because such an expression wouldn't properly go to NULL when evaluated above the outer join. Now, we wrap it in a PlaceHolderVar and arrange for the actual evaluation to occur below the outer join. When the resulting Var bubbles up through the join, it will be set to NULL if necessary, yielding the correct results. This fixes a planner limitation that's existed since 7.1. In future we might want to use this mechanism to re-introduce some form of Hellerstein's "expensive functions" optimization, ie place the evaluation of an expensive function at the most suitable point in the plan tree.
17 years ago
* If rtoffset == 0, we don't need to change any Vars, and if there
Implement UPDATE tab SET (col1,col2,...) = (SELECT ...), ... This SQL-standard feature allows a sub-SELECT yielding multiple columns (but only one row) to be used to compute the new values of several columns to be updated. While the same results can be had with an independent sub-SELECT per column, such a workaround can require a great deal of duplicated computation. The standard actually says that the source for a multi-column assignment could be any row-valued expression. The implementation used here is tightly tied to our existing sub-SELECT support and can't handle other cases; the Bison grammar would have some issues with them too. However, I don't feel too bad about this since other cases can be converted into sub-SELECTs. For instance, "SET (a,b,c) = row_valued_function(x)" could be written "SET (a,b,c) = (SELECT * FROM row_valued_function(x))".
11 years ago
* are no MULTIEXPR subqueries then we don't need to replace
* PARAM_MULTIEXPR Params, and if there are no placeholders anywhere
* we won't need to remove them. Then it's OK to just scribble on the
* input node tree instead of copying (since the only change, filling
* in any unset opfuncid fields, is harmless). This saves just enough
* cycles to be noticeable on trivial queries.
Change fix_scan_expr() to avoid copying the input node tree in the common case where rtoffset == 0. In that case there is no need to change Var nodes, and since filling in unset opfuncid fields is always safe, scribbling on the input tree to that extent is not objectionable. This brings the cost of this operation back down to what it was in 8.2 for simple queries. Per investigation of performance gripe from Guillaume Smet.
18 years ago
*/
(void) fix_scan_expr_walker(node, &context);
return node;
}
Teach the planner to remove SubqueryScan nodes from the plan if they aren't doing anything useful (ie, neither selection nor projection). Also, extend to SubqueryScan the hacks already in place to avoid unnecessary ExecProject calls when the result would just be the same tuple the subquery already delivered. This saves some overhead in UNION and other set operations, as well as avoiding overhead for unflatten-able subqueries. Per example from Sokolov Yura.
21 years ago
}
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
static Node *
Re-run pgindent with updated list of typedefs. (Updated README should avoid this problem in the future.)
18 years ago
fix_scan_expr_mutator(Node *node, fix_scan_expr_context *context)
Teach the planner to remove SubqueryScan nodes from the plan if they aren't doing anything useful (ie, neither selection nor projection). Also, extend to SubqueryScan the hacks already in place to avoid unnecessary ExecProject calls when the result would just be the same tuple the subquery already delivered. This saves some overhead in UNION and other set operations, as well as avoiding overhead for unflatten-able subqueries. Per example from Sokolov Yura.
21 years ago
{
if (node == NULL)
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
return NULL;
Teach the planner to remove SubqueryScan nodes from the plan if they aren't doing anything useful (ie, neither selection nor projection). Also, extend to SubqueryScan the hacks already in place to avoid unnecessary ExecProject calls when the result would just be the same tuple the subquery already delivered. This saves some overhead in UNION and other set operations, as well as avoiding overhead for unflatten-able subqueries. Per example from Sokolov Yura.
21 years ago
if (IsA(node, Var))
{
Marginal performance hack: use a dedicated routine instead of copyObject to copy nodes that are known to be Vars during plan reference adjustment. Saves useless memzero operation as well as the big switch in copyObject.
19 years ago
Var *var = copyVar((Var *) node);
Teach the planner to remove SubqueryScan nodes from the plan if they aren't doing anything useful (ie, neither selection nor projection). Also, extend to SubqueryScan the hacks already in place to avoid unnecessary ExecProject calls when the result would just be the same tuple the subquery already delivered. This saves some overhead in UNION and other set operations, as well as avoiding overhead for unflatten-able subqueries. Per example from Sokolov Yura.
21 years ago
Assert(var->varlevelsup == 0);
pgindent run for 8.3.
18 years ago
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
/*
Rearrange the implementation of index-only scans. This commit changes index-only scans so that data is read directly from the index tuple without first generating a faux heap tuple. The only immediate benefit is that indexes on system columns (such as OID) can be used in index-only scans, but this is necessary infrastructure if we are ever to support index-only scans on expression indexes. The executor is now ready for that, though the planner still needs substantial work to recognize the possibility. To do this, Vars in index-only plan nodes have to refer to index columns not heap columns. I introduced a new special varno, INDEX_VAR, to mark such Vars to avoid confusion. (In passing, this commit renames the two existing special varnos to OUTER_VAR and INNER_VAR.) This allows ruleutils.c to handle them with logic similar to what we use for subplan reference Vars. Since index-only scans are now fundamentally different from regular indexscans so far as their expression subtrees are concerned, I also chose to change them to have their own plan node type (and hence, their own executor source file).
14 years ago
* We should not see any Vars marked INNER_VAR or OUTER_VAR. But an
* indexqual expression could contain INDEX_VAR Vars.
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
*/
Rearrange the implementation of index-only scans. This commit changes index-only scans so that data is read directly from the index tuple without first generating a faux heap tuple. The only immediate benefit is that indexes on system columns (such as OID) can be used in index-only scans, but this is necessary infrastructure if we are ever to support index-only scans on expression indexes. The executor is now ready for that, though the planner still needs substantial work to recognize the possibility. To do this, Vars in index-only plan nodes have to refer to index columns not heap columns. I introduced a new special varno, INDEX_VAR, to mark such Vars to avoid confusion. (In passing, this commit renames the two existing special varnos to OUTER_VAR and INNER_VAR.) This allows ruleutils.c to handle them with logic similar to what we use for subplan reference Vars. Since index-only scans are now fundamentally different from regular indexscans so far as their expression subtrees are concerned, I also chose to change them to have their own plan node type (and hence, their own executor source file).
14 years ago
Assert(var->varno != INNER_VAR);
Assert(var->varno != OUTER_VAR);
if (!IS_SPECIAL_VARNO(var->varno))
var->varno += context->rtoffset;
Teach the planner to remove SubqueryScan nodes from the plan if they aren't doing anything useful (ie, neither selection nor projection). Also, extend to SubqueryScan the hacks already in place to avoid unnecessary ExecProject calls when the result would just be the same tuple the subquery already delivered. This saves some overhead in UNION and other set operations, as well as avoiding overhead for unflatten-able subqueries. Per example from Sokolov Yura.
21 years ago
if (var->varnoold > 0)
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
var->varnoold += context->rtoffset;
return (Node *) var;
Further planner/optimizer cleanups. Move all set_tlist_references and fix_opids processing to a single recursive pass over the plan tree executed at the very tail end of planning, rather than haphazardly here and there at different places. Now that tlist Vars do not get modified until the very end, it's possible to get rid of the klugy var_equal and match_varid partial-matching routines, and just use plain equal() throughout the optimizer. This is a step towards allowing merge and hash joins to be done on expressions instead of only Vars ...
26 years ago
}
Implement UPDATE tab SET (col1,col2,...) = (SELECT ...), ... This SQL-standard feature allows a sub-SELECT yielding multiple columns (but only one row) to be used to compute the new values of several columns to be updated. While the same results can be had with an independent sub-SELECT per column, such a workaround can require a great deal of duplicated computation. The standard actually says that the source for a multi-column assignment could be any row-valued expression. The implementation used here is tightly tied to our existing sub-SELECT support and can't handle other cases; the Bison grammar would have some issues with them too. However, I don't feel too bad about this since other cases can be converted into sub-SELECTs. For instance, "SET (a,b,c) = row_valued_function(x)" could be written "SET (a,b,c) = (SELECT * FROM row_valued_function(x))".
11 years ago
if (IsA(node, Param))
return fix_param_node(context->root, (Param *) node);
Support UPDATE/DELETE WHERE CURRENT OF cursor_name, per SQL standard. Along the way, allow FOR UPDATE in non-WITH-HOLD cursors; there may once have been a reason to disallow that, but it seems to work now, and it's really rather necessary if you want to select a row via a cursor and then update it in a concurrent-safe fashion. Original patch by Arul Shaji, rather heavily editorialized by Tom Lane.
19 years ago
if (IsA(node, CurrentOfExpr))
{
CurrentOfExpr *cexpr = (CurrentOfExpr *) copyObject(node);
Rearrange the implementation of index-only scans. This commit changes index-only scans so that data is read directly from the index tuple without first generating a faux heap tuple. The only immediate benefit is that indexes on system columns (such as OID) can be used in index-only scans, but this is necessary infrastructure if we are ever to support index-only scans on expression indexes. The executor is now ready for that, though the planner still needs substantial work to recognize the possibility. To do this, Vars in index-only plan nodes have to refer to index columns not heap columns. I introduced a new special varno, INDEX_VAR, to mark such Vars to avoid confusion. (In passing, this commit renames the two existing special varnos to OUTER_VAR and INNER_VAR.) This allows ruleutils.c to handle them with logic similar to what we use for subplan reference Vars. Since index-only scans are now fundamentally different from regular indexscans so far as their expression subtrees are concerned, I also chose to change them to have their own plan node type (and hence, their own executor source file).
14 years ago
Assert(cexpr->cvarno != INNER_VAR);
Assert(cexpr->cvarno != OUTER_VAR);
if (!IS_SPECIAL_VARNO(cexpr->cvarno))
cexpr->cvarno += context->rtoffset;
Support UPDATE/DELETE WHERE CURRENT OF cursor_name, per SQL standard. Along the way, allow FOR UPDATE in non-WITH-HOLD cursors; there may once have been a reason to disallow that, but it seems to work now, and it's really rather necessary if you want to select a row via a cursor and then update it in a concurrent-safe fashion. Original patch by Arul Shaji, rather heavily editorialized by Tom Lane.
19 years ago
return (Node *) cexpr;
}
Add a concept of "placeholder" variables to the planner. These are variables that represent some expression that we desire to compute below the top level of the plan, and then let that value "bubble up" as though it were a plain Var (ie, a column value). The immediate application is to allow sub-selects to be flattened even when they are below an outer join and have non-nullable output expressions. Formerly we couldn't flatten because such an expression wouldn't properly go to NULL when evaluated above the outer join. Now, we wrap it in a PlaceHolderVar and arrange for the actual evaluation to occur below the outer join. When the resulting Var bubbles up through the join, it will be set to NULL if necessary, yielding the correct results. This fixes a planner limitation that's existed since 7.1. In future we might want to use this mechanism to re-introduce some form of Hellerstein's "expensive functions" optimization, ie place the evaluation of an expensive function at the most suitable point in the plan tree.
17 years ago
if (IsA(node, PlaceHolderVar))
{
/* At scan level, we should always just evaluate the contained expr */
PlaceHolderVar *phv = (PlaceHolderVar *) node;
return fix_scan_expr_mutator((Node *) phv->phexpr, context);
}
Rearrange planner to save the whole PlannerInfo (subroot) for a subquery. Formerly, set_subquery_pathlist and other creators of plans for subqueries saved only the rangetable and rowMarks lists from the lower-level PlannerInfo. But there's no reason not to remember the whole PlannerInfo, and indeed this turns out to simplify matters in a number of places. The immediate reason for doing this was so that the subroot will still be accessible when we're trying to extract column statistics out of an already-planned subquery. But now that I've done it, it seems like a good code-beautification effort in its own right. I also chose to get rid of the transient subrtable and subrowmark fields in SubqueryScan nodes, in favor of having setrefs.c look up the subquery's RelOptInfo. That required changing all the APIs in setrefs.c to pass PlannerInfo not PlannerGlobal, which was a large but quite mechanical transformation. One side-effect not foreseen at the beginning is that this finally broke inheritance_planner's assumption that replanning the same subquery RTE N times would necessarily give interchangeable results each time. That assumption was always pretty risky, but now we really have to make a separate RTE for each instance so that there's a place to carry the separate subroots.
14 years ago
fix_expr_common(context->root, node);
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
return expression_tree_mutator(node, fix_scan_expr_mutator,
(void *) context);
Repair bug reported by ldm@apartia.com: Append nodes, which don't actually use their targetlist, are given a targetlist that is just a pointer to the first appended plan's targetlist. This is OK, but what is not OK is that any sub-select expressions in said tlist were being entered in the subPlan lists of both the Append and the first appended plan. That led to two startup and two shutdown calls for the same plan node at exec time, which led to crashes. Fix is to not generate a list of subPlans for an Append node. Same problem and fix apply to other node types that don't have a real, functioning targetlist: Material, Sort, Unique, Hash.
26 years ago
}
Change fix_scan_expr() to avoid copying the input node tree in the common case where rtoffset == 0. In that case there is no need to change Var nodes, and since filling in unset opfuncid fields is always safe, scribbling on the input tree to that extent is not objectionable. This brings the cost of this operation back down to what it was in 8.2 for simple queries. Per investigation of performance gripe from Guillaume Smet.
18 years ago
static bool
fix_scan_expr_walker(Node *node, fix_scan_expr_context *context)
{
if (node == NULL)
return false;
Add a concept of "placeholder" variables to the planner. These are variables that represent some expression that we desire to compute below the top level of the plan, and then let that value "bubble up" as though it were a plain Var (ie, a column value). The immediate application is to allow sub-selects to be flattened even when they are below an outer join and have non-nullable output expressions. Formerly we couldn't flatten because such an expression wouldn't properly go to NULL when evaluated above the outer join. Now, we wrap it in a PlaceHolderVar and arrange for the actual evaluation to occur below the outer join. When the resulting Var bubbles up through the join, it will be set to NULL if necessary, yielding the correct results. This fixes a planner limitation that's existed since 7.1. In future we might want to use this mechanism to re-introduce some form of Hellerstein's "expensive functions" optimization, ie place the evaluation of an expensive function at the most suitable point in the plan tree.
17 years ago
Assert(!IsA(node, PlaceHolderVar));
Rearrange planner to save the whole PlannerInfo (subroot) for a subquery. Formerly, set_subquery_pathlist and other creators of plans for subqueries saved only the rangetable and rowMarks lists from the lower-level PlannerInfo. But there's no reason not to remember the whole PlannerInfo, and indeed this turns out to simplify matters in a number of places. The immediate reason for doing this was so that the subroot will still be accessible when we're trying to extract column statistics out of an already-planned subquery. But now that I've done it, it seems like a good code-beautification effort in its own right. I also chose to get rid of the transient subrtable and subrowmark fields in SubqueryScan nodes, in favor of having setrefs.c look up the subquery's RelOptInfo. That required changing all the APIs in setrefs.c to pass PlannerInfo not PlannerGlobal, which was a large but quite mechanical transformation. One side-effect not foreseen at the beginning is that this finally broke inheritance_planner's assumption that replanning the same subquery RTE N times would necessarily give interchangeable results each time. That assumption was always pretty risky, but now we really have to make a separate RTE for each instance so that there's a place to carry the separate subroots.
14 years ago
fix_expr_common(context->root, node);
Change fix_scan_expr() to avoid copying the input node tree in the common case where rtoffset == 0. In that case there is no need to change Var nodes, and since filling in unset opfuncid fields is always safe, scribbling on the input tree to that extent is not objectionable. This brings the cost of this operation back down to what it was in 8.2 for simple queries. Per investigation of performance gripe from Guillaume Smet.
18 years ago
return expression_tree_walker(node, fix_scan_expr_walker,
(void *) context);
}
Massive commit to run PGINDENT on all *.c and *.h files.
28 years ago
/*
Further planner/optimizer cleanups. Move all set_tlist_references and fix_opids processing to a single recursive pass over the plan tree executed at the very tail end of planning, rather than haphazardly here and there at different places. Now that tlist Vars do not get modified until the very end, it's possible to get rid of the klugy var_equal and match_varid partial-matching routines, and just use plain equal() throughout the optimizer. This is a step towards allowing merge and hash joins to be done on expressions instead of only Vars ...
26 years ago
* set_join_references
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
* Modify the target list and quals of a join node to reference its
Rearrange the implementation of index-only scans. This commit changes index-only scans so that data is read directly from the index tuple without first generating a faux heap tuple. The only immediate benefit is that indexes on system columns (such as OID) can be used in index-only scans, but this is necessary infrastructure if we are ever to support index-only scans on expression indexes. The executor is now ready for that, though the planner still needs substantial work to recognize the possibility. To do this, Vars in index-only plan nodes have to refer to index columns not heap columns. I introduced a new special varno, INDEX_VAR, to mark such Vars to avoid confusion. (In passing, this commit renames the two existing special varnos to OUTER_VAR and INNER_VAR.) This allows ruleutils.c to handle them with logic similar to what we use for subplan reference Vars. Since index-only scans are now fundamentally different from regular indexscans so far as their expression subtrees are concerned, I also chose to change them to have their own plan node type (and hence, their own executor source file).
14 years ago
* subplans, by setting the varnos to OUTER_VAR or INNER_VAR and setting
* attno values to the result domain number of either the corresponding
* outer or inner join tuple item. Also perform opcode lookup for these
Rearrange planner to save the whole PlannerInfo (subroot) for a subquery. Formerly, set_subquery_pathlist and other creators of plans for subqueries saved only the rangetable and rowMarks lists from the lower-level PlannerInfo. But there's no reason not to remember the whole PlannerInfo, and indeed this turns out to simplify matters in a number of places. The immediate reason for doing this was so that the subroot will still be accessible when we're trying to extract column statistics out of an already-planned subquery. But now that I've done it, it seems like a good code-beautification effort in its own right. I also chose to get rid of the transient subrtable and subrowmark fields in SubqueryScan nodes, in favor of having setrefs.c look up the subquery's RelOptInfo. That required changing all the APIs in setrefs.c to pass PlannerInfo not PlannerGlobal, which was a large but quite mechanical transformation. One side-effect not foreseen at the beginning is that this finally broke inheritance_planner's assumption that replanning the same subquery RTE N times would necessarily give interchangeable results each time. That assumption was always pretty risky, but now we really have to make a separate RTE for each instance so that there's a place to carry the separate subroots.
14 years ago
* expressions. and add regclass OIDs to root->glob->relationOids.
Postgres95 1.01 Distribution - Virgin Sources
30 years ago
*/
static void
Rearrange planner to save the whole PlannerInfo (subroot) for a subquery. Formerly, set_subquery_pathlist and other creators of plans for subqueries saved only the rangetable and rowMarks lists from the lower-level PlannerInfo. But there's no reason not to remember the whole PlannerInfo, and indeed this turns out to simplify matters in a number of places. The immediate reason for doing this was so that the subroot will still be accessible when we're trying to extract column statistics out of an already-planned subquery. But now that I've done it, it seems like a good code-beautification effort in its own right. I also chose to get rid of the transient subrtable and subrowmark fields in SubqueryScan nodes, in favor of having setrefs.c look up the subquery's RelOptInfo. That required changing all the APIs in setrefs.c to pass PlannerInfo not PlannerGlobal, which was a large but quite mechanical transformation. One side-effect not foreseen at the beginning is that this finally broke inheritance_planner's assumption that replanning the same subquery RTE N times would necessarily give interchangeable results each time. That assumption was always pretty risky, but now we really have to make a separate RTE for each instance so that there's a place to carry the separate subroots.
14 years ago
set_join_references(PlannerInfo *root, Join *join, int rtoffset)
Postgres95 1.01 Distribution - Virgin Sources
30 years ago
{
Now that switch_outer processing no longer relies on being run after join_references(), it's practical to consolidate all join_references() processing into the set_plan_references traversal in setrefs.c. This seems considerably cleaner than the old way where we did it for join quals in createplan.c and for targetlists in setrefs.c.
23 years ago
Plan *outer_plan = join->plan.lefttree;
Plan *inner_plan = join->plan.righttree;
Revise searching of subplan target lists to use something more efficient than tlist_member calls. Building a large join tlist is still O(N^2), but with a much smaller constant factor than before.
21 years ago
indexed_tlist *outer_itlist;
indexed_tlist *inner_itlist;
IN clauses appearing at top level of WHERE can now be handled as joins. There are two implementation techniques: the executor understands a new JOIN_IN jointype, which emits at most one matching row per left-hand row, or the result of the IN's sub-select can be fed through a DISTINCT filter and then joined as an ordinary relation. Along the way, some minor code cleanup in the optimizer; notably, break out most of the jointree-rearrangement preprocessing in planner.c and put it in a new file prep/prepjointree.c.
23 years ago
Revise searching of subplan target lists to use something more efficient than tlist_member calls. Building a large join tlist is still O(N^2), but with a much smaller constant factor than before.
21 years ago
outer_itlist = build_tlist_index(outer_plan->targetlist);
inner_itlist = build_tlist_index(inner_plan->targetlist);
Postgres95 1.01 Distribution - Virgin Sources
30 years ago
Fix incorrect matching of subexpressions in outer-join plan nodes. Previously we would re-use input subexpressions in all expression trees attached to a Join plan node. However, if it's an outer join and the subexpression appears in the nullable-side input, this is potentially incorrect for apparently-matching subexpressions that came from above the outer join (ie, targetlist and qpqual expressions), because the executor will treat the subexpression value as NULL when maybe it should not be. The case is fairly hard to hit because (a) you need a non-strict subexpression (else NULL is correct), and (b) we don't usually compute expressions in the outputs of non-toplevel plan nodes. But we might do so if the expressions are sort keys for a mergejoin, for example. Probably in the long run we should make a more explicit distinction between Vars appearing above and below an outer join, but that will be a major planner redesign and not at all back-patchable. For the moment, just hack set_join_references so that it will not match any non-Var expressions coming from nullable inputs to expressions that came from above the join. (This is somewhat overkill, in that a strict expression could still be matched, but it doesn't seem worth the effort to check that.) Per report from Qingqing Zhou. The added regression test case is based on his example. This has been broken for a very long time, so back-patch to all active branches.
11 years ago
/*
* First process the joinquals (including merge or hash clauses). These
* are logically below the join so they can always use all values
* available from the input tlists. It's okay to also handle
* NestLoopParams now, because those couldn't refer to nullable
* subexpressions.
*/
Rearrange planner to save the whole PlannerInfo (subroot) for a subquery. Formerly, set_subquery_pathlist and other creators of plans for subqueries saved only the rangetable and rowMarks lists from the lower-level PlannerInfo. But there's no reason not to remember the whole PlannerInfo, and indeed this turns out to simplify matters in a number of places. The immediate reason for doing this was so that the subroot will still be accessible when we're trying to extract column statistics out of an already-planned subquery. But now that I've done it, it seems like a good code-beautification effort in its own right. I also chose to get rid of the transient subrtable and subrowmark fields in SubqueryScan nodes, in favor of having setrefs.c look up the subquery's RelOptInfo. That required changing all the APIs in setrefs.c to pass PlannerInfo not PlannerGlobal, which was a large but quite mechanical transformation. One side-effect not foreseen at the beginning is that this finally broke inheritance_planner's assumption that replanning the same subquery RTE N times would necessarily give interchangeable results each time. That assumption was always pretty risky, but now we really have to make a separate RTE for each instance so that there's a place to carry the separate subroots.
14 years ago
join->joinqual = fix_join_expr(root,
Fix the plan-invalidation mechanism to treat regclass constants that refer to a relation as a reason to invalidate a plan when the relation changes. This handles scenarios such as dropping/recreating a sequence that is referenced by nextval('seq') in a cached plan. Rather than teach plancache.c all about digging through plan trees to find regclass Consts, we charge the planner's setrefs.c with making a list of the relation OIDs on which each plan depends. That way the list can be built cheaply during a plan tree traversal that has to happen anyway. Per bug #3662 and subsequent discussion.
18 years ago
join->joinqual,
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
outer_itlist,
inner_itlist,
(Index) 0,
rtoffset);
Now that switch_outer processing no longer relies on being run after join_references(), it's practical to consolidate all join_references() processing into the set_plan_references traversal in setrefs.c. This seems considerably cleaner than the old way where we did it for join quals in createplan.c and for targetlists in setrefs.c.
23 years ago
/* Now do join-type-specific stuff */
if (IsA(join, NestLoop))
{
Make NestLoop plan nodes pass outer-relation variables into their inner relation using the general PARAM_EXEC executor parameter mechanism, rather than the ad-hoc kluge of passing the outer tuple down through ExecReScan. The previous method was hard to understand and could never be extended to handle parameters coming from multiple join levels. This patch doesn't change the set of possible plans nor have any significant performance effect, but it's necessary infrastructure for future generalization of the concept of an inner indexscan plan. ExecReScan's second parameter is now unused, so it's removed.
15 years ago
NestLoop *nl = (NestLoop *) join;
ListCell *lc;
foreach(lc, nl->nestParams)
{
NestLoopParam *nlp = (NestLoopParam *) lfirst(lc);
Rearrange planner to save the whole PlannerInfo (subroot) for a subquery. Formerly, set_subquery_pathlist and other creators of plans for subqueries saved only the rangetable and rowMarks lists from the lower-level PlannerInfo. But there's no reason not to remember the whole PlannerInfo, and indeed this turns out to simplify matters in a number of places. The immediate reason for doing this was so that the subroot will still be accessible when we're trying to extract column statistics out of an already-planned subquery. But now that I've done it, it seems like a good code-beautification effort in its own right. I also chose to get rid of the transient subrtable and subrowmark fields in SubqueryScan nodes, in favor of having setrefs.c look up the subquery's RelOptInfo. That required changing all the APIs in setrefs.c to pass PlannerInfo not PlannerGlobal, which was a large but quite mechanical transformation. One side-effect not foreseen at the beginning is that this finally broke inheritance_planner's assumption that replanning the same subquery RTE N times would necessarily give interchangeable results each time. That assumption was always pretty risky, but now we really have to make a separate RTE for each instance so that there's a place to carry the separate subroots.
14 years ago
nlp->paramval = (Var *) fix_upper_expr(root,
Make NestLoop plan nodes pass outer-relation variables into their inner relation using the general PARAM_EXEC executor parameter mechanism, rather than the ad-hoc kluge of passing the outer tuple down through ExecReScan. The previous method was hard to understand and could never be extended to handle parameters coming from multiple join levels. This patch doesn't change the set of possible plans nor have any significant performance effect, but it's necessary infrastructure for future generalization of the concept of an inner indexscan plan. ExecReScan's second parameter is now unused, so it's removed.
15 years ago
(Node *) nlp->paramval,
outer_itlist,
Rearrange the implementation of index-only scans. This commit changes index-only scans so that data is read directly from the index tuple without first generating a faux heap tuple. The only immediate benefit is that indexes on system columns (such as OID) can be used in index-only scans, but this is necessary infrastructure if we are ever to support index-only scans on expression indexes. The executor is now ready for that, though the planner still needs substantial work to recognize the possibility. To do this, Vars in index-only plan nodes have to refer to index columns not heap columns. I introduced a new special varno, INDEX_VAR, to mark such Vars to avoid confusion. (In passing, this commit renames the two existing special varnos to OUTER_VAR and INNER_VAR.) This allows ruleutils.c to handle them with logic similar to what we use for subplan reference Vars. Since index-only scans are now fundamentally different from regular indexscans so far as their expression subtrees are concerned, I also chose to change them to have their own plan node type (and hence, their own executor source file).
14 years ago
OUTER_VAR,
Make NestLoop plan nodes pass outer-relation variables into their inner relation using the general PARAM_EXEC executor parameter mechanism, rather than the ad-hoc kluge of passing the outer tuple down through ExecReScan. The previous method was hard to understand and could never be extended to handle parameters coming from multiple join levels. This patch doesn't change the set of possible plans nor have any significant performance effect, but it's necessary infrastructure for future generalization of the concept of an inner indexscan plan. ExecReScan's second parameter is now unused, so it's removed.
15 years ago
rtoffset);
Fix handling of PlaceHolderVars in nestloop parameter management. If we use a PlaceHolderVar from the outer relation in an inner indexscan, we need to reference the PlaceHolderVar as such as the value to be passed in from the outer relation. The previous code effectively tried to reconstruct the PHV from its component expression, which doesn't work since (a) the Vars therein aren't necessarily bubbled up far enough, and (b) it would be the wrong semantics anyway because of the possibility that the PHV is supposed to have gone to null at some point before the current join. Point (a) led to "variable not found in subplan target list" planner errors, but point (b) would have led to silently wrong answers. Per report from Roger Niederland.
14 years ago
/* Check we replaced any PlaceHolderVar with simple Var */
if (!(IsA(nlp->paramval, Var) &&
nlp->paramval->varno == OUTER_VAR))
elog(ERROR, "NestLoopParam was not reduced to a simple Var");
Make NestLoop plan nodes pass outer-relation variables into their inner relation using the general PARAM_EXEC executor parameter mechanism, rather than the ad-hoc kluge of passing the outer tuple down through ExecReScan. The previous method was hard to understand and could never be extended to handle parameters coming from multiple join levels. This patch doesn't change the set of possible plans nor have any significant performance effect, but it's necessary infrastructure for future generalization of the concept of an inner indexscan plan. ExecReScan's second parameter is now unused, so it's removed.
15 years ago
}
Now that switch_outer processing no longer relies on being run after join_references(), it's practical to consolidate all join_references() processing into the set_plan_references traversal in setrefs.c. This seems considerably cleaner than the old way where we did it for join quals in createplan.c and for targetlists in setrefs.c.
23 years ago
}
else if (IsA(join, MergeJoin))
{
MergeJoin *mj = (MergeJoin *) join;
Rearrange planner to save the whole PlannerInfo (subroot) for a subquery. Formerly, set_subquery_pathlist and other creators of plans for subqueries saved only the rangetable and rowMarks lists from the lower-level PlannerInfo. But there's no reason not to remember the whole PlannerInfo, and indeed this turns out to simplify matters in a number of places. The immediate reason for doing this was so that the subroot will still be accessible when we're trying to extract column statistics out of an already-planned subquery. But now that I've done it, it seems like a good code-beautification effort in its own right. I also chose to get rid of the transient subrtable and subrowmark fields in SubqueryScan nodes, in favor of having setrefs.c look up the subquery's RelOptInfo. That required changing all the APIs in setrefs.c to pass PlannerInfo not PlannerGlobal, which was a large but quite mechanical transformation. One side-effect not foreseen at the beginning is that this finally broke inheritance_planner's assumption that replanning the same subquery RTE N times would necessarily give interchangeable results each time. That assumption was always pretty risky, but now we really have to make a separate RTE for each instance so that there's a place to carry the separate subroots.
14 years ago
mj->mergeclauses = fix_join_expr(root,
Fix the plan-invalidation mechanism to treat regclass constants that refer to a relation as a reason to invalidate a plan when the relation changes. This handles scenarios such as dropping/recreating a sequence that is referenced by nextval('seq') in a cached plan. Rather than teach plancache.c all about digging through plan trees to find regclass Consts, we charge the planner's setrefs.c with making a list of the relation OIDs on which each plan depends. That way the list can be built cheaply during a plan tree traversal that has to happen anyway. Per bug #3662 and subsequent discussion.
18 years ago
mj->mergeclauses,
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
outer_itlist,
inner_itlist,
(Index) 0,
rtoffset);
Now that switch_outer processing no longer relies on being run after join_references(), it's practical to consolidate all join_references() processing into the set_plan_references traversal in setrefs.c. This seems considerably cleaner than the old way where we did it for join quals in createplan.c and for targetlists in setrefs.c.
23 years ago
}
else if (IsA(join, HashJoin))
{
HashJoin *hj = (HashJoin *) join;
Rearrange planner to save the whole PlannerInfo (subroot) for a subquery. Formerly, set_subquery_pathlist and other creators of plans for subqueries saved only the rangetable and rowMarks lists from the lower-level PlannerInfo. But there's no reason not to remember the whole PlannerInfo, and indeed this turns out to simplify matters in a number of places. The immediate reason for doing this was so that the subroot will still be accessible when we're trying to extract column statistics out of an already-planned subquery. But now that I've done it, it seems like a good code-beautification effort in its own right. I also chose to get rid of the transient subrtable and subrowmark fields in SubqueryScan nodes, in favor of having setrefs.c look up the subquery's RelOptInfo. That required changing all the APIs in setrefs.c to pass PlannerInfo not PlannerGlobal, which was a large but quite mechanical transformation. One side-effect not foreseen at the beginning is that this finally broke inheritance_planner's assumption that replanning the same subquery RTE N times would necessarily give interchangeable results each time. That assumption was always pretty risky, but now we really have to make a separate RTE for each instance so that there's a place to carry the separate subroots.
14 years ago
hj->hashclauses = fix_join_expr(root,
Fix the plan-invalidation mechanism to treat regclass constants that refer to a relation as a reason to invalidate a plan when the relation changes. This handles scenarios such as dropping/recreating a sequence that is referenced by nextval('seq') in a cached plan. Rather than teach plancache.c all about digging through plan trees to find regclass Consts, we charge the planner's setrefs.c with making a list of the relation OIDs on which each plan depends. That way the list can be built cheaply during a plan tree traversal that has to happen anyway. Per bug #3662 and subsequent discussion.
18 years ago
hj->hashclauses,
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
outer_itlist,
inner_itlist,
(Index) 0,
rtoffset);
Now that switch_outer processing no longer relies on being run after join_references(), it's practical to consolidate all join_references() processing into the set_plan_references traversal in setrefs.c. This seems considerably cleaner than the old way where we did it for join quals in createplan.c and for targetlists in setrefs.c.
23 years ago
}
Revise searching of subplan target lists to use something more efficient than tlist_member calls. Building a large join tlist is still O(N^2), but with a much smaller constant factor than before.
21 years ago
Fix incorrect matching of subexpressions in outer-join plan nodes. Previously we would re-use input subexpressions in all expression trees attached to a Join plan node. However, if it's an outer join and the subexpression appears in the nullable-side input, this is potentially incorrect for apparently-matching subexpressions that came from above the outer join (ie, targetlist and qpqual expressions), because the executor will treat the subexpression value as NULL when maybe it should not be. The case is fairly hard to hit because (a) you need a non-strict subexpression (else NULL is correct), and (b) we don't usually compute expressions in the outputs of non-toplevel plan nodes. But we might do so if the expressions are sort keys for a mergejoin, for example. Probably in the long run we should make a more explicit distinction between Vars appearing above and below an outer join, but that will be a major planner redesign and not at all back-patchable. For the moment, just hack set_join_references so that it will not match any non-Var expressions coming from nullable inputs to expressions that came from above the join. (This is somewhat overkill, in that a strict expression could still be matched, but it doesn't seem worth the effort to check that.) Per report from Qingqing Zhou. The added regression test case is based on his example. This has been broken for a very long time, so back-patch to all active branches.
11 years ago
/*
* Now we need to fix up the targetlist and qpqual, which are logically
* above the join. This means they should not re-use any input expression
* that was computed in the nullable side of an outer join. Vars and
* PlaceHolderVars are fine, so we can implement this restriction just by
* clearing has_non_vars in the indexed_tlist structs.
*
* XXX This is a grotty workaround for the fact that we don't clearly
* distinguish between a Var appearing below an outer join and the "same"
* Var appearing above it. If we did, we'd not need to hack the matching
* rules this way.
*/
switch (join->jointype)
{
case JOIN_LEFT:
case JOIN_SEMI:
case JOIN_ANTI:
inner_itlist->has_non_vars = false;
break;
case JOIN_RIGHT:
outer_itlist->has_non_vars = false;
break;
case JOIN_FULL:
outer_itlist->has_non_vars = false;
inner_itlist->has_non_vars = false;
break;
default:
break;
}
join->plan.targetlist = fix_join_expr(root,
join->plan.targetlist,
outer_itlist,
inner_itlist,
(Index) 0,
rtoffset);
join->plan.qual = fix_join_expr(root,
join->plan.qual,
outer_itlist,
inner_itlist,
(Index) 0,
rtoffset);
Revise searching of subplan target lists to use something more efficient than tlist_member calls. Building a large join tlist is still O(N^2), but with a much smaller constant factor than before.
21 years ago
pfree(outer_itlist);
pfree(inner_itlist);
Postgres95 1.01 Distribution - Virgin Sources
30 years ago
}
Massive commit to run PGINDENT on all *.c and *.h files.
28 years ago
/*
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
* set_upper_references
Further planner/optimizer cleanups. Move all set_tlist_references and fix_opids processing to a single recursive pass over the plan tree executed at the very tail end of planning, rather than haphazardly here and there at different places. Now that tlist Vars do not get modified until the very end, it's possible to get rid of the klugy var_equal and match_varid partial-matching routines, and just use plain equal() throughout the optimizer. This is a step towards allowing merge and hash joins to be done on expressions instead of only Vars ...
26 years ago
* Update the targetlist and quals of an upper-level plan node
* to refer to the tuples returned by its lefttree subplan.
Fix the plan-invalidation mechanism to treat regclass constants that refer to a relation as a reason to invalidate a plan when the relation changes. This handles scenarios such as dropping/recreating a sequence that is referenced by nextval('seq') in a cached plan. Rather than teach plancache.c all about digging through plan trees to find regclass Consts, we charge the planner's setrefs.c with making a list of the relation OIDs on which each plan depends. That way the list can be built cheaply during a plan tree traversal that has to happen anyway. Per bug #3662 and subsequent discussion.
18 years ago
* Also perform opcode lookup for these expressions, and
Rearrange planner to save the whole PlannerInfo (subroot) for a subquery. Formerly, set_subquery_pathlist and other creators of plans for subqueries saved only the rangetable and rowMarks lists from the lower-level PlannerInfo. But there's no reason not to remember the whole PlannerInfo, and indeed this turns out to simplify matters in a number of places. The immediate reason for doing this was so that the subroot will still be accessible when we're trying to extract column statistics out of an already-planned subquery. But now that I've done it, it seems like a good code-beautification effort in its own right. I also chose to get rid of the transient subrtable and subrowmark fields in SubqueryScan nodes, in favor of having setrefs.c look up the subquery's RelOptInfo. That required changing all the APIs in setrefs.c to pass PlannerInfo not PlannerGlobal, which was a large but quite mechanical transformation. One side-effect not foreseen at the beginning is that this finally broke inheritance_planner's assumption that replanning the same subquery RTE N times would necessarily give interchangeable results each time. That assumption was always pretty risky, but now we really have to make a separate RTE for each instance so that there's a place to carry the separate subroots.
14 years ago
* add regclass OIDs to root->glob->relationOids.
Massive commit to run PGINDENT on all *.c and *.h files.
28 years ago
*
Further planner/optimizer cleanups. Move all set_tlist_references and fix_opids processing to a single recursive pass over the plan tree executed at the very tail end of planning, rather than haphazardly here and there at different places. Now that tlist Vars do not get modified until the very end, it's possible to get rid of the klugy var_equal and match_varid partial-matching routines, and just use plain equal() throughout the optimizer. This is a step towards allowing merge and hash joins to be done on expressions instead of only Vars ...
26 years ago
* This is used for single-input plan types like Agg, Group, Result.
Fix oversight in planning of GROUP queries: when an expression is used as both a GROUP BY item and an output expression, the top-level Group node should just copy up the evaluated expression value from its input, rather than re-evaluating the expression. Aside from any performance benefit this might offer, this avoids a crash when there is a sub-SELECT in said expression.
25 years ago
*
* In most cases, we have to match up individual Vars in the tlist and
* qual expressions with elements of the subplan's tlist (which was
* generated by flatten_tlist() from these selfsame expressions, so it
* should have all the required variables). There is an important exception,
Fix problems with subselects used in GROUP BY expressions, per gripe from Philip Warner. Side effect of change is that GROUP BY expressions will not be re-evaluated at multiple plan levels anymore, whereas this sometimes happened with old code.
24 years ago
* however: GROUP BY and ORDER BY expressions will have been pushed into the
* subplan tlist unflattened. If these values are also needed in the output
* then we want to reference the subplan tlist element rather than recomputing
* the expression.
Postgres95 1.01 Distribution - Virgin Sources
30 years ago
*/
static void
Rearrange planner to save the whole PlannerInfo (subroot) for a subquery. Formerly, set_subquery_pathlist and other creators of plans for subqueries saved only the rangetable and rowMarks lists from the lower-level PlannerInfo. But there's no reason not to remember the whole PlannerInfo, and indeed this turns out to simplify matters in a number of places. The immediate reason for doing this was so that the subroot will still be accessible when we're trying to extract column statistics out of an already-planned subquery. But now that I've done it, it seems like a good code-beautification effort in its own right. I also chose to get rid of the transient subrtable and subrowmark fields in SubqueryScan nodes, in favor of having setrefs.c look up the subquery's RelOptInfo. That required changing all the APIs in setrefs.c to pass PlannerInfo not PlannerGlobal, which was a large but quite mechanical transformation. One side-effect not foreseen at the beginning is that this finally broke inheritance_planner's assumption that replanning the same subquery RTE N times would necessarily give interchangeable results each time. That assumption was always pretty risky, but now we really have to make a separate RTE for each instance so that there's a place to carry the separate subroots.
14 years ago
set_upper_references(PlannerInfo *root, Plan *plan, int rtoffset)
Postgres95 1.01 Distribution - Virgin Sources
30 years ago
{
Further planner/optimizer cleanups. Move all set_tlist_references and fix_opids processing to a single recursive pass over the plan tree executed at the very tail end of planning, rather than haphazardly here and there at different places. Now that tlist Vars do not get modified until the very end, it's possible to get rid of the klugy var_equal and match_varid partial-matching routines, and just use plain equal() throughout the optimizer. This is a step towards allowing merge and hash joins to be done on expressions instead of only Vars ...
26 years ago
Plan *subplan = plan->lefttree;
Revise searching of subplan target lists to use something more efficient than tlist_member calls. Building a large join tlist is still O(N^2), but with a much smaller constant factor than before.
21 years ago
indexed_tlist *subplan_itlist;
List *output_targetlist;
Reimplement the linked list data structure used throughout the backend. In the past, we used a 'Lispy' linked list implementation: a "list" was merely a pointer to the head node of the list. The problem with that design is that it makes lappend() and length() linear time. This patch fixes that problem (and others) by maintaining a count of the list length and a pointer to the tail node along with each head node pointer. A "list" is now a pointer to a structure containing some meta-data about the list; the head and tail pointers in that structure refer to ListCell structures that maintain the actual linked list of nodes. The function names of the list API have also been changed to, I hope, be more logically consistent. By default, the old function names are still available; they will be disabled-by-default once the rest of the tree has been updated to use the new API names.
22 years ago
ListCell *l;
Massive commit to run PGINDENT on all *.c and *.h files.
28 years ago
Now that plans have flat rangetable lists, it's a lot easier to get EXPLAIN to drill down into subplan targetlists to print the referent expression for an OUTER or INNER var in an upper plan node. Hence, make it do that always, and banish the old hack of showing "?columnN?" when things got too complicated. Along the way, fix an EXPLAIN bug I introduced by suppressing subqueries from execution-time range tables: get_name_for_var_field() assumed it could look at rte->subquery to find out the real type of a RECORD var. That doesn't work anymore, but instead we can look at the input plan of the SubqueryScan plan node.
19 years ago
subplan_itlist = build_tlist_index(subplan->targetlist);
Fix problems with subselects used in GROUP BY expressions, per gripe from Philip Warner. Side effect of change is that GROUP BY expressions will not be re-evaluated at multiple plan levels anymore, whereas this sometimes happened with old code.
24 years ago
output_targetlist = NIL;
foreach(l, plan->targetlist)
{
TargetEntry *tle = (TargetEntry *) lfirst(l);
Node *newexpr;
While doing the final setrefs.c pass over a plan tree, try to match up non-Var sort/group expressions using ressortgroupref labels instead of depending entirely on equal()-ity of the upper node's tlist expressions to the lower node's. This avoids emitting the wrong outputs in cases where there are textually identical volatile sort/group expressions, as for example select distinct random(),random() from generate_series(1,10); Per report from Andrew Gierth. Backpatch to 8.4. Arguably this is wrong all the way back, but the only known case where there's an observable problem is when using hash aggregation to implement DISTINCT, which is new as of 8.4. So for the moment I'll refrain from backpatching further.
16 years ago
/* If it's a non-Var sort/group item, first try to match by sortref */
if (tle->ressortgroupref != 0 && !IsA(tle->expr, Var))
{
newexpr = (Node *)
search_indexed_tlist_for_sortgroupref((Node *) tle->expr,
tle->ressortgroupref,
subplan_itlist,
Rearrange the implementation of index-only scans. This commit changes index-only scans so that data is read directly from the index tuple without first generating a faux heap tuple. The only immediate benefit is that indexes on system columns (such as OID) can be used in index-only scans, but this is necessary infrastructure if we are ever to support index-only scans on expression indexes. The executor is now ready for that, though the planner still needs substantial work to recognize the possibility. To do this, Vars in index-only plan nodes have to refer to index columns not heap columns. I introduced a new special varno, INDEX_VAR, to mark such Vars to avoid confusion. (In passing, this commit renames the two existing special varnos to OUTER_VAR and INNER_VAR.) This allows ruleutils.c to handle them with logic similar to what we use for subplan reference Vars. Since index-only scans are now fundamentally different from regular indexscans so far as their expression subtrees are concerned, I also chose to change them to have their own plan node type (and hence, their own executor source file).
14 years ago
OUTER_VAR);
While doing the final setrefs.c pass over a plan tree, try to match up non-Var sort/group expressions using ressortgroupref labels instead of depending entirely on equal()-ity of the upper node's tlist expressions to the lower node's. This avoids emitting the wrong outputs in cases where there are textually identical volatile sort/group expressions, as for example select distinct random(),random() from generate_series(1,10); Per report from Andrew Gierth. Backpatch to 8.4. Arguably this is wrong all the way back, but the only known case where there's an observable problem is when using hash aggregation to implement DISTINCT, which is new as of 8.4. So for the moment I'll refrain from backpatching further.
16 years ago
if (!newexpr)
Rearrange planner to save the whole PlannerInfo (subroot) for a subquery. Formerly, set_subquery_pathlist and other creators of plans for subqueries saved only the rangetable and rowMarks lists from the lower-level PlannerInfo. But there's no reason not to remember the whole PlannerInfo, and indeed this turns out to simplify matters in a number of places. The immediate reason for doing this was so that the subroot will still be accessible when we're trying to extract column statistics out of an already-planned subquery. But now that I've done it, it seems like a good code-beautification effort in its own right. I also chose to get rid of the transient subrtable and subrowmark fields in SubqueryScan nodes, in favor of having setrefs.c look up the subquery's RelOptInfo. That required changing all the APIs in setrefs.c to pass PlannerInfo not PlannerGlobal, which was a large but quite mechanical transformation. One side-effect not foreseen at the beginning is that this finally broke inheritance_planner's assumption that replanning the same subquery RTE N times would necessarily give interchangeable results each time. That assumption was always pretty risky, but now we really have to make a separate RTE for each instance so that there's a place to carry the separate subroots.
14 years ago
newexpr = fix_upper_expr(root,
While doing the final setrefs.c pass over a plan tree, try to match up non-Var sort/group expressions using ressortgroupref labels instead of depending entirely on equal()-ity of the upper node's tlist expressions to the lower node's. This avoids emitting the wrong outputs in cases where there are textually identical volatile sort/group expressions, as for example select distinct random(),random() from generate_series(1,10); Per report from Andrew Gierth. Backpatch to 8.4. Arguably this is wrong all the way back, but the only known case where there's an observable problem is when using hash aggregation to implement DISTINCT, which is new as of 8.4. So for the moment I'll refrain from backpatching further.
16 years ago
(Node *) tle->expr,
subplan_itlist,
Rearrange the implementation of index-only scans. This commit changes index-only scans so that data is read directly from the index tuple without first generating a faux heap tuple. The only immediate benefit is that indexes on system columns (such as OID) can be used in index-only scans, but this is necessary infrastructure if we are ever to support index-only scans on expression indexes. The executor is now ready for that, though the planner still needs substantial work to recognize the possibility. To do this, Vars in index-only plan nodes have to refer to index columns not heap columns. I introduced a new special varno, INDEX_VAR, to mark such Vars to avoid confusion. (In passing, this commit renames the two existing special varnos to OUTER_VAR and INNER_VAR.) This allows ruleutils.c to handle them with logic similar to what we use for subplan reference Vars. Since index-only scans are now fundamentally different from regular indexscans so far as their expression subtrees are concerned, I also chose to change them to have their own plan node type (and hence, their own executor source file).
14 years ago
OUTER_VAR,
While doing the final setrefs.c pass over a plan tree, try to match up non-Var sort/group expressions using ressortgroupref labels instead of depending entirely on equal()-ity of the upper node's tlist expressions to the lower node's. This avoids emitting the wrong outputs in cases where there are textually identical volatile sort/group expressions, as for example select distinct random(),random() from generate_series(1,10); Per report from Andrew Gierth. Backpatch to 8.4. Arguably this is wrong all the way back, but the only known case where there's an observable problem is when using hash aggregation to implement DISTINCT, which is new as of 8.4. So for the moment I'll refrain from backpatching further.
16 years ago
rtoffset);
}
else
Rearrange planner to save the whole PlannerInfo (subroot) for a subquery. Formerly, set_subquery_pathlist and other creators of plans for subqueries saved only the rangetable and rowMarks lists from the lower-level PlannerInfo. But there's no reason not to remember the whole PlannerInfo, and indeed this turns out to simplify matters in a number of places. The immediate reason for doing this was so that the subroot will still be accessible when we're trying to extract column statistics out of an already-planned subquery. But now that I've done it, it seems like a good code-beautification effort in its own right. I also chose to get rid of the transient subrtable and subrowmark fields in SubqueryScan nodes, in favor of having setrefs.c look up the subquery's RelOptInfo. That required changing all the APIs in setrefs.c to pass PlannerInfo not PlannerGlobal, which was a large but quite mechanical transformation. One side-effect not foreseen at the beginning is that this finally broke inheritance_planner's assumption that replanning the same subquery RTE N times would necessarily give interchangeable results each time. That assumption was always pretty risky, but now we really have to make a separate RTE for each instance so that there's a place to carry the separate subroots.
14 years ago
newexpr = fix_upper_expr(root,
While doing the final setrefs.c pass over a plan tree, try to match up non-Var sort/group expressions using ressortgroupref labels instead of depending entirely on equal()-ity of the upper node's tlist expressions to the lower node's. This avoids emitting the wrong outputs in cases where there are textually identical volatile sort/group expressions, as for example select distinct random(),random() from generate_series(1,10); Per report from Andrew Gierth. Backpatch to 8.4. Arguably this is wrong all the way back, but the only known case where there's an observable problem is when using hash aggregation to implement DISTINCT, which is new as of 8.4. So for the moment I'll refrain from backpatching further.
16 years ago
(Node *) tle->expr,
subplan_itlist,
Rearrange the implementation of index-only scans. This commit changes index-only scans so that data is read directly from the index tuple without first generating a faux heap tuple. The only immediate benefit is that indexes on system columns (such as OID) can be used in index-only scans, but this is necessary infrastructure if we are ever to support index-only scans on expression indexes. The executor is now ready for that, though the planner still needs substantial work to recognize the possibility. To do this, Vars in index-only plan nodes have to refer to index columns not heap columns. I introduced a new special varno, INDEX_VAR, to mark such Vars to avoid confusion. (In passing, this commit renames the two existing special varnos to OUTER_VAR and INNER_VAR.) This allows ruleutils.c to handle them with logic similar to what we use for subplan reference Vars. Since index-only scans are now fundamentally different from regular indexscans so far as their expression subtrees are concerned, I also chose to change them to have their own plan node type (and hence, their own executor source file).
14 years ago
OUTER_VAR,
While doing the final setrefs.c pass over a plan tree, try to match up non-Var sort/group expressions using ressortgroupref labels instead of depending entirely on equal()-ity of the upper node's tlist expressions to the lower node's. This avoids emitting the wrong outputs in cases where there are textually identical volatile sort/group expressions, as for example select distinct random(),random() from generate_series(1,10); Per report from Andrew Gierth. Backpatch to 8.4. Arguably this is wrong all the way back, but the only known case where there's an observable problem is when using hash aggregation to implement DISTINCT, which is new as of 8.4. So for the moment I'll refrain from backpatching further.
16 years ago
rtoffset);
Merge Resdom nodes into TargetEntry nodes to simplify code and save a few palloc's. I also chose to eliminate the restype and restypmod fields entirely, since they are redundant with information stored in the node's contained expression; re-examining the expression at need seems simpler and more reliable than trying to keep restype/restypmod up to date. initdb forced due to change in contents of stored rules.
21 years ago
tle = flatCopyTargetEntry(tle);
tle->expr = (Expr *) newexpr;
output_targetlist = lappend(output_targetlist, tle);
Fix problems with subselects used in GROUP BY expressions, per gripe from Philip Warner. Side effect of change is that GROUP BY expressions will not be re-evaluated at multiple plan levels anymore, whereas this sometimes happened with old code.
24 years ago
}
plan->targetlist = output_targetlist;
Further planner/optimizer cleanups. Move all set_tlist_references and fix_opids processing to a single recursive pass over the plan tree executed at the very tail end of planning, rather than haphazardly here and there at different places. Now that tlist Vars do not get modified until the very end, it's possible to get rid of the klugy var_equal and match_varid partial-matching routines, and just use plain equal() throughout the optimizer. This is a step towards allowing merge and hash joins to be done on expressions instead of only Vars ...
26 years ago
plan->qual = (List *)
Rearrange planner to save the whole PlannerInfo (subroot) for a subquery. Formerly, set_subquery_pathlist and other creators of plans for subqueries saved only the rangetable and rowMarks lists from the lower-level PlannerInfo. But there's no reason not to remember the whole PlannerInfo, and indeed this turns out to simplify matters in a number of places. The immediate reason for doing this was so that the subroot will still be accessible when we're trying to extract column statistics out of an already-planned subquery. But now that I've done it, it seems like a good code-beautification effort in its own right. I also chose to get rid of the transient subrtable and subrowmark fields in SubqueryScan nodes, in favor of having setrefs.c look up the subquery's RelOptInfo. That required changing all the APIs in setrefs.c to pass PlannerInfo not PlannerGlobal, which was a large but quite mechanical transformation. One side-effect not foreseen at the beginning is that this finally broke inheritance_planner's assumption that replanning the same subquery RTE N times would necessarily give interchangeable results each time. That assumption was always pretty risky, but now we really have to make a separate RTE for each instance so that there's a place to carry the separate subroots.
14 years ago
fix_upper_expr(root,
Fix the plan-invalidation mechanism to treat regclass constants that refer to a relation as a reason to invalidate a plan when the relation changes. This handles scenarios such as dropping/recreating a sequence that is referenced by nextval('seq') in a cached plan. Rather than teach plancache.c all about digging through plan trees to find regclass Consts, we charge the planner's setrefs.c with making a list of the relation OIDs on which each plan depends. That way the list can be built cheaply during a plan tree traversal that has to happen anyway. Per bug #3662 and subsequent discussion.
18 years ago
(Node *) plan->qual,
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
subplan_itlist,
Rearrange the implementation of index-only scans. This commit changes index-only scans so that data is read directly from the index tuple without first generating a faux heap tuple. The only immediate benefit is that indexes on system columns (such as OID) can be used in index-only scans, but this is necessary infrastructure if we are ever to support index-only scans on expression indexes. The executor is now ready for that, though the planner still needs substantial work to recognize the possibility. To do this, Vars in index-only plan nodes have to refer to index columns not heap columns. I introduced a new special varno, INDEX_VAR, to mark such Vars to avoid confusion. (In passing, this commit renames the two existing special varnos to OUTER_VAR and INNER_VAR.) This allows ruleutils.c to handle them with logic similar to what we use for subplan reference Vars. Since index-only scans are now fundamentally different from regular indexscans so far as their expression subtrees are concerned, I also chose to change them to have their own plan node type (and hence, their own executor source file).
14 years ago
OUTER_VAR,
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
rtoffset);
Revise searching of subplan target lists to use something more efficient than tlist_member calls. Building a large join tlist is still O(N^2), but with a much smaller constant factor than before.
21 years ago
pfree(subplan_itlist);
Postgres95 1.01 Distribution - Virgin Sources
30 years ago
}
Now that plans have flat rangetable lists, it's a lot easier to get EXPLAIN to drill down into subplan targetlists to print the referent expression for an OUTER or INNER var in an upper plan node. Hence, make it do that always, and banish the old hack of showing "?columnN?" when things got too complicated. Along the way, fix an EXPLAIN bug I introduced by suppressing subqueries from execution-time range tables: get_name_for_var_field() assumed it could look at rte->subquery to find out the real type of a RECORD var. That doesn't work anymore, but instead we can look at the input plan of the SubqueryScan plan node.
19 years ago
/*
* set_dummy_tlist_references
* Replace the targetlist of an upper-level plan node with a simple
Rearrange the implementation of index-only scans. This commit changes index-only scans so that data is read directly from the index tuple without first generating a faux heap tuple. The only immediate benefit is that indexes on system columns (such as OID) can be used in index-only scans, but this is necessary infrastructure if we are ever to support index-only scans on expression indexes. The executor is now ready for that, though the planner still needs substantial work to recognize the possibility. To do this, Vars in index-only plan nodes have to refer to index columns not heap columns. I introduced a new special varno, INDEX_VAR, to mark such Vars to avoid confusion. (In passing, this commit renames the two existing special varnos to OUTER_VAR and INNER_VAR.) This allows ruleutils.c to handle them with logic similar to what we use for subplan reference Vars. Since index-only scans are now fundamentally different from regular indexscans so far as their expression subtrees are concerned, I also chose to change them to have their own plan node type (and hence, their own executor source file).
14 years ago
* list of OUTER_VAR references to its child.
Now that plans have flat rangetable lists, it's a lot easier to get EXPLAIN to drill down into subplan targetlists to print the referent expression for an OUTER or INNER var in an upper plan node. Hence, make it do that always, and banish the old hack of showing "?columnN?" when things got too complicated. Along the way, fix an EXPLAIN bug I introduced by suppressing subqueries from execution-time range tables: get_name_for_var_field() assumed it could look at rte->subquery to find out the real type of a RECORD var. That doesn't work anymore, but instead we can look at the input plan of the SubqueryScan plan node.
19 years ago
*
* This is used for plan types like Sort and Append that don't evaluate
* their targetlists. Although the executor doesn't care at all what's in
* the tlist, EXPLAIN needs it to be realistic.
*
* Note: we could almost use set_upper_references() here, but it fails for
* Append for lack of a lefttree subplan. Single-purpose code is faster
* anyway.
*/
static void
set_dummy_tlist_references(Plan *plan, int rtoffset)
{
List *output_targetlist;
ListCell *l;
output_targetlist = NIL;
foreach(l, plan->targetlist)
{
TargetEntry *tle = (TargetEntry *) lfirst(l);
Var *oldvar = (Var *) tle->expr;
Var *newvar;
Rearrange the implementation of index-only scans. This commit changes index-only scans so that data is read directly from the index tuple without first generating a faux heap tuple. The only immediate benefit is that indexes on system columns (such as OID) can be used in index-only scans, but this is necessary infrastructure if we are ever to support index-only scans on expression indexes. The executor is now ready for that, though the planner still needs substantial work to recognize the possibility. To do this, Vars in index-only plan nodes have to refer to index columns not heap columns. I introduced a new special varno, INDEX_VAR, to mark such Vars to avoid confusion. (In passing, this commit renames the two existing special varnos to OUTER_VAR and INNER_VAR.) This allows ruleutils.c to handle them with logic similar to what we use for subplan reference Vars. Since index-only scans are now fundamentally different from regular indexscans so far as their expression subtrees are concerned, I also chose to change them to have their own plan node type (and hence, their own executor source file).
14 years ago
newvar = makeVar(OUTER_VAR,
Now that plans have flat rangetable lists, it's a lot easier to get EXPLAIN to drill down into subplan targetlists to print the referent expression for an OUTER or INNER var in an upper plan node. Hence, make it do that always, and banish the old hack of showing "?columnN?" when things got too complicated. Along the way, fix an EXPLAIN bug I introduced by suppressing subqueries from execution-time range tables: get_name_for_var_field() assumed it could look at rte->subquery to find out the real type of a RECORD var. That doesn't work anymore, but instead we can look at the input plan of the SubqueryScan plan node.
19 years ago
tle->resno,
exprType((Node *) oldvar),
exprTypmod((Node *) oldvar),
Per-column collation support This adds collation support for columns and domains, a COLLATE clause to override it per expression, and B-tree index support. Peter Eisentraut reviewed by Pavel Stehule, Itagaki Takahiro, Robert Haas, Noah Misch
15 years ago
exprCollation((Node *) oldvar),
Now that plans have flat rangetable lists, it's a lot easier to get EXPLAIN to drill down into subplan targetlists to print the referent expression for an OUTER or INNER var in an upper plan node. Hence, make it do that always, and banish the old hack of showing "?columnN?" when things got too complicated. Along the way, fix an EXPLAIN bug I introduced by suppressing subqueries from execution-time range tables: get_name_for_var_field() assumed it could look at rte->subquery to find out the real type of a RECORD var. That doesn't work anymore, but instead we can look at the input plan of the SubqueryScan plan node.
19 years ago
0);
if (IsA(oldvar, Var))
{
newvar->varnoold = oldvar->varno + rtoffset;
newvar->varoattno = oldvar->varattno;
}
else
{
pgindent run for 8.3.
18 years ago
newvar->varnoold = 0; /* wasn't ever a plain Var */
Now that plans have flat rangetable lists, it's a lot easier to get EXPLAIN to drill down into subplan targetlists to print the referent expression for an OUTER or INNER var in an upper plan node. Hence, make it do that always, and banish the old hack of showing "?columnN?" when things got too complicated. Along the way, fix an EXPLAIN bug I introduced by suppressing subqueries from execution-time range tables: get_name_for_var_field() assumed it could look at rte->subquery to find out the real type of a RECORD var. That doesn't work anymore, but instead we can look at the input plan of the SubqueryScan plan node.
19 years ago
newvar->varoattno = 0;
}
tle = flatCopyTargetEntry(tle);
tle->expr = (Expr *) newvar;
output_targetlist = lappend(output_targetlist, tle);
}
plan->targetlist = output_targetlist;
/* We don't touch plan->qual here */
}
IN clauses appearing at top level of WHERE can now be handled as joins. There are two implementation techniques: the executor understands a new JOIN_IN jointype, which emits at most one matching row per left-hand row, or the result of the IN's sub-select can be fed through a DISTINCT filter and then joined as an ordinary relation. Along the way, some minor code cleanup in the optimizer; notably, break out most of the jointree-rearrangement preprocessing in planner.c and put it in a new file prep/prepjointree.c.
23 years ago
/*
Revise searching of subplan target lists to use something more efficient than tlist_member calls. Building a large join tlist is still O(N^2), but with a much smaller constant factor than before.
21 years ago
* build_tlist_index --- build an index data structure for a child tlist
IN clauses appearing at top level of WHERE can now be handled as joins. There are two implementation techniques: the executor understands a new JOIN_IN jointype, which emits at most one matching row per left-hand row, or the result of the IN's sub-select can be fed through a DISTINCT filter and then joined as an ordinary relation. Along the way, some minor code cleanup in the optimizer; notably, break out most of the jointree-rearrangement preprocessing in planner.c and put it in a new file prep/prepjointree.c.
23 years ago
*
Revise searching of subplan target lists to use something more efficient than tlist_member calls. Building a large join tlist is still O(N^2), but with a much smaller constant factor than before.
21 years ago
* In most cases, subplan tlists will be "flat" tlists with only Vars,
* so we try to optimize that case by extracting information about Vars
pgindent run for 9.4 This includes removing tabs after periods in C comments, which was applied to back branches, so this change should not effect backpatching.
12 years ago
* in advance. Matching a parent tlist to a child is still an O(N^2)
Revise searching of subplan target lists to use something more efficient than tlist_member calls. Building a large join tlist is still O(N^2), but with a much smaller constant factor than before.
21 years ago
* operation, but at least with a much smaller constant factor than plain
* tlist_member() searches.
*
* The result of this function is an indexed_tlist struct to pass to
* search_indexed_tlist_for_var() or search_indexed_tlist_for_non_var().
* When done, the indexed_tlist may be freed with a single pfree().
IN clauses appearing at top level of WHERE can now be handled as joins. There are two implementation techniques: the executor understands a new JOIN_IN jointype, which emits at most one matching row per left-hand row, or the result of the IN's sub-select can be fed through a DISTINCT filter and then joined as an ordinary relation. Along the way, some minor code cleanup in the optimizer; notably, break out most of the jointree-rearrangement preprocessing in planner.c and put it in a new file prep/prepjointree.c.
23 years ago
*/
Revise searching of subplan target lists to use something more efficient than tlist_member calls. Building a large join tlist is still O(N^2), but with a much smaller constant factor than before.
21 years ago
static indexed_tlist *
build_tlist_index(List *tlist)
IN clauses appearing at top level of WHERE can now be handled as joins. There are two implementation techniques: the executor understands a new JOIN_IN jointype, which emits at most one matching row per left-hand row, or the result of the IN's sub-select can be fed through a DISTINCT filter and then joined as an ordinary relation. Along the way, some minor code cleanup in the optimizer; notably, break out most of the jointree-rearrangement preprocessing in planner.c and put it in a new file prep/prepjointree.c.
23 years ago
{
Revise searching of subplan target lists to use something more efficient than tlist_member calls. Building a large join tlist is still O(N^2), but with a much smaller constant factor than before.
21 years ago
indexed_tlist *itlist;
tlist_vinfo *vinfo;
Reimplement the linked list data structure used throughout the backend. In the past, we used a 'Lispy' linked list implementation: a "list" was merely a pointer to the head node of the list. The problem with that design is that it makes lappend() and length() linear time. This patch fixes that problem (and others) by maintaining a count of the list length and a pointer to the tail node along with each head node pointer. A "list" is now a pointer to a structure containing some meta-data about the list; the head and tail pointers in that structure refer to ListCell structures that maintain the actual linked list of nodes. The function names of the list API have also been changed to, I hope, be more logically consistent. By default, the old function names are still available; they will be disabled-by-default once the rest of the tree has been updated to use the new API names.
22 years ago
ListCell *l;
IN clauses appearing at top level of WHERE can now be handled as joins. There are two implementation techniques: the executor understands a new JOIN_IN jointype, which emits at most one matching row per left-hand row, or the result of the IN's sub-select can be fed through a DISTINCT filter and then joined as an ordinary relation. Along the way, some minor code cleanup in the optimizer; notably, break out most of the jointree-rearrangement preprocessing in planner.c and put it in a new file prep/prepjointree.c.
23 years ago
Revise searching of subplan target lists to use something more efficient than tlist_member calls. Building a large join tlist is still O(N^2), but with a much smaller constant factor than before.
21 years ago
/* Create data structure with enough slots for all tlist entries */
itlist = (indexed_tlist *)
palloc(offsetof(indexed_tlist, vars) +
list_length(tlist) * sizeof(tlist_vinfo));
itlist->tlist = tlist;
Add a concept of "placeholder" variables to the planner. These are variables that represent some expression that we desire to compute below the top level of the plan, and then let that value "bubble up" as though it were a plain Var (ie, a column value). The immediate application is to allow sub-selects to be flattened even when they are below an outer join and have non-nullable output expressions. Formerly we couldn't flatten because such an expression wouldn't properly go to NULL when evaluated above the outer join. Now, we wrap it in a PlaceHolderVar and arrange for the actual evaluation to occur below the outer join. When the resulting Var bubbles up through the join, it will be set to NULL if necessary, yielding the correct results. This fixes a planner limitation that's existed since 7.1. In future we might want to use this mechanism to re-introduce some form of Hellerstein's "expensive functions" optimization, ie place the evaluation of an expensive function at the most suitable point in the plan tree.
17 years ago
itlist->has_ph_vars = false;
Revise searching of subplan target lists to use something more efficient than tlist_member calls. Building a large join tlist is still O(N^2), but with a much smaller constant factor than before.
21 years ago
itlist->has_non_vars = false;
/* Find the Vars and fill in the index array */
vinfo = itlist->vars;
IN clauses appearing at top level of WHERE can now be handled as joins. There are two implementation techniques: the executor understands a new JOIN_IN jointype, which emits at most one matching row per left-hand row, or the result of the IN's sub-select can be fed through a DISTINCT filter and then joined as an ordinary relation. Along the way, some minor code cleanup in the optimizer; notably, break out most of the jointree-rearrangement preprocessing in planner.c and put it in a new file prep/prepjointree.c.
23 years ago
foreach(l, tlist)
{
TargetEntry *tle = (TargetEntry *) lfirst(l);
Revise searching of subplan target lists to use something more efficient than tlist_member calls. Building a large join tlist is still O(N^2), but with a much smaller constant factor than before.
21 years ago
if (tle->expr && IsA(tle->expr, Var))
{
Standard pgindent run for 8.1.
20 years ago
Var *var = (Var *) tle->expr;
Revise searching of subplan target lists to use something more efficient than tlist_member calls. Building a large join tlist is still O(N^2), but with a much smaller constant factor than before.
21 years ago
vinfo->varno = var->varno;
vinfo->varattno = var->varattno;
vinfo->resno = tle->resno;
vinfo++;
}
Add a concept of "placeholder" variables to the planner. These are variables that represent some expression that we desire to compute below the top level of the plan, and then let that value "bubble up" as though it were a plain Var (ie, a column value). The immediate application is to allow sub-selects to be flattened even when they are below an outer join and have non-nullable output expressions. Formerly we couldn't flatten because such an expression wouldn't properly go to NULL when evaluated above the outer join. Now, we wrap it in a PlaceHolderVar and arrange for the actual evaluation to occur below the outer join. When the resulting Var bubbles up through the join, it will be set to NULL if necessary, yielding the correct results. This fixes a planner limitation that's existed since 7.1. In future we might want to use this mechanism to re-introduce some form of Hellerstein's "expensive functions" optimization, ie place the evaluation of an expensive function at the most suitable point in the plan tree.
17 years ago
else if (tle->expr && IsA(tle->expr, PlaceHolderVar))
itlist->has_ph_vars = true;
Revise searching of subplan target lists to use something more efficient than tlist_member calls. Building a large join tlist is still O(N^2), but with a much smaller constant factor than before.
21 years ago
else
itlist->has_non_vars = true;
}
itlist->num_vars = (vinfo - itlist->vars);
return itlist;
}
Add INSERT/UPDATE/DELETE RETURNING, with basic docs and regression tests. plpgsql support to come later. Along the way, convert execMain's SELECT INTO support into a DestReceiver, in order to eliminate some ugly special cases. Jonah Harris and Tom Lane
19 years ago
/*
* build_tlist_index_other_vars --- build a restricted tlist index
*
* This is like build_tlist_index, but we only index tlist entries that
Add a concept of "placeholder" variables to the planner. These are variables that represent some expression that we desire to compute below the top level of the plan, and then let that value "bubble up" as though it were a plain Var (ie, a column value). The immediate application is to allow sub-selects to be flattened even when they are below an outer join and have non-nullable output expressions. Formerly we couldn't flatten because such an expression wouldn't properly go to NULL when evaluated above the outer join. Now, we wrap it in a PlaceHolderVar and arrange for the actual evaluation to occur below the outer join. When the resulting Var bubbles up through the join, it will be set to NULL if necessary, yielding the correct results. This fixes a planner limitation that's existed since 7.1. In future we might want to use this mechanism to re-introduce some form of Hellerstein's "expensive functions" optimization, ie place the evaluation of an expensive function at the most suitable point in the plan tree.
17 years ago
* are Vars belonging to some rel other than the one specified. We will set
* has_ph_vars (allowing PlaceHolderVars to be matched), but not has_non_vars
* (so nothing other than Vars and PlaceHolderVars can be matched).
Add INSERT/UPDATE/DELETE RETURNING, with basic docs and regression tests. plpgsql support to come later. Along the way, convert execMain's SELECT INTO support into a DestReceiver, in order to eliminate some ugly special cases. Jonah Harris and Tom Lane
19 years ago
*/
static indexed_tlist *
build_tlist_index_other_vars(List *tlist, Index ignore_rel)
{
indexed_tlist *itlist;
tlist_vinfo *vinfo;
ListCell *l;
/* Create data structure with enough slots for all tlist entries */
itlist = (indexed_tlist *)
palloc(offsetof(indexed_tlist, vars) +
list_length(tlist) * sizeof(tlist_vinfo));
itlist->tlist = tlist;
Add a concept of "placeholder" variables to the planner. These are variables that represent some expression that we desire to compute below the top level of the plan, and then let that value "bubble up" as though it were a plain Var (ie, a column value). The immediate application is to allow sub-selects to be flattened even when they are below an outer join and have non-nullable output expressions. Formerly we couldn't flatten because such an expression wouldn't properly go to NULL when evaluated above the outer join. Now, we wrap it in a PlaceHolderVar and arrange for the actual evaluation to occur below the outer join. When the resulting Var bubbles up through the join, it will be set to NULL if necessary, yielding the correct results. This fixes a planner limitation that's existed since 7.1. In future we might want to use this mechanism to re-introduce some form of Hellerstein's "expensive functions" optimization, ie place the evaluation of an expensive function at the most suitable point in the plan tree.
17 years ago
itlist->has_ph_vars = false;
Add INSERT/UPDATE/DELETE RETURNING, with basic docs and regression tests. plpgsql support to come later. Along the way, convert execMain's SELECT INTO support into a DestReceiver, in order to eliminate some ugly special cases. Jonah Harris and Tom Lane
19 years ago
itlist->has_non_vars = false;
/* Find the desired Vars and fill in the index array */
vinfo = itlist->vars;
foreach(l, tlist)
{
TargetEntry *tle = (TargetEntry *) lfirst(l);
if (tle->expr && IsA(tle->expr, Var))
{
Var *var = (Var *) tle->expr;
if (var->varno != ignore_rel)
{
vinfo->varno = var->varno;
vinfo->varattno = var->varattno;
vinfo->resno = tle->resno;
vinfo++;
}
}
Add a concept of "placeholder" variables to the planner. These are variables that represent some expression that we desire to compute below the top level of the plan, and then let that value "bubble up" as though it were a plain Var (ie, a column value). The immediate application is to allow sub-selects to be flattened even when they are below an outer join and have non-nullable output expressions. Formerly we couldn't flatten because such an expression wouldn't properly go to NULL when evaluated above the outer join. Now, we wrap it in a PlaceHolderVar and arrange for the actual evaluation to occur below the outer join. When the resulting Var bubbles up through the join, it will be set to NULL if necessary, yielding the correct results. This fixes a planner limitation that's existed since 7.1. In future we might want to use this mechanism to re-introduce some form of Hellerstein's "expensive functions" optimization, ie place the evaluation of an expensive function at the most suitable point in the plan tree.
17 years ago
else if (tle->expr && IsA(tle->expr, PlaceHolderVar))
itlist->has_ph_vars = true;
Add INSERT/UPDATE/DELETE RETURNING, with basic docs and regression tests. plpgsql support to come later. Along the way, convert execMain's SELECT INTO support into a DestReceiver, in order to eliminate some ugly special cases. Jonah Harris and Tom Lane
19 years ago
}
itlist->num_vars = (vinfo - itlist->vars);
return itlist;
}
Revise searching of subplan target lists to use something more efficient than tlist_member calls. Building a large join tlist is still O(N^2), but with a much smaller constant factor than before.
21 years ago
/*
* search_indexed_tlist_for_var --- find a Var in an indexed tlist
*
* If a match is found, return a copy of the given Var with suitably
* modified varno/varattno (to wit, newvarno and the resno of the TLE entry).
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
* Also ensure that varnoold is incremented by rtoffset.
Revise searching of subplan target lists to use something more efficient than tlist_member calls. Building a large join tlist is still O(N^2), but with a much smaller constant factor than before.
21 years ago
* If no match, return NULL.
*/
static Var *
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
search_indexed_tlist_for_var(Var *var, indexed_tlist *itlist,
Index newvarno, int rtoffset)
Revise searching of subplan target lists to use something more efficient than tlist_member calls. Building a large join tlist is still O(N^2), but with a much smaller constant factor than before.
21 years ago
{
Index varno = var->varno;
AttrNumber varattno = var->varattno;
tlist_vinfo *vinfo;
int i;
vinfo = itlist->vars;
i = itlist->num_vars;
while (i-- > 0)
{
if (vinfo->varno == varno && vinfo->varattno == varattno)
{
/* Found a match */
Marginal performance hack: use a dedicated routine instead of copyObject to copy nodes that are known to be Vars during plan reference adjustment. Saves useless memzero operation as well as the big switch in copyObject.
19 years ago
Var *newvar = copyVar(var);
Revise searching of subplan target lists to use something more efficient than tlist_member calls. Building a large join tlist is still O(N^2), but with a much smaller constant factor than before.
21 years ago
newvar->varno = newvarno;
newvar->varattno = vinfo->resno;
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
if (newvar->varnoold > 0)
newvar->varnoold += rtoffset;
Revise searching of subplan target lists to use something more efficient than tlist_member calls. Building a large join tlist is still O(N^2), but with a much smaller constant factor than before.
21 years ago
return newvar;
}
vinfo++;
IN clauses appearing at top level of WHERE can now be handled as joins. There are two implementation techniques: the executor understands a new JOIN_IN jointype, which emits at most one matching row per left-hand row, or the result of the IN's sub-select can be fed through a DISTINCT filter and then joined as an ordinary relation. Along the way, some minor code cleanup in the optimizer; notably, break out most of the jointree-rearrangement preprocessing in planner.c and put it in a new file prep/prepjointree.c.
23 years ago
}
Revise searching of subplan target lists to use something more efficient than tlist_member calls. Building a large join tlist is still O(N^2), but with a much smaller constant factor than before.
21 years ago
return NULL; /* no match */
}
/*
* search_indexed_tlist_for_non_var --- find a non-Var in an indexed tlist
*
* If a match is found, return a Var constructed to reference the tlist item.
* If no match, return NULL.
*
Add a concept of "placeholder" variables to the planner. These are variables that represent some expression that we desire to compute below the top level of the plan, and then let that value "bubble up" as though it were a plain Var (ie, a column value). The immediate application is to allow sub-selects to be flattened even when they are below an outer join and have non-nullable output expressions. Formerly we couldn't flatten because such an expression wouldn't properly go to NULL when evaluated above the outer join. Now, we wrap it in a PlaceHolderVar and arrange for the actual evaluation to occur below the outer join. When the resulting Var bubbles up through the join, it will be set to NULL if necessary, yielding the correct results. This fixes a planner limitation that's existed since 7.1. In future we might want to use this mechanism to re-introduce some form of Hellerstein's "expensive functions" optimization, ie place the evaluation of an expensive function at the most suitable point in the plan tree.
17 years ago
* NOTE: it is a waste of time to call this unless itlist->has_ph_vars or
Fix incorrect matching of subexpressions in outer-join plan nodes. Previously we would re-use input subexpressions in all expression trees attached to a Join plan node. However, if it's an outer join and the subexpression appears in the nullable-side input, this is potentially incorrect for apparently-matching subexpressions that came from above the outer join (ie, targetlist and qpqual expressions), because the executor will treat the subexpression value as NULL when maybe it should not be. The case is fairly hard to hit because (a) you need a non-strict subexpression (else NULL is correct), and (b) we don't usually compute expressions in the outputs of non-toplevel plan nodes. But we might do so if the expressions are sort keys for a mergejoin, for example. Probably in the long run we should make a more explicit distinction between Vars appearing above and below an outer join, but that will be a major planner redesign and not at all back-patchable. For the moment, just hack set_join_references so that it will not match any non-Var expressions coming from nullable inputs to expressions that came from above the join. (This is somewhat overkill, in that a strict expression could still be matched, but it doesn't seem worth the effort to check that.) Per report from Qingqing Zhou. The added regression test case is based on his example. This has been broken for a very long time, so back-patch to all active branches.
11 years ago
* itlist->has_non_vars. Furthermore, set_join_references() relies on being
* able to prevent matching of non-Vars by clearing itlist->has_non_vars,
* so there's a correctness reason not to call it unless that's set.
Revise searching of subplan target lists to use something more efficient than tlist_member calls. Building a large join tlist is still O(N^2), but with a much smaller constant factor than before.
21 years ago
*/
static Var *
search_indexed_tlist_for_non_var(Node *node,
indexed_tlist *itlist, Index newvarno)
{
TargetEntry *tle;
tle = tlist_member(node, itlist->tlist);
if (tle)
{
/* Found a matching subplan output expression */
Var *newvar;
Small refactoring of makeVar() from a TargetEntry
15 years ago
newvar = makeVarFromTargetEntry(newvarno, tle);
Standard pgindent run for 8.1.
20 years ago
newvar->varnoold = 0; /* wasn't ever a plain Var */
Revise searching of subplan target lists to use something more efficient than tlist_member calls. Building a large join tlist is still O(N^2), but with a much smaller constant factor than before.
21 years ago
newvar->varoattno = 0;
return newvar;
}
return NULL; /* no match */
IN clauses appearing at top level of WHERE can now be handled as joins. There are two implementation techniques: the executor understands a new JOIN_IN jointype, which emits at most one matching row per left-hand row, or the result of the IN's sub-select can be fed through a DISTINCT filter and then joined as an ordinary relation. Along the way, some minor code cleanup in the optimizer; notably, break out most of the jointree-rearrangement preprocessing in planner.c and put it in a new file prep/prepjointree.c.
23 years ago
}
While doing the final setrefs.c pass over a plan tree, try to match up non-Var sort/group expressions using ressortgroupref labels instead of depending entirely on equal()-ity of the upper node's tlist expressions to the lower node's. This avoids emitting the wrong outputs in cases where there are textually identical volatile sort/group expressions, as for example select distinct random(),random() from generate_series(1,10); Per report from Andrew Gierth. Backpatch to 8.4. Arguably this is wrong all the way back, but the only known case where there's an observable problem is when using hash aggregation to implement DISTINCT, which is new as of 8.4. So for the moment I'll refrain from backpatching further.
16 years ago
/*
* search_indexed_tlist_for_sortgroupref --- find a sort/group expression
* (which is assumed not to be just a Var)
*
* If a match is found, return a Var constructed to reference the tlist item.
* If no match, return NULL.
*
* This is needed to ensure that we select the right subplan TLE in cases
* where there are multiple textually-equal()-but-volatile sort expressions.
* And it's also faster than search_indexed_tlist_for_non_var.
*/
static Var *
search_indexed_tlist_for_sortgroupref(Node *node,
Index sortgroupref,
indexed_tlist *itlist,
Index newvarno)
{
ListCell *lc;
foreach(lc, itlist->tlist)
{
TargetEntry *tle = (TargetEntry *) lfirst(lc);
/* The equal() check should be redundant, but let's be paranoid */
if (tle->ressortgroupref == sortgroupref &&
equal(node, tle->expr))
{
/* Found a matching subplan output expression */
Var *newvar;
Small refactoring of makeVar() from a TargetEntry
15 years ago
newvar = makeVarFromTargetEntry(newvarno, tle);
pgindent run for 9.0
16 years ago
newvar->varnoold = 0; /* wasn't ever a plain Var */
While doing the final setrefs.c pass over a plan tree, try to match up non-Var sort/group expressions using ressortgroupref labels instead of depending entirely on equal()-ity of the upper node's tlist expressions to the lower node's. This avoids emitting the wrong outputs in cases where there are textually identical volatile sort/group expressions, as for example select distinct random(),random() from generate_series(1,10); Per report from Andrew Gierth. Backpatch to 8.4. Arguably this is wrong all the way back, but the only known case where there's an observable problem is when using hash aggregation to implement DISTINCT, which is new as of 8.4. So for the moment I'll refrain from backpatching further.
16 years ago
newvar->varoattno = 0;
return newvar;
}
}
return NULL; /* no match */
}
Massive commit to run PGINDENT on all *.c and *.h files.
28 years ago
/*
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
* fix_join_expr
* Create a new set of targetlist entries or join qual clauses by
Further planner/optimizer cleanups. Move all set_tlist_references and fix_opids processing to a single recursive pass over the plan tree executed at the very tail end of planning, rather than haphazardly here and there at different places. Now that tlist Vars do not get modified until the very end, it's possible to get rid of the klugy var_equal and match_varid partial-matching routines, and just use plain equal() throughout the optimizer. This is a step towards allowing merge and hash joins to be done on expressions instead of only Vars ...
26 years ago
* changing the varno/varattno values of variables in the clauses
* to reference target list values from the outer and inner join
Fix the plan-invalidation mechanism to treat regclass constants that refer to a relation as a reason to invalidate a plan when the relation changes. This handles scenarios such as dropping/recreating a sequence that is referenced by nextval('seq') in a cached plan. Rather than teach plancache.c all about digging through plan trees to find regclass Consts, we charge the planner's setrefs.c with making a list of the relation OIDs on which each plan depends. That way the list can be built cheaply during a plan tree traversal that has to happen anyway. Per bug #3662 and subsequent discussion.
18 years ago
* relation target lists. Also perform opcode lookup and add
Rearrange planner to save the whole PlannerInfo (subroot) for a subquery. Formerly, set_subquery_pathlist and other creators of plans for subqueries saved only the rangetable and rowMarks lists from the lower-level PlannerInfo. But there's no reason not to remember the whole PlannerInfo, and indeed this turns out to simplify matters in a number of places. The immediate reason for doing this was so that the subroot will still be accessible when we're trying to extract column statistics out of an already-planned subquery. But now that I've done it, it seems like a good code-beautification effort in its own right. I also chose to get rid of the transient subrtable and subrowmark fields in SubqueryScan nodes, in favor of having setrefs.c look up the subquery's RelOptInfo. That required changing all the APIs in setrefs.c to pass PlannerInfo not PlannerGlobal, which was a large but quite mechanical transformation. One side-effect not foreseen at the beginning is that this finally broke inheritance_planner's assumption that replanning the same subquery RTE N times would necessarily give interchangeable results each time. That assumption was always pretty risky, but now we really have to make a separate RTE for each instance so that there's a place to carry the separate subroots.
14 years ago
* regclass OIDs to root->glob->relationOids.
Further planner/optimizer cleanups. Move all set_tlist_references and fix_opids processing to a single recursive pass over the plan tree executed at the very tail end of planning, rather than haphazardly here and there at different places. Now that tlist Vars do not get modified until the very end, it's possible to get rid of the klugy var_equal and match_varid partial-matching routines, and just use plain equal() throughout the optimizer. This is a step towards allowing merge and hash joins to be done on expressions instead of only Vars ...
26 years ago
*
Rearrange the implementation of index-only scans. This commit changes index-only scans so that data is read directly from the index tuple without first generating a faux heap tuple. The only immediate benefit is that indexes on system columns (such as OID) can be used in index-only scans, but this is necessary infrastructure if we are ever to support index-only scans on expression indexes. The executor is now ready for that, though the planner still needs substantial work to recognize the possibility. To do this, Vars in index-only plan nodes have to refer to index columns not heap columns. I introduced a new special varno, INDEX_VAR, to mark such Vars to avoid confusion. (In passing, this commit renames the two existing special varnos to OUTER_VAR and INNER_VAR.) This allows ruleutils.c to handle them with logic similar to what we use for subplan reference Vars. Since index-only scans are now fundamentally different from regular indexscans so far as their expression subtrees are concerned, I also chose to change them to have their own plan node type (and hence, their own executor source file).
14 years ago
* This is used in two different scenarios: a normal join clause, where all
* the Vars in the clause *must* be replaced by OUTER_VAR or INNER_VAR
* references; and a RETURNING clause, which may contain both Vars of the
* target relation and Vars of other relations. In the latter case we want
* to replace the other-relation Vars by OUTER_VAR references, while leaving
* target Vars alone.
Further planner/optimizer cleanups. Move all set_tlist_references and fix_opids processing to a single recursive pass over the plan tree executed at the very tail end of planning, rather than haphazardly here and there at different places. Now that tlist Vars do not get modified until the very end, it's possible to get rid of the klugy var_equal and match_varid partial-matching routines, and just use plain equal() throughout the optimizer. This is a step towards allowing merge and hash joins to be done on expressions instead of only Vars ...
26 years ago
*
* For a normal join, acceptable_rel should be zero so that any failure to
Make NestLoop plan nodes pass outer-relation variables into their inner relation using the general PARAM_EXEC executor parameter mechanism, rather than the ad-hoc kluge of passing the outer tuple down through ExecReScan. The previous method was hard to understand and could never be extended to handle parameters coming from multiple join levels. This patch doesn't change the set of possible plans nor have any significant performance effect, but it's necessary infrastructure for future generalization of the concept of an inner indexscan plan. ExecReScan's second parameter is now unused, so it's removed.
15 years ago
* match a Var will be reported as an error. For the RETURNING case, pass
* inner_itlist = NULL and acceptable_rel = the ID of the target relation.
Further planner/optimizer cleanups. Move all set_tlist_references and fix_opids processing to a single recursive pass over the plan tree executed at the very tail end of planning, rather than haphazardly here and there at different places. Now that tlist Vars do not get modified until the very end, it's possible to get rid of the klugy var_equal and match_varid partial-matching routines, and just use plain equal() throughout the optimizer. This is a step towards allowing merge and hash joins to be done on expressions instead of only Vars ...
26 years ago
*
* 'clauses' is the targetlist or list of join clauses
Add support for INSERT ... ON CONFLICT DO NOTHING/UPDATE. The newly added ON CONFLICT clause allows to specify an alternative to raising a unique or exclusion constraint violation error when inserting. ON CONFLICT refers to constraints that can either be specified using a inference clause (by specifying the columns of a unique constraint) or by naming a unique or exclusion constraint. DO NOTHING avoids the constraint violation, without touching the pre-existing row. DO UPDATE SET ... [WHERE ...] updates the pre-existing tuple, and has access to both the tuple proposed for insertion and the existing tuple; the optional WHERE clause can be used to prevent an update from being executed. The UPDATE SET and WHERE clauses have access to the tuple proposed for insertion using the "magic" EXCLUDED alias, and to the pre-existing tuple using the table name or its alias. This feature is often referred to as upsert. This is implemented using a new infrastructure called "speculative insertion". It is an optimistic variant of regular insertion that first does a pre-check for existing tuples and then attempts an insert. If a violating tuple was inserted concurrently, the speculatively inserted tuple is deleted and a new attempt is made. If the pre-check finds a matching tuple the alternative DO NOTHING or DO UPDATE action is taken. If the insertion succeeds without detecting a conflict, the tuple is deemed inserted. To handle the possible ambiguity between the excluded alias and a table named excluded, and for convenience with long relation names, INSERT INTO now can alias its target table. Bumps catversion as stored rules change. Author: Peter Geoghegan, with significant contributions from Heikki Linnakangas and Andres Freund. Testing infrastructure by Jeff Janes. Reviewed-By: Heikki Linnakangas, Andres Freund, Robert Haas, Simon Riggs, Dean Rasheed, Stephen Frost and many others.
11 years ago
* 'outer_itlist' is the indexed target list of the outer join relation,
* or NULL
Revise searching of subplan target lists to use something more efficient than tlist_member calls. Building a large join tlist is still O(N^2), but with a much smaller constant factor than before.
21 years ago
* 'inner_itlist' is the indexed target list of the inner join relation,
* or NULL
Further planner/optimizer cleanups. Move all set_tlist_references and fix_opids processing to a single recursive pass over the plan tree executed at the very tail end of planning, rather than haphazardly here and there at different places. Now that tlist Vars do not get modified until the very end, it's possible to get rid of the klugy var_equal and match_varid partial-matching routines, and just use plain equal() throughout the optimizer. This is a step towards allowing merge and hash joins to be done on expressions instead of only Vars ...
26 years ago
* 'acceptable_rel' is either zero or the rangetable index of a relation
Make NestLoop plan nodes pass outer-relation variables into their inner relation using the general PARAM_EXEC executor parameter mechanism, rather than the ad-hoc kluge of passing the outer tuple down through ExecReScan. The previous method was hard to understand and could never be extended to handle parameters coming from multiple join levels. This patch doesn't change the set of possible plans nor have any significant performance effect, but it's necessary infrastructure for future generalization of the concept of an inner indexscan plan. ExecReScan's second parameter is now unused, so it's removed.
15 years ago
* whose Vars may appear in the clause without provoking an error
* 'rtoffset': how much to increment varnoold by
Massive commit to run PGINDENT on all *.c and *.h files.
28 years ago
*
Further planner/optimizer cleanups. Move all set_tlist_references and fix_opids processing to a single recursive pass over the plan tree executed at the very tail end of planning, rather than haphazardly here and there at different places. Now that tlist Vars do not get modified until the very end, it's possible to get rid of the klugy var_equal and match_varid partial-matching routines, and just use plain equal() throughout the optimizer. This is a step towards allowing merge and hash joins to be done on expressions instead of only Vars ...
26 years ago
* Returns the new expression tree. The original clause structure is
Fix oversights in flatten_tlistentry and replace_clause_joinvar_refs that led to CASE expressions not working very well in joined queries.
27 years ago
* not modified.
Postgres95 1.01 Distribution - Virgin Sources
30 years ago
*/
Now that switch_outer processing no longer relies on being run after join_references(), it's practical to consolidate all join_references() processing into the set_plan_references traversal in setrefs.c. This seems considerably cleaner than the old way where we did it for join quals in createplan.c and for targetlists in setrefs.c.
23 years ago
static List *
Rearrange planner to save the whole PlannerInfo (subroot) for a subquery. Formerly, set_subquery_pathlist and other creators of plans for subqueries saved only the rangetable and rowMarks lists from the lower-level PlannerInfo. But there's no reason not to remember the whole PlannerInfo, and indeed this turns out to simplify matters in a number of places. The immediate reason for doing this was so that the subroot will still be accessible when we're trying to extract column statistics out of an already-planned subquery. But now that I've done it, it seems like a good code-beautification effort in its own right. I also chose to get rid of the transient subrtable and subrowmark fields in SubqueryScan nodes, in favor of having setrefs.c look up the subquery's RelOptInfo. That required changing all the APIs in setrefs.c to pass PlannerInfo not PlannerGlobal, which was a large but quite mechanical transformation. One side-effect not foreseen at the beginning is that this finally broke inheritance_planner's assumption that replanning the same subquery RTE N times would necessarily give interchangeable results each time. That assumption was always pretty risky, but now we really have to make a separate RTE for each instance so that there's a place to carry the separate subroots.
14 years ago
fix_join_expr(PlannerInfo *root,
Fix the plan-invalidation mechanism to treat regclass constants that refer to a relation as a reason to invalidate a plan when the relation changes. This handles scenarios such as dropping/recreating a sequence that is referenced by nextval('seq') in a cached plan. Rather than teach plancache.c all about digging through plan trees to find regclass Consts, we charge the planner's setrefs.c with making a list of the relation OIDs on which each plan depends. That way the list can be built cheaply during a plan tree traversal that has to happen anyway. Per bug #3662 and subsequent discussion.
18 years ago
List *clauses,
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
indexed_tlist *outer_itlist,
indexed_tlist *inner_itlist,
Index acceptable_rel,
int rtoffset)
Postgres95 1.01 Distribution - Virgin Sources
30 years ago
{
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
fix_join_expr_context context;
pgindent run over code.
27 years ago
Rearrange planner to save the whole PlannerInfo (subroot) for a subquery. Formerly, set_subquery_pathlist and other creators of plans for subqueries saved only the rangetable and rowMarks lists from the lower-level PlannerInfo. But there's no reason not to remember the whole PlannerInfo, and indeed this turns out to simplify matters in a number of places. The immediate reason for doing this was so that the subroot will still be accessible when we're trying to extract column statistics out of an already-planned subquery. But now that I've done it, it seems like a good code-beautification effort in its own right. I also chose to get rid of the transient subrtable and subrowmark fields in SubqueryScan nodes, in favor of having setrefs.c look up the subquery's RelOptInfo. That required changing all the APIs in setrefs.c to pass PlannerInfo not PlannerGlobal, which was a large but quite mechanical transformation. One side-effect not foreseen at the beginning is that this finally broke inheritance_planner's assumption that replanning the same subquery RTE N times would necessarily give interchangeable results each time. That assumption was always pretty risky, but now we really have to make a separate RTE for each instance so that there's a place to carry the separate subroots.
14 years ago
context.root = root;
Revise searching of subplan target lists to use something more efficient than tlist_member calls. Building a large join tlist is still O(N^2), but with a much smaller constant factor than before.
21 years ago
context.outer_itlist = outer_itlist;
context.inner_itlist = inner_itlist;
Further planner/optimizer cleanups. Move all set_tlist_references and fix_opids processing to a single recursive pass over the plan tree executed at the very tail end of planning, rather than haphazardly here and there at different places. Now that tlist Vars do not get modified until the very end, it's possible to get rid of the klugy var_equal and match_varid partial-matching routines, and just use plain equal() throughout the optimizer. This is a step towards allowing merge and hash joins to be done on expressions instead of only Vars ...
26 years ago
context.acceptable_rel = acceptable_rel;
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
context.rtoffset = rtoffset;
return (List *) fix_join_expr_mutator((Node *) clauses, &context);
Postgres95 1.01 Distribution - Virgin Sources
30 years ago
}
Clean up routines in setrefs.c by replacing individual tree walking logic with expression_tree_walker/mutator calls.
26 years ago
static Node *
Re-run pgindent with updated list of typedefs. (Updated README should avoid this problem in the future.)
18 years ago
fix_join_expr_mutator(Node *node, fix_join_expr_context *context)
Postgres95 1.01 Distribution - Virgin Sources
30 years ago
{
Revise searching of subplan target lists to use something more efficient than tlist_member calls. Building a large join tlist is still O(N^2), but with a much smaller constant factor than before.
21 years ago
Var *newvar;
Clean up routines in setrefs.c by replacing individual tree walking logic with expression_tree_walker/mutator calls.
26 years ago
if (node == NULL)
return NULL;
if (IsA(node, Var))
{
Var *var = (Var *) node;
Postgres95 1.01 Distribution - Virgin Sources
30 years ago
Restructure representation of join alias variables. An explicit JOIN now has an RTE of its own, and references to its outputs now are Vars referencing the JOIN RTE, rather than CASE-expressions. This allows reverse-listing in ruleutils.c to use the correct alias easily, rather than painfully reverse-engineering the alias namespace as it used to do. Also, nested FULL JOINs work correctly, because the result of the inner joins are simple Vars that the planner can cope with. This fixes a bug reported a couple times now, notably by Tatsuo on 18-Nov-01. The alias Vars are expanded into COALESCE expressions where needed at the very end of planning, rather than during parsing. Also, beginnings of support for showing plan qualifier expressions in EXPLAIN. There are probably still cases that need work. initdb forced due to change of stored-rule representation.
24 years ago
/* First look for the var in the input tlists */
Add support for INSERT ... ON CONFLICT DO NOTHING/UPDATE. The newly added ON CONFLICT clause allows to specify an alternative to raising a unique or exclusion constraint violation error when inserting. ON CONFLICT refers to constraints that can either be specified using a inference clause (by specifying the columns of a unique constraint) or by naming a unique or exclusion constraint. DO NOTHING avoids the constraint violation, without touching the pre-existing row. DO UPDATE SET ... [WHERE ...] updates the pre-existing tuple, and has access to both the tuple proposed for insertion and the existing tuple; the optional WHERE clause can be used to prevent an update from being executed. The UPDATE SET and WHERE clauses have access to the tuple proposed for insertion using the "magic" EXCLUDED alias, and to the pre-existing tuple using the table name or its alias. This feature is often referred to as upsert. This is implemented using a new infrastructure called "speculative insertion". It is an optimistic variant of regular insertion that first does a pre-check for existing tuples and then attempts an insert. If a violating tuple was inserted concurrently, the speculatively inserted tuple is deleted and a new attempt is made. If the pre-check finds a matching tuple the alternative DO NOTHING or DO UPDATE action is taken. If the insertion succeeds without detecting a conflict, the tuple is deemed inserted. To handle the possible ambiguity between the excluded alias and a table named excluded, and for convenience with long relation names, INSERT INTO now can alias its target table. Bumps catversion as stored rules change. Author: Peter Geoghegan, with significant contributions from Heikki Linnakangas and Andres Freund. Testing infrastructure by Jeff Janes. Reviewed-By: Heikki Linnakangas, Andres Freund, Robert Haas, Simon Riggs, Dean Rasheed, Stephen Frost and many others.
11 years ago
if (context->outer_itlist)
{
newvar = search_indexed_tlist_for_var(var,
context->outer_itlist,
OUTER_VAR,
context->rtoffset);
if (newvar)
return (Node *) newvar;
}
/* Then in the outer */
Revise searching of subplan target lists to use something more efficient than tlist_member calls. Building a large join tlist is still O(N^2), but with a much smaller constant factor than before.
21 years ago
if (context->inner_itlist)
Further planner/optimizer cleanups. Move all set_tlist_references and fix_opids processing to a single recursive pass over the plan tree executed at the very tail end of planning, rather than haphazardly here and there at different places. Now that tlist Vars do not get modified until the very end, it's possible to get rid of the klugy var_equal and match_varid partial-matching routines, and just use plain equal() throughout the optimizer. This is a step towards allowing merge and hash joins to be done on expressions instead of only Vars ...
26 years ago
{
Revise searching of subplan target lists to use something more efficient than tlist_member calls. Building a large join tlist is still O(N^2), but with a much smaller constant factor than before.
21 years ago
newvar = search_indexed_tlist_for_var(var,
context->inner_itlist,
Rearrange the implementation of index-only scans. This commit changes index-only scans so that data is read directly from the index tuple without first generating a faux heap tuple. The only immediate benefit is that indexes on system columns (such as OID) can be used in index-only scans, but this is necessary infrastructure if we are ever to support index-only scans on expression indexes. The executor is now ready for that, though the planner still needs substantial work to recognize the possibility. To do this, Vars in index-only plan nodes have to refer to index columns not heap columns. I introduced a new special varno, INDEX_VAR, to mark such Vars to avoid confusion. (In passing, this commit renames the two existing special varnos to OUTER_VAR and INNER_VAR.) This allows ruleutils.c to handle them with logic similar to what we use for subplan reference Vars. Since index-only scans are now fundamentally different from regular indexscans so far as their expression subtrees are concerned, I also chose to change them to have their own plan node type (and hence, their own executor source file).
14 years ago
INNER_VAR,
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
context->rtoffset);
Revise searching of subplan target lists to use something more efficient than tlist_member calls. Building a large join tlist is still O(N^2), but with a much smaller constant factor than before.
21 years ago
if (newvar)
return (Node *) newvar;
Further planner/optimizer cleanups. Move all set_tlist_references and fix_opids processing to a single recursive pass over the plan tree executed at the very tail end of planning, rather than haphazardly here and there at different places. Now that tlist Vars do not get modified until the very end, it's possible to get rid of the klugy var_equal and match_varid partial-matching routines, and just use plain equal() throughout the optimizer. This is a step towards allowing merge and hash joins to be done on expressions instead of only Vars ...
26 years ago
}
Ye-old pgindent run. Same 4-space tabs.
26 years ago
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
/* If it's for acceptable_rel, adjust and return it */
Add an at-least-marginally-plausible method of estimating the number of groups produced by GROUP BY. This improves the accuracy of planning estimates for grouped subselects, and is needed to check whether a hashed aggregation plan risks memory overflow.
23 years ago
if (var->varno == context->acceptable_rel)
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
{
Marginal performance hack: use a dedicated routine instead of copyObject to copy nodes that are known to be Vars during plan reference adjustment. Saves useless memzero operation as well as the big switch in copyObject.
19 years ago
var = copyVar(var);
Fix planner's handling of RETURNING lists in writable CTEs. setrefs.c failed to do "rtoffset" adjustment of Vars in RETURNING lists, which meant they were left with the wrong varnos when the RETURNING list was in a subquery. That was never possible before writable CTEs, of course, but now it's broken. The executor fails to notice any problem because ExecEvalVar just references the ecxt_scantuple for any normal varno; but EXPLAIN breaks when the varno is wrong, as illustrated in a recent complaint from Bartosz Dmytrak. Since the eventual rtoffset of the subquery is not known at the time we are preparing its plan node, the previous scheme of executing set_returning_clause_references() at that time cannot handle this adjustment. Fortunately, it turns out that we don't really need to do it that way, because all the needed information is available during normal setrefs.c execution; we just have to dig it out of the ModifyTable node. So, do that, and get rid of the kluge of early setrefs processing of RETURNING lists. (This is a little bit of a cheat in the case of inherited UPDATE/DELETE, because we are not passing a "root" struct that corresponds exactly to what the subplan was built with. But that doesn't matter, and anyway this is less ugly than early setrefs processing was.) Back-patch to 9.1, where the problem became possible to hit.
14 years ago
var->varno += context->rtoffset;
Make NestLoop plan nodes pass outer-relation variables into their inner relation using the general PARAM_EXEC executor parameter mechanism, rather than the ad-hoc kluge of passing the outer tuple down through ExecReScan. The previous method was hard to understand and could never be extended to handle parameters coming from multiple join levels. This patch doesn't change the set of possible plans nor have any significant performance effect, but it's necessary infrastructure for future generalization of the concept of an inner indexscan plan. ExecReScan's second parameter is now unused, so it's removed.
15 years ago
if (var->varnoold > 0)
var->varnoold += context->rtoffset;
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
return (Node *) var;
}
Add an at-least-marginally-plausible method of estimating the number of groups produced by GROUP BY. This improves the accuracy of planning estimates for grouped subselects, and is needed to check whether a hashed aggregation plan risks memory overflow.
23 years ago
/* No referent found for Var */
Error message editing in backend/optimizer, backend/rewrite.
22 years ago
elog(ERROR, "variable not found in subplan target lists");
Massive commit to run PGINDENT on all *.c and *.h files.
28 years ago
}
Add a concept of "placeholder" variables to the planner. These are variables that represent some expression that we desire to compute below the top level of the plan, and then let that value "bubble up" as though it were a plain Var (ie, a column value). The immediate application is to allow sub-selects to be flattened even when they are below an outer join and have non-nullable output expressions. Formerly we couldn't flatten because such an expression wouldn't properly go to NULL when evaluated above the outer join. Now, we wrap it in a PlaceHolderVar and arrange for the actual evaluation to occur below the outer join. When the resulting Var bubbles up through the join, it will be set to NULL if necessary, yielding the correct results. This fixes a planner limitation that's existed since 7.1. In future we might want to use this mechanism to re-introduce some form of Hellerstein's "expensive functions" optimization, ie place the evaluation of an expensive function at the most suitable point in the plan tree.
17 years ago
if (IsA(node, PlaceHolderVar))
{
PlaceHolderVar *phv = (PlaceHolderVar *) node;
/* See if the PlaceHolderVar has bubbled up from a lower plan node */
Add support for INSERT ... ON CONFLICT DO NOTHING/UPDATE. The newly added ON CONFLICT clause allows to specify an alternative to raising a unique or exclusion constraint violation error when inserting. ON CONFLICT refers to constraints that can either be specified using a inference clause (by specifying the columns of a unique constraint) or by naming a unique or exclusion constraint. DO NOTHING avoids the constraint violation, without touching the pre-existing row. DO UPDATE SET ... [WHERE ...] updates the pre-existing tuple, and has access to both the tuple proposed for insertion and the existing tuple; the optional WHERE clause can be used to prevent an update from being executed. The UPDATE SET and WHERE clauses have access to the tuple proposed for insertion using the "magic" EXCLUDED alias, and to the pre-existing tuple using the table name or its alias. This feature is often referred to as upsert. This is implemented using a new infrastructure called "speculative insertion". It is an optimistic variant of regular insertion that first does a pre-check for existing tuples and then attempts an insert. If a violating tuple was inserted concurrently, the speculatively inserted tuple is deleted and a new attempt is made. If the pre-check finds a matching tuple the alternative DO NOTHING or DO UPDATE action is taken. If the insertion succeeds without detecting a conflict, the tuple is deemed inserted. To handle the possible ambiguity between the excluded alias and a table named excluded, and for convenience with long relation names, INSERT INTO now can alias its target table. Bumps catversion as stored rules change. Author: Peter Geoghegan, with significant contributions from Heikki Linnakangas and Andres Freund. Testing infrastructure by Jeff Janes. Reviewed-By: Heikki Linnakangas, Andres Freund, Robert Haas, Simon Riggs, Dean Rasheed, Stephen Frost and many others.
11 years ago
if (context->outer_itlist && context->outer_itlist->has_ph_vars)
Add a concept of "placeholder" variables to the planner. These are variables that represent some expression that we desire to compute below the top level of the plan, and then let that value "bubble up" as though it were a plain Var (ie, a column value). The immediate application is to allow sub-selects to be flattened even when they are below an outer join and have non-nullable output expressions. Formerly we couldn't flatten because such an expression wouldn't properly go to NULL when evaluated above the outer join. Now, we wrap it in a PlaceHolderVar and arrange for the actual evaluation to occur below the outer join. When the resulting Var bubbles up through the join, it will be set to NULL if necessary, yielding the correct results. This fixes a planner limitation that's existed since 7.1. In future we might want to use this mechanism to re-introduce some form of Hellerstein's "expensive functions" optimization, ie place the evaluation of an expensive function at the most suitable point in the plan tree.
17 years ago
{
newvar = search_indexed_tlist_for_non_var((Node *) phv,
context->outer_itlist,
Rearrange the implementation of index-only scans. This commit changes index-only scans so that data is read directly from the index tuple without first generating a faux heap tuple. The only immediate benefit is that indexes on system columns (such as OID) can be used in index-only scans, but this is necessary infrastructure if we are ever to support index-only scans on expression indexes. The executor is now ready for that, though the planner still needs substantial work to recognize the possibility. To do this, Vars in index-only plan nodes have to refer to index columns not heap columns. I introduced a new special varno, INDEX_VAR, to mark such Vars to avoid confusion. (In passing, this commit renames the two existing special varnos to OUTER_VAR and INNER_VAR.) This allows ruleutils.c to handle them with logic similar to what we use for subplan reference Vars. Since index-only scans are now fundamentally different from regular indexscans so far as their expression subtrees are concerned, I also chose to change them to have their own plan node type (and hence, their own executor source file).
14 years ago
OUTER_VAR);
Add a concept of "placeholder" variables to the planner. These are variables that represent some expression that we desire to compute below the top level of the plan, and then let that value "bubble up" as though it were a plain Var (ie, a column value). The immediate application is to allow sub-selects to be flattened even when they are below an outer join and have non-nullable output expressions. Formerly we couldn't flatten because such an expression wouldn't properly go to NULL when evaluated above the outer join. Now, we wrap it in a PlaceHolderVar and arrange for the actual evaluation to occur below the outer join. When the resulting Var bubbles up through the join, it will be set to NULL if necessary, yielding the correct results. This fixes a planner limitation that's existed since 7.1. In future we might want to use this mechanism to re-introduce some form of Hellerstein's "expensive functions" optimization, ie place the evaluation of an expensive function at the most suitable point in the plan tree.
17 years ago
if (newvar)
return (Node *) newvar;
}
if (context->inner_itlist && context->inner_itlist->has_ph_vars)
{
newvar = search_indexed_tlist_for_non_var((Node *) phv,
context->inner_itlist,
Rearrange the implementation of index-only scans. This commit changes index-only scans so that data is read directly from the index tuple without first generating a faux heap tuple. The only immediate benefit is that indexes on system columns (such as OID) can be used in index-only scans, but this is necessary infrastructure if we are ever to support index-only scans on expression indexes. The executor is now ready for that, though the planner still needs substantial work to recognize the possibility. To do this, Vars in index-only plan nodes have to refer to index columns not heap columns. I introduced a new special varno, INDEX_VAR, to mark such Vars to avoid confusion. (In passing, this commit renames the two existing special varnos to OUTER_VAR and INNER_VAR.) This allows ruleutils.c to handle them with logic similar to what we use for subplan reference Vars. Since index-only scans are now fundamentally different from regular indexscans so far as their expression subtrees are concerned, I also chose to change them to have their own plan node type (and hence, their own executor source file).
14 years ago
INNER_VAR);
Add a concept of "placeholder" variables to the planner. These are variables that represent some expression that we desire to compute below the top level of the plan, and then let that value "bubble up" as though it were a plain Var (ie, a column value). The immediate application is to allow sub-selects to be flattened even when they are below an outer join and have non-nullable output expressions. Formerly we couldn't flatten because such an expression wouldn't properly go to NULL when evaluated above the outer join. Now, we wrap it in a PlaceHolderVar and arrange for the actual evaluation to occur below the outer join. When the resulting Var bubbles up through the join, it will be set to NULL if necessary, yielding the correct results. This fixes a planner limitation that's existed since 7.1. In future we might want to use this mechanism to re-introduce some form of Hellerstein's "expensive functions" optimization, ie place the evaluation of an expensive function at the most suitable point in the plan tree.
17 years ago
if (newvar)
return (Node *) newvar;
}
/* If not supplied by input plans, evaluate the contained expr */
return fix_join_expr_mutator((Node *) phv->phexpr, context);
}
Implement UPDATE tab SET (col1,col2,...) = (SELECT ...), ... This SQL-standard feature allows a sub-SELECT yielding multiple columns (but only one row) to be used to compute the new values of several columns to be updated. While the same results can be had with an independent sub-SELECT per column, such a workaround can require a great deal of duplicated computation. The standard actually says that the source for a multi-column assignment could be any row-valued expression. The implementation used here is tightly tied to our existing sub-SELECT support and can't handle other cases; the Bison grammar would have some issues with them too. However, I don't feel too bad about this since other cases can be converted into sub-SELECTs. For instance, "SET (a,b,c) = row_valued_function(x)" could be written "SET (a,b,c) = (SELECT * FROM row_valued_function(x))".
11 years ago
if (IsA(node, Param))
return fix_param_node(context->root, (Param *) node);
IN clauses appearing at top level of WHERE can now be handled as joins. There are two implementation techniques: the executor understands a new JOIN_IN jointype, which emits at most one matching row per left-hand row, or the result of the IN's sub-select can be fed through a DISTINCT filter and then joined as an ordinary relation. Along the way, some minor code cleanup in the optimizer; notably, break out most of the jointree-rearrangement preprocessing in planner.c and put it in a new file prep/prepjointree.c.
23 years ago
/* Try matching more complex expressions too, if tlists have any */
Add support for INSERT ... ON CONFLICT DO NOTHING/UPDATE. The newly added ON CONFLICT clause allows to specify an alternative to raising a unique or exclusion constraint violation error when inserting. ON CONFLICT refers to constraints that can either be specified using a inference clause (by specifying the columns of a unique constraint) or by naming a unique or exclusion constraint. DO NOTHING avoids the constraint violation, without touching the pre-existing row. DO UPDATE SET ... [WHERE ...] updates the pre-existing tuple, and has access to both the tuple proposed for insertion and the existing tuple; the optional WHERE clause can be used to prevent an update from being executed. The UPDATE SET and WHERE clauses have access to the tuple proposed for insertion using the "magic" EXCLUDED alias, and to the pre-existing tuple using the table name or its alias. This feature is often referred to as upsert. This is implemented using a new infrastructure called "speculative insertion". It is an optimistic variant of regular insertion that first does a pre-check for existing tuples and then attempts an insert. If a violating tuple was inserted concurrently, the speculatively inserted tuple is deleted and a new attempt is made. If the pre-check finds a matching tuple the alternative DO NOTHING or DO UPDATE action is taken. If the insertion succeeds without detecting a conflict, the tuple is deemed inserted. To handle the possible ambiguity between the excluded alias and a table named excluded, and for convenience with long relation names, INSERT INTO now can alias its target table. Bumps catversion as stored rules change. Author: Peter Geoghegan, with significant contributions from Heikki Linnakangas and Andres Freund. Testing infrastructure by Jeff Janes. Reviewed-By: Heikki Linnakangas, Andres Freund, Robert Haas, Simon Riggs, Dean Rasheed, Stephen Frost and many others.
11 years ago
if (context->outer_itlist && context->outer_itlist->has_non_vars)
IN clauses appearing at top level of WHERE can now be handled as joins. There are two implementation techniques: the executor understands a new JOIN_IN jointype, which emits at most one matching row per left-hand row, or the result of the IN's sub-select can be fed through a DISTINCT filter and then joined as an ordinary relation. Along the way, some minor code cleanup in the optimizer; notably, break out most of the jointree-rearrangement preprocessing in planner.c and put it in a new file prep/prepjointree.c.
23 years ago
{
Revise searching of subplan target lists to use something more efficient than tlist_member calls. Building a large join tlist is still O(N^2), but with a much smaller constant factor than before.
21 years ago
newvar = search_indexed_tlist_for_non_var(node,
context->outer_itlist,
Rearrange the implementation of index-only scans. This commit changes index-only scans so that data is read directly from the index tuple without first generating a faux heap tuple. The only immediate benefit is that indexes on system columns (such as OID) can be used in index-only scans, but this is necessary infrastructure if we are ever to support index-only scans on expression indexes. The executor is now ready for that, though the planner still needs substantial work to recognize the possibility. To do this, Vars in index-only plan nodes have to refer to index columns not heap columns. I introduced a new special varno, INDEX_VAR, to mark such Vars to avoid confusion. (In passing, this commit renames the two existing special varnos to OUTER_VAR and INNER_VAR.) This allows ruleutils.c to handle them with logic similar to what we use for subplan reference Vars. Since index-only scans are now fundamentally different from regular indexscans so far as their expression subtrees are concerned, I also chose to change them to have their own plan node type (and hence, their own executor source file).
14 years ago
OUTER_VAR);
Revise searching of subplan target lists to use something more efficient than tlist_member calls. Building a large join tlist is still O(N^2), but with a much smaller constant factor than before.
21 years ago
if (newvar)
IN clauses appearing at top level of WHERE can now be handled as joins. There are two implementation techniques: the executor understands a new JOIN_IN jointype, which emits at most one matching row per left-hand row, or the result of the IN's sub-select can be fed through a DISTINCT filter and then joined as an ordinary relation. Along the way, some minor code cleanup in the optimizer; notably, break out most of the jointree-rearrangement preprocessing in planner.c and put it in a new file prep/prepjointree.c.
23 years ago
return (Node *) newvar;
Revise searching of subplan target lists to use something more efficient than tlist_member calls. Building a large join tlist is still O(N^2), but with a much smaller constant factor than before.
21 years ago
}
if (context->inner_itlist && context->inner_itlist->has_non_vars)
{
newvar = search_indexed_tlist_for_non_var(node,
context->inner_itlist,
Rearrange the implementation of index-only scans. This commit changes index-only scans so that data is read directly from the index tuple without first generating a faux heap tuple. The only immediate benefit is that indexes on system columns (such as OID) can be used in index-only scans, but this is necessary infrastructure if we are ever to support index-only scans on expression indexes. The executor is now ready for that, though the planner still needs substantial work to recognize the possibility. To do this, Vars in index-only plan nodes have to refer to index columns not heap columns. I introduced a new special varno, INDEX_VAR, to mark such Vars to avoid confusion. (In passing, this commit renames the two existing special varnos to OUTER_VAR and INNER_VAR.) This allows ruleutils.c to handle them with logic similar to what we use for subplan reference Vars. Since index-only scans are now fundamentally different from regular indexscans so far as their expression subtrees are concerned, I also chose to change them to have their own plan node type (and hence, their own executor source file).
14 years ago
INNER_VAR);
Revise searching of subplan target lists to use something more efficient than tlist_member calls. Building a large join tlist is still O(N^2), but with a much smaller constant factor than before.
21 years ago
if (newvar)
IN clauses appearing at top level of WHERE can now be handled as joins. There are two implementation techniques: the executor understands a new JOIN_IN jointype, which emits at most one matching row per left-hand row, or the result of the IN's sub-select can be fed through a DISTINCT filter and then joined as an ordinary relation. Along the way, some minor code cleanup in the optimizer; notably, break out most of the jointree-rearrangement preprocessing in planner.c and put it in a new file prep/prepjointree.c.
23 years ago
return (Node *) newvar;
}
Rearrange planner to save the whole PlannerInfo (subroot) for a subquery. Formerly, set_subquery_pathlist and other creators of plans for subqueries saved only the rangetable and rowMarks lists from the lower-level PlannerInfo. But there's no reason not to remember the whole PlannerInfo, and indeed this turns out to simplify matters in a number of places. The immediate reason for doing this was so that the subroot will still be accessible when we're trying to extract column statistics out of an already-planned subquery. But now that I've done it, it seems like a good code-beautification effort in its own right. I also chose to get rid of the transient subrtable and subrowmark fields in SubqueryScan nodes, in favor of having setrefs.c look up the subquery's RelOptInfo. That required changing all the APIs in setrefs.c to pass PlannerInfo not PlannerGlobal, which was a large but quite mechanical transformation. One side-effect not foreseen at the beginning is that this finally broke inheritance_planner's assumption that replanning the same subquery RTE N times would necessarily give interchangeable results each time. That assumption was always pretty risky, but now we really have to make a separate RTE for each instance so that there's a place to carry the separate subroots.
14 years ago
fix_expr_common(context->root, node);
Further planner/optimizer cleanups. Move all set_tlist_references and fix_opids processing to a single recursive pass over the plan tree executed at the very tail end of planning, rather than haphazardly here and there at different places. Now that tlist Vars do not get modified until the very end, it's possible to get rid of the klugy var_equal and match_varid partial-matching routines, and just use plain equal() throughout the optimizer. This is a step towards allowing merge and hash joins to be done on expressions instead of only Vars ...
26 years ago
return expression_tree_mutator(node,
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
fix_join_expr_mutator,
Further planner/optimizer cleanups. Move all set_tlist_references and fix_opids processing to a single recursive pass over the plan tree executed at the very tail end of planning, rather than haphazardly here and there at different places. Now that tlist Vars do not get modified until the very end, it's possible to get rid of the klugy var_equal and match_varid partial-matching routines, and just use plain equal() throughout the optimizer. This is a step towards allowing merge and hash joins to be done on expressions instead of only Vars ...
26 years ago
(void *) context);
Postgres95 1.01 Distribution - Virgin Sources
30 years ago
}
Further planner/optimizer cleanups. Move all set_tlist_references and fix_opids processing to a single recursive pass over the plan tree executed at the very tail end of planning, rather than haphazardly here and there at different places. Now that tlist Vars do not get modified until the very end, it's possible to get rid of the klugy var_equal and match_varid partial-matching routines, and just use plain equal() throughout the optimizer. This is a step towards allowing merge and hash joins to be done on expressions instead of only Vars ...
26 years ago
/*
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
* fix_upper_expr
* Modifies an expression tree so that all Var nodes reference outputs
Fix the plan-invalidation mechanism to treat regclass constants that refer to a relation as a reason to invalidate a plan when the relation changes. This handles scenarios such as dropping/recreating a sequence that is referenced by nextval('seq') in a cached plan. Rather than teach plancache.c all about digging through plan trees to find regclass Consts, we charge the planner's setrefs.c with making a list of the relation OIDs on which each plan depends. That way the list can be built cheaply during a plan tree traversal that has to happen anyway. Per bug #3662 and subsequent discussion.
18 years ago
* of a subplan. Also performs opcode lookup, and adds regclass OIDs to
Rearrange planner to save the whole PlannerInfo (subroot) for a subquery. Formerly, set_subquery_pathlist and other creators of plans for subqueries saved only the rangetable and rowMarks lists from the lower-level PlannerInfo. But there's no reason not to remember the whole PlannerInfo, and indeed this turns out to simplify matters in a number of places. The immediate reason for doing this was so that the subroot will still be accessible when we're trying to extract column statistics out of an already-planned subquery. But now that I've done it, it seems like a good code-beautification effort in its own right. I also chose to get rid of the transient subrtable and subrowmark fields in SubqueryScan nodes, in favor of having setrefs.c look up the subquery's RelOptInfo. That required changing all the APIs in setrefs.c to pass PlannerInfo not PlannerGlobal, which was a large but quite mechanical transformation. One side-effect not foreseen at the beginning is that this finally broke inheritance_planner's assumption that replanning the same subquery RTE N times would necessarily give interchangeable results each time. That assumption was always pretty risky, but now we really have to make a separate RTE for each instance so that there's a place to carry the separate subroots.
14 years ago
* root->glob->relationOids.
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
*
* This is used to fix up target and qual expressions of non-join upper-level
Rearrange the implementation of index-only scans. This commit changes index-only scans so that data is read directly from the index tuple without first generating a faux heap tuple. The only immediate benefit is that indexes on system columns (such as OID) can be used in index-only scans, but this is necessary infrastructure if we are ever to support index-only scans on expression indexes. The executor is now ready for that, though the planner still needs substantial work to recognize the possibility. To do this, Vars in index-only plan nodes have to refer to index columns not heap columns. I introduced a new special varno, INDEX_VAR, to mark such Vars to avoid confusion. (In passing, this commit renames the two existing special varnos to OUTER_VAR and INNER_VAR.) This allows ruleutils.c to handle them with logic similar to what we use for subplan reference Vars. Since index-only scans are now fundamentally different from regular indexscans so far as their expression subtrees are concerned, I also chose to change them to have their own plan node type (and hence, their own executor source file).
14 years ago
* plan nodes, as well as index-only scan nodes.
Revise union_planner and associated routines to clean up breakage from EXCEPT/HAVING patch. Cases involving nontrivial GROUP BY expressions now work again. Also, the code is at least somewhat better documented...
27 years ago
*
Further planner/optimizer cleanups. Move all set_tlist_references and fix_opids processing to a single recursive pass over the plan tree executed at the very tail end of planning, rather than haphazardly here and there at different places. Now that tlist Vars do not get modified until the very end, it's possible to get rid of the klugy var_equal and match_varid partial-matching routines, and just use plain equal() throughout the optimizer. This is a step towards allowing merge and hash joins to be done on expressions instead of only Vars ...
26 years ago
* An error is raised if no matching var can be found in the subplan tlist
* --- so this routine should only be applied to nodes whose subplans'
* targetlists were generated via flatten_tlist() or some such method.
Postgres95 1.01 Distribution - Virgin Sources
30 years ago
*
Revise searching of subplan target lists to use something more efficient than tlist_member calls. Building a large join tlist is still O(N^2), but with a much smaller constant factor than before.
21 years ago
* If itlist->has_non_vars is true, then we try to match whole subexpressions
Fix problems with subselects used in GROUP BY expressions, per gripe from Philip Warner. Side effect of change is that GROUP BY expressions will not be re-evaluated at multiple plan levels anymore, whereas this sometimes happened with old code.
24 years ago
* against elements of the subplan tlist, so that we can avoid recomputing
* expressions that were already computed by the subplan. (This is relatively
* expensive, so we don't want to try it in the common case where the
* subplan tlist is just a flattened list of Vars.)
*
* 'node': the tree to be fixed (a target item or qual)
Rearrange the implementation of index-only scans. This commit changes index-only scans so that data is read directly from the index tuple without first generating a faux heap tuple. The only immediate benefit is that indexes on system columns (such as OID) can be used in index-only scans, but this is necessary infrastructure if we are ever to support index-only scans on expression indexes. The executor is now ready for that, though the planner still needs substantial work to recognize the possibility. To do this, Vars in index-only plan nodes have to refer to index columns not heap columns. I introduced a new special varno, INDEX_VAR, to mark such Vars to avoid confusion. (In passing, this commit renames the two existing special varnos to OUTER_VAR and INNER_VAR.) This allows ruleutils.c to handle them with logic similar to what we use for subplan reference Vars. Since index-only scans are now fundamentally different from regular indexscans so far as their expression subtrees are concerned, I also chose to change them to have their own plan node type (and hence, their own executor source file).
14 years ago
* 'subplan_itlist': indexed target list for subplan (or index)
* 'newvarno': varno to use for Vars referencing tlist elements
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
* 'rtoffset': how much to increment varnoold by
Massive commit to run PGINDENT on all *.c and *.h files.
28 years ago
*
Further planner/optimizer cleanups. Move all set_tlist_references and fix_opids processing to a single recursive pass over the plan tree executed at the very tail end of planning, rather than haphazardly here and there at different places. Now that tlist Vars do not get modified until the very end, it's possible to get rid of the klugy var_equal and match_varid partial-matching routines, and just use plain equal() throughout the optimizer. This is a step towards allowing merge and hash joins to be done on expressions instead of only Vars ...
26 years ago
* The resulting tree is a copy of the original in which all Var nodes have
Rearrange the implementation of index-only scans. This commit changes index-only scans so that data is read directly from the index tuple without first generating a faux heap tuple. The only immediate benefit is that indexes on system columns (such as OID) can be used in index-only scans, but this is necessary infrastructure if we are ever to support index-only scans on expression indexes. The executor is now ready for that, though the planner still needs substantial work to recognize the possibility. To do this, Vars in index-only plan nodes have to refer to index columns not heap columns. I introduced a new special varno, INDEX_VAR, to mark such Vars to avoid confusion. (In passing, this commit renames the two existing special varnos to OUTER_VAR and INNER_VAR.) This allows ruleutils.c to handle them with logic similar to what we use for subplan reference Vars. Since index-only scans are now fundamentally different from regular indexscans so far as their expression subtrees are concerned, I also chose to change them to have their own plan node type (and hence, their own executor source file).
14 years ago
* varno = newvarno, varattno = resno of corresponding targetlist element.
Further planner/optimizer cleanups. Move all set_tlist_references and fix_opids processing to a single recursive pass over the plan tree executed at the very tail end of planning, rather than haphazardly here and there at different places. Now that tlist Vars do not get modified until the very end, it's possible to get rid of the klugy var_equal and match_varid partial-matching routines, and just use plain equal() throughout the optimizer. This is a step towards allowing merge and hash joins to be done on expressions instead of only Vars ...
26 years ago
* The original tree is not modified.
Postgres95 1.01 Distribution - Virgin Sources
30 years ago
*/
Further planner/optimizer cleanups. Move all set_tlist_references and fix_opids processing to a single recursive pass over the plan tree executed at the very tail end of planning, rather than haphazardly here and there at different places. Now that tlist Vars do not get modified until the very end, it's possible to get rid of the klugy var_equal and match_varid partial-matching routines, and just use plain equal() throughout the optimizer. This is a step towards allowing merge and hash joins to be done on expressions instead of only Vars ...
26 years ago
static Node *
Rearrange planner to save the whole PlannerInfo (subroot) for a subquery. Formerly, set_subquery_pathlist and other creators of plans for subqueries saved only the rangetable and rowMarks lists from the lower-level PlannerInfo. But there's no reason not to remember the whole PlannerInfo, and indeed this turns out to simplify matters in a number of places. The immediate reason for doing this was so that the subroot will still be accessible when we're trying to extract column statistics out of an already-planned subquery. But now that I've done it, it seems like a good code-beautification effort in its own right. I also chose to get rid of the transient subrtable and subrowmark fields in SubqueryScan nodes, in favor of having setrefs.c look up the subquery's RelOptInfo. That required changing all the APIs in setrefs.c to pass PlannerInfo not PlannerGlobal, which was a large but quite mechanical transformation. One side-effect not foreseen at the beginning is that this finally broke inheritance_planner's assumption that replanning the same subquery RTE N times would necessarily give interchangeable results each time. That assumption was always pretty risky, but now we really have to make a separate RTE for each instance so that there's a place to carry the separate subroots.
14 years ago
fix_upper_expr(PlannerInfo *root,
Fix the plan-invalidation mechanism to treat regclass constants that refer to a relation as a reason to invalidate a plan when the relation changes. This handles scenarios such as dropping/recreating a sequence that is referenced by nextval('seq') in a cached plan. Rather than teach plancache.c all about digging through plan trees to find regclass Consts, we charge the planner's setrefs.c with making a list of the relation OIDs on which each plan depends. That way the list can be built cheaply during a plan tree traversal that has to happen anyway. Per bug #3662 and subsequent discussion.
18 years ago
Node *node,
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
indexed_tlist *subplan_itlist,
Rearrange the implementation of index-only scans. This commit changes index-only scans so that data is read directly from the index tuple without first generating a faux heap tuple. The only immediate benefit is that indexes on system columns (such as OID) can be used in index-only scans, but this is necessary infrastructure if we are ever to support index-only scans on expression indexes. The executor is now ready for that, though the planner still needs substantial work to recognize the possibility. To do this, Vars in index-only plan nodes have to refer to index columns not heap columns. I introduced a new special varno, INDEX_VAR, to mark such Vars to avoid confusion. (In passing, this commit renames the two existing special varnos to OUTER_VAR and INNER_VAR.) This allows ruleutils.c to handle them with logic similar to what we use for subplan reference Vars. Since index-only scans are now fundamentally different from regular indexscans so far as their expression subtrees are concerned, I also chose to change them to have their own plan node type (and hence, their own executor source file).
14 years ago
Index newvarno,
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
int rtoffset)
Postgres95 1.01 Distribution - Virgin Sources
30 years ago
{
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
fix_upper_expr_context context;
Postgres95 1.01 Distribution - Virgin Sources
30 years ago
Rearrange planner to save the whole PlannerInfo (subroot) for a subquery. Formerly, set_subquery_pathlist and other creators of plans for subqueries saved only the rangetable and rowMarks lists from the lower-level PlannerInfo. But there's no reason not to remember the whole PlannerInfo, and indeed this turns out to simplify matters in a number of places. The immediate reason for doing this was so that the subroot will still be accessible when we're trying to extract column statistics out of an already-planned subquery. But now that I've done it, it seems like a good code-beautification effort in its own right. I also chose to get rid of the transient subrtable and subrowmark fields in SubqueryScan nodes, in favor of having setrefs.c look up the subquery's RelOptInfo. That required changing all the APIs in setrefs.c to pass PlannerInfo not PlannerGlobal, which was a large but quite mechanical transformation. One side-effect not foreseen at the beginning is that this finally broke inheritance_planner's assumption that replanning the same subquery RTE N times would necessarily give interchangeable results each time. That assumption was always pretty risky, but now we really have to make a separate RTE for each instance so that there's a place to carry the separate subroots.
14 years ago
context.root = root;
Revise searching of subplan target lists to use something more efficient than tlist_member calls. Building a large join tlist is still O(N^2), but with a much smaller constant factor than before.
21 years ago
context.subplan_itlist = subplan_itlist;
Rearrange the implementation of index-only scans. This commit changes index-only scans so that data is read directly from the index tuple without first generating a faux heap tuple. The only immediate benefit is that indexes on system columns (such as OID) can be used in index-only scans, but this is necessary infrastructure if we are ever to support index-only scans on expression indexes. The executor is now ready for that, though the planner still needs substantial work to recognize the possibility. To do this, Vars in index-only plan nodes have to refer to index columns not heap columns. I introduced a new special varno, INDEX_VAR, to mark such Vars to avoid confusion. (In passing, this commit renames the two existing special varnos to OUTER_VAR and INNER_VAR.) This allows ruleutils.c to handle them with logic similar to what we use for subplan reference Vars. Since index-only scans are now fundamentally different from regular indexscans so far as their expression subtrees are concerned, I also chose to change them to have their own plan node type (and hence, their own executor source file).
14 years ago
context.newvarno = newvarno;
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
context.rtoffset = rtoffset;
return fix_upper_expr_mutator(node, &context);
Clean up routines in setrefs.c by replacing individual tree walking logic with expression_tree_walker/mutator calls.
26 years ago
}
pgindent run over code.
27 years ago
Further planner/optimizer cleanups. Move all set_tlist_references and fix_opids processing to a single recursive pass over the plan tree executed at the very tail end of planning, rather than haphazardly here and there at different places. Now that tlist Vars do not get modified until the very end, it's possible to get rid of the klugy var_equal and match_varid partial-matching routines, and just use plain equal() throughout the optimizer. This is a step towards allowing merge and hash joins to be done on expressions instead of only Vars ...
26 years ago
static Node *
Re-run pgindent with updated list of typedefs. (Updated README should avoid this problem in the future.)
18 years ago
fix_upper_expr_mutator(Node *node, fix_upper_expr_context *context)
Clean up routines in setrefs.c by replacing individual tree walking logic with expression_tree_walker/mutator calls.
26 years ago
{
Revise searching of subplan target lists to use something more efficient than tlist_member calls. Building a large join tlist is still O(N^2), but with a much smaller constant factor than before.
21 years ago
Var *newvar;
Clean up routines in setrefs.c by replacing individual tree walking logic with expression_tree_walker/mutator calls.
26 years ago
if (node == NULL)
Further planner/optimizer cleanups. Move all set_tlist_references and fix_opids processing to a single recursive pass over the plan tree executed at the very tail end of planning, rather than haphazardly here and there at different places. Now that tlist Vars do not get modified until the very end, it's possible to get rid of the klugy var_equal and match_varid partial-matching routines, and just use plain equal() throughout the optimizer. This is a step towards allowing merge and hash joins to be done on expressions instead of only Vars ...
26 years ago
return NULL;
Clean up routines in setrefs.c by replacing individual tree walking logic with expression_tree_walker/mutator calls.
26 years ago
if (IsA(node, Var))
{
Var *var = (Var *) node;
Revise union_planner and associated routines to clean up breakage from EXCEPT/HAVING patch. Cases involving nontrivial GROUP BY expressions now work again. Also, the code is at least somewhat better documented...
27 years ago
Revise searching of subplan target lists to use something more efficient than tlist_member calls. Building a large join tlist is still O(N^2), but with a much smaller constant factor than before.
21 years ago
newvar = search_indexed_tlist_for_var(var,
context->subplan_itlist,
Rearrange the implementation of index-only scans. This commit changes index-only scans so that data is read directly from the index tuple without first generating a faux heap tuple. The only immediate benefit is that indexes on system columns (such as OID) can be used in index-only scans, but this is necessary infrastructure if we are ever to support index-only scans on expression indexes. The executor is now ready for that, though the planner still needs substantial work to recognize the possibility. To do this, Vars in index-only plan nodes have to refer to index columns not heap columns. I introduced a new special varno, INDEX_VAR, to mark such Vars to avoid confusion. (In passing, this commit renames the two existing special varnos to OUTER_VAR and INNER_VAR.) This allows ruleutils.c to handle them with logic similar to what we use for subplan reference Vars. Since index-only scans are now fundamentally different from regular indexscans so far as their expression subtrees are concerned, I also chose to change them to have their own plan node type (and hence, their own executor source file).
14 years ago
context->newvarno,
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
context->rtoffset);
Revise searching of subplan target lists to use something more efficient than tlist_member calls. Building a large join tlist is still O(N^2), but with a much smaller constant factor than before.
21 years ago
if (!newvar)
Error message editing in backend/optimizer, backend/rewrite.
22 years ago
elog(ERROR, "variable not found in subplan target list");
Further planner/optimizer cleanups. Move all set_tlist_references and fix_opids processing to a single recursive pass over the plan tree executed at the very tail end of planning, rather than haphazardly here and there at different places. Now that tlist Vars do not get modified until the very end, it's possible to get rid of the klugy var_equal and match_varid partial-matching routines, and just use plain equal() throughout the optimizer. This is a step towards allowing merge and hash joins to be done on expressions instead of only Vars ...
26 years ago
return (Node *) newvar;
}
Add a concept of "placeholder" variables to the planner. These are variables that represent some expression that we desire to compute below the top level of the plan, and then let that value "bubble up" as though it were a plain Var (ie, a column value). The immediate application is to allow sub-selects to be flattened even when they are below an outer join and have non-nullable output expressions. Formerly we couldn't flatten because such an expression wouldn't properly go to NULL when evaluated above the outer join. Now, we wrap it in a PlaceHolderVar and arrange for the actual evaluation to occur below the outer join. When the resulting Var bubbles up through the join, it will be set to NULL if necessary, yielding the correct results. This fixes a planner limitation that's existed since 7.1. In future we might want to use this mechanism to re-introduce some form of Hellerstein's "expensive functions" optimization, ie place the evaluation of an expensive function at the most suitable point in the plan tree.
17 years ago
if (IsA(node, PlaceHolderVar))
{
PlaceHolderVar *phv = (PlaceHolderVar *) node;
/* See if the PlaceHolderVar has bubbled up from a lower plan node */
if (context->subplan_itlist->has_ph_vars)
{
newvar = search_indexed_tlist_for_non_var((Node *) phv,
context->subplan_itlist,
Rearrange the implementation of index-only scans. This commit changes index-only scans so that data is read directly from the index tuple without first generating a faux heap tuple. The only immediate benefit is that indexes on system columns (such as OID) can be used in index-only scans, but this is necessary infrastructure if we are ever to support index-only scans on expression indexes. The executor is now ready for that, though the planner still needs substantial work to recognize the possibility. To do this, Vars in index-only plan nodes have to refer to index columns not heap columns. I introduced a new special varno, INDEX_VAR, to mark such Vars to avoid confusion. (In passing, this commit renames the two existing special varnos to OUTER_VAR and INNER_VAR.) This allows ruleutils.c to handle them with logic similar to what we use for subplan reference Vars. Since index-only scans are now fundamentally different from regular indexscans so far as their expression subtrees are concerned, I also chose to change them to have their own plan node type (and hence, their own executor source file).
14 years ago
context->newvarno);
Add a concept of "placeholder" variables to the planner. These are variables that represent some expression that we desire to compute below the top level of the plan, and then let that value "bubble up" as though it were a plain Var (ie, a column value). The immediate application is to allow sub-selects to be flattened even when they are below an outer join and have non-nullable output expressions. Formerly we couldn't flatten because such an expression wouldn't properly go to NULL when evaluated above the outer join. Now, we wrap it in a PlaceHolderVar and arrange for the actual evaluation to occur below the outer join. When the resulting Var bubbles up through the join, it will be set to NULL if necessary, yielding the correct results. This fixes a planner limitation that's existed since 7.1. In future we might want to use this mechanism to re-introduce some form of Hellerstein's "expensive functions" optimization, ie place the evaluation of an expensive function at the most suitable point in the plan tree.
17 years ago
if (newvar)
return (Node *) newvar;
}
/* If not supplied by input plan, evaluate the contained expr */
return fix_upper_expr_mutator((Node *) phv->phexpr, context);
}
Implement UPDATE tab SET (col1,col2,...) = (SELECT ...), ... This SQL-standard feature allows a sub-SELECT yielding multiple columns (but only one row) to be used to compute the new values of several columns to be updated. While the same results can be had with an independent sub-SELECT per column, such a workaround can require a great deal of duplicated computation. The standard actually says that the source for a multi-column assignment could be any row-valued expression. The implementation used here is tightly tied to our existing sub-SELECT support and can't handle other cases; the Bison grammar would have some issues with them too. However, I don't feel too bad about this since other cases can be converted into sub-SELECTs. For instance, "SET (a,b,c) = row_valued_function(x)" could be written "SET (a,b,c) = (SELECT * FROM row_valued_function(x))".
11 years ago
if (IsA(node, Param))
return fix_param_node(context->root, (Param *) node);
Fix problems with subselects used in GROUP BY expressions, per gripe from Philip Warner. Side effect of change is that GROUP BY expressions will not be re-evaluated at multiple plan levels anymore, whereas this sometimes happened with old code.
24 years ago
/* Try matching more complex expressions too, if tlist has any */
Revise searching of subplan target lists to use something more efficient than tlist_member calls. Building a large join tlist is still O(N^2), but with a much smaller constant factor than before.
21 years ago
if (context->subplan_itlist->has_non_vars)
Fix problems with subselects used in GROUP BY expressions, per gripe from Philip Warner. Side effect of change is that GROUP BY expressions will not be re-evaluated at multiple plan levels anymore, whereas this sometimes happened with old code.
24 years ago
{
Revise searching of subplan target lists to use something more efficient than tlist_member calls. Building a large join tlist is still O(N^2), but with a much smaller constant factor than before.
21 years ago
newvar = search_indexed_tlist_for_non_var(node,
context->subplan_itlist,
Rearrange the implementation of index-only scans. This commit changes index-only scans so that data is read directly from the index tuple without first generating a faux heap tuple. The only immediate benefit is that indexes on system columns (such as OID) can be used in index-only scans, but this is necessary infrastructure if we are ever to support index-only scans on expression indexes. The executor is now ready for that, though the planner still needs substantial work to recognize the possibility. To do this, Vars in index-only plan nodes have to refer to index columns not heap columns. I introduced a new special varno, INDEX_VAR, to mark such Vars to avoid confusion. (In passing, this commit renames the two existing special varnos to OUTER_VAR and INNER_VAR.) This allows ruleutils.c to handle them with logic similar to what we use for subplan reference Vars. Since index-only scans are now fundamentally different from regular indexscans so far as their expression subtrees are concerned, I also chose to change them to have their own plan node type (and hence, their own executor source file).
14 years ago
context->newvarno);
Revise searching of subplan target lists to use something more efficient than tlist_member calls. Building a large join tlist is still O(N^2), but with a much smaller constant factor than before.
21 years ago
if (newvar)
Fix problems with subselects used in GROUP BY expressions, per gripe from Philip Warner. Side effect of change is that GROUP BY expressions will not be re-evaluated at multiple plan levels anymore, whereas this sometimes happened with old code.
24 years ago
return (Node *) newvar;
}
Rearrange planner to save the whole PlannerInfo (subroot) for a subquery. Formerly, set_subquery_pathlist and other creators of plans for subqueries saved only the rangetable and rowMarks lists from the lower-level PlannerInfo. But there's no reason not to remember the whole PlannerInfo, and indeed this turns out to simplify matters in a number of places. The immediate reason for doing this was so that the subroot will still be accessible when we're trying to extract column statistics out of an already-planned subquery. But now that I've done it, it seems like a good code-beautification effort in its own right. I also chose to get rid of the transient subrtable and subrowmark fields in SubqueryScan nodes, in favor of having setrefs.c look up the subquery's RelOptInfo. That required changing all the APIs in setrefs.c to pass PlannerInfo not PlannerGlobal, which was a large but quite mechanical transformation. One side-effect not foreseen at the beginning is that this finally broke inheritance_planner's assumption that replanning the same subquery RTE N times would necessarily give interchangeable results each time. That assumption was always pretty risky, but now we really have to make a separate RTE for each instance so that there's a place to carry the separate subroots.
14 years ago
fix_expr_common(context->root, node);
Further planner/optimizer cleanups. Move all set_tlist_references and fix_opids processing to a single recursive pass over the plan tree executed at the very tail end of planning, rather than haphazardly here and there at different places. Now that tlist Vars do not get modified until the very end, it's possible to get rid of the klugy var_equal and match_varid partial-matching routines, and just use plain equal() throughout the optimizer. This is a step towards allowing merge and hash joins to be done on expressions instead of only Vars ...
26 years ago
return expression_tree_mutator(node,
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
fix_upper_expr_mutator,
Further planner/optimizer cleanups. Move all set_tlist_references and fix_opids processing to a single recursive pass over the plan tree executed at the very tail end of planning, rather than haphazardly here and there at different places. Now that tlist Vars do not get modified until the very end, it's possible to get rid of the klugy var_equal and match_varid partial-matching routines, and just use plain equal() throughout the optimizer. This is a step towards allowing merge and hash joins to be done on expressions instead of only Vars ...
26 years ago
(void *) context);
Postgres95 1.01 Distribution - Virgin Sources
30 years ago
}
Add INSERT/UPDATE/DELETE RETURNING, with basic docs and regression tests. plpgsql support to come later. Along the way, convert execMain's SELECT INTO support into a DestReceiver, in order to eliminate some ugly special cases. Jonah Harris and Tom Lane
19 years ago
/*
* set_returning_clause_references
* Perform setrefs.c's work on a RETURNING targetlist
*
* If the query involves more than just the result table, we have to
* adjust any Vars that refer to other tables to reference junk tlist
Split the processing of INSERT/UPDATE/DELETE operations out of execMain.c. They are now handled by a new plan node type called ModifyTable, which is placed at the top of the plan tree. In itself this change doesn't do much, except perhaps make the handling of RETURNING lists and inherited UPDATEs a tad less klugy. But it is necessary preparation for the intended extension of allowing RETURNING queries inside WITH. Marko Tiikkaja
16 years ago
* entries in the top subplan's targetlist. Vars referencing the result
Add INSERT/UPDATE/DELETE RETURNING, with basic docs and regression tests. plpgsql support to come later. Along the way, convert execMain's SELECT INTO support into a DestReceiver, in order to eliminate some ugly special cases. Jonah Harris and Tom Lane
19 years ago
* table should be left alone, however (the executor will evaluate them
pgindent run for 9.4 This includes removing tabs after periods in C comments, which was applied to back branches, so this change should not effect backpatching.
12 years ago
* using the actual heap tuple, after firing triggers if any). In the
Fix planner's handling of RETURNING lists in writable CTEs. setrefs.c failed to do "rtoffset" adjustment of Vars in RETURNING lists, which meant they were left with the wrong varnos when the RETURNING list was in a subquery. That was never possible before writable CTEs, of course, but now it's broken. The executor fails to notice any problem because ExecEvalVar just references the ecxt_scantuple for any normal varno; but EXPLAIN breaks when the varno is wrong, as illustrated in a recent complaint from Bartosz Dmytrak. Since the eventual rtoffset of the subquery is not known at the time we are preparing its plan node, the previous scheme of executing set_returning_clause_references() at that time cannot handle this adjustment. Fortunately, it turns out that we don't really need to do it that way, because all the needed information is available during normal setrefs.c execution; we just have to dig it out of the ModifyTable node. So, do that, and get rid of the kluge of early setrefs processing of RETURNING lists. (This is a little bit of a cheat in the case of inherited UPDATE/DELETE, because we are not passing a "root" struct that corresponds exactly to what the subplan was built with. But that doesn't matter, and anyway this is less ugly than early setrefs processing was.) Back-patch to 9.1, where the problem became possible to hit.
14 years ago
* adjusted RETURNING list, result-table Vars will have their original
* varno (plus rtoffset), but Vars for other rels will have varno OUTER_VAR.
Add INSERT/UPDATE/DELETE RETURNING, with basic docs and regression tests. plpgsql support to come later. Along the way, convert execMain's SELECT INTO support into a DestReceiver, in order to eliminate some ugly special cases. Jonah Harris and Tom Lane
19 years ago
*
Fix the plan-invalidation mechanism to treat regclass constants that refer to a relation as a reason to invalidate a plan when the relation changes. This handles scenarios such as dropping/recreating a sequence that is referenced by nextval('seq') in a cached plan. Rather than teach plancache.c all about digging through plan trees to find regclass Consts, we charge the planner's setrefs.c with making a list of the relation OIDs on which each plan depends. That way the list can be built cheaply during a plan tree traversal that has to happen anyway. Per bug #3662 and subsequent discussion.
18 years ago
* We also must perform opcode lookup and add regclass OIDs to
Rearrange planner to save the whole PlannerInfo (subroot) for a subquery. Formerly, set_subquery_pathlist and other creators of plans for subqueries saved only the rangetable and rowMarks lists from the lower-level PlannerInfo. But there's no reason not to remember the whole PlannerInfo, and indeed this turns out to simplify matters in a number of places. The immediate reason for doing this was so that the subroot will still be accessible when we're trying to extract column statistics out of an already-planned subquery. But now that I've done it, it seems like a good code-beautification effort in its own right. I also chose to get rid of the transient subrtable and subrowmark fields in SubqueryScan nodes, in favor of having setrefs.c look up the subquery's RelOptInfo. That required changing all the APIs in setrefs.c to pass PlannerInfo not PlannerGlobal, which was a large but quite mechanical transformation. One side-effect not foreseen at the beginning is that this finally broke inheritance_planner's assumption that replanning the same subquery RTE N times would necessarily give interchangeable results each time. That assumption was always pretty risky, but now we really have to make a separate RTE for each instance so that there's a place to carry the separate subroots.
14 years ago
* root->glob->relationOids.
Add INSERT/UPDATE/DELETE RETURNING, with basic docs and regression tests. plpgsql support to come later. Along the way, convert execMain's SELECT INTO support into a DestReceiver, in order to eliminate some ugly special cases. Jonah Harris and Tom Lane
19 years ago
*
* 'rlist': the RETURNING targetlist to be fixed
Split the processing of INSERT/UPDATE/DELETE operations out of execMain.c. They are now handled by a new plan node type called ModifyTable, which is placed at the top of the plan tree. In itself this change doesn't do much, except perhaps make the handling of RETURNING lists and inherited UPDATEs a tad less klugy. But it is necessary preparation for the intended extension of allowing RETURNING queries inside WITH. Marko Tiikkaja
16 years ago
* 'topplan': the top subplan node that will be just below the ModifyTable
Fix planner's handling of RETURNING lists in writable CTEs. setrefs.c failed to do "rtoffset" adjustment of Vars in RETURNING lists, which meant they were left with the wrong varnos when the RETURNING list was in a subquery. That was never possible before writable CTEs, of course, but now it's broken. The executor fails to notice any problem because ExecEvalVar just references the ecxt_scantuple for any normal varno; but EXPLAIN breaks when the varno is wrong, as illustrated in a recent complaint from Bartosz Dmytrak. Since the eventual rtoffset of the subquery is not known at the time we are preparing its plan node, the previous scheme of executing set_returning_clause_references() at that time cannot handle this adjustment. Fortunately, it turns out that we don't really need to do it that way, because all the needed information is available during normal setrefs.c execution; we just have to dig it out of the ModifyTable node. So, do that, and get rid of the kluge of early setrefs processing of RETURNING lists. (This is a little bit of a cheat in the case of inherited UPDATE/DELETE, because we are not passing a "root" struct that corresponds exactly to what the subplan was built with. But that doesn't matter, and anyway this is less ugly than early setrefs processing was.) Back-patch to 9.1, where the problem became possible to hit.
14 years ago
* node (note it's not yet passed through set_plan_refs)
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
* 'resultRelation': RT index of the associated result relation
Fix planner's handling of RETURNING lists in writable CTEs. setrefs.c failed to do "rtoffset" adjustment of Vars in RETURNING lists, which meant they were left with the wrong varnos when the RETURNING list was in a subquery. That was never possible before writable CTEs, of course, but now it's broken. The executor fails to notice any problem because ExecEvalVar just references the ecxt_scantuple for any normal varno; but EXPLAIN breaks when the varno is wrong, as illustrated in a recent complaint from Bartosz Dmytrak. Since the eventual rtoffset of the subquery is not known at the time we are preparing its plan node, the previous scheme of executing set_returning_clause_references() at that time cannot handle this adjustment. Fortunately, it turns out that we don't really need to do it that way, because all the needed information is available during normal setrefs.c execution; we just have to dig it out of the ModifyTable node. So, do that, and get rid of the kluge of early setrefs processing of RETURNING lists. (This is a little bit of a cheat in the case of inherited UPDATE/DELETE, because we are not passing a "root" struct that corresponds exactly to what the subplan was built with. But that doesn't matter, and anyway this is less ugly than early setrefs processing was.) Back-patch to 9.1, where the problem became possible to hit.
14 years ago
* 'rtoffset': how much to increment varnos by
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
*
Fix planner's handling of RETURNING lists in writable CTEs. setrefs.c failed to do "rtoffset" adjustment of Vars in RETURNING lists, which meant they were left with the wrong varnos when the RETURNING list was in a subquery. That was never possible before writable CTEs, of course, but now it's broken. The executor fails to notice any problem because ExecEvalVar just references the ecxt_scantuple for any normal varno; but EXPLAIN breaks when the varno is wrong, as illustrated in a recent complaint from Bartosz Dmytrak. Since the eventual rtoffset of the subquery is not known at the time we are preparing its plan node, the previous scheme of executing set_returning_clause_references() at that time cannot handle this adjustment. Fortunately, it turns out that we don't really need to do it that way, because all the needed information is available during normal setrefs.c execution; we just have to dig it out of the ModifyTable node. So, do that, and get rid of the kluge of early setrefs processing of RETURNING lists. (This is a little bit of a cheat in the case of inherited UPDATE/DELETE, because we are not passing a "root" struct that corresponds exactly to what the subplan was built with. But that doesn't matter, and anyway this is less ugly than early setrefs processing was.) Back-patch to 9.1, where the problem became possible to hit.
14 years ago
* Note: the given 'root' is for the parent query level, not the 'topplan'.
* This does not matter currently since we only access the dependency-item
* lists in root->glob, but it would need some hacking if we wanted a root
* that actually matches the subplan.
*
* Note: resultRelation is not yet adjusted by rtoffset.
Add INSERT/UPDATE/DELETE RETURNING, with basic docs and regression tests. plpgsql support to come later. Along the way, convert execMain's SELECT INTO support into a DestReceiver, in order to eliminate some ugly special cases. Jonah Harris and Tom Lane
19 years ago
*/
Fix planner's handling of RETURNING lists in writable CTEs. setrefs.c failed to do "rtoffset" adjustment of Vars in RETURNING lists, which meant they were left with the wrong varnos when the RETURNING list was in a subquery. That was never possible before writable CTEs, of course, but now it's broken. The executor fails to notice any problem because ExecEvalVar just references the ecxt_scantuple for any normal varno; but EXPLAIN breaks when the varno is wrong, as illustrated in a recent complaint from Bartosz Dmytrak. Since the eventual rtoffset of the subquery is not known at the time we are preparing its plan node, the previous scheme of executing set_returning_clause_references() at that time cannot handle this adjustment. Fortunately, it turns out that we don't really need to do it that way, because all the needed information is available during normal setrefs.c execution; we just have to dig it out of the ModifyTable node. So, do that, and get rid of the kluge of early setrefs processing of RETURNING lists. (This is a little bit of a cheat in the case of inherited UPDATE/DELETE, because we are not passing a "root" struct that corresponds exactly to what the subplan was built with. But that doesn't matter, and anyway this is less ugly than early setrefs processing was.) Back-patch to 9.1, where the problem became possible to hit.
14 years ago
static List *
Rearrange planner to save the whole PlannerInfo (subroot) for a subquery. Formerly, set_subquery_pathlist and other creators of plans for subqueries saved only the rangetable and rowMarks lists from the lower-level PlannerInfo. But there's no reason not to remember the whole PlannerInfo, and indeed this turns out to simplify matters in a number of places. The immediate reason for doing this was so that the subroot will still be accessible when we're trying to extract column statistics out of an already-planned subquery. But now that I've done it, it seems like a good code-beautification effort in its own right. I also chose to get rid of the transient subrtable and subrowmark fields in SubqueryScan nodes, in favor of having setrefs.c look up the subquery's RelOptInfo. That required changing all the APIs in setrefs.c to pass PlannerInfo not PlannerGlobal, which was a large but quite mechanical transformation. One side-effect not foreseen at the beginning is that this finally broke inheritance_planner's assumption that replanning the same subquery RTE N times would necessarily give interchangeable results each time. That assumption was always pretty risky, but now we really have to make a separate RTE for each instance so that there's a place to carry the separate subroots.
14 years ago
set_returning_clause_references(PlannerInfo *root,
Fix the plan-invalidation mechanism to treat regclass constants that refer to a relation as a reason to invalidate a plan when the relation changes. This handles scenarios such as dropping/recreating a sequence that is referenced by nextval('seq') in a cached plan. Rather than teach plancache.c all about digging through plan trees to find regclass Consts, we charge the planner's setrefs.c with making a list of the relation OIDs on which each plan depends. That way the list can be built cheaply during a plan tree traversal that has to happen anyway. Per bug #3662 and subsequent discussion.
18 years ago
List *rlist,
Add INSERT/UPDATE/DELETE RETURNING, with basic docs and regression tests. plpgsql support to come later. Along the way, convert execMain's SELECT INTO support into a DestReceiver, in order to eliminate some ugly special cases. Jonah Harris and Tom Lane
19 years ago
Plan *topplan,
Fix planner's handling of RETURNING lists in writable CTEs. setrefs.c failed to do "rtoffset" adjustment of Vars in RETURNING lists, which meant they were left with the wrong varnos when the RETURNING list was in a subquery. That was never possible before writable CTEs, of course, but now it's broken. The executor fails to notice any problem because ExecEvalVar just references the ecxt_scantuple for any normal varno; but EXPLAIN breaks when the varno is wrong, as illustrated in a recent complaint from Bartosz Dmytrak. Since the eventual rtoffset of the subquery is not known at the time we are preparing its plan node, the previous scheme of executing set_returning_clause_references() at that time cannot handle this adjustment. Fortunately, it turns out that we don't really need to do it that way, because all the needed information is available during normal setrefs.c execution; we just have to dig it out of the ModifyTable node. So, do that, and get rid of the kluge of early setrefs processing of RETURNING lists. (This is a little bit of a cheat in the case of inherited UPDATE/DELETE, because we are not passing a "root" struct that corresponds exactly to what the subplan was built with. But that doesn't matter, and anyway this is less ugly than early setrefs processing was.) Back-patch to 9.1, where the problem became possible to hit.
14 years ago
Index resultRelation,
int rtoffset)
Add INSERT/UPDATE/DELETE RETURNING, with basic docs and regression tests. plpgsql support to come later. Along the way, convert execMain's SELECT INTO support into a DestReceiver, in order to eliminate some ugly special cases. Jonah Harris and Tom Lane
19 years ago
{
indexed_tlist *itlist;
/*
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
* We can perform the desired Var fixup by abusing the fix_join_expr
Make NestLoop plan nodes pass outer-relation variables into their inner relation using the general PARAM_EXEC executor parameter mechanism, rather than the ad-hoc kluge of passing the outer tuple down through ExecReScan. The previous method was hard to understand and could never be extended to handle parameters coming from multiple join levels. This patch doesn't change the set of possible plans nor have any significant performance effect, but it's necessary infrastructure for future generalization of the concept of an inner indexscan plan. ExecReScan's second parameter is now unused, so it's removed.
15 years ago
* machinery that formerly handled inner indexscan fixup. We search the
pgindent run for 8.2.
19 years ago
* top plan's targetlist for Vars of non-result relations, and use
pgindent run for 8.3.
18 years ago
* fix_join_expr to convert RETURNING Vars into references to those tlist
* entries, while leaving result-rel Vars as-is.
Add a concept of "placeholder" variables to the planner. These are variables that represent some expression that we desire to compute below the top level of the plan, and then let that value "bubble up" as though it were a plain Var (ie, a column value). The immediate application is to allow sub-selects to be flattened even when they are below an outer join and have non-nullable output expressions. Formerly we couldn't flatten because such an expression wouldn't properly go to NULL when evaluated above the outer join. Now, we wrap it in a PlaceHolderVar and arrange for the actual evaluation to occur below the outer join. When the resulting Var bubbles up through the join, it will be set to NULL if necessary, yielding the correct results. This fixes a planner limitation that's existed since 7.1. In future we might want to use this mechanism to re-introduce some form of Hellerstein's "expensive functions" optimization, ie place the evaluation of an expensive function at the most suitable point in the plan tree.
17 years ago
*
* PlaceHolderVars will also be sought in the targetlist, but no
8.4 pgindent run, with new combined Linux/FreeBSD/MinGW typedef list provided by Andrew.
16 years ago
* more-complex expressions will be. Note that it is not possible for a
* PlaceHolderVar to refer to the result relation, since the result is
* never below an outer join. If that case could happen, we'd have to be
* prepared to pick apart the PlaceHolderVar and evaluate its contained
* expression instead.
Add INSERT/UPDATE/DELETE RETURNING, with basic docs and regression tests. plpgsql support to come later. Along the way, convert execMain's SELECT INTO support into a DestReceiver, in order to eliminate some ugly special cases. Jonah Harris and Tom Lane
19 years ago
*/
itlist = build_tlist_index_other_vars(topplan->targetlist, resultRelation);
Rearrange planner to save the whole PlannerInfo (subroot) for a subquery. Formerly, set_subquery_pathlist and other creators of plans for subqueries saved only the rangetable and rowMarks lists from the lower-level PlannerInfo. But there's no reason not to remember the whole PlannerInfo, and indeed this turns out to simplify matters in a number of places. The immediate reason for doing this was so that the subroot will still be accessible when we're trying to extract column statistics out of an already-planned subquery. But now that I've done it, it seems like a good code-beautification effort in its own right. I also chose to get rid of the transient subrtable and subrowmark fields in SubqueryScan nodes, in favor of having setrefs.c look up the subquery's RelOptInfo. That required changing all the APIs in setrefs.c to pass PlannerInfo not PlannerGlobal, which was a large but quite mechanical transformation. One side-effect not foreseen at the beginning is that this finally broke inheritance_planner's assumption that replanning the same subquery RTE N times would necessarily give interchangeable results each time. That assumption was always pretty risky, but now we really have to make a separate RTE for each instance so that there's a place to carry the separate subroots.
14 years ago
rlist = fix_join_expr(root,
Fix the plan-invalidation mechanism to treat regclass constants that refer to a relation as a reason to invalidate a plan when the relation changes. This handles scenarios such as dropping/recreating a sequence that is referenced by nextval('seq') in a cached plan. Rather than teach plancache.c all about digging through plan trees to find regclass Consts, we charge the planner's setrefs.c with making a list of the relation OIDs on which each plan depends. That way the list can be built cheaply during a plan tree traversal that has to happen anyway. Per bug #3662 and subsequent discussion.
18 years ago
rlist,
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
itlist,
NULL,
resultRelation,
Fix planner's handling of RETURNING lists in writable CTEs. setrefs.c failed to do "rtoffset" adjustment of Vars in RETURNING lists, which meant they were left with the wrong varnos when the RETURNING list was in a subquery. That was never possible before writable CTEs, of course, but now it's broken. The executor fails to notice any problem because ExecEvalVar just references the ecxt_scantuple for any normal varno; but EXPLAIN breaks when the varno is wrong, as illustrated in a recent complaint from Bartosz Dmytrak. Since the eventual rtoffset of the subquery is not known at the time we are preparing its plan node, the previous scheme of executing set_returning_clause_references() at that time cannot handle this adjustment. Fortunately, it turns out that we don't really need to do it that way, because all the needed information is available during normal setrefs.c execution; we just have to dig it out of the ModifyTable node. So, do that, and get rid of the kluge of early setrefs processing of RETURNING lists. (This is a little bit of a cheat in the case of inherited UPDATE/DELETE, because we are not passing a "root" struct that corresponds exactly to what the subplan was built with. But that doesn't matter, and anyway this is less ugly than early setrefs processing was.) Back-patch to 9.1, where the problem became possible to hit.
14 years ago
rtoffset);
Add INSERT/UPDATE/DELETE RETURNING, with basic docs and regression tests. plpgsql support to come later. Along the way, convert execMain's SELECT INTO support into a DestReceiver, in order to eliminate some ugly special cases. Jonah Harris and Tom Lane
19 years ago
pfree(itlist);
return rlist;
}
Support GROUPING SETS, CUBE and ROLLUP. This SQL standard functionality allows to aggregate data by different GROUP BY clauses at once. Each grouping set returns rows with columns grouped by in other sets set to NULL. This could previously be achieved by doing each grouping as a separate query, conjoined by UNION ALLs. Besides being considerably more concise, grouping sets will in many cases be faster, requiring only one scan over the underlying data. The current implementation of grouping sets only supports using sorting for input. Individual sets that share a sort order are computed in one pass. If there are sets that don't share a sort order, additional sort & aggregation steps are performed. These additional passes are sourced by the previous sort step; thus avoiding repeated scans of the source data. The code is structured in a way that adding support for purely using hash aggregation or a mix of hashing and sorting is possible. Sorting was chosen to be supported first, as it is the most generic method of implementation. Instead of, as in an earlier versions of the patch, representing the chain of sort and aggregation steps as full blown planner and executor nodes, all but the first sort are performed inside the aggregation node itself. This avoids the need to do some unusual gymnastics to handle having to return aggregated and non-aggregated tuples from underlying nodes, as well as having to shut down underlying nodes early to limit memory usage. The optimizer still builds Sort/Agg node to describe each phase, but they're not part of the plan tree, but instead additional data for the aggregation node. They're a convenient and preexisting way to describe aggregation and sorting. The first (and possibly only) sort step is still performed as a separate execution step. That retains similarity with existing group by plans, makes rescans fairly simple, avoids very deep plans (leading to slow explains) and easily allows to avoid the sorting step if the underlying data is sorted by other means. A somewhat ugly side of this patch is having to deal with a grammar ambiguity between the new CUBE keyword and the cube extension/functions named cube (and rollup). To avoid breaking existing deployments of the cube extension it has not been renamed, neither has cube been made a reserved keyword. Instead precedence hacking is used to make GROUP BY cube(..) refer to the CUBE grouping sets feature, and not the function cube(). To actually group by a function cube(), unlikely as that might be, the function name has to be quoted. Needs a catversion bump because stored rules may change. Author: Andrew Gierth and Atri Sharma, with contributions from Andres Freund Reviewed-By: Andres Freund, Noah Misch, Tom Lane, Svenne Krap, Tomas Vondra, Erik Rijkers, Marti Raudsepp, Pavel Stehule Discussion: CAOeZVidmVRe2jU6aMk_5qkxnB7dfmPROzM7Ur8JPW5j8Y5X-Lw@mail.gmail.com
10 years ago
Postgres95 1.01 Distribution - Virgin Sources
30 years ago
/*****************************************************************************
Further planner/optimizer cleanups. Move all set_tlist_references and fix_opids processing to a single recursive pass over the plan tree executed at the very tail end of planning, rather than haphazardly here and there at different places. Now that tlist Vars do not get modified until the very end, it's possible to get rid of the klugy var_equal and match_varid partial-matching routines, and just use plain equal() throughout the optimizer. This is a step towards allowing merge and hash joins to be done on expressions instead of only Vars ...
26 years ago
* OPERATOR REGPROC LOOKUP
Postgres95 1.01 Distribution - Virgin Sources
30 years ago
*****************************************************************************/
Revise union_planner and associated routines to clean up breakage from EXCEPT/HAVING patch. Cases involving nontrivial GROUP BY expressions now work again. Also, the code is at least somewhat better documented...
27 years ago
/*
Phase 2 of read-only-plans project: restructure expression-tree nodes so that all executable expression nodes inherit from a common supertype Expr. This is somewhat of an exercise in code purity rather than any real functional advance, but getting rid of the extra Oper or Func node formerly used in each operator or function call should provide at least a little space and speed improvement. initdb forced by changes in stored-rules representation.
23 years ago
* fix_opfuncids
* Calculate opfuncid field from opno for each OpExpr node in given tree.
Further planner/optimizer cleanups. Move all set_tlist_references and fix_opids processing to a single recursive pass over the plan tree executed at the very tail end of planning, rather than haphazardly here and there at different places. Now that tlist Vars do not get modified until the very end, it's possible to get rid of the klugy var_equal and match_varid partial-matching routines, and just use plain equal() throughout the optimizer. This is a step towards allowing merge and hash joins to be done on expressions instead of only Vars ...
26 years ago
* The given tree can be anything expression_tree_walker handles.
Clean up routines in setrefs.c by replacing individual tree walking logic with expression_tree_walker/mutator calls.
26 years ago
*
Further planner/optimizer cleanups. Move all set_tlist_references and fix_opids processing to a single recursive pass over the plan tree executed at the very tail end of planning, rather than haphazardly here and there at different places. Now that tlist Vars do not get modified until the very end, it's possible to get rid of the klugy var_equal and match_varid partial-matching routines, and just use plain equal() throughout the optimizer. This is a step towards allowing merge and hash joins to be done on expressions instead of only Vars ...
26 years ago
* The argument is modified in-place. (This is OK since we'd want the
* same change for any node, even if it gets visited more than once due to
* shared structure.)
Revise union_planner and associated routines to clean up breakage from EXCEPT/HAVING patch. Cases involving nontrivial GROUP BY expressions now work again. Also, the code is at least somewhat better documented...
27 years ago
*/
Repair some problems in planner's handling of HAVING clauses. This fixes a few of the problems Hiroshi Inoue complained of, but I have not touched the rewrite-related issues.
27 years ago
void
Phase 2 of read-only-plans project: restructure expression-tree nodes so that all executable expression nodes inherit from a common supertype Expr. This is somewhat of an exercise in code purity rather than any real functional advance, but getting rid of the extra Oper or Func node formerly used in each operator or function call should provide at least a little space and speed improvement. initdb forced by changes in stored-rules representation.
23 years ago
fix_opfuncids(Node *node)
Repair some problems in planner's handling of HAVING clauses. This fixes a few of the problems Hiroshi Inoue complained of, but I have not touched the rewrite-related issues.
27 years ago
{
Further planner/optimizer cleanups. Move all set_tlist_references and fix_opids processing to a single recursive pass over the plan tree executed at the very tail end of planning, rather than haphazardly here and there at different places. Now that tlist Vars do not get modified until the very end, it's possible to get rid of the klugy var_equal and match_varid partial-matching routines, and just use plain equal() throughout the optimizer. This is a step towards allowing merge and hash joins to be done on expressions instead of only Vars ...
26 years ago
/* This tree walk requires no special setup, so away we go... */
Phase 2 of read-only-plans project: restructure expression-tree nodes so that all executable expression nodes inherit from a common supertype Expr. This is somewhat of an exercise in code purity rather than any real functional advance, but getting rid of the extra Oper or Func node formerly used in each operator or function call should provide at least a little space and speed improvement. initdb forced by changes in stored-rules representation.
23 years ago
fix_opfuncids_walker(node, NULL);
Clean up routines in setrefs.c by replacing individual tree walking logic with expression_tree_walker/mutator calls.
26 years ago
}
pgindent run over code.
27 years ago
Clean up routines in setrefs.c by replacing individual tree walking logic with expression_tree_walker/mutator calls.
26 years ago
static bool
Phase 2 of read-only-plans project: restructure expression-tree nodes so that all executable expression nodes inherit from a common supertype Expr. This is somewhat of an exercise in code purity rather than any real functional advance, but getting rid of the extra Oper or Func node formerly used in each operator or function call should provide at least a little space and speed improvement. initdb forced by changes in stored-rules representation.
23 years ago
fix_opfuncids_walker(Node *node, void *context)
Clean up routines in setrefs.c by replacing individual tree walking logic with expression_tree_walker/mutator calls.
26 years ago
{
if (node == NULL)
return false;
Phase 2 of read-only-plans project: restructure expression-tree nodes so that all executable expression nodes inherit from a common supertype Expr. This is somewhat of an exercise in code purity rather than any real functional advance, but getting rid of the extra Oper or Func node formerly used in each operator or function call should provide at least a little space and speed improvement. initdb forced by changes in stored-rules representation.
23 years ago
if (IsA(node, OpExpr))
set_opfuncid((OpExpr *) node);
else if (IsA(node, DistinctExpr))
pgindent run.
22 years ago
set_opfuncid((OpExpr *) node); /* rely on struct equivalence */
COALESCE() and NULLIF() are now first-class expressions, not macros that turn into CASE expressions. They evaluate their arguments at most once. Patch by Kris Jurka, review and (very light) editorializing by me.
23 years ago
else if (IsA(node, NullIfExpr))
pgindent run.
22 years ago
set_opfuncid((OpExpr *) node); /* rely on struct equivalence */
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
19 years ago
else if (IsA(node, ScalarArrayOpExpr))
set_sa_opfuncid((ScalarArrayOpExpr *) node);
Phase 2 of read-only-plans project: restructure expression-tree nodes so that all executable expression nodes inherit from a common supertype Expr. This is somewhat of an exercise in code purity rather than any real functional advance, but getting rid of the extra Oper or Func node formerly used in each operator or function call should provide at least a little space and speed improvement. initdb forced by changes in stored-rules representation.
23 years ago
return expression_tree_walker(node, fix_opfuncids_walker, context);
OK, folks, here is the pgindent output.
27 years ago
}
Support expressions of the form 'scalar op ANY (array)' and 'scalar op ALL (array)', where the operator is applied between the lefthand scalar and each element of the array. The operator must yield boolean; the result of the construct is the OR or AND of the per-element results, respectively. Original coding by Joe Conway, after an idea of Peter's. Rewritten by Tom to keep the implementation strictly separate from subqueries.
23 years ago
/*
* set_opfuncid
* Set the opfuncid (procedure OID) in an OpExpr node,
* if it hasn't been set already.
*
* Because of struct equivalence, this can also be used for
* DistinctExpr and NullIfExpr nodes.
*/
void
Another pgindent run with updated typedefs.
22 years ago
set_opfuncid(OpExpr *opexpr)
Support expressions of the form 'scalar op ANY (array)' and 'scalar op ALL (array)', where the operator is applied between the lefthand scalar and each element of the array. The operator must yield boolean; the result of the construct is the OR or AND of the per-element results, respectively. Original coding by Joe Conway, after an idea of Peter's. Rewritten by Tom to keep the implementation strictly separate from subqueries.
23 years ago
{
if (opexpr->opfuncid == InvalidOid)
opexpr->opfuncid = get_opcode(opexpr->opno);
}
/*
* set_sa_opfuncid
* As above, for ScalarArrayOpExpr nodes.
*/
Add COST and ROWS options to CREATE/ALTER FUNCTION, plus underlying pg_proc columns procost and prorows, to allow simple user adjustment of the estimated cost of a function call, as well as control of the estimated number of rows returned by a set-returning function. We might eventually wish to extend this to allow function-specific estimation routines, but there seems to be consensus that we should try a simple constant estimate first. In particular this provides a relatively simple way to control the order in which different WHERE clauses are applied in a plan node, which is a Good Thing in view of the fact that the recent EquivalenceClass planner rewrite made that much less predictable than before.
19 years ago
void
Another pgindent run with updated typedefs.
22 years ago
set_sa_opfuncid(ScalarArrayOpExpr *opexpr)
Support expressions of the form 'scalar op ANY (array)' and 'scalar op ALL (array)', where the operator is applied between the lefthand scalar and each element of the array. The operator must yield boolean; the result of the construct is the OR or AND of the per-element results, respectively. Original coding by Joe Conway, after an idea of Peter's. Rewritten by Tom to keep the implementation strictly separate from subqueries.
23 years ago
{
if (opexpr->opfuncid == InvalidOid)
opexpr->opfuncid = get_opcode(opexpr->opno);
}
Improve the plan cache invalidation mechanism to make it invalidate plans when user-defined functions used in a plan are modified. Also invalidate plans when schemas, operators, or operator classes are modified; but for these cases we just invalidate everything rather than tracking exact dependencies, since these types of objects seldom change in a production database. Tom Lane; loosely based on a patch by Martin Pihlak.
17 years ago
/*****************************************************************************
* QUERY DEPENDENCY MANAGEMENT
*****************************************************************************/
/*
* record_plan_function_dependency
* Mark the current plan as depending on a particular function.
*
* This is exported so that the function-inlining code can record a
* dependency on a function that it's removed from the plan tree.
*/
void
Rearrange planner to save the whole PlannerInfo (subroot) for a subquery. Formerly, set_subquery_pathlist and other creators of plans for subqueries saved only the rangetable and rowMarks lists from the lower-level PlannerInfo. But there's no reason not to remember the whole PlannerInfo, and indeed this turns out to simplify matters in a number of places. The immediate reason for doing this was so that the subroot will still be accessible when we're trying to extract column statistics out of an already-planned subquery. But now that I've done it, it seems like a good code-beautification effort in its own right. I also chose to get rid of the transient subrtable and subrowmark fields in SubqueryScan nodes, in favor of having setrefs.c look up the subquery's RelOptInfo. That required changing all the APIs in setrefs.c to pass PlannerInfo not PlannerGlobal, which was a large but quite mechanical transformation. One side-effect not foreseen at the beginning is that this finally broke inheritance_planner's assumption that replanning the same subquery RTE N times would necessarily give interchangeable results each time. That assumption was always pretty risky, but now we really have to make a separate RTE for each instance so that there's a place to carry the separate subroots.
14 years ago
record_plan_function_dependency(PlannerInfo *root, Oid funcid)
Improve the plan cache invalidation mechanism to make it invalidate plans when user-defined functions used in a plan are modified. Also invalidate plans when schemas, operators, or operator classes are modified; but for these cases we just invalidate everything rather than tracking exact dependencies, since these types of objects seldom change in a production database. Tom Lane; loosely based on a patch by Martin Pihlak.
17 years ago
{
/*
* For performance reasons, we don't bother to track built-in functions;
* we just assume they'll never change (or at least not in ways that'd
* invalidate plans using them). For this purpose we can consider a
* built-in function to be one with OID less than FirstBootstrapObjectId.
8.4 pgindent run, with new combined Linux/FreeBSD/MinGW typedef list provided by Andrew.
16 years ago
* Note that the OID generator guarantees never to generate such an OID
* after startup, even at OID wraparound.
Improve the plan cache invalidation mechanism to make it invalidate plans when user-defined functions used in a plan are modified. Also invalidate plans when schemas, operators, or operator classes are modified; but for these cases we just invalidate everything rather than tracking exact dependencies, since these types of objects seldom change in a production database. Tom Lane; loosely based on a patch by Martin Pihlak.
17 years ago
*/
if (funcid >= (Oid) FirstBootstrapObjectId)
{
Revise sinval code to remove no-longer-used tuple TID from inval messages. This requires adjusting the API for syscache callback functions: they now get a hash value, not a TID, to identify the target tuple. Most of them weren't paying any attention to that argument anyway, but plancache did require a small amount of fixing. Also, improve performance a trifle by avoiding sending duplicate inval messages when a heap_update isn't changing the catcache lookup columns.
14 years ago
PlanInvalItem *inval_item = makeNode(PlanInvalItem);
Improve the plan cache invalidation mechanism to make it invalidate plans when user-defined functions used in a plan are modified. Also invalidate plans when schemas, operators, or operator classes are modified; but for these cases we just invalidate everything rather than tracking exact dependencies, since these types of objects seldom change in a production database. Tom Lane; loosely based on a patch by Martin Pihlak.
17 years ago
/*
Revise sinval code to remove no-longer-used tuple TID from inval messages. This requires adjusting the API for syscache callback functions: they now get a hash value, not a TID, to identify the target tuple. Most of them weren't paying any attention to that argument anyway, but plancache did require a small amount of fixing. Also, improve performance a trifle by avoiding sending duplicate inval messages when a heap_update isn't changing the catcache lookup columns.
14 years ago
* It would work to use any syscache on pg_proc, but the easiest is
* PROCOID since we already have the function's OID at hand. Note
Expose an API for calculating catcache hash values. Now that cache invalidation callbacks get only a hash value, and not a tuple TID (per commits 632ae6829f7abda34e15082c91d9dfb3fc0f298b and b5282aa893e565b7844f8237462cb843438cdd5e), the only way they can restrict what they invalidate is to know what the hash values mean. setrefs.c was doing this via a hard-wired assumption but that seems pretty grotty, and it'll only get worse as more cases come up. So let's expose a calculation function that takes the same parameters as SearchSysCache. Per complaint from Marko Kreen.
14 years ago
* that plancache.c knows we use PROCOID.
Improve the plan cache invalidation mechanism to make it invalidate plans when user-defined functions used in a plan are modified. Also invalidate plans when schemas, operators, or operator classes are modified; but for these cases we just invalidate everything rather than tracking exact dependencies, since these types of objects seldom change in a production database. Tom Lane; loosely based on a patch by Martin Pihlak.
17 years ago
*/
inval_item->cacheId = PROCOID;
Expose an API for calculating catcache hash values. Now that cache invalidation callbacks get only a hash value, and not a tuple TID (per commits 632ae6829f7abda34e15082c91d9dfb3fc0f298b and b5282aa893e565b7844f8237462cb843438cdd5e), the only way they can restrict what they invalidate is to know what the hash values mean. setrefs.c was doing this via a hard-wired assumption but that seems pretty grotty, and it'll only get worse as more cases come up. So let's expose a calculation function that takes the same parameters as SearchSysCache. Per complaint from Marko Kreen.
14 years ago
inval_item->hashValue = GetSysCacheHashValue1(PROCOID,
Run pgindent on 9.2 source tree in preparation for first 9.3 commit-fest.
13 years ago
ObjectIdGetDatum(funcid));
Improve the plan cache invalidation mechanism to make it invalidate plans when user-defined functions used in a plan are modified. Also invalidate plans when schemas, operators, or operator classes are modified; but for these cases we just invalidate everything rather than tracking exact dependencies, since these types of objects seldom change in a production database. Tom Lane; loosely based on a patch by Martin Pihlak.
17 years ago
Rearrange planner to save the whole PlannerInfo (subroot) for a subquery. Formerly, set_subquery_pathlist and other creators of plans for subqueries saved only the rangetable and rowMarks lists from the lower-level PlannerInfo. But there's no reason not to remember the whole PlannerInfo, and indeed this turns out to simplify matters in a number of places. The immediate reason for doing this was so that the subroot will still be accessible when we're trying to extract column statistics out of an already-planned subquery. But now that I've done it, it seems like a good code-beautification effort in its own right. I also chose to get rid of the transient subrtable and subrowmark fields in SubqueryScan nodes, in favor of having setrefs.c look up the subquery's RelOptInfo. That required changing all the APIs in setrefs.c to pass PlannerInfo not PlannerGlobal, which was a large but quite mechanical transformation. One side-effect not foreseen at the beginning is that this finally broke inheritance_planner's assumption that replanning the same subquery RTE N times would necessarily give interchangeable results each time. That assumption was always pretty risky, but now we really have to make a separate RTE for each instance so that there's a place to carry the separate subroots.
14 years ago
root->glob->invalItems = lappend(root->glob->invalItems, inval_item);
Improve the plan cache invalidation mechanism to make it invalidate plans when user-defined functions used in a plan are modified. Also invalidate plans when schemas, operators, or operator classes are modified; but for these cases we just invalidate everything rather than tracking exact dependencies, since these types of objects seldom change in a production database. Tom Lane; loosely based on a patch by Martin Pihlak.
17 years ago
}
}
/*
* extract_query_dependencies
Do parse analysis of an EXPLAIN's contained statement during the normal parse analysis phase, rather than at execution time. This makes parameter handling work the same as it does in ordinary plannable queries, and in particular fixes the incompatibility that Pavel pointed out with plpgsql's new handling of variable references. plancache.c gets a little bit grottier, but the alternatives seem worse.
16 years ago
* Given a not-yet-planned query or queries (i.e. a Query node or list
* of Query nodes), extract dependencies just as set_plan_references
* would do.
Improve the plan cache invalidation mechanism to make it invalidate plans when user-defined functions used in a plan are modified. Also invalidate plans when schemas, operators, or operator classes are modified; but for these cases we just invalidate everything rather than tracking exact dependencies, since these types of objects seldom change in a production database. Tom Lane; loosely based on a patch by Martin Pihlak.
17 years ago
*
* This is needed by plancache.c to handle invalidation of cached unplanned
* queries.
*/
void
Do parse analysis of an EXPLAIN's contained statement during the normal parse analysis phase, rather than at execution time. This makes parameter handling work the same as it does in ordinary plannable queries, and in particular fixes the incompatibility that Pavel pointed out with plpgsql's new handling of variable references. plancache.c gets a little bit grottier, but the alternatives seem worse.
16 years ago
extract_query_dependencies(Node *query,
Improve the plan cache invalidation mechanism to make it invalidate plans when user-defined functions used in a plan are modified. Also invalidate plans when schemas, operators, or operator classes are modified; but for these cases we just invalidate everything rather than tracking exact dependencies, since these types of objects seldom change in a production database. Tom Lane; loosely based on a patch by Martin Pihlak.
17 years ago
List **relationOids,
Row-Level Security Policies (RLS) Building on the updatable security-barrier views work, add the ability to define policies on tables to limit the set of rows which are returned from a query and which are allowed to be added to a table. Expressions defined by the policy for filtering are added to the security barrier quals of the query, while expressions defined to check records being added to a table are added to the with-check options of the query. New top-level commands are CREATE/ALTER/DROP POLICY and are controlled by the table owner. Row Security is able to be enabled and disabled by the owner on a per-table basis using ALTER TABLE .. ENABLE/DISABLE ROW SECURITY. Per discussion, ROW SECURITY is disabled on tables by default and must be enabled for policies on the table to be used. If no policies exist on a table with ROW SECURITY enabled, a default-deny policy is used and no records will be visible. By default, row security is applied at all times except for the table owner and the superuser. A new GUC, row_security, is added which can be set to ON, OFF, or FORCE. When set to FORCE, row security will be applied even for the table owner and superusers. When set to OFF, row security will be disabled when allowed and an error will be thrown if the user does not have rights to bypass row security. Per discussion, pg_dump sets row_security = OFF by default to ensure that exports and backups will have all data in the table or will error if there are insufficient privileges to bypass row security. A new option has been added to pg_dump, --enable-row-security, to ask pg_dump to export with row security enabled. A new role capability, BYPASSRLS, which can only be set by the superuser, is added to allow other users to be able to bypass row security using row_security = OFF. Many thanks to the various individuals who have helped with the design, particularly Robert Haas for his feedback. Authors include Craig Ringer, KaiGai Kohei, Adam Brightwell, Dean Rasheed, with additional changes and rework by me. Reviewers have included all of the above, Greg Smith, Jeff McCormick, and Robert Haas.
11 years ago
List **invalItems,
bool *hasRowSecurity)
Improve the plan cache invalidation mechanism to make it invalidate plans when user-defined functions used in a plan are modified. Also invalidate plans when schemas, operators, or operator classes are modified; but for these cases we just invalidate everything rather than tracking exact dependencies, since these types of objects seldom change in a production database. Tom Lane; loosely based on a patch by Martin Pihlak.
17 years ago
{
PlannerGlobal glob;
Rearrange planner to save the whole PlannerInfo (subroot) for a subquery. Formerly, set_subquery_pathlist and other creators of plans for subqueries saved only the rangetable and rowMarks lists from the lower-level PlannerInfo. But there's no reason not to remember the whole PlannerInfo, and indeed this turns out to simplify matters in a number of places. The immediate reason for doing this was so that the subroot will still be accessible when we're trying to extract column statistics out of an already-planned subquery. But now that I've done it, it seems like a good code-beautification effort in its own right. I also chose to get rid of the transient subrtable and subrowmark fields in SubqueryScan nodes, in favor of having setrefs.c look up the subquery's RelOptInfo. That required changing all the APIs in setrefs.c to pass PlannerInfo not PlannerGlobal, which was a large but quite mechanical transformation. One side-effect not foreseen at the beginning is that this finally broke inheritance_planner's assumption that replanning the same subquery RTE N times would necessarily give interchangeable results each time. That assumption was always pretty risky, but now we really have to make a separate RTE for each instance so that there's a place to carry the separate subroots.
14 years ago
PlannerInfo root;
Improve the plan cache invalidation mechanism to make it invalidate plans when user-defined functions used in a plan are modified. Also invalidate plans when schemas, operators, or operator classes are modified; but for these cases we just invalidate everything rather than tracking exact dependencies, since these types of objects seldom change in a production database. Tom Lane; loosely based on a patch by Martin Pihlak.
17 years ago
Rearrange planner to save the whole PlannerInfo (subroot) for a subquery. Formerly, set_subquery_pathlist and other creators of plans for subqueries saved only the rangetable and rowMarks lists from the lower-level PlannerInfo. But there's no reason not to remember the whole PlannerInfo, and indeed this turns out to simplify matters in a number of places. The immediate reason for doing this was so that the subroot will still be accessible when we're trying to extract column statistics out of an already-planned subquery. But now that I've done it, it seems like a good code-beautification effort in its own right. I also chose to get rid of the transient subrtable and subrowmark fields in SubqueryScan nodes, in favor of having setrefs.c look up the subquery's RelOptInfo. That required changing all the APIs in setrefs.c to pass PlannerInfo not PlannerGlobal, which was a large but quite mechanical transformation. One side-effect not foreseen at the beginning is that this finally broke inheritance_planner's assumption that replanning the same subquery RTE N times would necessarily give interchangeable results each time. That assumption was always pretty risky, but now we really have to make a separate RTE for each instance so that there's a place to carry the separate subroots.
14 years ago
/* Make up dummy planner state so we can use this module's machinery */
Improve the plan cache invalidation mechanism to make it invalidate plans when user-defined functions used in a plan are modified. Also invalidate plans when schemas, operators, or operator classes are modified; but for these cases we just invalidate everything rather than tracking exact dependencies, since these types of objects seldom change in a production database. Tom Lane; loosely based on a patch by Martin Pihlak.
17 years ago
MemSet(&glob, 0, sizeof(glob));
glob.type = T_PlannerGlobal;
glob.relationOids = NIL;
glob.invalItems = NIL;
Rename pg_rowsecurity -> pg_policy and other fixes As pointed out by Robert, we should really have named pg_rowsecurity pg_policy, as the objects stored in that catalog are policies. This patch fixes that and updates the column names to start with 'pol' to match the new catalog name. The security consideration for COPY with row level security, also pointed out by Robert, has also been addressed by remembering and re-checking the OID of the relation initially referenced during COPY processing, to make sure it hasn't changed under us by the time we finish planning out the query which has been built. Robert and Alvaro also commented on missing OCLASS and OBJECT entries for POLICY (formerly ROWSECURITY or POLICY, depending) in various places. This patch fixes that too, which also happens to add the ability to COMMENT on policies. In passing, attempt to improve the consistency of messages, comments, and documentation as well. This removes various incarnations of 'row-security', 'row-level security', 'Row-security', etc, in favor of 'policy', 'row level security' or 'row_security' as appropriate. Happy Thanksgiving!
11 years ago
glob.hasRowSecurity = false;
Improve the plan cache invalidation mechanism to make it invalidate plans when user-defined functions used in a plan are modified. Also invalidate plans when schemas, operators, or operator classes are modified; but for these cases we just invalidate everything rather than tracking exact dependencies, since these types of objects seldom change in a production database. Tom Lane; loosely based on a patch by Martin Pihlak.
17 years ago
Rearrange planner to save the whole PlannerInfo (subroot) for a subquery. Formerly, set_subquery_pathlist and other creators of plans for subqueries saved only the rangetable and rowMarks lists from the lower-level PlannerInfo. But there's no reason not to remember the whole PlannerInfo, and indeed this turns out to simplify matters in a number of places. The immediate reason for doing this was so that the subroot will still be accessible when we're trying to extract column statistics out of an already-planned subquery. But now that I've done it, it seems like a good code-beautification effort in its own right. I also chose to get rid of the transient subrtable and subrowmark fields in SubqueryScan nodes, in favor of having setrefs.c look up the subquery's RelOptInfo. That required changing all the APIs in setrefs.c to pass PlannerInfo not PlannerGlobal, which was a large but quite mechanical transformation. One side-effect not foreseen at the beginning is that this finally broke inheritance_planner's assumption that replanning the same subquery RTE N times would necessarily give interchangeable results each time. That assumption was always pretty risky, but now we really have to make a separate RTE for each instance so that there's a place to carry the separate subroots.
14 years ago
MemSet(&root, 0, sizeof(root));
root.type = T_PlannerInfo;
root.glob = &glob;
(void) extract_query_dependencies_walker(query, &root);
Improve the plan cache invalidation mechanism to make it invalidate plans when user-defined functions used in a plan are modified. Also invalidate plans when schemas, operators, or operator classes are modified; but for these cases we just invalidate everything rather than tracking exact dependencies, since these types of objects seldom change in a production database. Tom Lane; loosely based on a patch by Martin Pihlak.
17 years ago
*relationOids = glob.relationOids;
*invalItems = glob.invalItems;
Rename pg_rowsecurity -> pg_policy and other fixes As pointed out by Robert, we should really have named pg_rowsecurity pg_policy, as the objects stored in that catalog are policies. This patch fixes that and updates the column names to start with 'pol' to match the new catalog name. The security consideration for COPY with row level security, also pointed out by Robert, has also been addressed by remembering and re-checking the OID of the relation initially referenced during COPY processing, to make sure it hasn't changed under us by the time we finish planning out the query which has been built. Robert and Alvaro also commented on missing OCLASS and OBJECT entries for POLICY (formerly ROWSECURITY or POLICY, depending) in various places. This patch fixes that too, which also happens to add the ability to COMMENT on policies. In passing, attempt to improve the consistency of messages, comments, and documentation as well. This removes various incarnations of 'row-security', 'row-level security', 'Row-security', etc, in favor of 'policy', 'row level security' or 'row_security' as appropriate. Happy Thanksgiving!
11 years ago
*hasRowSecurity = glob.hasRowSecurity;
Improve the plan cache invalidation mechanism to make it invalidate plans when user-defined functions used in a plan are modified. Also invalidate plans when schemas, operators, or operator classes are modified; but for these cases we just invalidate everything rather than tracking exact dependencies, since these types of objects seldom change in a production database. Tom Lane; loosely based on a patch by Martin Pihlak.
17 years ago
}
static bool
Rearrange planner to save the whole PlannerInfo (subroot) for a subquery. Formerly, set_subquery_pathlist and other creators of plans for subqueries saved only the rangetable and rowMarks lists from the lower-level PlannerInfo. But there's no reason not to remember the whole PlannerInfo, and indeed this turns out to simplify matters in a number of places. The immediate reason for doing this was so that the subroot will still be accessible when we're trying to extract column statistics out of an already-planned subquery. But now that I've done it, it seems like a good code-beautification effort in its own right. I also chose to get rid of the transient subrtable and subrowmark fields in SubqueryScan nodes, in favor of having setrefs.c look up the subquery's RelOptInfo. That required changing all the APIs in setrefs.c to pass PlannerInfo not PlannerGlobal, which was a large but quite mechanical transformation. One side-effect not foreseen at the beginning is that this finally broke inheritance_planner's assumption that replanning the same subquery RTE N times would necessarily give interchangeable results each time. That assumption was always pretty risky, but now we really have to make a separate RTE for each instance so that there's a place to carry the separate subroots.
14 years ago
extract_query_dependencies_walker(Node *node, PlannerInfo *context)
Improve the plan cache invalidation mechanism to make it invalidate plans when user-defined functions used in a plan are modified. Also invalidate plans when schemas, operators, or operator classes are modified; but for these cases we just invalidate everything rather than tracking exact dependencies, since these types of objects seldom change in a production database. Tom Lane; loosely based on a patch by Martin Pihlak.
17 years ago
{
if (node == NULL)
return false;
Add a concept of "placeholder" variables to the planner. These are variables that represent some expression that we desire to compute below the top level of the plan, and then let that value "bubble up" as though it were a plain Var (ie, a column value). The immediate application is to allow sub-selects to be flattened even when they are below an outer join and have non-nullable output expressions. Formerly we couldn't flatten because such an expression wouldn't properly go to NULL when evaluated above the outer join. Now, we wrap it in a PlaceHolderVar and arrange for the actual evaluation to occur below the outer join. When the resulting Var bubbles up through the join, it will be set to NULL if necessary, yielding the correct results. This fixes a planner limitation that's existed since 7.1. In future we might want to use this mechanism to re-introduce some form of Hellerstein's "expensive functions" optimization, ie place the evaluation of an expensive function at the most suitable point in the plan tree.
17 years ago
Assert(!IsA(node, PlaceHolderVar));
Improve the plan cache invalidation mechanism to make it invalidate plans when user-defined functions used in a plan are modified. Also invalidate plans when schemas, operators, or operator classes are modified; but for these cases we just invalidate everything rather than tracking exact dependencies, since these types of objects seldom change in a production database. Tom Lane; loosely based on a patch by Martin Pihlak.
17 years ago
/* Extract function dependencies and check for regclass Consts */
fix_expr_common(context, node);
if (IsA(node, Query))
{
Query *query = (Query *) node;
ListCell *lc;
Rename pg_rowsecurity -> pg_policy and other fixes As pointed out by Robert, we should really have named pg_rowsecurity pg_policy, as the objects stored in that catalog are policies. This patch fixes that and updates the column names to start with 'pol' to match the new catalog name. The security consideration for COPY with row level security, also pointed out by Robert, has also been addressed by remembering and re-checking the OID of the relation initially referenced during COPY processing, to make sure it hasn't changed under us by the time we finish planning out the query which has been built. Robert and Alvaro also commented on missing OCLASS and OBJECT entries for POLICY (formerly ROWSECURITY or POLICY, depending) in various places. This patch fixes that too, which also happens to add the ability to COMMENT on policies. In passing, attempt to improve the consistency of messages, comments, and documentation as well. This removes various incarnations of 'row-security', 'row-level security', 'Row-security', etc, in favor of 'policy', 'row level security' or 'row_security' as appropriate. Happy Thanksgiving!
11 years ago
/* Collect row security information */
context->glob->hasRowSecurity = query->hasRowSecurity;
Row-Level Security Policies (RLS) Building on the updatable security-barrier views work, add the ability to define policies on tables to limit the set of rows which are returned from a query and which are allowed to be added to a table. Expressions defined by the policy for filtering are added to the security barrier quals of the query, while expressions defined to check records being added to a table are added to the with-check options of the query. New top-level commands are CREATE/ALTER/DROP POLICY and are controlled by the table owner. Row Security is able to be enabled and disabled by the owner on a per-table basis using ALTER TABLE .. ENABLE/DISABLE ROW SECURITY. Per discussion, ROW SECURITY is disabled on tables by default and must be enabled for policies on the table to be used. If no policies exist on a table with ROW SECURITY enabled, a default-deny policy is used and no records will be visible. By default, row security is applied at all times except for the table owner and the superuser. A new GUC, row_security, is added which can be set to ON, OFF, or FORCE. When set to FORCE, row security will be applied even for the table owner and superusers. When set to OFF, row security will be disabled when allowed and an error will be thrown if the user does not have rights to bypass row security. Per discussion, pg_dump sets row_security = OFF by default to ensure that exports and backups will have all data in the table or will error if there are insufficient privileges to bypass row security. A new option has been added to pg_dump, --enable-row-security, to ask pg_dump to export with row security enabled. A new role capability, BYPASSRLS, which can only be set by the superuser, is added to allow other users to be able to bypass row security using row_security = OFF. Many thanks to the various individuals who have helped with the design, particularly Robert Haas for his feedback. Authors include Craig Ringer, KaiGai Kohei, Adam Brightwell, Dean Rasheed, with additional changes and rework by me. Reviewers have included all of the above, Greg Smith, Jeff McCormick, and Robert Haas.
11 years ago
Make UtilityContainsQuery recurse until it finds a non-utility Query. The callers of UtilityContainsQuery want it to return a non-utility Query if it returns anything at all. However, since we made CREATE TABLE AS/SELECT INTO into a utility command instead of a variant of SELECT, a command like "EXPLAIN SELECT INTO" results in two nested utility statements. So what we need UtilityContainsQuery to do is drill down to the bottom non-utility Query. I had thought of this possibility in setrefs.c, and fixed it there by looping around the UtilityContainsQuery call; but overlooked that the call sites in plancache.c have a similar issue. In those cases it's notationally inconvenient to provide an external loop, so let's redefine UtilityContainsQuery as recursing down to a non-utility Query instead. Noted by Rushabh Lathia. This is a somewhat cleaned-up version of his proposed patch.
13 years ago
if (query->commandType == CMD_UTILITY)
Do parse analysis of an EXPLAIN's contained statement during the normal parse analysis phase, rather than at execution time. This makes parameter handling work the same as it does in ordinary plannable queries, and in particular fixes the incompatibility that Pavel pointed out with plpgsql's new handling of variable references. plancache.c gets a little bit grottier, but the alternatives seem worse.
16 years ago
{
Restructure SELECT INTO's parsetree representation into CreateTableAsStmt. Making this operation look like a utility statement seems generally a good idea, and particularly so in light of the desire to provide command triggers for utility statements. The original choice of representing it as SELECT with an IntoClause appendage had metastasized into rather a lot of places, unfortunately, so that this patch is a great deal more complicated than one might at first expect. In particular, keeping EXPLAIN working for SELECT INTO and CREATE TABLE AS subcommands required restructuring some EXPLAIN-related APIs. Add-on code that calls ExplainOnePlan or ExplainOneUtility, or uses ExplainOneQuery_hook, will need adjustment. Also, the cases PREPARE ... SELECT INTO and CREATE RULE ... SELECT INTO, which formerly were accepted though undocumented, are no longer accepted. The PREPARE case can be replaced with use of CREATE TABLE AS EXECUTE. The CREATE RULE case doesn't seem to have much real-world use (since the rule would work only once before failing with "table already exists"), so we'll not bother with that one. Both SELECT INTO and CREATE TABLE AS still return a command tag of "SELECT nnnn". There was some discussion of returning "CREATE TABLE nnnn", but for the moment backwards compatibility wins the day. Andres Freund and Tom Lane
14 years ago
/*
* Ignore utility statements, except those (such as EXPLAIN) that
* contain a parsed-but-not-planned query.
*/
query = UtilityContainsQuery(query->utilityStmt);
if (query == NULL)
Do parse analysis of an EXPLAIN's contained statement during the normal parse analysis phase, rather than at execution time. This makes parameter handling work the same as it does in ordinary plannable queries, and in particular fixes the incompatibility that Pavel pointed out with plpgsql's new handling of variable references. plancache.c gets a little bit grottier, but the alternatives seem worse.
16 years ago
return false;
}
Improve the plan cache invalidation mechanism to make it invalidate plans when user-defined functions used in a plan are modified. Also invalidate plans when schemas, operators, or operator classes are modified; but for these cases we just invalidate everything rather than tracking exact dependencies, since these types of objects seldom change in a production database. Tom Lane; loosely based on a patch by Martin Pihlak.
17 years ago
/* Collect relation OIDs in this Query's rtable */
foreach(lc, query->rtable)
{
RangeTblEntry *rte = (RangeTblEntry *) lfirst(lc);
if (rte->rtekind == RTE_RELATION)
Rearrange planner to save the whole PlannerInfo (subroot) for a subquery. Formerly, set_subquery_pathlist and other creators of plans for subqueries saved only the rangetable and rowMarks lists from the lower-level PlannerInfo. But there's no reason not to remember the whole PlannerInfo, and indeed this turns out to simplify matters in a number of places. The immediate reason for doing this was so that the subroot will still be accessible when we're trying to extract column statistics out of an already-planned subquery. But now that I've done it, it seems like a good code-beautification effort in its own right. I also chose to get rid of the transient subrtable and subrowmark fields in SubqueryScan nodes, in favor of having setrefs.c look up the subquery's RelOptInfo. That required changing all the APIs in setrefs.c to pass PlannerInfo not PlannerGlobal, which was a large but quite mechanical transformation. One side-effect not foreseen at the beginning is that this finally broke inheritance_planner's assumption that replanning the same subquery RTE N times would necessarily give interchangeable results each time. That assumption was always pretty risky, but now we really have to make a separate RTE for each instance so that there's a place to carry the separate subroots.
14 years ago
context->glob->relationOids =
lappend_oid(context->glob->relationOids, rte->relid);
Improve the plan cache invalidation mechanism to make it invalidate plans when user-defined functions used in a plan are modified. Also invalidate plans when schemas, operators, or operator classes are modified; but for these cases we just invalidate everything rather than tracking exact dependencies, since these types of objects seldom change in a production database. Tom Lane; loosely based on a patch by Martin Pihlak.
17 years ago
}
/* And recurse into the query's subexpressions */
return query_tree_walker(query, extract_query_dependencies_walker,
(void *) context, 0);
}
return expression_tree_walker(node, extract_query_dependencies_walker,
(void *) context);
}