|
|
|
|
/*-------------------------------------------------------------------------
|
|
|
|
|
*
|
|
|
|
|
* prepunion.c
|
|
|
|
|
* Routines to plan set-operation queries. The filename is a leftover
|
|
|
|
|
* from a time when only UNIONs were implemented.
|
|
|
|
|
*
|
|
|
|
|
* There are two code paths in the planner for set-operation queries.
|
|
|
|
|
* If a subquery consists entirely of simple UNION ALL operations, it
|
|
|
|
|
* is converted into an "append relation". Otherwise, it is handled
|
|
|
|
|
* by the general code in this module (plan_set_operations and its
|
|
|
|
|
* subroutines). There is some support code here for the append-relation
|
|
|
|
|
* case, but most of the heavy lifting for that is done elsewhere,
|
|
|
|
|
* notably in prepjointree.c and allpaths.c.
|
|
|
|
|
*
|
|
|
|
|
* There is also some code here to support planning of queries that use
|
|
|
|
|
* inheritance (SELECT FROM foo*). Inheritance trees are converted into
|
|
|
|
|
* append relations, and thenceforth share code with the UNION ALL case.
|
|
|
|
|
*
|
|
|
|
|
*
|
|
|
|
|
* Portions Copyright (c) 1996-2018, PostgreSQL Global Development Group
|
|
|
|
|
* Portions Copyright (c) 1994, Regents of the University of California
|
|
|
|
|
*
|
|
|
|
|
*
|
|
|
|
|
* IDENTIFICATION
|
|
|
|
|
* src/backend/optimizer/prep/prepunion.c
|
|
|
|
|
*
|
|
|
|
|
*-------------------------------------------------------------------------
|
|
|
|
|
*/
|
|
|
|
|
#include "postgres.h"
|
|
|
|
|
|
|
|
|
|
#include <limits.h>
|
|
|
|
|
|
|
|
|
|
#include "access/heapam.h"
|
|
|
|
|
#include "access/htup_details.h"
|
|
|
|
|
#include "access/sysattr.h"
|
|
|
|
|
#include "catalog/partition.h"
|
|
|
|
|
#include "catalog/pg_inherits.h"
|
|
|
|
|
#include "catalog/pg_type.h"
|
|
|
|
|
#include "miscadmin.h"
|
|
|
|
|
#include "nodes/makefuncs.h"
|
|
|
|
|
#include "nodes/nodeFuncs.h"
|
|
|
|
|
#include "optimizer/cost.h"
|
|
|
|
|
#include "optimizer/pathnode.h"
|
Make the upper part of the planner work by generating and comparing Paths.
I've been saying we needed to do this for more than five years, and here it
finally is. This patch removes the ever-growing tangle of spaghetti logic
that grouping_planner() used to use to try to identify the best plan for
post-scan/join query steps. Now, there is (nearly) independent
consideration of each execution step, and entirely separate construction of
Paths to represent each of the possible ways to do that step. We choose
the best Path or set of Paths using the same add_path() logic that's been
used inside query_planner() for years.
In addition, this patch removes the old restriction that subquery_planner()
could return only a single Plan. It now returns a RelOptInfo containing a
set of Paths, just as query_planner() does, and the parent query level can
use each of those Paths as the basis of a SubqueryScanPath at its level.
This allows finding some optimizations that we missed before, wherein a
subquery was capable of returning presorted data and thereby avoiding a
sort in the parent level, making the overall cost cheaper even though
delivering sorted output was not the cheapest plan for the subquery in
isolation. (A couple of regression test outputs change in consequence of
that. However, there is very little change in visible planner behavior
overall, because the point of this patch is not to get immediate planning
benefits but to create the infrastructure for future improvements.)
There is a great deal left to do here. This patch unblocks a lot of
planner work that was basically impractical in the old code structure,
such as allowing FDWs to implement remote aggregation, or rewriting
plan_set_operations() to allow consideration of multiple implementation
orders for set operations. (The latter will likely require a full
rewrite of plan_set_operations(); what I've done here is only to fix it
to return Paths not Plans.) I have also left unfinished some localized
refactoring in createplan.c and planner.c, because it was not necessary
to get this patch to a working state.
Thanks to Robert Haas, David Rowley, and Amit Kapila for review.
9 years ago
|
|
|
#include "optimizer/paths.h"
|
|
|
|
|
#include "optimizer/planmain.h"
|
|
|
|
|
#include "optimizer/planner.h"
|
|
|
|
|
#include "optimizer/prep.h"
|
|
|
|
|
#include "optimizer/tlist.h"
|
|
|
|
|
#include "parser/parse_coerce.h"
|
|
|
|
|
#include "parser/parsetree.h"
|
|
|
|
|
#include "utils/lsyscache.h"
|
|
|
|
|
#include "utils/rel.h"
|
|
|
|
|
#include "utils/selfuncs.h"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
typedef struct
|
|
|
|
|
{
|
|
|
|
|
PlannerInfo *root;
|
Teach adjust_appendrel_attrs(_multilevel) to do multiple translations.
Currently, child relations are always base relations, so when we
translate parent relids to child relids, we only need to translate
a singler relid. However, the proposed partition-wise join feature
will create child joins, which will mean we need to translate a set
of parent relids to the corresponding child relids. This is
preliminary refactoring to make that possible.
Ashutosh Bapat. Review and testing of the larger patch set of which
this is a part by Amit Langote, Rajkumar Raghuwanshi, Rafia Sabih,
Thomas Munro, Dilip Kumar, and me. Some adjustments, mostly
cosmetic, by me.
Discussion: http://postgr.es/m/CA+TgmobQK80vtXjAsPZWWXd7c8u13G86gmuLupN+uUJjA+i4nA@mail.gmail.com
8 years ago
|
|
|
int nappinfos;
|
|
|
|
|
AppendRelInfo **appinfos;
|
|
|
|
|
} adjust_appendrel_attrs_context;
|
|
|
|
|
|
|
|
|
|
static RelOptInfo *recurse_set_operations(Node *setOp, PlannerInfo *root,
|
|
|
|
|
List *colTypes, List *colCollations,
|
|
|
|
|
bool junkOK,
|
|
|
|
|
int flag, List *refnames_tlist,
|
Make the upper part of the planner work by generating and comparing Paths.
I've been saying we needed to do this for more than five years, and here it
finally is. This patch removes the ever-growing tangle of spaghetti logic
that grouping_planner() used to use to try to identify the best plan for
post-scan/join query steps. Now, there is (nearly) independent
consideration of each execution step, and entirely separate construction of
Paths to represent each of the possible ways to do that step. We choose
the best Path or set of Paths using the same add_path() logic that's been
used inside query_planner() for years.
In addition, this patch removes the old restriction that subquery_planner()
could return only a single Plan. It now returns a RelOptInfo containing a
set of Paths, just as query_planner() does, and the parent query level can
use each of those Paths as the basis of a SubqueryScanPath at its level.
This allows finding some optimizations that we missed before, wherein a
subquery was capable of returning presorted data and thereby avoiding a
sort in the parent level, making the overall cost cheaper even though
delivering sorted output was not the cheapest plan for the subquery in
isolation. (A couple of regression test outputs change in consequence of
that. However, there is very little change in visible planner behavior
overall, because the point of this patch is not to get immediate planning
benefits but to create the infrastructure for future improvements.)
There is a great deal left to do here. This patch unblocks a lot of
planner work that was basically impractical in the old code structure,
such as allowing FDWs to implement remote aggregation, or rewriting
plan_set_operations() to allow consideration of multiple implementation
orders for set operations. (The latter will likely require a full
rewrite of plan_set_operations(); what I've done here is only to fix it
to return Paths not Plans.) I have also left unfinished some localized
refactoring in createplan.c and planner.c, because it was not necessary
to get this patch to a working state.
Thanks to Robert Haas, David Rowley, and Amit Kapila for review.
9 years ago
|
|
|
List **pTargetList,
|
|
|
|
|
double *pNumGroups);
|
|
|
|
|
static RelOptInfo *generate_recursion_path(SetOperationStmt *setOp,
|
Make the upper part of the planner work by generating and comparing Paths.
I've been saying we needed to do this for more than five years, and here it
finally is. This patch removes the ever-growing tangle of spaghetti logic
that grouping_planner() used to use to try to identify the best plan for
post-scan/join query steps. Now, there is (nearly) independent
consideration of each execution step, and entirely separate construction of
Paths to represent each of the possible ways to do that step. We choose
the best Path or set of Paths using the same add_path() logic that's been
used inside query_planner() for years.
In addition, this patch removes the old restriction that subquery_planner()
could return only a single Plan. It now returns a RelOptInfo containing a
set of Paths, just as query_planner() does, and the parent query level can
use each of those Paths as the basis of a SubqueryScanPath at its level.
This allows finding some optimizations that we missed before, wherein a
subquery was capable of returning presorted data and thereby avoiding a
sort in the parent level, making the overall cost cheaper even though
delivering sorted output was not the cheapest plan for the subquery in
isolation. (A couple of regression test outputs change in consequence of
that. However, there is very little change in visible planner behavior
overall, because the point of this patch is not to get immediate planning
benefits but to create the infrastructure for future improvements.)
There is a great deal left to do here. This patch unblocks a lot of
planner work that was basically impractical in the old code structure,
such as allowing FDWs to implement remote aggregation, or rewriting
plan_set_operations() to allow consideration of multiple implementation
orders for set operations. (The latter will likely require a full
rewrite of plan_set_operations(); what I've done here is only to fix it
to return Paths not Plans.) I have also left unfinished some localized
refactoring in createplan.c and planner.c, because it was not necessary
to get this patch to a working state.
Thanks to Robert Haas, David Rowley, and Amit Kapila for review.
9 years ago
|
|
|
PlannerInfo *root,
|
|
|
|
|
List *refnames_tlist,
|
Make the upper part of the planner work by generating and comparing Paths.
I've been saying we needed to do this for more than five years, and here it
finally is. This patch removes the ever-growing tangle of spaghetti logic
that grouping_planner() used to use to try to identify the best plan for
post-scan/join query steps. Now, there is (nearly) independent
consideration of each execution step, and entirely separate construction of
Paths to represent each of the possible ways to do that step. We choose
the best Path or set of Paths using the same add_path() logic that's been
used inside query_planner() for years.
In addition, this patch removes the old restriction that subquery_planner()
could return only a single Plan. It now returns a RelOptInfo containing a
set of Paths, just as query_planner() does, and the parent query level can
use each of those Paths as the basis of a SubqueryScanPath at its level.
This allows finding some optimizations that we missed before, wherein a
subquery was capable of returning presorted data and thereby avoiding a
sort in the parent level, making the overall cost cheaper even though
delivering sorted output was not the cheapest plan for the subquery in
isolation. (A couple of regression test outputs change in consequence of
that. However, there is very little change in visible planner behavior
overall, because the point of this patch is not to get immediate planning
benefits but to create the infrastructure for future improvements.)
There is a great deal left to do here. This patch unblocks a lot of
planner work that was basically impractical in the old code structure,
such as allowing FDWs to implement remote aggregation, or rewriting
plan_set_operations() to allow consideration of multiple implementation
orders for set operations. (The latter will likely require a full
rewrite of plan_set_operations(); what I've done here is only to fix it
to return Paths not Plans.) I have also left unfinished some localized
refactoring in createplan.c and planner.c, because it was not necessary
to get this patch to a working state.
Thanks to Robert Haas, David Rowley, and Amit Kapila for review.
9 years ago
|
|
|
List **pTargetList);
|
|
|
|
|
static RelOptInfo *generate_union_paths(SetOperationStmt *op, PlannerInfo *root,
|
|
|
|
|
List *refnames_tlist,
|
|
|
|
|
List **pTargetList);
|
|
|
|
|
static RelOptInfo *generate_nonunion_paths(SetOperationStmt *op, PlannerInfo *root,
|
|
|
|
|
List *refnames_tlist,
|
|
|
|
|
List **pTargetList);
|
|
|
|
|
static List *plan_union_children(PlannerInfo *root,
|
|
|
|
|
SetOperationStmt *top_union,
|
|
|
|
|
List *refnames_tlist,
|
|
|
|
|
List **tlist_list);
|
Make the upper part of the planner work by generating and comparing Paths.
I've been saying we needed to do this for more than five years, and here it
finally is. This patch removes the ever-growing tangle of spaghetti logic
that grouping_planner() used to use to try to identify the best plan for
post-scan/join query steps. Now, there is (nearly) independent
consideration of each execution step, and entirely separate construction of
Paths to represent each of the possible ways to do that step. We choose
the best Path or set of Paths using the same add_path() logic that's been
used inside query_planner() for years.
In addition, this patch removes the old restriction that subquery_planner()
could return only a single Plan. It now returns a RelOptInfo containing a
set of Paths, just as query_planner() does, and the parent query level can
use each of those Paths as the basis of a SubqueryScanPath at its level.
This allows finding some optimizations that we missed before, wherein a
subquery was capable of returning presorted data and thereby avoiding a
sort in the parent level, making the overall cost cheaper even though
delivering sorted output was not the cheapest plan for the subquery in
isolation. (A couple of regression test outputs change in consequence of
that. However, there is very little change in visible planner behavior
overall, because the point of this patch is not to get immediate planning
benefits but to create the infrastructure for future improvements.)
There is a great deal left to do here. This patch unblocks a lot of
planner work that was basically impractical in the old code structure,
such as allowing FDWs to implement remote aggregation, or rewriting
plan_set_operations() to allow consideration of multiple implementation
orders for set operations. (The latter will likely require a full
rewrite of plan_set_operations(); what I've done here is only to fix it
to return Paths not Plans.) I have also left unfinished some localized
refactoring in createplan.c and planner.c, because it was not necessary
to get this patch to a working state.
Thanks to Robert Haas, David Rowley, and Amit Kapila for review.
9 years ago
|
|
|
static Path *make_union_unique(SetOperationStmt *op, Path *path, List *tlist,
|
|
|
|
|
PlannerInfo *root);
|
|
|
|
|
static void postprocess_setop_rel(PlannerInfo *root, RelOptInfo *rel);
|
|
|
|
|
static bool choose_hashed_setop(PlannerInfo *root, List *groupClauses,
|
Make the upper part of the planner work by generating and comparing Paths.
I've been saying we needed to do this for more than five years, and here it
finally is. This patch removes the ever-growing tangle of spaghetti logic
that grouping_planner() used to use to try to identify the best plan for
post-scan/join query steps. Now, there is (nearly) independent
consideration of each execution step, and entirely separate construction of
Paths to represent each of the possible ways to do that step. We choose
the best Path or set of Paths using the same add_path() logic that's been
used inside query_planner() for years.
In addition, this patch removes the old restriction that subquery_planner()
could return only a single Plan. It now returns a RelOptInfo containing a
set of Paths, just as query_planner() does, and the parent query level can
use each of those Paths as the basis of a SubqueryScanPath at its level.
This allows finding some optimizations that we missed before, wherein a
subquery was capable of returning presorted data and thereby avoiding a
sort in the parent level, making the overall cost cheaper even though
delivering sorted output was not the cheapest plan for the subquery in
isolation. (A couple of regression test outputs change in consequence of
that. However, there is very little change in visible planner behavior
overall, because the point of this patch is not to get immediate planning
benefits but to create the infrastructure for future improvements.)
There is a great deal left to do here. This patch unblocks a lot of
planner work that was basically impractical in the old code structure,
such as allowing FDWs to implement remote aggregation, or rewriting
plan_set_operations() to allow consideration of multiple implementation
orders for set operations. (The latter will likely require a full
rewrite of plan_set_operations(); what I've done here is only to fix it
to return Paths not Plans.) I have also left unfinished some localized
refactoring in createplan.c and planner.c, because it was not necessary
to get this patch to a working state.
Thanks to Robert Haas, David Rowley, and Amit Kapila for review.
9 years ago
|
|
|
Path *input_path,
|
|
|
|
|
double dNumGroups, double dNumOutputRows,
|
|
|
|
|
const char *construct);
|
|
|
|
|
static List *generate_setop_tlist(List *colTypes, List *colCollations,
|
|
|
|
|
int flag,
|
|
|
|
|
Index varno,
|
|
|
|
|
bool hack_constants,
|
|
|
|
|
List *input_tlist,
|
|
|
|
|
List *refnames_tlist);
|
|
|
|
|
static List *generate_append_tlist(List *colTypes, List *colCollations,
|
|
|
|
|
bool flag,
|
Make the upper part of the planner work by generating and comparing Paths.
I've been saying we needed to do this for more than five years, and here it
finally is. This patch removes the ever-growing tangle of spaghetti logic
that grouping_planner() used to use to try to identify the best plan for
post-scan/join query steps. Now, there is (nearly) independent
consideration of each execution step, and entirely separate construction of
Paths to represent each of the possible ways to do that step. We choose
the best Path or set of Paths using the same add_path() logic that's been
used inside query_planner() for years.
In addition, this patch removes the old restriction that subquery_planner()
could return only a single Plan. It now returns a RelOptInfo containing a
set of Paths, just as query_planner() does, and the parent query level can
use each of those Paths as the basis of a SubqueryScanPath at its level.
This allows finding some optimizations that we missed before, wherein a
subquery was capable of returning presorted data and thereby avoiding a
sort in the parent level, making the overall cost cheaper even though
delivering sorted output was not the cheapest plan for the subquery in
isolation. (A couple of regression test outputs change in consequence of
that. However, there is very little change in visible planner behavior
overall, because the point of this patch is not to get immediate planning
benefits but to create the infrastructure for future improvements.)
There is a great deal left to do here. This patch unblocks a lot of
planner work that was basically impractical in the old code structure,
such as allowing FDWs to implement remote aggregation, or rewriting
plan_set_operations() to allow consideration of multiple implementation
orders for set operations. (The latter will likely require a full
rewrite of plan_set_operations(); what I've done here is only to fix it
to return Paths not Plans.) I have also left unfinished some localized
refactoring in createplan.c and planner.c, because it was not necessary
to get this patch to a working state.
Thanks to Robert Haas, David Rowley, and Amit Kapila for review.
9 years ago
|
|
|
List *input_tlists,
|
|
|
|
|
List *refnames_tlist);
|
|
|
|
|
static List *generate_setop_grouplist(SetOperationStmt *op, List *targetlist);
|
|
|
|
|
static void expand_inherited_rtentry(PlannerInfo *root, RangeTblEntry *rte,
|
|
|
|
|
Index rti);
|
|
|
|
|
static void expand_partitioned_rtentry(PlannerInfo *root,
|
|
|
|
|
RangeTblEntry *parentrte,
|
|
|
|
|
Index parentRTindex, Relation parentrel,
|
|
|
|
|
PlanRowMark *top_parentrc, LOCKMODE lockmode,
|
Faster partition pruning
Add a new module backend/partitioning/partprune.c, implementing a more
sophisticated algorithm for partition pruning. The new module uses each
partition's "boundinfo" for pruning instead of constraint exclusion,
based on an idea proposed by Robert Haas of a "pruning program": a list
of steps generated from the query quals which are run iteratively to
obtain a list of partitions that must be scanned in order to satisfy
those quals.
At present, this targets planner-time partition pruning, but there exist
further patches to apply partition pruning at execution time as well.
This commit also moves some definitions from include/catalog/partition.h
to a new file include/partitioning/partbounds.h, in an attempt to
rationalize partitioning related code.
Authors: Amit Langote, David Rowley, Dilip Kumar
Reviewers: Robert Haas, Kyotaro Horiguchi, Ashutosh Bapat, Jesper Pedersen.
Discussion: https://postgr.es/m/098b9c71-1915-1a2a-8d52-1a7a50ce79e8@lab.ntt.co.jp
7 years ago
|
|
|
List **appinfos);
|
|
|
|
|
static void expand_single_inheritance_child(PlannerInfo *root,
|
|
|
|
|
RangeTblEntry *parentrte,
|
|
|
|
|
Index parentRTindex, Relation parentrel,
|
|
|
|
|
PlanRowMark *top_parentrc, Relation childrel,
|
|
|
|
|
List **appinfos, RangeTblEntry **childrte_p,
|
|
|
|
|
Index *childRTindex_p);
|
|
|
|
|
static void make_inh_translation_list(Relation oldrelation,
|
|
|
|
|
Relation newrelation,
|
|
|
|
|
Index newvarno,
|
|
|
|
|
List **translated_vars);
|
|
|
|
|
static Bitmapset *translate_col_privs(const Bitmapset *parent_privs,
|
|
|
|
|
List *translated_vars);
|
|
|
|
|
static Node *adjust_appendrel_attrs_mutator(Node *node,
|
|
|
|
|
adjust_appendrel_attrs_context *context);
|
Teach adjust_appendrel_attrs(_multilevel) to do multiple translations.
Currently, child relations are always base relations, so when we
translate parent relids to child relids, we only need to translate
a singler relid. However, the proposed partition-wise join feature
will create child joins, which will mean we need to translate a set
of parent relids to the corresponding child relids. This is
preliminary refactoring to make that possible.
Ashutosh Bapat. Review and testing of the larger patch set of which
this is a part by Amit Langote, Rajkumar Raghuwanshi, Rafia Sabih,
Thomas Munro, Dilip Kumar, and me. Some adjustments, mostly
cosmetic, by me.
Discussion: http://postgr.es/m/CA+TgmobQK80vtXjAsPZWWXd7c8u13G86gmuLupN+uUJjA+i4nA@mail.gmail.com
8 years ago
|
|
|
static Relids adjust_child_relids(Relids relids, int nappinfos,
|
|
|
|
|
AppendRelInfo **appinfos);
|
|
|
|
|
static List *adjust_inherited_tlist(List *tlist,
|
|
|
|
|
AppendRelInfo *context);
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* plan_set_operations
|
|
|
|
|
*
|
|
|
|
|
* Plans the queries for a tree of set operations (UNION/INTERSECT/EXCEPT)
|
|
|
|
|
*
|
|
|
|
|
* This routine only deals with the setOperations tree of the given query.
|
Make the upper part of the planner work by generating and comparing Paths.
I've been saying we needed to do this for more than five years, and here it
finally is. This patch removes the ever-growing tangle of spaghetti logic
that grouping_planner() used to use to try to identify the best plan for
post-scan/join query steps. Now, there is (nearly) independent
consideration of each execution step, and entirely separate construction of
Paths to represent each of the possible ways to do that step. We choose
the best Path or set of Paths using the same add_path() logic that's been
used inside query_planner() for years.
In addition, this patch removes the old restriction that subquery_planner()
could return only a single Plan. It now returns a RelOptInfo containing a
set of Paths, just as query_planner() does, and the parent query level can
use each of those Paths as the basis of a SubqueryScanPath at its level.
This allows finding some optimizations that we missed before, wherein a
subquery was capable of returning presorted data and thereby avoiding a
sort in the parent level, making the overall cost cheaper even though
delivering sorted output was not the cheapest plan for the subquery in
isolation. (A couple of regression test outputs change in consequence of
that. However, there is very little change in visible planner behavior
overall, because the point of this patch is not to get immediate planning
benefits but to create the infrastructure for future improvements.)
There is a great deal left to do here. This patch unblocks a lot of
planner work that was basically impractical in the old code structure,
such as allowing FDWs to implement remote aggregation, or rewriting
plan_set_operations() to allow consideration of multiple implementation
orders for set operations. (The latter will likely require a full
rewrite of plan_set_operations(); what I've done here is only to fix it
to return Paths not Plans.) I have also left unfinished some localized
refactoring in createplan.c and planner.c, because it was not necessary
to get this patch to a working state.
Thanks to Robert Haas, David Rowley, and Amit Kapila for review.
9 years ago
|
|
|
* Any top-level ORDER BY requested in root->parse->sortClause will be handled
|
|
|
|
|
* when we return to grouping_planner; likewise for LIMIT.
|
|
|
|
|
*
|
Make the upper part of the planner work by generating and comparing Paths.
I've been saying we needed to do this for more than five years, and here it
finally is. This patch removes the ever-growing tangle of spaghetti logic
that grouping_planner() used to use to try to identify the best plan for
post-scan/join query steps. Now, there is (nearly) independent
consideration of each execution step, and entirely separate construction of
Paths to represent each of the possible ways to do that step. We choose
the best Path or set of Paths using the same add_path() logic that's been
used inside query_planner() for years.
In addition, this patch removes the old restriction that subquery_planner()
could return only a single Plan. It now returns a RelOptInfo containing a
set of Paths, just as query_planner() does, and the parent query level can
use each of those Paths as the basis of a SubqueryScanPath at its level.
This allows finding some optimizations that we missed before, wherein a
subquery was capable of returning presorted data and thereby avoiding a
sort in the parent level, making the overall cost cheaper even though
delivering sorted output was not the cheapest plan for the subquery in
isolation. (A couple of regression test outputs change in consequence of
that. However, there is very little change in visible planner behavior
overall, because the point of this patch is not to get immediate planning
benefits but to create the infrastructure for future improvements.)
There is a great deal left to do here. This patch unblocks a lot of
planner work that was basically impractical in the old code structure,
such as allowing FDWs to implement remote aggregation, or rewriting
plan_set_operations() to allow consideration of multiple implementation
orders for set operations. (The latter will likely require a full
rewrite of plan_set_operations(); what I've done here is only to fix it
to return Paths not Plans.) I have also left unfinished some localized
refactoring in createplan.c and planner.c, because it was not necessary
to get this patch to a working state.
Thanks to Robert Haas, David Rowley, and Amit Kapila for review.
9 years ago
|
|
|
* What we return is an "upperrel" RelOptInfo containing at least one Path
|
|
|
|
|
* that implements the set-operation tree. In addition, root->processed_tlist
|
|
|
|
|
* receives a targetlist representing the output of the topmost setop node.
|
|
|
|
|
*/
|
Make the upper part of the planner work by generating and comparing Paths.
I've been saying we needed to do this for more than five years, and here it
finally is. This patch removes the ever-growing tangle of spaghetti logic
that grouping_planner() used to use to try to identify the best plan for
post-scan/join query steps. Now, there is (nearly) independent
consideration of each execution step, and entirely separate construction of
Paths to represent each of the possible ways to do that step. We choose
the best Path or set of Paths using the same add_path() logic that's been
used inside query_planner() for years.
In addition, this patch removes the old restriction that subquery_planner()
could return only a single Plan. It now returns a RelOptInfo containing a
set of Paths, just as query_planner() does, and the parent query level can
use each of those Paths as the basis of a SubqueryScanPath at its level.
This allows finding some optimizations that we missed before, wherein a
subquery was capable of returning presorted data and thereby avoiding a
sort in the parent level, making the overall cost cheaper even though
delivering sorted output was not the cheapest plan for the subquery in
isolation. (A couple of regression test outputs change in consequence of
that. However, there is very little change in visible planner behavior
overall, because the point of this patch is not to get immediate planning
benefits but to create the infrastructure for future improvements.)
There is a great deal left to do here. This patch unblocks a lot of
planner work that was basically impractical in the old code structure,
such as allowing FDWs to implement remote aggregation, or rewriting
plan_set_operations() to allow consideration of multiple implementation
orders for set operations. (The latter will likely require a full
rewrite of plan_set_operations(); what I've done here is only to fix it
to return Paths not Plans.) I have also left unfinished some localized
refactoring in createplan.c and planner.c, because it was not necessary
to get this patch to a working state.
Thanks to Robert Haas, David Rowley, and Amit Kapila for review.
9 years ago
|
|
|
RelOptInfo *
|
|
|
|
|
plan_set_operations(PlannerInfo *root)
|
|
|
|
|
{
|
|
|
|
|
Query *parse = root->parse;
|
|
|
|
|
SetOperationStmt *topop = castNode(SetOperationStmt, parse->setOperations);
|
|
|
|
|
Node *node;
|
|
|
|
|
RangeTblEntry *leftmostRTE;
|
|
|
|
|
Query *leftmostQuery;
|
Make the upper part of the planner work by generating and comparing Paths.
I've been saying we needed to do this for more than five years, and here it
finally is. This patch removes the ever-growing tangle of spaghetti logic
that grouping_planner() used to use to try to identify the best plan for
post-scan/join query steps. Now, there is (nearly) independent
consideration of each execution step, and entirely separate construction of
Paths to represent each of the possible ways to do that step. We choose
the best Path or set of Paths using the same add_path() logic that's been
used inside query_planner() for years.
In addition, this patch removes the old restriction that subquery_planner()
could return only a single Plan. It now returns a RelOptInfo containing a
set of Paths, just as query_planner() does, and the parent query level can
use each of those Paths as the basis of a SubqueryScanPath at its level.
This allows finding some optimizations that we missed before, wherein a
subquery was capable of returning presorted data and thereby avoiding a
sort in the parent level, making the overall cost cheaper even though
delivering sorted output was not the cheapest plan for the subquery in
isolation. (A couple of regression test outputs change in consequence of
that. However, there is very little change in visible planner behavior
overall, because the point of this patch is not to get immediate planning
benefits but to create the infrastructure for future improvements.)
There is a great deal left to do here. This patch unblocks a lot of
planner work that was basically impractical in the old code structure,
such as allowing FDWs to implement remote aggregation, or rewriting
plan_set_operations() to allow consideration of multiple implementation
orders for set operations. (The latter will likely require a full
rewrite of plan_set_operations(); what I've done here is only to fix it
to return Paths not Plans.) I have also left unfinished some localized
refactoring in createplan.c and planner.c, because it was not necessary
to get this patch to a working state.
Thanks to Robert Haas, David Rowley, and Amit Kapila for review.
9 years ago
|
|
|
RelOptInfo *setop_rel;
|
|
|
|
|
List *top_tlist;
|
|
|
|
|
|
|
|
|
|
Assert(topop);
|
|
|
|
|
|
|
|
|
|
/* check for unsupported stuff */
|
|
|
|
|
Assert(parse->jointree->fromlist == NIL);
|
|
|
|
|
Assert(parse->jointree->quals == NULL);
|
|
|
|
|
Assert(parse->groupClause == NIL);
|
|
|
|
|
Assert(parse->havingQual == NULL);
|
|
|
|
|
Assert(parse->windowClause == NIL);
|
|
|
|
|
Assert(parse->distinctClause == NIL);
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* We'll need to build RelOptInfos for each of the leaf subqueries, which
|
|
|
|
|
* are RTE_SUBQUERY rangetable entries in this Query. Prepare the index
|
|
|
|
|
* arrays for that.
|
|
|
|
|
*/
|
|
|
|
|
setup_simple_rel_arrays(root);
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Populate append_rel_array with each AppendRelInfo to allow direct
|
|
|
|
|
* lookups by child relid.
|
|
|
|
|
*/
|
|
|
|
|
setup_append_rel_array(root);
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Find the leftmost component Query. We need to use its column names for
|
|
|
|
|
* all generated tlists (else SELECT INTO won't work right).
|
|
|
|
|
*/
|
|
|
|
|
node = topop->larg;
|
|
|
|
|
while (node && IsA(node, SetOperationStmt))
|
|
|
|
|
node = ((SetOperationStmt *) node)->larg;
|
|
|
|
|
Assert(node && IsA(node, RangeTblRef));
|
|
|
|
|
leftmostRTE = root->simple_rte_array[((RangeTblRef *) node)->rtindex];
|
|
|
|
|
leftmostQuery = leftmostRTE->subquery;
|
|
|
|
|
Assert(leftmostQuery != NULL);
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* If the topmost node is a recursive union, it needs special processing.
|
|
|
|
|
*/
|
|
|
|
|
if (root->hasRecursion)
|
Make the upper part of the planner work by generating and comparing Paths.
I've been saying we needed to do this for more than five years, and here it
finally is. This patch removes the ever-growing tangle of spaghetti logic
that grouping_planner() used to use to try to identify the best plan for
post-scan/join query steps. Now, there is (nearly) independent
consideration of each execution step, and entirely separate construction of
Paths to represent each of the possible ways to do that step. We choose
the best Path or set of Paths using the same add_path() logic that's been
used inside query_planner() for years.
In addition, this patch removes the old restriction that subquery_planner()
could return only a single Plan. It now returns a RelOptInfo containing a
set of Paths, just as query_planner() does, and the parent query level can
use each of those Paths as the basis of a SubqueryScanPath at its level.
This allows finding some optimizations that we missed before, wherein a
subquery was capable of returning presorted data and thereby avoiding a
sort in the parent level, making the overall cost cheaper even though
delivering sorted output was not the cheapest plan for the subquery in
isolation. (A couple of regression test outputs change in consequence of
that. However, there is very little change in visible planner behavior
overall, because the point of this patch is not to get immediate planning
benefits but to create the infrastructure for future improvements.)
There is a great deal left to do here. This patch unblocks a lot of
planner work that was basically impractical in the old code structure,
such as allowing FDWs to implement remote aggregation, or rewriting
plan_set_operations() to allow consideration of multiple implementation
orders for set operations. (The latter will likely require a full
rewrite of plan_set_operations(); what I've done here is only to fix it
to return Paths not Plans.) I have also left unfinished some localized
refactoring in createplan.c and planner.c, because it was not necessary
to get this patch to a working state.
Thanks to Robert Haas, David Rowley, and Amit Kapila for review.
9 years ago
|
|
|
{
|
|
|
|
|
setop_rel = generate_recursion_path(topop, root,
|
|
|
|
|
leftmostQuery->targetList,
|
|
|
|
|
&top_tlist);
|
Make the upper part of the planner work by generating and comparing Paths.
I've been saying we needed to do this for more than five years, and here it
finally is. This patch removes the ever-growing tangle of spaghetti logic
that grouping_planner() used to use to try to identify the best plan for
post-scan/join query steps. Now, there is (nearly) independent
consideration of each execution step, and entirely separate construction of
Paths to represent each of the possible ways to do that step. We choose
the best Path or set of Paths using the same add_path() logic that's been
used inside query_planner() for years.
In addition, this patch removes the old restriction that subquery_planner()
could return only a single Plan. It now returns a RelOptInfo containing a
set of Paths, just as query_planner() does, and the parent query level can
use each of those Paths as the basis of a SubqueryScanPath at its level.
This allows finding some optimizations that we missed before, wherein a
subquery was capable of returning presorted data and thereby avoiding a
sort in the parent level, making the overall cost cheaper even though
delivering sorted output was not the cheapest plan for the subquery in
isolation. (A couple of regression test outputs change in consequence of
that. However, there is very little change in visible planner behavior
overall, because the point of this patch is not to get immediate planning
benefits but to create the infrastructure for future improvements.)
There is a great deal left to do here. This patch unblocks a lot of
planner work that was basically impractical in the old code structure,
such as allowing FDWs to implement remote aggregation, or rewriting
plan_set_operations() to allow consideration of multiple implementation
orders for set operations. (The latter will likely require a full
rewrite of plan_set_operations(); what I've done here is only to fix it
to return Paths not Plans.) I have also left unfinished some localized
refactoring in createplan.c and planner.c, because it was not necessary
to get this patch to a working state.
Thanks to Robert Haas, David Rowley, and Amit Kapila for review.
9 years ago
|
|
|
}
|
|
|
|
|
else
|
|
|
|
|
{
|
|
|
|
|
/*
|
|
|
|
|
* Recurse on setOperations tree to generate paths for set ops. The
|
|
|
|
|
* final output paths should have just the column types shown as the
|
Make the upper part of the planner work by generating and comparing Paths.
I've been saying we needed to do this for more than five years, and here it
finally is. This patch removes the ever-growing tangle of spaghetti logic
that grouping_planner() used to use to try to identify the best plan for
post-scan/join query steps. Now, there is (nearly) independent
consideration of each execution step, and entirely separate construction of
Paths to represent each of the possible ways to do that step. We choose
the best Path or set of Paths using the same add_path() logic that's been
used inside query_planner() for years.
In addition, this patch removes the old restriction that subquery_planner()
could return only a single Plan. It now returns a RelOptInfo containing a
set of Paths, just as query_planner() does, and the parent query level can
use each of those Paths as the basis of a SubqueryScanPath at its level.
This allows finding some optimizations that we missed before, wherein a
subquery was capable of returning presorted data and thereby avoiding a
sort in the parent level, making the overall cost cheaper even though
delivering sorted output was not the cheapest plan for the subquery in
isolation. (A couple of regression test outputs change in consequence of
that. However, there is very little change in visible planner behavior
overall, because the point of this patch is not to get immediate planning
benefits but to create the infrastructure for future improvements.)
There is a great deal left to do here. This patch unblocks a lot of
planner work that was basically impractical in the old code structure,
such as allowing FDWs to implement remote aggregation, or rewriting
plan_set_operations() to allow consideration of multiple implementation
orders for set operations. (The latter will likely require a full
rewrite of plan_set_operations(); what I've done here is only to fix it
to return Paths not Plans.) I have also left unfinished some localized
refactoring in createplan.c and planner.c, because it was not necessary
to get this patch to a working state.
Thanks to Robert Haas, David Rowley, and Amit Kapila for review.
9 years ago
|
|
|
* output from the top-level node, plus possibly resjunk working
|
|
|
|
|
* columns (we can rely on upper-level nodes to deal with that).
|
|
|
|
|
*/
|
|
|
|
|
setop_rel = recurse_set_operations((Node *) topop, root,
|
|
|
|
|
topop->colTypes, topop->colCollations,
|
|
|
|
|
true, -1,
|
|
|
|
|
leftmostQuery->targetList,
|
|
|
|
|
&top_tlist,
|
|
|
|
|
NULL);
|
Make the upper part of the planner work by generating and comparing Paths.
I've been saying we needed to do this for more than five years, and here it
finally is. This patch removes the ever-growing tangle of spaghetti logic
that grouping_planner() used to use to try to identify the best plan for
post-scan/join query steps. Now, there is (nearly) independent
consideration of each execution step, and entirely separate construction of
Paths to represent each of the possible ways to do that step. We choose
the best Path or set of Paths using the same add_path() logic that's been
used inside query_planner() for years.
In addition, this patch removes the old restriction that subquery_planner()
could return only a single Plan. It now returns a RelOptInfo containing a
set of Paths, just as query_planner() does, and the parent query level can
use each of those Paths as the basis of a SubqueryScanPath at its level.
This allows finding some optimizations that we missed before, wherein a
subquery was capable of returning presorted data and thereby avoiding a
sort in the parent level, making the overall cost cheaper even though
delivering sorted output was not the cheapest plan for the subquery in
isolation. (A couple of regression test outputs change in consequence of
that. However, there is very little change in visible planner behavior
overall, because the point of this patch is not to get immediate planning
benefits but to create the infrastructure for future improvements.)
There is a great deal left to do here. This patch unblocks a lot of
planner work that was basically impractical in the old code structure,
such as allowing FDWs to implement remote aggregation, or rewriting
plan_set_operations() to allow consideration of multiple implementation
orders for set operations. (The latter will likely require a full
rewrite of plan_set_operations(); what I've done here is only to fix it
to return Paths not Plans.) I have also left unfinished some localized
refactoring in createplan.c and planner.c, because it was not necessary
to get this patch to a working state.
Thanks to Robert Haas, David Rowley, and Amit Kapila for review.
9 years ago
|
|
|
}
|
|
|
|
|
|
Make the upper part of the planner work by generating and comparing Paths.
I've been saying we needed to do this for more than five years, and here it
finally is. This patch removes the ever-growing tangle of spaghetti logic
that grouping_planner() used to use to try to identify the best plan for
post-scan/join query steps. Now, there is (nearly) independent
consideration of each execution step, and entirely separate construction of
Paths to represent each of the possible ways to do that step. We choose
the best Path or set of Paths using the same add_path() logic that's been
used inside query_planner() for years.
In addition, this patch removes the old restriction that subquery_planner()
could return only a single Plan. It now returns a RelOptInfo containing a
set of Paths, just as query_planner() does, and the parent query level can
use each of those Paths as the basis of a SubqueryScanPath at its level.
This allows finding some optimizations that we missed before, wherein a
subquery was capable of returning presorted data and thereby avoiding a
sort in the parent level, making the overall cost cheaper even though
delivering sorted output was not the cheapest plan for the subquery in
isolation. (A couple of regression test outputs change in consequence of
that. However, there is very little change in visible planner behavior
overall, because the point of this patch is not to get immediate planning
benefits but to create the infrastructure for future improvements.)
There is a great deal left to do here. This patch unblocks a lot of
planner work that was basically impractical in the old code structure,
such as allowing FDWs to implement remote aggregation, or rewriting
plan_set_operations() to allow consideration of multiple implementation
orders for set operations. (The latter will likely require a full
rewrite of plan_set_operations(); what I've done here is only to fix it
to return Paths not Plans.) I have also left unfinished some localized
refactoring in createplan.c and planner.c, because it was not necessary
to get this patch to a working state.
Thanks to Robert Haas, David Rowley, and Amit Kapila for review.
9 years ago
|
|
|
/* Must return the built tlist into root->processed_tlist. */
|
|
|
|
|
root->processed_tlist = top_tlist;
|
|
|
|
|
|
|
|
|
|
return setop_rel;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* recurse_set_operations
|
|
|
|
|
* Recursively handle one step in a tree of set operations
|
|
|
|
|
*
|
|
|
|
|
* colTypes: OID list of set-op's result column datatypes
|
|
|
|
|
* colCollations: OID list of set-op's result column collations
|
|
|
|
|
* junkOK: if true, child resjunk columns may be left in the result
|
|
|
|
|
* flag: if >= 0, add a resjunk output column indicating value of flag
|
|
|
|
|
* refnames_tlist: targetlist to take column names from
|
|
|
|
|
*
|
|
|
|
|
* Returns a RelOptInfo for the subtree, as well as these output parameters:
|
Make the upper part of the planner work by generating and comparing Paths.
I've been saying we needed to do this for more than five years, and here it
finally is. This patch removes the ever-growing tangle of spaghetti logic
that grouping_planner() used to use to try to identify the best plan for
post-scan/join query steps. Now, there is (nearly) independent
consideration of each execution step, and entirely separate construction of
Paths to represent each of the possible ways to do that step. We choose
the best Path or set of Paths using the same add_path() logic that's been
used inside query_planner() for years.
In addition, this patch removes the old restriction that subquery_planner()
could return only a single Plan. It now returns a RelOptInfo containing a
set of Paths, just as query_planner() does, and the parent query level can
use each of those Paths as the basis of a SubqueryScanPath at its level.
This allows finding some optimizations that we missed before, wherein a
subquery was capable of returning presorted data and thereby avoiding a
sort in the parent level, making the overall cost cheaper even though
delivering sorted output was not the cheapest plan for the subquery in
isolation. (A couple of regression test outputs change in consequence of
that. However, there is very little change in visible planner behavior
overall, because the point of this patch is not to get immediate planning
benefits but to create the infrastructure for future improvements.)
There is a great deal left to do here. This patch unblocks a lot of
planner work that was basically impractical in the old code structure,
such as allowing FDWs to implement remote aggregation, or rewriting
plan_set_operations() to allow consideration of multiple implementation
orders for set operations. (The latter will likely require a full
rewrite of plan_set_operations(); what I've done here is only to fix it
to return Paths not Plans.) I have also left unfinished some localized
refactoring in createplan.c and planner.c, because it was not necessary
to get this patch to a working state.
Thanks to Robert Haas, David Rowley, and Amit Kapila for review.
9 years ago
|
|
|
* *pTargetList: receives the fully-fledged tlist for the subtree's top plan
|
|
|
|
|
* *pNumGroups: if not NULL, we estimate the number of distinct groups
|
|
|
|
|
* in the result, and store it there
|
|
|
|
|
*
|
Make the upper part of the planner work by generating and comparing Paths.
I've been saying we needed to do this for more than five years, and here it
finally is. This patch removes the ever-growing tangle of spaghetti logic
that grouping_planner() used to use to try to identify the best plan for
post-scan/join query steps. Now, there is (nearly) independent
consideration of each execution step, and entirely separate construction of
Paths to represent each of the possible ways to do that step. We choose
the best Path or set of Paths using the same add_path() logic that's been
used inside query_planner() for years.
In addition, this patch removes the old restriction that subquery_planner()
could return only a single Plan. It now returns a RelOptInfo containing a
set of Paths, just as query_planner() does, and the parent query level can
use each of those Paths as the basis of a SubqueryScanPath at its level.
This allows finding some optimizations that we missed before, wherein a
subquery was capable of returning presorted data and thereby avoiding a
sort in the parent level, making the overall cost cheaper even though
delivering sorted output was not the cheapest plan for the subquery in
isolation. (A couple of regression test outputs change in consequence of
that. However, there is very little change in visible planner behavior
overall, because the point of this patch is not to get immediate planning
benefits but to create the infrastructure for future improvements.)
There is a great deal left to do here. This patch unblocks a lot of
planner work that was basically impractical in the old code structure,
such as allowing FDWs to implement remote aggregation, or rewriting
plan_set_operations() to allow consideration of multiple implementation
orders for set operations. (The latter will likely require a full
rewrite of plan_set_operations(); what I've done here is only to fix it
to return Paths not Plans.) I have also left unfinished some localized
refactoring in createplan.c and planner.c, because it was not necessary
to get this patch to a working state.
Thanks to Robert Haas, David Rowley, and Amit Kapila for review.
9 years ago
|
|
|
* The pTargetList output parameter is mostly redundant with the pathtarget
|
|
|
|
|
* of the returned RelOptInfo, but for the moment we need it because much of
|
|
|
|
|
* the logic in this file depends on flag columns being marked resjunk.
|
|
|
|
|
* Pending a redesign of how that works, this is the easy way out.
|
Make the upper part of the planner work by generating and comparing Paths.
I've been saying we needed to do this for more than five years, and here it
finally is. This patch removes the ever-growing tangle of spaghetti logic
that grouping_planner() used to use to try to identify the best plan for
post-scan/join query steps. Now, there is (nearly) independent
consideration of each execution step, and entirely separate construction of
Paths to represent each of the possible ways to do that step. We choose
the best Path or set of Paths using the same add_path() logic that's been
used inside query_planner() for years.
In addition, this patch removes the old restriction that subquery_planner()
could return only a single Plan. It now returns a RelOptInfo containing a
set of Paths, just as query_planner() does, and the parent query level can
use each of those Paths as the basis of a SubqueryScanPath at its level.
This allows finding some optimizations that we missed before, wherein a
subquery was capable of returning presorted data and thereby avoiding a
sort in the parent level, making the overall cost cheaper even though
delivering sorted output was not the cheapest plan for the subquery in
isolation. (A couple of regression test outputs change in consequence of
that. However, there is very little change in visible planner behavior
overall, because the point of this patch is not to get immediate planning
benefits but to create the infrastructure for future improvements.)
There is a great deal left to do here. This patch unblocks a lot of
planner work that was basically impractical in the old code structure,
such as allowing FDWs to implement remote aggregation, or rewriting
plan_set_operations() to allow consideration of multiple implementation
orders for set operations. (The latter will likely require a full
rewrite of plan_set_operations(); what I've done here is only to fix it
to return Paths not Plans.) I have also left unfinished some localized
refactoring in createplan.c and planner.c, because it was not necessary
to get this patch to a working state.
Thanks to Robert Haas, David Rowley, and Amit Kapila for review.
9 years ago
|
|
|
*
|
|
|
|
|
* We don't have to care about typmods here: the only allowed difference
|
|
|
|
|
* between set-op input and output typmods is input is a specific typmod
|
|
|
|
|
* and output is -1, and that does not require a coercion.
|
|
|
|
|
*/
|
|
|
|
|
static RelOptInfo *
|
|
|
|
|
recurse_set_operations(Node *setOp, PlannerInfo *root,
|
|
|
|
|
List *colTypes, List *colCollations,
|
|
|
|
|
bool junkOK,
|
|
|
|
|
int flag, List *refnames_tlist,
|
Make the upper part of the planner work by generating and comparing Paths.
I've been saying we needed to do this for more than five years, and here it
finally is. This patch removes the ever-growing tangle of spaghetti logic
that grouping_planner() used to use to try to identify the best plan for
post-scan/join query steps. Now, there is (nearly) independent
consideration of each execution step, and entirely separate construction of
Paths to represent each of the possible ways to do that step. We choose
the best Path or set of Paths using the same add_path() logic that's been
used inside query_planner() for years.
In addition, this patch removes the old restriction that subquery_planner()
could return only a single Plan. It now returns a RelOptInfo containing a
set of Paths, just as query_planner() does, and the parent query level can
use each of those Paths as the basis of a SubqueryScanPath at its level.
This allows finding some optimizations that we missed before, wherein a
subquery was capable of returning presorted data and thereby avoiding a
sort in the parent level, making the overall cost cheaper even though
delivering sorted output was not the cheapest plan for the subquery in
isolation. (A couple of regression test outputs change in consequence of
that. However, there is very little change in visible planner behavior
overall, because the point of this patch is not to get immediate planning
benefits but to create the infrastructure for future improvements.)
There is a great deal left to do here. This patch unblocks a lot of
planner work that was basically impractical in the old code structure,
such as allowing FDWs to implement remote aggregation, or rewriting
plan_set_operations() to allow consideration of multiple implementation
orders for set operations. (The latter will likely require a full
rewrite of plan_set_operations(); what I've done here is only to fix it
to return Paths not Plans.) I have also left unfinished some localized
refactoring in createplan.c and planner.c, because it was not necessary
to get this patch to a working state.
Thanks to Robert Haas, David Rowley, and Amit Kapila for review.
9 years ago
|
|
|
List **pTargetList,
|
|
|
|
|
double *pNumGroups)
|
|
|
|
|
{
|
|
|
|
|
RelOptInfo *rel = NULL; /* keep compiler quiet */
|
|
|
|
|
|
|
|
|
|
/* Guard against stack overflow due to overly complex setop nests */
|
|
|
|
|
check_stack_depth();
|
|
|
|
|
|
|
|
|
|
if (IsA(setOp, RangeTblRef))
|
|
|
|
|
{
|
|
|
|
|
RangeTblRef *rtr = (RangeTblRef *) setOp;
|
|
|
|
|
RangeTblEntry *rte = root->simple_rte_array[rtr->rtindex];
|
|
|
|
|
Query *subquery = rte->subquery;
|
|
|
|
|
PlannerInfo *subroot;
|
Make the upper part of the planner work by generating and comparing Paths.
I've been saying we needed to do this for more than five years, and here it
finally is. This patch removes the ever-growing tangle of spaghetti logic
that grouping_planner() used to use to try to identify the best plan for
post-scan/join query steps. Now, there is (nearly) independent
consideration of each execution step, and entirely separate construction of
Paths to represent each of the possible ways to do that step. We choose
the best Path or set of Paths using the same add_path() logic that's been
used inside query_planner() for years.
In addition, this patch removes the old restriction that subquery_planner()
could return only a single Plan. It now returns a RelOptInfo containing a
set of Paths, just as query_planner() does, and the parent query level can
use each of those Paths as the basis of a SubqueryScanPath at its level.
This allows finding some optimizations that we missed before, wherein a
subquery was capable of returning presorted data and thereby avoiding a
sort in the parent level, making the overall cost cheaper even though
delivering sorted output was not the cheapest plan for the subquery in
isolation. (A couple of regression test outputs change in consequence of
that. However, there is very little change in visible planner behavior
overall, because the point of this patch is not to get immediate planning
benefits but to create the infrastructure for future improvements.)
There is a great deal left to do here. This patch unblocks a lot of
planner work that was basically impractical in the old code structure,
such as allowing FDWs to implement remote aggregation, or rewriting
plan_set_operations() to allow consideration of multiple implementation
orders for set operations. (The latter will likely require a full
rewrite of plan_set_operations(); what I've done here is only to fix it
to return Paths not Plans.) I have also left unfinished some localized
refactoring in createplan.c and planner.c, because it was not necessary
to get this patch to a working state.
Thanks to Robert Haas, David Rowley, and Amit Kapila for review.
9 years ago
|
|
|
RelOptInfo *final_rel;
|
|
|
|
|
Path *subpath;
|
|
|
|
|
Path *path;
|
|
|
|
|
List *tlist;
|
|
|
|
|
|
|
|
|
|
Assert(subquery != NULL);
|
|
|
|
|
|
|
|
|
|
/* Build a RelOptInfo for this leaf subquery. */
|
Abstract logic to allow for multiple kinds of child rels.
Currently, the only type of child relation is an "other member rel",
which is the child of a baserel, but in the future joins and even
upper relations may have child rels. To facilitate that, introduce
macros that test to test for particular RelOptKind values, and use
them in various places where they help to clarify the sense of a test.
(For example, a test may allow RELOPT_OTHER_MEMBER_REL either because
it intends to allow child rels, or because it intends to allow simple
rels.)
Also, remove find_childrel_top_parent, which will not work for a
child rel that is not a baserel. Instead, add a new RelOptInfo
member top_parent_relids to track the same kind of information in a
more generic manner.
Ashutosh Bapat, slightly tweaked by me. Review and testing of the
patch set from which this was taken by Rajkumar Raghuwanshi and Rafia
Sabih.
Discussion: http://postgr.es/m/CA+TgmoagTnF2yqR3PT2rv=om=wJiZ4-A+ATwdnriTGku1CLYxA@mail.gmail.com
8 years ago
|
|
|
rel = build_simple_rel(root, rtr->rtindex, NULL);
|
|
|
|
|
|
Fix PARAM_EXEC assignment mechanism to be safe in the presence of WITH.
The planner previously assumed that parameter Vars having the same absolute
query level, varno, and varattno could safely be assigned the same runtime
PARAM_EXEC slot, even though they might be different Vars appearing in
different subqueries. This was (probably) safe before the introduction of
CTEs, but the lazy-evalution mechanism used for CTEs means that a CTE can
be executed during execution of some other subquery, causing the lifespan
of Params at the same syntactic nesting level as the CTE to overlap with
use of the same slots inside the CTE. In 9.1 we created additional hazards
by using the same parameter-assignment technology for nestloop inner scan
parameters, but it was broken before that, as illustrated by the added
regression test.
To fix, restructure the planner's management of PlannerParamItems so that
items having different semantic lifespans are kept rigorously separated.
This will probably result in complex queries using more runtime PARAM_EXEC
slots than before, but the slots are cheap enough that this hardly matters.
Also, stop generating PlannerParamItems containing Params for subquery
outputs: all we really need to do is reserve the PARAM_EXEC slot number,
and that now only takes incrementing a counter. The planning code is
simpler and probably faster than before, as well as being more correct.
Per report from Vik Reykja.
These changes will mostly also need to be made in the back branches, but
I'm going to hold off on that until after 9.2.0 wraps.
13 years ago
|
|
|
/* plan_params should not be in use in current query level */
|
|
|
|
|
Assert(root->plan_params == NIL);
|
|
|
|
|
|
Make the upper part of the planner work by generating and comparing Paths.
I've been saying we needed to do this for more than five years, and here it
finally is. This patch removes the ever-growing tangle of spaghetti logic
that grouping_planner() used to use to try to identify the best plan for
post-scan/join query steps. Now, there is (nearly) independent
consideration of each execution step, and entirely separate construction of
Paths to represent each of the possible ways to do that step. We choose
the best Path or set of Paths using the same add_path() logic that's been
used inside query_planner() for years.
In addition, this patch removes the old restriction that subquery_planner()
could return only a single Plan. It now returns a RelOptInfo containing a
set of Paths, just as query_planner() does, and the parent query level can
use each of those Paths as the basis of a SubqueryScanPath at its level.
This allows finding some optimizations that we missed before, wherein a
subquery was capable of returning presorted data and thereby avoiding a
sort in the parent level, making the overall cost cheaper even though
delivering sorted output was not the cheapest plan for the subquery in
isolation. (A couple of regression test outputs change in consequence of
that. However, there is very little change in visible planner behavior
overall, because the point of this patch is not to get immediate planning
benefits but to create the infrastructure for future improvements.)
There is a great deal left to do here. This patch unblocks a lot of
planner work that was basically impractical in the old code structure,
such as allowing FDWs to implement remote aggregation, or rewriting
plan_set_operations() to allow consideration of multiple implementation
orders for set operations. (The latter will likely require a full
rewrite of plan_set_operations(); what I've done here is only to fix it
to return Paths not Plans.) I have also left unfinished some localized
refactoring in createplan.c and planner.c, because it was not necessary
to get this patch to a working state.
Thanks to Robert Haas, David Rowley, and Amit Kapila for review.
9 years ago
|
|
|
/* Generate a subroot and Paths for the subquery */
|
|
|
|
|
subroot = rel->subroot = subquery_planner(root->glob, subquery,
|
|
|
|
|
root,
|
|
|
|
|
false,
|
|
|
|
|
root->tuple_fraction);
|
|
|
|
|
|
Fix PARAM_EXEC assignment mechanism to be safe in the presence of WITH.
The planner previously assumed that parameter Vars having the same absolute
query level, varno, and varattno could safely be assigned the same runtime
PARAM_EXEC slot, even though they might be different Vars appearing in
different subqueries. This was (probably) safe before the introduction of
CTEs, but the lazy-evalution mechanism used for CTEs means that a CTE can
be executed during execution of some other subquery, causing the lifespan
of Params at the same syntactic nesting level as the CTE to overlap with
use of the same slots inside the CTE. In 9.1 we created additional hazards
by using the same parameter-assignment technology for nestloop inner scan
parameters, but it was broken before that, as illustrated by the added
regression test.
To fix, restructure the planner's management of PlannerParamItems so that
items having different semantic lifespans are kept rigorously separated.
This will probably result in complex queries using more runtime PARAM_EXEC
slots than before, but the slots are cheap enough that this hardly matters.
Also, stop generating PlannerParamItems containing Params for subquery
outputs: all we really need to do is reserve the PARAM_EXEC slot number,
and that now only takes incrementing a counter. The planning code is
simpler and probably faster than before, as well as being more correct.
Per report from Vik Reykja.
These changes will mostly also need to be made in the back branches, but
I'm going to hold off on that until after 9.2.0 wraps.
13 years ago
|
|
|
/*
|
|
|
|
|
* It should not be possible for the primitive query to contain any
|
|
|
|
|
* cross-references to other primitive queries in the setop tree.
|
|
|
|
|
*/
|
|
|
|
|
if (root->plan_params)
|
|
|
|
|
elog(ERROR, "unexpected outer reference in set operation subquery");
|
|
|
|
|
|
|
|
|
|
/* Figure out the appropriate target list for this subquery. */
|
|
|
|
|
tlist = generate_setop_tlist(colTypes, colCollations,
|
|
|
|
|
flag,
|
|
|
|
|
rtr->rtindex,
|
|
|
|
|
true,
|
|
|
|
|
subroot->processed_tlist,
|
|
|
|
|
refnames_tlist);
|
|
|
|
|
rel->reltarget = create_pathtarget(root, tlist);
|
|
|
|
|
|
|
|
|
|
/* Return the fully-fledged tlist to caller, too */
|
|
|
|
|
*pTargetList = tlist;
|
|
|
|
|
|
Make the upper part of the planner work by generating and comparing Paths.
I've been saying we needed to do this for more than five years, and here it
finally is. This patch removes the ever-growing tangle of spaghetti logic
that grouping_planner() used to use to try to identify the best plan for
post-scan/join query steps. Now, there is (nearly) independent
consideration of each execution step, and entirely separate construction of
Paths to represent each of the possible ways to do that step. We choose
the best Path or set of Paths using the same add_path() logic that's been
used inside query_planner() for years.
In addition, this patch removes the old restriction that subquery_planner()
could return only a single Plan. It now returns a RelOptInfo containing a
set of Paths, just as query_planner() does, and the parent query level can
use each of those Paths as the basis of a SubqueryScanPath at its level.
This allows finding some optimizations that we missed before, wherein a
subquery was capable of returning presorted data and thereby avoiding a
sort in the parent level, making the overall cost cheaper even though
delivering sorted output was not the cheapest plan for the subquery in
isolation. (A couple of regression test outputs change in consequence of
that. However, there is very little change in visible planner behavior
overall, because the point of this patch is not to get immediate planning
benefits but to create the infrastructure for future improvements.)
There is a great deal left to do here. This patch unblocks a lot of
planner work that was basically impractical in the old code structure,
such as allowing FDWs to implement remote aggregation, or rewriting
plan_set_operations() to allow consideration of multiple implementation
orders for set operations. (The latter will likely require a full
rewrite of plan_set_operations(); what I've done here is only to fix it
to return Paths not Plans.) I have also left unfinished some localized
refactoring in createplan.c and planner.c, because it was not necessary
to get this patch to a working state.
Thanks to Robert Haas, David Rowley, and Amit Kapila for review.
9 years ago
|
|
|
/*
|
|
|
|
|
* Mark rel with estimated output rows, width, etc. Note that we have
|
|
|
|
|
* to do this before generating outer-query paths, else
|
|
|
|
|
* cost_subqueryscan is not happy.
|
|
|
|
|
*/
|
|
|
|
|
set_subquery_size_estimates(root, rel);
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Since we may want to add a partial path to this relation, we must
|
|
|
|
|
* set its consider_parallel flag correctly.
|
|
|
|
|
*/
|
|
|
|
|
final_rel = fetch_upper_rel(subroot, UPPERREL_FINAL, NULL);
|
|
|
|
|
rel->consider_parallel = final_rel->consider_parallel;
|
|
|
|
|
|
Make the upper part of the planner work by generating and comparing Paths.
I've been saying we needed to do this for more than five years, and here it
finally is. This patch removes the ever-growing tangle of spaghetti logic
that grouping_planner() used to use to try to identify the best plan for
post-scan/join query steps. Now, there is (nearly) independent
consideration of each execution step, and entirely separate construction of
Paths to represent each of the possible ways to do that step. We choose
the best Path or set of Paths using the same add_path() logic that's been
used inside query_planner() for years.
In addition, this patch removes the old restriction that subquery_planner()
could return only a single Plan. It now returns a RelOptInfo containing a
set of Paths, just as query_planner() does, and the parent query level can
use each of those Paths as the basis of a SubqueryScanPath at its level.
This allows finding some optimizations that we missed before, wherein a
subquery was capable of returning presorted data and thereby avoiding a
sort in the parent level, making the overall cost cheaper even though
delivering sorted output was not the cheapest plan for the subquery in
isolation. (A couple of regression test outputs change in consequence of
that. However, there is very little change in visible planner behavior
overall, because the point of this patch is not to get immediate planning
benefits but to create the infrastructure for future improvements.)
There is a great deal left to do here. This patch unblocks a lot of
planner work that was basically impractical in the old code structure,
such as allowing FDWs to implement remote aggregation, or rewriting
plan_set_operations() to allow consideration of multiple implementation
orders for set operations. (The latter will likely require a full
rewrite of plan_set_operations(); what I've done here is only to fix it
to return Paths not Plans.) I have also left unfinished some localized
refactoring in createplan.c and planner.c, because it was not necessary
to get this patch to a working state.
Thanks to Robert Haas, David Rowley, and Amit Kapila for review.
9 years ago
|
|
|
/*
|
|
|
|
|
* For the moment, we consider only a single Path for the subquery.
|
|
|
|
|
* This should change soon (make it look more like
|
|
|
|
|
* set_subquery_pathlist).
|
|
|
|
|
*/
|
|
|
|
|
subpath = get_cheapest_fractional_path(final_rel,
|
|
|
|
|
root->tuple_fraction);
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Stick a SubqueryScanPath atop that.
|
|
|
|
|
*
|
|
|
|
|
* We don't bother to determine the subquery's output ordering since
|
|
|
|
|
* it won't be reflected in the set-op result anyhow; so just label
|
|
|
|
|
* the SubqueryScanPath with nil pathkeys. (XXX that should change
|
|
|
|
|
* soon too, likely.)
|
|
|
|
|
*/
|
|
|
|
|
path = (Path *) create_subqueryscan_path(root, rel, subpath,
|
|
|
|
|
NIL, NULL);
|
|
|
|
|
|
|
|
|
|
add_path(rel, path);
|
Make the upper part of the planner work by generating and comparing Paths.
I've been saying we needed to do this for more than five years, and here it
finally is. This patch removes the ever-growing tangle of spaghetti logic
that grouping_planner() used to use to try to identify the best plan for
post-scan/join query steps. Now, there is (nearly) independent
consideration of each execution step, and entirely separate construction of
Paths to represent each of the possible ways to do that step. We choose
the best Path or set of Paths using the same add_path() logic that's been
used inside query_planner() for years.
In addition, this patch removes the old restriction that subquery_planner()
could return only a single Plan. It now returns a RelOptInfo containing a
set of Paths, just as query_planner() does, and the parent query level can
use each of those Paths as the basis of a SubqueryScanPath at its level.
This allows finding some optimizations that we missed before, wherein a
subquery was capable of returning presorted data and thereby avoiding a
sort in the parent level, making the overall cost cheaper even though
delivering sorted output was not the cheapest plan for the subquery in
isolation. (A couple of regression test outputs change in consequence of
that. However, there is very little change in visible planner behavior
overall, because the point of this patch is not to get immediate planning
benefits but to create the infrastructure for future improvements.)
There is a great deal left to do here. This patch unblocks a lot of
planner work that was basically impractical in the old code structure,
such as allowing FDWs to implement remote aggregation, or rewriting
plan_set_operations() to allow consideration of multiple implementation
orders for set operations. (The latter will likely require a full
rewrite of plan_set_operations(); what I've done here is only to fix it
to return Paths not Plans.) I have also left unfinished some localized
refactoring in createplan.c and planner.c, because it was not necessary
to get this patch to a working state.
Thanks to Robert Haas, David Rowley, and Amit Kapila for review.
9 years ago
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* If we have a partial path for the child relation, we can use that
|
|
|
|
|
* to build a partial path for this relation. But there's no point in
|
|
|
|
|
* considering any path but the cheapest.
|
|
|
|
|
*/
|
|
|
|
|
if (rel->consider_parallel && bms_is_empty(rel->lateral_relids) &&
|
|
|
|
|
final_rel->partial_pathlist != NIL)
|
|
|
|
|
{
|
|
|
|
|
Path *partial_subpath;
|
|
|
|
|
Path *partial_path;
|
|
|
|
|
|
|
|
|
|
partial_subpath = linitial(final_rel->partial_pathlist);
|
|
|
|
|
partial_path = (Path *)
|
|
|
|
|
create_subqueryscan_path(root, rel, partial_subpath,
|
|
|
|
|
NIL, NULL);
|
|
|
|
|
add_partial_path(rel, partial_path);
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Estimate number of groups if caller wants it. If the subquery used
|
|
|
|
|
* grouping or aggregation, its output is probably mostly unique
|
|
|
|
|
* anyway; otherwise do statistical estimation.
|
|
|
|
|
*
|
|
|
|
|
* XXX you don't really want to know about this: we do the estimation
|
|
|
|
|
* using the subquery's original targetlist expressions, not the
|
|
|
|
|
* subroot->processed_tlist which might seem more appropriate. The
|
|
|
|
|
* reason is that if the subquery is itself a setop, it may return a
|
|
|
|
|
* processed_tlist containing "varno 0" Vars generated by
|
|
|
|
|
* generate_append_tlist, and those would confuse estimate_num_groups
|
|
|
|
|
* mightily. We ought to get rid of the "varno 0" hack, but that
|
|
|
|
|
* requires a redesign of the parsetree representation of setops, so
|
|
|
|
|
* that there can be an RTE corresponding to each setop's output.
|
|
|
|
|
*/
|
|
|
|
|
if (pNumGroups)
|
|
|
|
|
{
|
Support GROUPING SETS, CUBE and ROLLUP.
This SQL standard functionality allows to aggregate data by different
GROUP BY clauses at once. Each grouping set returns rows with columns
grouped by in other sets set to NULL.
This could previously be achieved by doing each grouping as a separate
query, conjoined by UNION ALLs. Besides being considerably more concise,
grouping sets will in many cases be faster, requiring only one scan over
the underlying data.
The current implementation of grouping sets only supports using sorting
for input. Individual sets that share a sort order are computed in one
pass. If there are sets that don't share a sort order, additional sort &
aggregation steps are performed. These additional passes are sourced by
the previous sort step; thus avoiding repeated scans of the source data.
The code is structured in a way that adding support for purely using
hash aggregation or a mix of hashing and sorting is possible. Sorting
was chosen to be supported first, as it is the most generic method of
implementation.
Instead of, as in an earlier versions of the patch, representing the
chain of sort and aggregation steps as full blown planner and executor
nodes, all but the first sort are performed inside the aggregation node
itself. This avoids the need to do some unusual gymnastics to handle
having to return aggregated and non-aggregated tuples from underlying
nodes, as well as having to shut down underlying nodes early to limit
memory usage. The optimizer still builds Sort/Agg node to describe each
phase, but they're not part of the plan tree, but instead additional
data for the aggregation node. They're a convenient and preexisting way
to describe aggregation and sorting. The first (and possibly only) sort
step is still performed as a separate execution step. That retains
similarity with existing group by plans, makes rescans fairly simple,
avoids very deep plans (leading to slow explains) and easily allows to
avoid the sorting step if the underlying data is sorted by other means.
A somewhat ugly side of this patch is having to deal with a grammar
ambiguity between the new CUBE keyword and the cube extension/functions
named cube (and rollup). To avoid breaking existing deployments of the
cube extension it has not been renamed, neither has cube been made a
reserved keyword. Instead precedence hacking is used to make GROUP BY
cube(..) refer to the CUBE grouping sets feature, and not the function
cube(). To actually group by a function cube(), unlikely as that might
be, the function name has to be quoted.
Needs a catversion bump because stored rules may change.
Author: Andrew Gierth and Atri Sharma, with contributions from Andres Freund
Reviewed-By: Andres Freund, Noah Misch, Tom Lane, Svenne Krap, Tomas
Vondra, Erik Rijkers, Marti Raudsepp, Pavel Stehule
Discussion: CAOeZVidmVRe2jU6aMk_5qkxnB7dfmPROzM7Ur8JPW5j8Y5X-Lw@mail.gmail.com
10 years ago
|
|
|
if (subquery->groupClause || subquery->groupingSets ||
|
|
|
|
|
subquery->distinctClause ||
|
|
|
|
|
subroot->hasHavingQual || subquery->hasAggs)
|
Make the upper part of the planner work by generating and comparing Paths.
I've been saying we needed to do this for more than five years, and here it
finally is. This patch removes the ever-growing tangle of spaghetti logic
that grouping_planner() used to use to try to identify the best plan for
post-scan/join query steps. Now, there is (nearly) independent
consideration of each execution step, and entirely separate construction of
Paths to represent each of the possible ways to do that step. We choose
the best Path or set of Paths using the same add_path() logic that's been
used inside query_planner() for years.
In addition, this patch removes the old restriction that subquery_planner()
could return only a single Plan. It now returns a RelOptInfo containing a
set of Paths, just as query_planner() does, and the parent query level can
use each of those Paths as the basis of a SubqueryScanPath at its level.
This allows finding some optimizations that we missed before, wherein a
subquery was capable of returning presorted data and thereby avoiding a
sort in the parent level, making the overall cost cheaper even though
delivering sorted output was not the cheapest plan for the subquery in
isolation. (A couple of regression test outputs change in consequence of
that. However, there is very little change in visible planner behavior
overall, because the point of this patch is not to get immediate planning
benefits but to create the infrastructure for future improvements.)
There is a great deal left to do here. This patch unblocks a lot of
planner work that was basically impractical in the old code structure,
such as allowing FDWs to implement remote aggregation, or rewriting
plan_set_operations() to allow consideration of multiple implementation
orders for set operations. (The latter will likely require a full
rewrite of plan_set_operations(); what I've done here is only to fix it
to return Paths not Plans.) I have also left unfinished some localized
refactoring in createplan.c and planner.c, because it was not necessary
to get this patch to a working state.
Thanks to Robert Haas, David Rowley, and Amit Kapila for review.
9 years ago
|
|
|
*pNumGroups = subpath->rows;
|
|
|
|
|
else
|
|
|
|
|
*pNumGroups = estimate_num_groups(subroot,
|
Phase 3 of pgindent updates.
Don't move parenthesized lines to the left, even if that means they
flow past the right margin.
By default, BSD indent lines up statement continuation lines that are
within parentheses so that they start just to the right of the preceding
left parenthesis. However, traditionally, if that resulted in the
continuation line extending to the right of the desired right margin,
then indent would push it left just far enough to not overrun the margin,
if it could do so without making the continuation line start to the left of
the current statement indent. That makes for a weird mix of indentations
unless one has been completely rigid about never violating the 80-column
limit.
This behavior has been pretty universally panned by Postgres developers.
Hence, disable it with indent's new -lpl switch, so that parenthesized
lines are always lined up with the preceding left paren.
This patch is much less interesting than the first round of indent
changes, but also bulkier, so I thought it best to separate the effects.
Discussion: https://postgr.es/m/E1dAmxK-0006EE-1r@gemulon.postgresql.org
Discussion: https://postgr.es/m/30527.1495162840@sss.pgh.pa.us
8 years ago
|
|
|
get_tlist_exprs(subquery->targetList, false),
|
Make the upper part of the planner work by generating and comparing Paths.
I've been saying we needed to do this for more than five years, and here it
finally is. This patch removes the ever-growing tangle of spaghetti logic
that grouping_planner() used to use to try to identify the best plan for
post-scan/join query steps. Now, there is (nearly) independent
consideration of each execution step, and entirely separate construction of
Paths to represent each of the possible ways to do that step. We choose
the best Path or set of Paths using the same add_path() logic that's been
used inside query_planner() for years.
In addition, this patch removes the old restriction that subquery_planner()
could return only a single Plan. It now returns a RelOptInfo containing a
set of Paths, just as query_planner() does, and the parent query level can
use each of those Paths as the basis of a SubqueryScanPath at its level.
This allows finding some optimizations that we missed before, wherein a
subquery was capable of returning presorted data and thereby avoiding a
sort in the parent level, making the overall cost cheaper even though
delivering sorted output was not the cheapest plan for the subquery in
isolation. (A couple of regression test outputs change in consequence of
that. However, there is very little change in visible planner behavior
overall, because the point of this patch is not to get immediate planning
benefits but to create the infrastructure for future improvements.)
There is a great deal left to do here. This patch unblocks a lot of
planner work that was basically impractical in the old code structure,
such as allowing FDWs to implement remote aggregation, or rewriting
plan_set_operations() to allow consideration of multiple implementation
orders for set operations. (The latter will likely require a full
rewrite of plan_set_operations(); what I've done here is only to fix it
to return Paths not Plans.) I have also left unfinished some localized
refactoring in createplan.c and planner.c, because it was not necessary
to get this patch to a working state.
Thanks to Robert Haas, David Rowley, and Amit Kapila for review.
9 years ago
|
|
|
subpath->rows,
|
Support GROUPING SETS, CUBE and ROLLUP.
This SQL standard functionality allows to aggregate data by different
GROUP BY clauses at once. Each grouping set returns rows with columns
grouped by in other sets set to NULL.
This could previously be achieved by doing each grouping as a separate
query, conjoined by UNION ALLs. Besides being considerably more concise,
grouping sets will in many cases be faster, requiring only one scan over
the underlying data.
The current implementation of grouping sets only supports using sorting
for input. Individual sets that share a sort order are computed in one
pass. If there are sets that don't share a sort order, additional sort &
aggregation steps are performed. These additional passes are sourced by
the previous sort step; thus avoiding repeated scans of the source data.
The code is structured in a way that adding support for purely using
hash aggregation or a mix of hashing and sorting is possible. Sorting
was chosen to be supported first, as it is the most generic method of
implementation.
Instead of, as in an earlier versions of the patch, representing the
chain of sort and aggregation steps as full blown planner and executor
nodes, all but the first sort are performed inside the aggregation node
itself. This avoids the need to do some unusual gymnastics to handle
having to return aggregated and non-aggregated tuples from underlying
nodes, as well as having to shut down underlying nodes early to limit
memory usage. The optimizer still builds Sort/Agg node to describe each
phase, but they're not part of the plan tree, but instead additional
data for the aggregation node. They're a convenient and preexisting way
to describe aggregation and sorting. The first (and possibly only) sort
step is still performed as a separate execution step. That retains
similarity with existing group by plans, makes rescans fairly simple,
avoids very deep plans (leading to slow explains) and easily allows to
avoid the sorting step if the underlying data is sorted by other means.
A somewhat ugly side of this patch is having to deal with a grammar
ambiguity between the new CUBE keyword and the cube extension/functions
named cube (and rollup). To avoid breaking existing deployments of the
cube extension it has not been renamed, neither has cube been made a
reserved keyword. Instead precedence hacking is used to make GROUP BY
cube(..) refer to the CUBE grouping sets feature, and not the function
cube(). To actually group by a function cube(), unlikely as that might
be, the function name has to be quoted.
Needs a catversion bump because stored rules may change.
Author: Andrew Gierth and Atri Sharma, with contributions from Andres Freund
Reviewed-By: Andres Freund, Noah Misch, Tom Lane, Svenne Krap, Tomas
Vondra, Erik Rijkers, Marti Raudsepp, Pavel Stehule
Discussion: CAOeZVidmVRe2jU6aMk_5qkxnB7dfmPROzM7Ur8JPW5j8Y5X-Lw@mail.gmail.com
10 years ago
|
|
|
NULL);
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
else if (IsA(setOp, SetOperationStmt))
|
|
|
|
|
{
|
|
|
|
|
SetOperationStmt *op = (SetOperationStmt *) setOp;
|
|
|
|
|
|
|
|
|
|
/* UNIONs are much different from INTERSECT/EXCEPT */
|
|
|
|
|
if (op->op == SETOP_UNION)
|
|
|
|
|
rel = generate_union_paths(op, root,
|
|
|
|
|
refnames_tlist,
|
|
|
|
|
pTargetList);
|
|
|
|
|
else
|
|
|
|
|
rel = generate_nonunion_paths(op, root,
|
|
|
|
|
refnames_tlist,
|
|
|
|
|
pTargetList);
|
|
|
|
|
if (pNumGroups)
|
|
|
|
|
*pNumGroups = rel->rows;
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* If necessary, add a Result node to project the caller-requested
|
|
|
|
|
* output columns.
|
|
|
|
|
*
|
|
|
|
|
* XXX you don't really want to know about this: setrefs.c will apply
|
|
|
|
|
* fix_upper_expr() to the Result node's tlist. This would fail if the
|
|
|
|
|
* Vars generated by generate_setop_tlist() were not exactly equal()
|
|
|
|
|
* to the corresponding tlist entries of the subplan. However, since
|
|
|
|
|
* the subplan was generated by generate_union_plan() or
|
|
|
|
|
* generate_nonunion_plan(), and hence its tlist was generated by
|
|
|
|
|
* generate_append_tlist(), this will work. We just tell
|
|
|
|
|
* generate_setop_tlist() to use varno 0.
|
|
|
|
|
*/
|
|
|
|
|
if (flag >= 0 ||
|
Make the upper part of the planner work by generating and comparing Paths.
I've been saying we needed to do this for more than five years, and here it
finally is. This patch removes the ever-growing tangle of spaghetti logic
that grouping_planner() used to use to try to identify the best plan for
post-scan/join query steps. Now, there is (nearly) independent
consideration of each execution step, and entirely separate construction of
Paths to represent each of the possible ways to do that step. We choose
the best Path or set of Paths using the same add_path() logic that's been
used inside query_planner() for years.
In addition, this patch removes the old restriction that subquery_planner()
could return only a single Plan. It now returns a RelOptInfo containing a
set of Paths, just as query_planner() does, and the parent query level can
use each of those Paths as the basis of a SubqueryScanPath at its level.
This allows finding some optimizations that we missed before, wherein a
subquery was capable of returning presorted data and thereby avoiding a
sort in the parent level, making the overall cost cheaper even though
delivering sorted output was not the cheapest plan for the subquery in
isolation. (A couple of regression test outputs change in consequence of
that. However, there is very little change in visible planner behavior
overall, because the point of this patch is not to get immediate planning
benefits but to create the infrastructure for future improvements.)
There is a great deal left to do here. This patch unblocks a lot of
planner work that was basically impractical in the old code structure,
such as allowing FDWs to implement remote aggregation, or rewriting
plan_set_operations() to allow consideration of multiple implementation
orders for set operations. (The latter will likely require a full
rewrite of plan_set_operations(); what I've done here is only to fix it
to return Paths not Plans.) I have also left unfinished some localized
refactoring in createplan.c and planner.c, because it was not necessary
to get this patch to a working state.
Thanks to Robert Haas, David Rowley, and Amit Kapila for review.
9 years ago
|
|
|
!tlist_same_datatypes(*pTargetList, colTypes, junkOK) ||
|
|
|
|
|
!tlist_same_collations(*pTargetList, colCollations, junkOK))
|
|
|
|
|
{
|
|
|
|
|
PathTarget *target;
|
|
|
|
|
ListCell *lc;
|
|
|
|
|
|
Make the upper part of the planner work by generating and comparing Paths.
I've been saying we needed to do this for more than five years, and here it
finally is. This patch removes the ever-growing tangle of spaghetti logic
that grouping_planner() used to use to try to identify the best plan for
post-scan/join query steps. Now, there is (nearly) independent
consideration of each execution step, and entirely separate construction of
Paths to represent each of the possible ways to do that step. We choose
the best Path or set of Paths using the same add_path() logic that's been
used inside query_planner() for years.
In addition, this patch removes the old restriction that subquery_planner()
could return only a single Plan. It now returns a RelOptInfo containing a
set of Paths, just as query_planner() does, and the parent query level can
use each of those Paths as the basis of a SubqueryScanPath at its level.
This allows finding some optimizations that we missed before, wherein a
subquery was capable of returning presorted data and thereby avoiding a
sort in the parent level, making the overall cost cheaper even though
delivering sorted output was not the cheapest plan for the subquery in
isolation. (A couple of regression test outputs change in consequence of
that. However, there is very little change in visible planner behavior
overall, because the point of this patch is not to get immediate planning
benefits but to create the infrastructure for future improvements.)
There is a great deal left to do here. This patch unblocks a lot of
planner work that was basically impractical in the old code structure,
such as allowing FDWs to implement remote aggregation, or rewriting
plan_set_operations() to allow consideration of multiple implementation
orders for set operations. (The latter will likely require a full
rewrite of plan_set_operations(); what I've done here is only to fix it
to return Paths not Plans.) I have also left unfinished some localized
refactoring in createplan.c and planner.c, because it was not necessary
to get this patch to a working state.
Thanks to Robert Haas, David Rowley, and Amit Kapila for review.
9 years ago
|
|
|
*pTargetList = generate_setop_tlist(colTypes, colCollations,
|
|
|
|
|
flag,
|
|
|
|
|
0,
|
|
|
|
|
false,
|
|
|
|
|
*pTargetList,
|
|
|
|
|
refnames_tlist);
|
|
|
|
|
target = create_pathtarget(root, *pTargetList);
|
|
|
|
|
|
|
|
|
|
/* Apply projection to each path */
|
|
|
|
|
foreach(lc, rel->pathlist)
|
|
|
|
|
{
|
|
|
|
|
Path *subpath = (Path *) lfirst(lc);
|
|
|
|
|
Path *path;
|
|
|
|
|
|
|
|
|
|
Assert(subpath->param_info == NULL);
|
|
|
|
|
path = apply_projection_to_path(root, subpath->parent,
|
|
|
|
|
subpath, target);
|
|
|
|
|
/* If we had to add a Result, path is different from subpath */
|
|
|
|
|
if (path != subpath)
|
|
|
|
|
lfirst(lc) = path;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/* Apply projection to each partial path */
|
|
|
|
|
foreach(lc, rel->partial_pathlist)
|
|
|
|
|
{
|
|
|
|
|
Path *subpath = (Path *) lfirst(lc);
|
|
|
|
|
Path *path;
|
|
|
|
|
|
|
|
|
|
Assert(subpath->param_info == NULL);
|
|
|
|
|
|
|
|
|
|
/* avoid apply_projection_to_path, in case of multiple refs */
|
|
|
|
|
path = (Path *) create_projection_path(root, subpath->parent,
|
|
|
|
|
subpath, target);
|
|
|
|
|
lfirst(lc) = path;
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
else
|
|
|
|
|
{
|
|
|
|
|
elog(ERROR, "unrecognized node type: %d",
|
|
|
|
|
(int) nodeTag(setOp));
|
Make the upper part of the planner work by generating and comparing Paths.
I've been saying we needed to do this for more than five years, and here it
finally is. This patch removes the ever-growing tangle of spaghetti logic
that grouping_planner() used to use to try to identify the best plan for
post-scan/join query steps. Now, there is (nearly) independent
consideration of each execution step, and entirely separate construction of
Paths to represent each of the possible ways to do that step. We choose
the best Path or set of Paths using the same add_path() logic that's been
used inside query_planner() for years.
In addition, this patch removes the old restriction that subquery_planner()
could return only a single Plan. It now returns a RelOptInfo containing a
set of Paths, just as query_planner() does, and the parent query level can
use each of those Paths as the basis of a SubqueryScanPath at its level.
This allows finding some optimizations that we missed before, wherein a
subquery was capable of returning presorted data and thereby avoiding a
sort in the parent level, making the overall cost cheaper even though
delivering sorted output was not the cheapest plan for the subquery in
isolation. (A couple of regression test outputs change in consequence of
that. However, there is very little change in visible planner behavior
overall, because the point of this patch is not to get immediate planning
benefits but to create the infrastructure for future improvements.)
There is a great deal left to do here. This patch unblocks a lot of
planner work that was basically impractical in the old code structure,
such as allowing FDWs to implement remote aggregation, or rewriting
plan_set_operations() to allow consideration of multiple implementation
orders for set operations. (The latter will likely require a full
rewrite of plan_set_operations(); what I've done here is only to fix it
to return Paths not Plans.) I have also left unfinished some localized
refactoring in createplan.c and planner.c, because it was not necessary
to get this patch to a working state.
Thanks to Robert Haas, David Rowley, and Amit Kapila for review.
9 years ago
|
|
|
*pTargetList = NIL;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
postprocess_setop_rel(root, rel);
|
|
|
|
|
|
|
|
|
|
return rel;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Generate paths for a recursive UNION node
|
|
|
|
|
*/
|
|
|
|
|
static RelOptInfo *
|
Make the upper part of the planner work by generating and comparing Paths.
I've been saying we needed to do this for more than five years, and here it
finally is. This patch removes the ever-growing tangle of spaghetti logic
that grouping_planner() used to use to try to identify the best plan for
post-scan/join query steps. Now, there is (nearly) independent
consideration of each execution step, and entirely separate construction of
Paths to represent each of the possible ways to do that step. We choose
the best Path or set of Paths using the same add_path() logic that's been
used inside query_planner() for years.
In addition, this patch removes the old restriction that subquery_planner()
could return only a single Plan. It now returns a RelOptInfo containing a
set of Paths, just as query_planner() does, and the parent query level can
use each of those Paths as the basis of a SubqueryScanPath at its level.
This allows finding some optimizations that we missed before, wherein a
subquery was capable of returning presorted data and thereby avoiding a
sort in the parent level, making the overall cost cheaper even though
delivering sorted output was not the cheapest plan for the subquery in
isolation. (A couple of regression test outputs change in consequence of
that. However, there is very little change in visible planner behavior
overall, because the point of this patch is not to get immediate planning
benefits but to create the infrastructure for future improvements.)
There is a great deal left to do here. This patch unblocks a lot of
planner work that was basically impractical in the old code structure,
such as allowing FDWs to implement remote aggregation, or rewriting
plan_set_operations() to allow consideration of multiple implementation
orders for set operations. (The latter will likely require a full
rewrite of plan_set_operations(); what I've done here is only to fix it
to return Paths not Plans.) I have also left unfinished some localized
refactoring in createplan.c and planner.c, because it was not necessary
to get this patch to a working state.
Thanks to Robert Haas, David Rowley, and Amit Kapila for review.
9 years ago
|
|
|
generate_recursion_path(SetOperationStmt *setOp, PlannerInfo *root,
|
|
|
|
|
List *refnames_tlist,
|
Make the upper part of the planner work by generating and comparing Paths.
I've been saying we needed to do this for more than five years, and here it
finally is. This patch removes the ever-growing tangle of spaghetti logic
that grouping_planner() used to use to try to identify the best plan for
post-scan/join query steps. Now, there is (nearly) independent
consideration of each execution step, and entirely separate construction of
Paths to represent each of the possible ways to do that step. We choose
the best Path or set of Paths using the same add_path() logic that's been
used inside query_planner() for years.
In addition, this patch removes the old restriction that subquery_planner()
could return only a single Plan. It now returns a RelOptInfo containing a
set of Paths, just as query_planner() does, and the parent query level can
use each of those Paths as the basis of a SubqueryScanPath at its level.
This allows finding some optimizations that we missed before, wherein a
subquery was capable of returning presorted data and thereby avoiding a
sort in the parent level, making the overall cost cheaper even though
delivering sorted output was not the cheapest plan for the subquery in
isolation. (A couple of regression test outputs change in consequence of
that. However, there is very little change in visible planner behavior
overall, because the point of this patch is not to get immediate planning
benefits but to create the infrastructure for future improvements.)
There is a great deal left to do here. This patch unblocks a lot of
planner work that was basically impractical in the old code structure,
such as allowing FDWs to implement remote aggregation, or rewriting
plan_set_operations() to allow consideration of multiple implementation
orders for set operations. (The latter will likely require a full
rewrite of plan_set_operations(); what I've done here is only to fix it
to return Paths not Plans.) I have also left unfinished some localized
refactoring in createplan.c and planner.c, because it was not necessary
to get this patch to a working state.
Thanks to Robert Haas, David Rowley, and Amit Kapila for review.
9 years ago
|
|
|
List **pTargetList)
|
|
|
|
|
{
|
|
|
|
|
RelOptInfo *result_rel;
|
Make the upper part of the planner work by generating and comparing Paths.
I've been saying we needed to do this for more than five years, and here it
finally is. This patch removes the ever-growing tangle of spaghetti logic
that grouping_planner() used to use to try to identify the best plan for
post-scan/join query steps. Now, there is (nearly) independent
consideration of each execution step, and entirely separate construction of
Paths to represent each of the possible ways to do that step. We choose
the best Path or set of Paths using the same add_path() logic that's been
used inside query_planner() for years.
In addition, this patch removes the old restriction that subquery_planner()
could return only a single Plan. It now returns a RelOptInfo containing a
set of Paths, just as query_planner() does, and the parent query level can
use each of those Paths as the basis of a SubqueryScanPath at its level.
This allows finding some optimizations that we missed before, wherein a
subquery was capable of returning presorted data and thereby avoiding a
sort in the parent level, making the overall cost cheaper even though
delivering sorted output was not the cheapest plan for the subquery in
isolation. (A couple of regression test outputs change in consequence of
that. However, there is very little change in visible planner behavior
overall, because the point of this patch is not to get immediate planning
benefits but to create the infrastructure for future improvements.)
There is a great deal left to do here. This patch unblocks a lot of
planner work that was basically impractical in the old code structure,
such as allowing FDWs to implement remote aggregation, or rewriting
plan_set_operations() to allow consideration of multiple implementation
orders for set operations. (The latter will likely require a full
rewrite of plan_set_operations(); what I've done here is only to fix it
to return Paths not Plans.) I have also left unfinished some localized
refactoring in createplan.c and planner.c, because it was not necessary
to get this patch to a working state.
Thanks to Robert Haas, David Rowley, and Amit Kapila for review.
9 years ago
|
|
|
Path *path;
|
|
|
|
|
RelOptInfo *lrel,
|
|
|
|
|
*rrel;
|
Make the upper part of the planner work by generating and comparing Paths.
I've been saying we needed to do this for more than five years, and here it
finally is. This patch removes the ever-growing tangle of spaghetti logic
that grouping_planner() used to use to try to identify the best plan for
post-scan/join query steps. Now, there is (nearly) independent
consideration of each execution step, and entirely separate construction of
Paths to represent each of the possible ways to do that step. We choose
the best Path or set of Paths using the same add_path() logic that's been
used inside query_planner() for years.
In addition, this patch removes the old restriction that subquery_planner()
could return only a single Plan. It now returns a RelOptInfo containing a
set of Paths, just as query_planner() does, and the parent query level can
use each of those Paths as the basis of a SubqueryScanPath at its level.
This allows finding some optimizations that we missed before, wherein a
subquery was capable of returning presorted data and thereby avoiding a
sort in the parent level, making the overall cost cheaper even though
delivering sorted output was not the cheapest plan for the subquery in
isolation. (A couple of regression test outputs change in consequence of
that. However, there is very little change in visible planner behavior
overall, because the point of this patch is not to get immediate planning
benefits but to create the infrastructure for future improvements.)
There is a great deal left to do here. This patch unblocks a lot of
planner work that was basically impractical in the old code structure,
such as allowing FDWs to implement remote aggregation, or rewriting
plan_set_operations() to allow consideration of multiple implementation
orders for set operations. (The latter will likely require a full
rewrite of plan_set_operations(); what I've done here is only to fix it
to return Paths not Plans.) I have also left unfinished some localized
refactoring in createplan.c and planner.c, because it was not necessary
to get this patch to a working state.
Thanks to Robert Haas, David Rowley, and Amit Kapila for review.
9 years ago
|
|
|
Path *lpath;
|
|
|
|
|
Path *rpath;
|
|
|
|
|
List *lpath_tlist;
|
|
|
|
|
List *rpath_tlist;
|
|
|
|
|
List *tlist;
|
|
|
|
|
List *groupList;
|
Make the upper part of the planner work by generating and comparing Paths.
I've been saying we needed to do this for more than five years, and here it
finally is. This patch removes the ever-growing tangle of spaghetti logic
that grouping_planner() used to use to try to identify the best plan for
post-scan/join query steps. Now, there is (nearly) independent
consideration of each execution step, and entirely separate construction of
Paths to represent each of the possible ways to do that step. We choose
the best Path or set of Paths using the same add_path() logic that's been
used inside query_planner() for years.
In addition, this patch removes the old restriction that subquery_planner()
could return only a single Plan. It now returns a RelOptInfo containing a
set of Paths, just as query_planner() does, and the parent query level can
use each of those Paths as the basis of a SubqueryScanPath at its level.
This allows finding some optimizations that we missed before, wherein a
subquery was capable of returning presorted data and thereby avoiding a
sort in the parent level, making the overall cost cheaper even though
delivering sorted output was not the cheapest plan for the subquery in
isolation. (A couple of regression test outputs change in consequence of
that. However, there is very little change in visible planner behavior
overall, because the point of this patch is not to get immediate planning
benefits but to create the infrastructure for future improvements.)
There is a great deal left to do here. This patch unblocks a lot of
planner work that was basically impractical in the old code structure,
such as allowing FDWs to implement remote aggregation, or rewriting
plan_set_operations() to allow consideration of multiple implementation
orders for set operations. (The latter will likely require a full
rewrite of plan_set_operations(); what I've done here is only to fix it
to return Paths not Plans.) I have also left unfinished some localized
refactoring in createplan.c and planner.c, because it was not necessary
to get this patch to a working state.
Thanks to Robert Haas, David Rowley, and Amit Kapila for review.
9 years ago
|
|
|
double dNumGroups;
|
|
|
|
|
|
|
|
|
|
/* Parser should have rejected other cases */
|
|
|
|
|
if (setOp->op != SETOP_UNION)
|
|
|
|
|
elog(ERROR, "only UNION queries can be recursive");
|
|
|
|
|
/* Worktable ID should be assigned */
|
|
|
|
|
Assert(root->wt_param_id >= 0);
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Unlike a regular UNION node, process the left and right inputs
|
|
|
|
|
* separately without any intention of combining them into one Append.
|
|
|
|
|
*/
|
|
|
|
|
lrel = recurse_set_operations(setOp->larg, root,
|
|
|
|
|
setOp->colTypes, setOp->colCollations,
|
|
|
|
|
false, -1,
|
|
|
|
|
refnames_tlist,
|
|
|
|
|
&lpath_tlist,
|
|
|
|
|
NULL);
|
|
|
|
|
lpath = lrel->cheapest_total_path;
|
Make the upper part of the planner work by generating and comparing Paths.
I've been saying we needed to do this for more than five years, and here it
finally is. This patch removes the ever-growing tangle of spaghetti logic
that grouping_planner() used to use to try to identify the best plan for
post-scan/join query steps. Now, there is (nearly) independent
consideration of each execution step, and entirely separate construction of
Paths to represent each of the possible ways to do that step. We choose
the best Path or set of Paths using the same add_path() logic that's been
used inside query_planner() for years.
In addition, this patch removes the old restriction that subquery_planner()
could return only a single Plan. It now returns a RelOptInfo containing a
set of Paths, just as query_planner() does, and the parent query level can
use each of those Paths as the basis of a SubqueryScanPath at its level.
This allows finding some optimizations that we missed before, wherein a
subquery was capable of returning presorted data and thereby avoiding a
sort in the parent level, making the overall cost cheaper even though
delivering sorted output was not the cheapest plan for the subquery in
isolation. (A couple of regression test outputs change in consequence of
that. However, there is very little change in visible planner behavior
overall, because the point of this patch is not to get immediate planning
benefits but to create the infrastructure for future improvements.)
There is a great deal left to do here. This patch unblocks a lot of
planner work that was basically impractical in the old code structure,
such as allowing FDWs to implement remote aggregation, or rewriting
plan_set_operations() to allow consideration of multiple implementation
orders for set operations. (The latter will likely require a full
rewrite of plan_set_operations(); what I've done here is only to fix it
to return Paths not Plans.) I have also left unfinished some localized
refactoring in createplan.c and planner.c, because it was not necessary
to get this patch to a working state.
Thanks to Robert Haas, David Rowley, and Amit Kapila for review.
9 years ago
|
|
|
/* The right path will want to look at the left one ... */
|
|
|
|
|
root->non_recursive_path = lpath;
|
|
|
|
|
rrel = recurse_set_operations(setOp->rarg, root,
|
|
|
|
|
setOp->colTypes, setOp->colCollations,
|
|
|
|
|
false, -1,
|
|
|
|
|
refnames_tlist,
|
|
|
|
|
&rpath_tlist,
|
|
|
|
|
NULL);
|
|
|
|
|
rpath = rrel->cheapest_total_path;
|
Make the upper part of the planner work by generating and comparing Paths.
I've been saying we needed to do this for more than five years, and here it
finally is. This patch removes the ever-growing tangle of spaghetti logic
that grouping_planner() used to use to try to identify the best plan for
post-scan/join query steps. Now, there is (nearly) independent
consideration of each execution step, and entirely separate construction of
Paths to represent each of the possible ways to do that step. We choose
the best Path or set of Paths using the same add_path() logic that's been
used inside query_planner() for years.
In addition, this patch removes the old restriction that subquery_planner()
could return only a single Plan. It now returns a RelOptInfo containing a
set of Paths, just as query_planner() does, and the parent query level can
use each of those Paths as the basis of a SubqueryScanPath at its level.
This allows finding some optimizations that we missed before, wherein a
subquery was capable of returning presorted data and thereby avoiding a
sort in the parent level, making the overall cost cheaper even though
delivering sorted output was not the cheapest plan for the subquery in
isolation. (A couple of regression test outputs change in consequence of
that. However, there is very little change in visible planner behavior
overall, because the point of this patch is not to get immediate planning
benefits but to create the infrastructure for future improvements.)
There is a great deal left to do here. This patch unblocks a lot of
planner work that was basically impractical in the old code structure,
such as allowing FDWs to implement remote aggregation, or rewriting
plan_set_operations() to allow consideration of multiple implementation
orders for set operations. (The latter will likely require a full
rewrite of plan_set_operations(); what I've done here is only to fix it
to return Paths not Plans.) I have also left unfinished some localized
refactoring in createplan.c and planner.c, because it was not necessary
to get this patch to a working state.
Thanks to Robert Haas, David Rowley, and Amit Kapila for review.
9 years ago
|
|
|
root->non_recursive_path = NULL;
|
|
|
|
|
|
|
|
|
|
/*
|
Make the upper part of the planner work by generating and comparing Paths.
I've been saying we needed to do this for more than five years, and here it
finally is. This patch removes the ever-growing tangle of spaghetti logic
that grouping_planner() used to use to try to identify the best plan for
post-scan/join query steps. Now, there is (nearly) independent
consideration of each execution step, and entirely separate construction of
Paths to represent each of the possible ways to do that step. We choose
the best Path or set of Paths using the same add_path() logic that's been
used inside query_planner() for years.
In addition, this patch removes the old restriction that subquery_planner()
could return only a single Plan. It now returns a RelOptInfo containing a
set of Paths, just as query_planner() does, and the parent query level can
use each of those Paths as the basis of a SubqueryScanPath at its level.
This allows finding some optimizations that we missed before, wherein a
subquery was capable of returning presorted data and thereby avoiding a
sort in the parent level, making the overall cost cheaper even though
delivering sorted output was not the cheapest plan for the subquery in
isolation. (A couple of regression test outputs change in consequence of
that. However, there is very little change in visible planner behavior
overall, because the point of this patch is not to get immediate planning
benefits but to create the infrastructure for future improvements.)
There is a great deal left to do here. This patch unblocks a lot of
planner work that was basically impractical in the old code structure,
such as allowing FDWs to implement remote aggregation, or rewriting
plan_set_operations() to allow consideration of multiple implementation
orders for set operations. (The latter will likely require a full
rewrite of plan_set_operations(); what I've done here is only to fix it
to return Paths not Plans.) I have also left unfinished some localized
refactoring in createplan.c and planner.c, because it was not necessary
to get this patch to a working state.
Thanks to Robert Haas, David Rowley, and Amit Kapila for review.
9 years ago
|
|
|
* Generate tlist for RecursiveUnion path node --- same as in Append cases
|
|
|
|
|
*/
|
|
|
|
|
tlist = generate_append_tlist(setOp->colTypes, setOp->colCollations, false,
|
Make the upper part of the planner work by generating and comparing Paths.
I've been saying we needed to do this for more than five years, and here it
finally is. This patch removes the ever-growing tangle of spaghetti logic
that grouping_planner() used to use to try to identify the best plan for
post-scan/join query steps. Now, there is (nearly) independent
consideration of each execution step, and entirely separate construction of
Paths to represent each of the possible ways to do that step. We choose
the best Path or set of Paths using the same add_path() logic that's been
used inside query_planner() for years.
In addition, this patch removes the old restriction that subquery_planner()
could return only a single Plan. It now returns a RelOptInfo containing a
set of Paths, just as query_planner() does, and the parent query level can
use each of those Paths as the basis of a SubqueryScanPath at its level.
This allows finding some optimizations that we missed before, wherein a
subquery was capable of returning presorted data and thereby avoiding a
sort in the parent level, making the overall cost cheaper even though
delivering sorted output was not the cheapest plan for the subquery in
isolation. (A couple of regression test outputs change in consequence of
that. However, there is very little change in visible planner behavior
overall, because the point of this patch is not to get immediate planning
benefits but to create the infrastructure for future improvements.)
There is a great deal left to do here. This patch unblocks a lot of
planner work that was basically impractical in the old code structure,
such as allowing FDWs to implement remote aggregation, or rewriting
plan_set_operations() to allow consideration of multiple implementation
orders for set operations. (The latter will likely require a full
rewrite of plan_set_operations(); what I've done here is only to fix it
to return Paths not Plans.) I have also left unfinished some localized
refactoring in createplan.c and planner.c, because it was not necessary
to get this patch to a working state.
Thanks to Robert Haas, David Rowley, and Amit Kapila for review.
9 years ago
|
|
|
list_make2(lpath_tlist, rpath_tlist),
|
|
|
|
|
refnames_tlist);
|
|
|
|
|
|
Make the upper part of the planner work by generating and comparing Paths.
I've been saying we needed to do this for more than five years, and here it
finally is. This patch removes the ever-growing tangle of spaghetti logic
that grouping_planner() used to use to try to identify the best plan for
post-scan/join query steps. Now, there is (nearly) independent
consideration of each execution step, and entirely separate construction of
Paths to represent each of the possible ways to do that step. We choose
the best Path or set of Paths using the same add_path() logic that's been
used inside query_planner() for years.
In addition, this patch removes the old restriction that subquery_planner()
could return only a single Plan. It now returns a RelOptInfo containing a
set of Paths, just as query_planner() does, and the parent query level can
use each of those Paths as the basis of a SubqueryScanPath at its level.
This allows finding some optimizations that we missed before, wherein a
subquery was capable of returning presorted data and thereby avoiding a
sort in the parent level, making the overall cost cheaper even though
delivering sorted output was not the cheapest plan for the subquery in
isolation. (A couple of regression test outputs change in consequence of
that. However, there is very little change in visible planner behavior
overall, because the point of this patch is not to get immediate planning
benefits but to create the infrastructure for future improvements.)
There is a great deal left to do here. This patch unblocks a lot of
planner work that was basically impractical in the old code structure,
such as allowing FDWs to implement remote aggregation, or rewriting
plan_set_operations() to allow consideration of multiple implementation
orders for set operations. (The latter will likely require a full
rewrite of plan_set_operations(); what I've done here is only to fix it
to return Paths not Plans.) I have also left unfinished some localized
refactoring in createplan.c and planner.c, because it was not necessary
to get this patch to a working state.
Thanks to Robert Haas, David Rowley, and Amit Kapila for review.
9 years ago
|
|
|
*pTargetList = tlist;
|
|
|
|
|
|
|
|
|
|
/* Build result relation. */
|
|
|
|
|
result_rel = fetch_upper_rel(root, UPPERREL_SETOP,
|
|
|
|
|
bms_union(lrel->relids, rrel->relids));
|
|
|
|
|
result_rel->reltarget = create_pathtarget(root, tlist);
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* If UNION, identify the grouping operators
|
|
|
|
|
*/
|
|
|
|
|
if (setOp->all)
|
|
|
|
|
{
|
|
|
|
|
groupList = NIL;
|
Make the upper part of the planner work by generating and comparing Paths.
I've been saying we needed to do this for more than five years, and here it
finally is. This patch removes the ever-growing tangle of spaghetti logic
that grouping_planner() used to use to try to identify the best plan for
post-scan/join query steps. Now, there is (nearly) independent
consideration of each execution step, and entirely separate construction of
Paths to represent each of the possible ways to do that step. We choose
the best Path or set of Paths using the same add_path() logic that's been
used inside query_planner() for years.
In addition, this patch removes the old restriction that subquery_planner()
could return only a single Plan. It now returns a RelOptInfo containing a
set of Paths, just as query_planner() does, and the parent query level can
use each of those Paths as the basis of a SubqueryScanPath at its level.
This allows finding some optimizations that we missed before, wherein a
subquery was capable of returning presorted data and thereby avoiding a
sort in the parent level, making the overall cost cheaper even though
delivering sorted output was not the cheapest plan for the subquery in
isolation. (A couple of regression test outputs change in consequence of
that. However, there is very little change in visible planner behavior
overall, because the point of this patch is not to get immediate planning
benefits but to create the infrastructure for future improvements.)
There is a great deal left to do here. This patch unblocks a lot of
planner work that was basically impractical in the old code structure,
such as allowing FDWs to implement remote aggregation, or rewriting
plan_set_operations() to allow consideration of multiple implementation
orders for set operations. (The latter will likely require a full
rewrite of plan_set_operations(); what I've done here is only to fix it
to return Paths not Plans.) I have also left unfinished some localized
refactoring in createplan.c and planner.c, because it was not necessary
to get this patch to a working state.
Thanks to Robert Haas, David Rowley, and Amit Kapila for review.
9 years ago
|
|
|
dNumGroups = 0;
|
|
|
|
|
}
|
|
|
|
|
else
|
|
|
|
|
{
|
|
|
|
|
/* Identify the grouping semantics */
|
|
|
|
|
groupList = generate_setop_grouplist(setOp, tlist);
|
|
|
|
|
|
|
|
|
|
/* We only support hashing here */
|
|
|
|
|
if (!grouping_is_hashable(groupList))
|
|
|
|
|
ereport(ERROR,
|
|
|
|
|
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
|
|
|
|
|
errmsg("could not implement recursive UNION"),
|
|
|
|
|
errdetail("All column datatypes must be hashable.")));
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* For the moment, take the number of distinct groups as equal to the
|
|
|
|
|
* total input size, ie, the worst case.
|
|
|
|
|
*/
|
Make the upper part of the planner work by generating and comparing Paths.
I've been saying we needed to do this for more than five years, and here it
finally is. This patch removes the ever-growing tangle of spaghetti logic
that grouping_planner() used to use to try to identify the best plan for
post-scan/join query steps. Now, there is (nearly) independent
consideration of each execution step, and entirely separate construction of
Paths to represent each of the possible ways to do that step. We choose
the best Path or set of Paths using the same add_path() logic that's been
used inside query_planner() for years.
In addition, this patch removes the old restriction that subquery_planner()
could return only a single Plan. It now returns a RelOptInfo containing a
set of Paths, just as query_planner() does, and the parent query level can
use each of those Paths as the basis of a SubqueryScanPath at its level.
This allows finding some optimizations that we missed before, wherein a
subquery was capable of returning presorted data and thereby avoiding a
sort in the parent level, making the overall cost cheaper even though
delivering sorted output was not the cheapest plan for the subquery in
isolation. (A couple of regression test outputs change in consequence of
that. However, there is very little change in visible planner behavior
overall, because the point of this patch is not to get immediate planning
benefits but to create the infrastructure for future improvements.)
There is a great deal left to do here. This patch unblocks a lot of
planner work that was basically impractical in the old code structure,
such as allowing FDWs to implement remote aggregation, or rewriting
plan_set_operations() to allow consideration of multiple implementation
orders for set operations. (The latter will likely require a full
rewrite of plan_set_operations(); what I've done here is only to fix it
to return Paths not Plans.) I have also left unfinished some localized
refactoring in createplan.c and planner.c, because it was not necessary
to get this patch to a working state.
Thanks to Robert Haas, David Rowley, and Amit Kapila for review.
9 years ago
|
|
|
dNumGroups = lpath->rows + rpath->rows * 10;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/*
|
Make the upper part of the planner work by generating and comparing Paths.
I've been saying we needed to do this for more than five years, and here it
finally is. This patch removes the ever-growing tangle of spaghetti logic
that grouping_planner() used to use to try to identify the best plan for
post-scan/join query steps. Now, there is (nearly) independent
consideration of each execution step, and entirely separate construction of
Paths to represent each of the possible ways to do that step. We choose
the best Path or set of Paths using the same add_path() logic that's been
used inside query_planner() for years.
In addition, this patch removes the old restriction that subquery_planner()
could return only a single Plan. It now returns a RelOptInfo containing a
set of Paths, just as query_planner() does, and the parent query level can
use each of those Paths as the basis of a SubqueryScanPath at its level.
This allows finding some optimizations that we missed before, wherein a
subquery was capable of returning presorted data and thereby avoiding a
sort in the parent level, making the overall cost cheaper even though
delivering sorted output was not the cheapest plan for the subquery in
isolation. (A couple of regression test outputs change in consequence of
that. However, there is very little change in visible planner behavior
overall, because the point of this patch is not to get immediate planning
benefits but to create the infrastructure for future improvements.)
There is a great deal left to do here. This patch unblocks a lot of
planner work that was basically impractical in the old code structure,
such as allowing FDWs to implement remote aggregation, or rewriting
plan_set_operations() to allow consideration of multiple implementation
orders for set operations. (The latter will likely require a full
rewrite of plan_set_operations(); what I've done here is only to fix it
to return Paths not Plans.) I have also left unfinished some localized
refactoring in createplan.c and planner.c, because it was not necessary
to get this patch to a working state.
Thanks to Robert Haas, David Rowley, and Amit Kapila for review.
9 years ago
|
|
|
* And make the path node.
|
|
|
|
|
*/
|
Make the upper part of the planner work by generating and comparing Paths.
I've been saying we needed to do this for more than five years, and here it
finally is. This patch removes the ever-growing tangle of spaghetti logic
that grouping_planner() used to use to try to identify the best plan for
post-scan/join query steps. Now, there is (nearly) independent
consideration of each execution step, and entirely separate construction of
Paths to represent each of the possible ways to do that step. We choose
the best Path or set of Paths using the same add_path() logic that's been
used inside query_planner() for years.
In addition, this patch removes the old restriction that subquery_planner()
could return only a single Plan. It now returns a RelOptInfo containing a
set of Paths, just as query_planner() does, and the parent query level can
use each of those Paths as the basis of a SubqueryScanPath at its level.
This allows finding some optimizations that we missed before, wherein a
subquery was capable of returning presorted data and thereby avoiding a
sort in the parent level, making the overall cost cheaper even though
delivering sorted output was not the cheapest plan for the subquery in
isolation. (A couple of regression test outputs change in consequence of
that. However, there is very little change in visible planner behavior
overall, because the point of this patch is not to get immediate planning
benefits but to create the infrastructure for future improvements.)
There is a great deal left to do here. This patch unblocks a lot of
planner work that was basically impractical in the old code structure,
such as allowing FDWs to implement remote aggregation, or rewriting
plan_set_operations() to allow consideration of multiple implementation
orders for set operations. (The latter will likely require a full
rewrite of plan_set_operations(); what I've done here is only to fix it
to return Paths not Plans.) I have also left unfinished some localized
refactoring in createplan.c and planner.c, because it was not necessary
to get this patch to a working state.
Thanks to Robert Haas, David Rowley, and Amit Kapila for review.
9 years ago
|
|
|
path = (Path *) create_recursiveunion_path(root,
|
|
|
|
|
result_rel,
|
|
|
|
|
lpath,
|
|
|
|
|
rpath,
|
|
|
|
|
result_rel->reltarget,
|
Make the upper part of the planner work by generating and comparing Paths.
I've been saying we needed to do this for more than five years, and here it
finally is. This patch removes the ever-growing tangle of spaghetti logic
that grouping_planner() used to use to try to identify the best plan for
post-scan/join query steps. Now, there is (nearly) independent
consideration of each execution step, and entirely separate construction of
Paths to represent each of the possible ways to do that step. We choose
the best Path or set of Paths using the same add_path() logic that's been
used inside query_planner() for years.
In addition, this patch removes the old restriction that subquery_planner()
could return only a single Plan. It now returns a RelOptInfo containing a
set of Paths, just as query_planner() does, and the parent query level can
use each of those Paths as the basis of a SubqueryScanPath at its level.
This allows finding some optimizations that we missed before, wherein a
subquery was capable of returning presorted data and thereby avoiding a
sort in the parent level, making the overall cost cheaper even though
delivering sorted output was not the cheapest plan for the subquery in
isolation. (A couple of regression test outputs change in consequence of
that. However, there is very little change in visible planner behavior
overall, because the point of this patch is not to get immediate planning
benefits but to create the infrastructure for future improvements.)
There is a great deal left to do here. This patch unblocks a lot of
planner work that was basically impractical in the old code structure,
such as allowing FDWs to implement remote aggregation, or rewriting
plan_set_operations() to allow consideration of multiple implementation
orders for set operations. (The latter will likely require a full
rewrite of plan_set_operations(); what I've done here is only to fix it
to return Paths not Plans.) I have also left unfinished some localized
refactoring in createplan.c and planner.c, because it was not necessary
to get this patch to a working state.
Thanks to Robert Haas, David Rowley, and Amit Kapila for review.
9 years ago
|
|
|
groupList,
|
|
|
|
|
root->wt_param_id,
|
|
|
|
|
dNumGroups);
|
|
|
|
|
|
|
|
|
|
add_path(result_rel, path);
|
|
|
|
|
postprocess_setop_rel(root, result_rel);
|
|
|
|
|
return result_rel;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Generate paths for a UNION or UNION ALL node
|
|
|
|
|
*/
|
|
|
|
|
static RelOptInfo *
|
|
|
|
|
generate_union_paths(SetOperationStmt *op, PlannerInfo *root,
|
|
|
|
|
List *refnames_tlist,
|
|
|
|
|
List **pTargetList)
|
|
|
|
|
{
|
|
|
|
|
Relids relids = NULL;
|
|
|
|
|
RelOptInfo *result_rel;
|
Make the upper part of the planner work by generating and comparing Paths.
I've been saying we needed to do this for more than five years, and here it
finally is. This patch removes the ever-growing tangle of spaghetti logic
that grouping_planner() used to use to try to identify the best plan for
post-scan/join query steps. Now, there is (nearly) independent
consideration of each execution step, and entirely separate construction of
Paths to represent each of the possible ways to do that step. We choose
the best Path or set of Paths using the same add_path() logic that's been
used inside query_planner() for years.
In addition, this patch removes the old restriction that subquery_planner()
could return only a single Plan. It now returns a RelOptInfo containing a
set of Paths, just as query_planner() does, and the parent query level can
use each of those Paths as the basis of a SubqueryScanPath at its level.
This allows finding some optimizations that we missed before, wherein a
subquery was capable of returning presorted data and thereby avoiding a
sort in the parent level, making the overall cost cheaper even though
delivering sorted output was not the cheapest plan for the subquery in
isolation. (A couple of regression test outputs change in consequence of
that. However, there is very little change in visible planner behavior
overall, because the point of this patch is not to get immediate planning
benefits but to create the infrastructure for future improvements.)
There is a great deal left to do here. This patch unblocks a lot of
planner work that was basically impractical in the old code structure,
such as allowing FDWs to implement remote aggregation, or rewriting
plan_set_operations() to allow consideration of multiple implementation
orders for set operations. (The latter will likely require a full
rewrite of plan_set_operations(); what I've done here is only to fix it
to return Paths not Plans.) I have also left unfinished some localized
refactoring in createplan.c and planner.c, because it was not necessary
to get this patch to a working state.
Thanks to Robert Haas, David Rowley, and Amit Kapila for review.
9 years ago
|
|
|
double save_fraction = root->tuple_fraction;
|
|
|
|
|
ListCell *lc;
|
|
|
|
|
List *pathlist = NIL;
|
|
|
|
|
List *partial_pathlist = NIL;
|
|
|
|
|
bool partial_paths_valid = true;
|
|
|
|
|
bool consider_parallel = true;
|
|
|
|
|
List *rellist;
|
Make the upper part of the planner work by generating and comparing Paths.
I've been saying we needed to do this for more than five years, and here it
finally is. This patch removes the ever-growing tangle of spaghetti logic
that grouping_planner() used to use to try to identify the best plan for
post-scan/join query steps. Now, there is (nearly) independent
consideration of each execution step, and entirely separate construction of
Paths to represent each of the possible ways to do that step. We choose
the best Path or set of Paths using the same add_path() logic that's been
used inside query_planner() for years.
In addition, this patch removes the old restriction that subquery_planner()
could return only a single Plan. It now returns a RelOptInfo containing a
set of Paths, just as query_planner() does, and the parent query level can
use each of those Paths as the basis of a SubqueryScanPath at its level.
This allows finding some optimizations that we missed before, wherein a
subquery was capable of returning presorted data and thereby avoiding a
sort in the parent level, making the overall cost cheaper even though
delivering sorted output was not the cheapest plan for the subquery in
isolation. (A couple of regression test outputs change in consequence of
that. However, there is very little change in visible planner behavior
overall, because the point of this patch is not to get immediate planning
benefits but to create the infrastructure for future improvements.)
There is a great deal left to do here. This patch unblocks a lot of
planner work that was basically impractical in the old code structure,
such as allowing FDWs to implement remote aggregation, or rewriting
plan_set_operations() to allow consideration of multiple implementation
orders for set operations. (The latter will likely require a full
rewrite of plan_set_operations(); what I've done here is only to fix it
to return Paths not Plans.) I have also left unfinished some localized
refactoring in createplan.c and planner.c, because it was not necessary
to get this patch to a working state.
Thanks to Robert Haas, David Rowley, and Amit Kapila for review.
9 years ago
|
|
|
List *tlist_list;
|
|
|
|
|
List *tlist;
|
Make the upper part of the planner work by generating and comparing Paths.
I've been saying we needed to do this for more than five years, and here it
finally is. This patch removes the ever-growing tangle of spaghetti logic
that grouping_planner() used to use to try to identify the best plan for
post-scan/join query steps. Now, there is (nearly) independent
consideration of each execution step, and entirely separate construction of
Paths to represent each of the possible ways to do that step. We choose
the best Path or set of Paths using the same add_path() logic that's been
used inside query_planner() for years.
In addition, this patch removes the old restriction that subquery_planner()
could return only a single Plan. It now returns a RelOptInfo containing a
set of Paths, just as query_planner() does, and the parent query level can
use each of those Paths as the basis of a SubqueryScanPath at its level.
This allows finding some optimizations that we missed before, wherein a
subquery was capable of returning presorted data and thereby avoiding a
sort in the parent level, making the overall cost cheaper even though
delivering sorted output was not the cheapest plan for the subquery in
isolation. (A couple of regression test outputs change in consequence of
that. However, there is very little change in visible planner behavior
overall, because the point of this patch is not to get immediate planning
benefits but to create the infrastructure for future improvements.)
There is a great deal left to do here. This patch unblocks a lot of
planner work that was basically impractical in the old code structure,
such as allowing FDWs to implement remote aggregation, or rewriting
plan_set_operations() to allow consideration of multiple implementation
orders for set operations. (The latter will likely require a full
rewrite of plan_set_operations(); what I've done here is only to fix it
to return Paths not Plans.) I have also left unfinished some localized
refactoring in createplan.c and planner.c, because it was not necessary
to get this patch to a working state.
Thanks to Robert Haas, David Rowley, and Amit Kapila for review.
9 years ago
|
|
|
Path *path;
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* If plain UNION, tell children to fetch all tuples.
|
|
|
|
|
*
|
|
|
|
|
* Note: in UNION ALL, we pass the top-level tuple_fraction unmodified to
|
|
|
|
|
* each arm of the UNION ALL. One could make a case for reducing the
|
|
|
|
|
* tuple fraction for later arms (discounting by the expected size of the
|
|
|
|
|
* earlier arms' results) but it seems not worth the trouble. The normal
|
|
|
|
|
* case where tuple_fraction isn't already zero is a LIMIT at top level,
|
|
|
|
|
* and passing it down as-is is usually enough to get the desired result
|
|
|
|
|
* of preferring fast-start plans.
|
|
|
|
|
*/
|
|
|
|
|
if (!op->all)
|
Make the upper part of the planner work by generating and comparing Paths.
I've been saying we needed to do this for more than five years, and here it
finally is. This patch removes the ever-growing tangle of spaghetti logic
that grouping_planner() used to use to try to identify the best plan for
post-scan/join query steps. Now, there is (nearly) independent
consideration of each execution step, and entirely separate construction of
Paths to represent each of the possible ways to do that step. We choose
the best Path or set of Paths using the same add_path() logic that's been
used inside query_planner() for years.
In addition, this patch removes the old restriction that subquery_planner()
could return only a single Plan. It now returns a RelOptInfo containing a
set of Paths, just as query_planner() does, and the parent query level can
use each of those Paths as the basis of a SubqueryScanPath at its level.
This allows finding some optimizations that we missed before, wherein a
subquery was capable of returning presorted data and thereby avoiding a
sort in the parent level, making the overall cost cheaper even though
delivering sorted output was not the cheapest plan for the subquery in
isolation. (A couple of regression test outputs change in consequence of
that. However, there is very little change in visible planner behavior
overall, because the point of this patch is not to get immediate planning
benefits but to create the infrastructure for future improvements.)
There is a great deal left to do here. This patch unblocks a lot of
planner work that was basically impractical in the old code structure,
such as allowing FDWs to implement remote aggregation, or rewriting
plan_set_operations() to allow consideration of multiple implementation
orders for set operations. (The latter will likely require a full
rewrite of plan_set_operations(); what I've done here is only to fix it
to return Paths not Plans.) I have also left unfinished some localized
refactoring in createplan.c and planner.c, because it was not necessary
to get this patch to a working state.
Thanks to Robert Haas, David Rowley, and Amit Kapila for review.
9 years ago
|
|
|
root->tuple_fraction = 0.0;
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* If any of my children are identical UNION nodes (same op, all-flag, and
|
|
|
|
|
* colTypes) then they can be merged into this node so that we generate
|
|
|
|
|
* only one Append and unique-ification for the lot. Recurse to find such
|
Make the upper part of the planner work by generating and comparing Paths.
I've been saying we needed to do this for more than five years, and here it
finally is. This patch removes the ever-growing tangle of spaghetti logic
that grouping_planner() used to use to try to identify the best plan for
post-scan/join query steps. Now, there is (nearly) independent
consideration of each execution step, and entirely separate construction of
Paths to represent each of the possible ways to do that step. We choose
the best Path or set of Paths using the same add_path() logic that's been
used inside query_planner() for years.
In addition, this patch removes the old restriction that subquery_planner()
could return only a single Plan. It now returns a RelOptInfo containing a
set of Paths, just as query_planner() does, and the parent query level can
use each of those Paths as the basis of a SubqueryScanPath at its level.
This allows finding some optimizations that we missed before, wherein a
subquery was capable of returning presorted data and thereby avoiding a
sort in the parent level, making the overall cost cheaper even though
delivering sorted output was not the cheapest plan for the subquery in
isolation. (A couple of regression test outputs change in consequence of
that. However, there is very little change in visible planner behavior
overall, because the point of this patch is not to get immediate planning
benefits but to create the infrastructure for future improvements.)
There is a great deal left to do here. This patch unblocks a lot of
planner work that was basically impractical in the old code structure,
such as allowing FDWs to implement remote aggregation, or rewriting
plan_set_operations() to allow consideration of multiple implementation
orders for set operations. (The latter will likely require a full
rewrite of plan_set_operations(); what I've done here is only to fix it
to return Paths not Plans.) I have also left unfinished some localized
refactoring in createplan.c and planner.c, because it was not necessary
to get this patch to a working state.
Thanks to Robert Haas, David Rowley, and Amit Kapila for review.
9 years ago
|
|
|
* nodes and compute their children's paths.
|
|
|
|
|
*/
|
|
|
|
|
rellist = plan_union_children(root, op, refnames_tlist, &tlist_list);
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Generate tlist for Append plan node.
|
|
|
|
|
*
|
|
|
|
|
* The tlist for an Append plan isn't important as far as the Append is
|
|
|
|
|
* concerned, but we must make it look real anyway for the benefit of the
|
|
|
|
|
* next plan level up.
|
|
|
|
|
*/
|
|
|
|
|
tlist = generate_append_tlist(op->colTypes, op->colCollations, false,
|
Make the upper part of the planner work by generating and comparing Paths.
I've been saying we needed to do this for more than five years, and here it
finally is. This patch removes the ever-growing tangle of spaghetti logic
that grouping_planner() used to use to try to identify the best plan for
post-scan/join query steps. Now, there is (nearly) independent
consideration of each execution step, and entirely separate construction of
Paths to represent each of the possible ways to do that step. We choose
the best Path or set of Paths using the same add_path() logic that's been
used inside query_planner() for years.
In addition, this patch removes the old restriction that subquery_planner()
could return only a single Plan. It now returns a RelOptInfo containing a
set of Paths, just as query_planner() does, and the parent query level can
use each of those Paths as the basis of a SubqueryScanPath at its level.
This allows finding some optimizations that we missed before, wherein a
subquery was capable of returning presorted data and thereby avoiding a
sort in the parent level, making the overall cost cheaper even though
delivering sorted output was not the cheapest plan for the subquery in
isolation. (A couple of regression test outputs change in consequence of
that. However, there is very little change in visible planner behavior
overall, because the point of this patch is not to get immediate planning
benefits but to create the infrastructure for future improvements.)
There is a great deal left to do here. This patch unblocks a lot of
planner work that was basically impractical in the old code structure,
such as allowing FDWs to implement remote aggregation, or rewriting
plan_set_operations() to allow consideration of multiple implementation
orders for set operations. (The latter will likely require a full
rewrite of plan_set_operations(); what I've done here is only to fix it
to return Paths not Plans.) I have also left unfinished some localized
refactoring in createplan.c and planner.c, because it was not necessary
to get this patch to a working state.
Thanks to Robert Haas, David Rowley, and Amit Kapila for review.
9 years ago
|
|
|
tlist_list, refnames_tlist);
|
|
|
|
|
|
|
|
|
|
*pTargetList = tlist;
|
|
|
|
|
|
|
|
|
|
/* Build path lists and relid set. */
|
|
|
|
|
foreach(lc, rellist)
|
|
|
|
|
{
|
|
|
|
|
RelOptInfo *rel = lfirst(lc);
|
|
|
|
|
|
|
|
|
|
pathlist = lappend(pathlist, rel->cheapest_total_path);
|
|
|
|
|
|
|
|
|
|
if (consider_parallel)
|
|
|
|
|
{
|
|
|
|
|
if (!rel->consider_parallel)
|
|
|
|
|
{
|
|
|
|
|
consider_parallel = false;
|
|
|
|
|
partial_paths_valid = false;
|
|
|
|
|
}
|
|
|
|
|
else if (rel->partial_pathlist == NIL)
|
|
|
|
|
partial_paths_valid = false;
|
|
|
|
|
else
|
|
|
|
|
partial_pathlist = lappend(partial_pathlist,
|
|
|
|
|
linitial(rel->partial_pathlist));
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
relids = bms_union(relids, rel->relids);
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/* Build result relation. */
|
|
|
|
|
result_rel = fetch_upper_rel(root, UPPERREL_SETOP, relids);
|
|
|
|
|
result_rel->reltarget = create_pathtarget(root, tlist);
|
|
|
|
|
result_rel->consider_parallel = consider_parallel;
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Append the child results together.
|
|
|
|
|
*/
|
Support partition pruning at execution time
Existing partition pruning is only able to work at plan time, for query
quals that appear in the parsed query. This is good but limiting, as
there can be parameters that appear later that can be usefully used to
further prune partitions.
This commit adds support for pruning subnodes of Append which cannot
possibly contain any matching tuples, during execution, by evaluating
Params to determine the minimum set of subnodes that can possibly match.
We support more than just simple Params in WHERE clauses. Support
additionally includes:
1. Parameterized Nested Loop Joins: The parameter from the outer side of the
join can be used to determine the minimum set of inner side partitions to
scan.
2. Initplans: Once an initplan has been executed we can then determine which
partitions match the value from the initplan.
Partition pruning is performed in two ways. When Params external to the plan
are found to match the partition key we attempt to prune away unneeded Append
subplans during the initialization of the executor. This allows us to bypass
the initialization of non-matching subplans meaning they won't appear in the
EXPLAIN or EXPLAIN ANALYZE output.
For parameters whose value is only known during the actual execution
then the pruning of these subplans must wait. Subplans which are
eliminated during this stage of pruning are still visible in the EXPLAIN
output. In order to determine if pruning has actually taken place, the
EXPLAIN ANALYZE must be viewed. If a certain Append subplan was never
executed due to the elimination of the partition then the execution
timing area will state "(never executed)". Whereas, if, for example in
the case of parameterized nested loops, the number of loops stated in
the EXPLAIN ANALYZE output for certain subplans may appear lower than
others due to the subplan having been scanned fewer times. This is due
to the list of matching subnodes having to be evaluated whenever a
parameter which was found to match the partition key changes.
This commit required some additional infrastructure that permits the
building of a data structure which is able to perform the translation of
the matching partition IDs, as returned by get_matching_partitions, into
the list index of a subpaths list, as exist in node types such as
Append, MergeAppend and ModifyTable. This allows us to translate a list
of clauses into a Bitmapset of all the subpath indexes which must be
included to satisfy the clause list.
Author: David Rowley, based on an earlier effort by Beena Emerson
Reviewers: Amit Langote, Robert Haas, Amul Sul, Rajkumar Raghuwanshi,
Jesper Pedersen
Discussion: https://postgr.es/m/CAOG9ApE16ac-_VVZVvv0gePSgkg_BwYEV1NBqZFqDR2bBE0X0A@mail.gmail.com
7 years ago
|
|
|
path = (Path *) create_append_path(root, result_rel, pathlist, NIL,
|
Support Parallel Append plan nodes.
When we create an Append node, we can spread out the workers over the
subplans instead of piling on to each subplan one at a time, which
should typically be a bit more efficient, both because the startup
cost of any plan executed entirely by one worker is paid only once and
also because of reduced contention. We can also construct Append
plans using a mix of partial and non-partial subplans, which may allow
for parallelism in places that otherwise couldn't support it.
Unfortunately, this patch doesn't handle the important case of
parallelizing UNION ALL by running each branch in a separate worker;
the executor infrastructure is added here, but more planner work is
needed.
Amit Khandekar, Robert Haas, Amul Sul, reviewed and tested by
Ashutosh Bapat, Amit Langote, Rafia Sabih, Amit Kapila, and
Rajkumar Raghuwanshi.
Discussion: http://postgr.es/m/CAJ3gD9dy0K_E8r727heqXoBmWZ83HwLFwdcaSSmBQ1+S+vRuUQ@mail.gmail.com
8 years ago
|
|
|
NULL, 0, false, NIL, -1);
|
|
|
|
|
|
|
|
|
|
/*
|
Make the upper part of the planner work by generating and comparing Paths.
I've been saying we needed to do this for more than five years, and here it
finally is. This patch removes the ever-growing tangle of spaghetti logic
that grouping_planner() used to use to try to identify the best plan for
post-scan/join query steps. Now, there is (nearly) independent
consideration of each execution step, and entirely separate construction of
Paths to represent each of the possible ways to do that step. We choose
the best Path or set of Paths using the same add_path() logic that's been
used inside query_planner() for years.
In addition, this patch removes the old restriction that subquery_planner()
could return only a single Plan. It now returns a RelOptInfo containing a
set of Paths, just as query_planner() does, and the parent query level can
use each of those Paths as the basis of a SubqueryScanPath at its level.
This allows finding some optimizations that we missed before, wherein a
subquery was capable of returning presorted data and thereby avoiding a
sort in the parent level, making the overall cost cheaper even though
delivering sorted output was not the cheapest plan for the subquery in
isolation. (A couple of regression test outputs change in consequence of
that. However, there is very little change in visible planner behavior
overall, because the point of this patch is not to get immediate planning
benefits but to create the infrastructure for future improvements.)
There is a great deal left to do here. This patch unblocks a lot of
planner work that was basically impractical in the old code structure,
such as allowing FDWs to implement remote aggregation, or rewriting
plan_set_operations() to allow consideration of multiple implementation
orders for set operations. (The latter will likely require a full
rewrite of plan_set_operations(); what I've done here is only to fix it
to return Paths not Plans.) I have also left unfinished some localized
refactoring in createplan.c and planner.c, because it was not necessary
to get this patch to a working state.
Thanks to Robert Haas, David Rowley, and Amit Kapila for review.
9 years ago
|
|
|
* For UNION ALL, we just need the Append path. For UNION, need to add
|
|
|
|
|
* node(s) to remove duplicates.
|
|
|
|
|
*/
|
Make the upper part of the planner work by generating and comparing Paths.
I've been saying we needed to do this for more than five years, and here it
finally is. This patch removes the ever-growing tangle of spaghetti logic
that grouping_planner() used to use to try to identify the best plan for
post-scan/join query steps. Now, there is (nearly) independent
consideration of each execution step, and entirely separate construction of
Paths to represent each of the possible ways to do that step. We choose
the best Path or set of Paths using the same add_path() logic that's been
used inside query_planner() for years.
In addition, this patch removes the old restriction that subquery_planner()
could return only a single Plan. It now returns a RelOptInfo containing a
set of Paths, just as query_planner() does, and the parent query level can
use each of those Paths as the basis of a SubqueryScanPath at its level.
This allows finding some optimizations that we missed before, wherein a
subquery was capable of returning presorted data and thereby avoiding a
sort in the parent level, making the overall cost cheaper even though
delivering sorted output was not the cheapest plan for the subquery in
isolation. (A couple of regression test outputs change in consequence of
that. However, there is very little change in visible planner behavior
overall, because the point of this patch is not to get immediate planning
benefits but to create the infrastructure for future improvements.)
There is a great deal left to do here. This patch unblocks a lot of
planner work that was basically impractical in the old code structure,
such as allowing FDWs to implement remote aggregation, or rewriting
plan_set_operations() to allow consideration of multiple implementation
orders for set operations. (The latter will likely require a full
rewrite of plan_set_operations(); what I've done here is only to fix it
to return Paths not Plans.) I have also left unfinished some localized
refactoring in createplan.c and planner.c, because it was not necessary
to get this patch to a working state.
Thanks to Robert Haas, David Rowley, and Amit Kapila for review.
9 years ago
|
|
|
if (!op->all)
|
|
|
|
|
path = make_union_unique(op, path, tlist, root);
|
|
|
|
|
|
|
|
|
|
add_path(result_rel, path);
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Estimate number of groups. For now we just assume the output is unique
|
|
|
|
|
* --- this is certainly true for the UNION case, and we want worst-case
|
|
|
|
|
* estimates anyway.
|
|
|
|
|
*/
|
|
|
|
|
result_rel->rows = path->rows;
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Now consider doing the same thing using the partial paths plus Append
|
|
|
|
|
* plus Gather.
|
|
|
|
|
*/
|
|
|
|
|
if (partial_paths_valid)
|
|
|
|
|
{
|
|
|
|
|
Path *ppath;
|
|
|
|
|
ListCell *lc;
|
|
|
|
|
int parallel_workers = 0;
|
|
|
|
|
|
|
|
|
|
/* Find the highest number of workers requested for any subpath. */
|
|
|
|
|
foreach(lc, partial_pathlist)
|
|
|
|
|
{
|
|
|
|
|
Path *path = lfirst(lc);
|
|
|
|
|
|
|
|
|
|
parallel_workers = Max(parallel_workers, path->parallel_workers);
|
|
|
|
|
}
|
|
|
|
|
Assert(parallel_workers > 0);
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* If the use of parallel append is permitted, always request at least
|
|
|
|
|
* log2(# of children) paths. We assume it can be useful to have
|
|
|
|
|
* extra workers in this case because they will be spread out across
|
|
|
|
|
* the children. The precise formula is just a guess; see
|
|
|
|
|
* add_paths_to_append_rel.
|
|
|
|
|
*/
|
|
|
|
|
if (enable_parallel_append)
|
|
|
|
|
{
|
|
|
|
|
parallel_workers = Max(parallel_workers,
|
|
|
|
|
fls(list_length(partial_pathlist)));
|
|
|
|
|
parallel_workers = Min(parallel_workers,
|
|
|
|
|
max_parallel_workers_per_gather);
|
|
|
|
|
}
|
|
|
|
|
Assert(parallel_workers > 0);
|
|
|
|
|
|
|
|
|
|
ppath = (Path *)
|
Support partition pruning at execution time
Existing partition pruning is only able to work at plan time, for query
quals that appear in the parsed query. This is good but limiting, as
there can be parameters that appear later that can be usefully used to
further prune partitions.
This commit adds support for pruning subnodes of Append which cannot
possibly contain any matching tuples, during execution, by evaluating
Params to determine the minimum set of subnodes that can possibly match.
We support more than just simple Params in WHERE clauses. Support
additionally includes:
1. Parameterized Nested Loop Joins: The parameter from the outer side of the
join can be used to determine the minimum set of inner side partitions to
scan.
2. Initplans: Once an initplan has been executed we can then determine which
partitions match the value from the initplan.
Partition pruning is performed in two ways. When Params external to the plan
are found to match the partition key we attempt to prune away unneeded Append
subplans during the initialization of the executor. This allows us to bypass
the initialization of non-matching subplans meaning they won't appear in the
EXPLAIN or EXPLAIN ANALYZE output.
For parameters whose value is only known during the actual execution
then the pruning of these subplans must wait. Subplans which are
eliminated during this stage of pruning are still visible in the EXPLAIN
output. In order to determine if pruning has actually taken place, the
EXPLAIN ANALYZE must be viewed. If a certain Append subplan was never
executed due to the elimination of the partition then the execution
timing area will state "(never executed)". Whereas, if, for example in
the case of parameterized nested loops, the number of loops stated in
the EXPLAIN ANALYZE output for certain subplans may appear lower than
others due to the subplan having been scanned fewer times. This is due
to the list of matching subnodes having to be evaluated whenever a
parameter which was found to match the partition key changes.
This commit required some additional infrastructure that permits the
building of a data structure which is able to perform the translation of
the matching partition IDs, as returned by get_matching_partitions, into
the list index of a subpaths list, as exist in node types such as
Append, MergeAppend and ModifyTable. This allows us to translate a list
of clauses into a Bitmapset of all the subpath indexes which must be
included to satisfy the clause list.
Author: David Rowley, based on an earlier effort by Beena Emerson
Reviewers: Amit Langote, Robert Haas, Amul Sul, Rajkumar Raghuwanshi,
Jesper Pedersen
Discussion: https://postgr.es/m/CAOG9ApE16ac-_VVZVvv0gePSgkg_BwYEV1NBqZFqDR2bBE0X0A@mail.gmail.com
7 years ago
|
|
|
create_append_path(root, result_rel, NIL, partial_pathlist,
|
|
|
|
|
NULL, parallel_workers, enable_parallel_append,
|
|
|
|
|
NIL, -1);
|
|
|
|
|
ppath = (Path *)
|
|
|
|
|
create_gather_path(root, result_rel, ppath,
|
|
|
|
|
result_rel->reltarget, NULL, NULL);
|
|
|
|
|
if (!op->all)
|
|
|
|
|
ppath = make_union_unique(op, ppath, tlist, root);
|
|
|
|
|
add_path(result_rel, ppath);
|
|
|
|
|
}
|
|
|
|
|
|
Make the upper part of the planner work by generating and comparing Paths.
I've been saying we needed to do this for more than five years, and here it
finally is. This patch removes the ever-growing tangle of spaghetti logic
that grouping_planner() used to use to try to identify the best plan for
post-scan/join query steps. Now, there is (nearly) independent
consideration of each execution step, and entirely separate construction of
Paths to represent each of the possible ways to do that step. We choose
the best Path or set of Paths using the same add_path() logic that's been
used inside query_planner() for years.
In addition, this patch removes the old restriction that subquery_planner()
could return only a single Plan. It now returns a RelOptInfo containing a
set of Paths, just as query_planner() does, and the parent query level can
use each of those Paths as the basis of a SubqueryScanPath at its level.
This allows finding some optimizations that we missed before, wherein a
subquery was capable of returning presorted data and thereby avoiding a
sort in the parent level, making the overall cost cheaper even though
delivering sorted output was not the cheapest plan for the subquery in
isolation. (A couple of regression test outputs change in consequence of
that. However, there is very little change in visible planner behavior
overall, because the point of this patch is not to get immediate planning
benefits but to create the infrastructure for future improvements.)
There is a great deal left to do here. This patch unblocks a lot of
planner work that was basically impractical in the old code structure,
such as allowing FDWs to implement remote aggregation, or rewriting
plan_set_operations() to allow consideration of multiple implementation
orders for set operations. (The latter will likely require a full
rewrite of plan_set_operations(); what I've done here is only to fix it
to return Paths not Plans.) I have also left unfinished some localized
refactoring in createplan.c and planner.c, because it was not necessary
to get this patch to a working state.
Thanks to Robert Haas, David Rowley, and Amit Kapila for review.
9 years ago
|
|
|
/* Undo effects of possibly forcing tuple_fraction to 0 */
|
|
|
|
|
root->tuple_fraction = save_fraction;
|
|
|
|
|
|
|
|
|
|
return result_rel;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Generate paths for an INTERSECT, INTERSECT ALL, EXCEPT, or EXCEPT ALL node
|
|
|
|
|
*/
|
|
|
|
|
static RelOptInfo *
|
|
|
|
|
generate_nonunion_paths(SetOperationStmt *op, PlannerInfo *root,
|
|
|
|
|
List *refnames_tlist,
|
|
|
|
|
List **pTargetList)
|
|
|
|
|
{
|
|
|
|
|
RelOptInfo *result_rel;
|
|
|
|
|
RelOptInfo *lrel,
|
|
|
|
|
*rrel;
|
Make the upper part of the planner work by generating and comparing Paths.
I've been saying we needed to do this for more than five years, and here it
finally is. This patch removes the ever-growing tangle of spaghetti logic
that grouping_planner() used to use to try to identify the best plan for
post-scan/join query steps. Now, there is (nearly) independent
consideration of each execution step, and entirely separate construction of
Paths to represent each of the possible ways to do that step. We choose
the best Path or set of Paths using the same add_path() logic that's been
used inside query_planner() for years.
In addition, this patch removes the old restriction that subquery_planner()
could return only a single Plan. It now returns a RelOptInfo containing a
set of Paths, just as query_planner() does, and the parent query level can
use each of those Paths as the basis of a SubqueryScanPath at its level.
This allows finding some optimizations that we missed before, wherein a
subquery was capable of returning presorted data and thereby avoiding a
sort in the parent level, making the overall cost cheaper even though
delivering sorted output was not the cheapest plan for the subquery in
isolation. (A couple of regression test outputs change in consequence of
that. However, there is very little change in visible planner behavior
overall, because the point of this patch is not to get immediate planning
benefits but to create the infrastructure for future improvements.)
There is a great deal left to do here. This patch unblocks a lot of
planner work that was basically impractical in the old code structure,
such as allowing FDWs to implement remote aggregation, or rewriting
plan_set_operations() to allow consideration of multiple implementation
orders for set operations. (The latter will likely require a full
rewrite of plan_set_operations(); what I've done here is only to fix it
to return Paths not Plans.) I have also left unfinished some localized
refactoring in createplan.c and planner.c, because it was not necessary
to get this patch to a working state.
Thanks to Robert Haas, David Rowley, and Amit Kapila for review.
9 years ago
|
|
|
double save_fraction = root->tuple_fraction;
|
|
|
|
|
Path *lpath,
|
|
|
|
|
*rpath,
|
|
|
|
|
*path;
|
|
|
|
|
List *lpath_tlist,
|
|
|
|
|
*rpath_tlist,
|
|
|
|
|
*tlist_list,
|
|
|
|
|
*tlist,
|
|
|
|
|
*groupList,
|
Make the upper part of the planner work by generating and comparing Paths.
I've been saying we needed to do this for more than five years, and here it
finally is. This patch removes the ever-growing tangle of spaghetti logic
that grouping_planner() used to use to try to identify the best plan for
post-scan/join query steps. Now, there is (nearly) independent
consideration of each execution step, and entirely separate construction of
Paths to represent each of the possible ways to do that step. We choose
the best Path or set of Paths using the same add_path() logic that's been
used inside query_planner() for years.
In addition, this patch removes the old restriction that subquery_planner()
could return only a single Plan. It now returns a RelOptInfo containing a
set of Paths, just as query_planner() does, and the parent query level can
use each of those Paths as the basis of a SubqueryScanPath at its level.
This allows finding some optimizations that we missed before, wherein a
subquery was capable of returning presorted data and thereby avoiding a
sort in the parent level, making the overall cost cheaper even though
delivering sorted output was not the cheapest plan for the subquery in
isolation. (A couple of regression test outputs change in consequence of
that. However, there is very little change in visible planner behavior
overall, because the point of this patch is not to get immediate planning
benefits but to create the infrastructure for future improvements.)
There is a great deal left to do here. This patch unblocks a lot of
planner work that was basically impractical in the old code structure,
such as allowing FDWs to implement remote aggregation, or rewriting
plan_set_operations() to allow consideration of multiple implementation
orders for set operations. (The latter will likely require a full
rewrite of plan_set_operations(); what I've done here is only to fix it
to return Paths not Plans.) I have also left unfinished some localized
refactoring in createplan.c and planner.c, because it was not necessary
to get this patch to a working state.
Thanks to Robert Haas, David Rowley, and Amit Kapila for review.
9 years ago
|
|
|
*pathlist;
|
|
|
|
|
double dLeftGroups,
|
|
|
|
|
dRightGroups,
|
|
|
|
|
dNumGroups,
|
|
|
|
|
dNumOutputRows;
|
|
|
|
|
bool use_hash;
|
|
|
|
|
SetOpCmd cmd;
|
|
|
|
|
int firstFlag;
|
|
|
|
|
|
Make the upper part of the planner work by generating and comparing Paths.
I've been saying we needed to do this for more than five years, and here it
finally is. This patch removes the ever-growing tangle of spaghetti logic
that grouping_planner() used to use to try to identify the best plan for
post-scan/join query steps. Now, there is (nearly) independent
consideration of each execution step, and entirely separate construction of
Paths to represent each of the possible ways to do that step. We choose
the best Path or set of Paths using the same add_path() logic that's been
used inside query_planner() for years.
In addition, this patch removes the old restriction that subquery_planner()
could return only a single Plan. It now returns a RelOptInfo containing a
set of Paths, just as query_planner() does, and the parent query level can
use each of those Paths as the basis of a SubqueryScanPath at its level.
This allows finding some optimizations that we missed before, wherein a
subquery was capable of returning presorted data and thereby avoiding a
sort in the parent level, making the overall cost cheaper even though
delivering sorted output was not the cheapest plan for the subquery in
isolation. (A couple of regression test outputs change in consequence of
that. However, there is very little change in visible planner behavior
overall, because the point of this patch is not to get immediate planning
benefits but to create the infrastructure for future improvements.)
There is a great deal left to do here. This patch unblocks a lot of
planner work that was basically impractical in the old code structure,
such as allowing FDWs to implement remote aggregation, or rewriting
plan_set_operations() to allow consideration of multiple implementation
orders for set operations. (The latter will likely require a full
rewrite of plan_set_operations(); what I've done here is only to fix it
to return Paths not Plans.) I have also left unfinished some localized
refactoring in createplan.c and planner.c, because it was not necessary
to get this patch to a working state.
Thanks to Robert Haas, David Rowley, and Amit Kapila for review.
9 years ago
|
|
|
/*
|
|
|
|
|
* Tell children to fetch all tuples.
|
|
|
|
|
*/
|
|
|
|
|
root->tuple_fraction = 0.0;
|
|
|
|
|
|
|
|
|
|
/* Recurse on children, ensuring their outputs are marked */
|
|
|
|
|
lrel = recurse_set_operations(op->larg, root,
|
|
|
|
|
op->colTypes, op->colCollations,
|
|
|
|
|
false, 0,
|
|
|
|
|
refnames_tlist,
|
|
|
|
|
&lpath_tlist,
|
|
|
|
|
&dLeftGroups);
|
|
|
|
|
lpath = lrel->cheapest_total_path;
|
|
|
|
|
rrel = recurse_set_operations(op->rarg, root,
|
|
|
|
|
op->colTypes, op->colCollations,
|
|
|
|
|
false, 1,
|
|
|
|
|
refnames_tlist,
|
|
|
|
|
&rpath_tlist,
|
|
|
|
|
&dRightGroups);
|
|
|
|
|
rpath = rrel->cheapest_total_path;
|
Make the upper part of the planner work by generating and comparing Paths.
I've been saying we needed to do this for more than five years, and here it
finally is. This patch removes the ever-growing tangle of spaghetti logic
that grouping_planner() used to use to try to identify the best plan for
post-scan/join query steps. Now, there is (nearly) independent
consideration of each execution step, and entirely separate construction of
Paths to represent each of the possible ways to do that step. We choose
the best Path or set of Paths using the same add_path() logic that's been
used inside query_planner() for years.
In addition, this patch removes the old restriction that subquery_planner()
could return only a single Plan. It now returns a RelOptInfo containing a
set of Paths, just as query_planner() does, and the parent query level can
use each of those Paths as the basis of a SubqueryScanPath at its level.
This allows finding some optimizations that we missed before, wherein a
subquery was capable of returning presorted data and thereby avoiding a
sort in the parent level, making the overall cost cheaper even though
delivering sorted output was not the cheapest plan for the subquery in
isolation. (A couple of regression test outputs change in consequence of
that. However, there is very little change in visible planner behavior
overall, because the point of this patch is not to get immediate planning
benefits but to create the infrastructure for future improvements.)
There is a great deal left to do here. This patch unblocks a lot of
planner work that was basically impractical in the old code structure,
such as allowing FDWs to implement remote aggregation, or rewriting
plan_set_operations() to allow consideration of multiple implementation
orders for set operations. (The latter will likely require a full
rewrite of plan_set_operations(); what I've done here is only to fix it
to return Paths not Plans.) I have also left unfinished some localized
refactoring in createplan.c and planner.c, because it was not necessary
to get this patch to a working state.
Thanks to Robert Haas, David Rowley, and Amit Kapila for review.
9 years ago
|
|
|
|
|
|
|
|
/* Undo effects of forcing tuple_fraction to 0 */
|
|
|
|
|
root->tuple_fraction = save_fraction;
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* For EXCEPT, we must put the left input first. For INTERSECT, either
|
|
|
|
|
* order should give the same results, and we prefer to put the smaller
|
|
|
|
|
* input first in order to minimize the size of the hash table in the
|
|
|
|
|
* hashing case. "Smaller" means the one with the fewer groups.
|
|
|
|
|
*/
|
|
|
|
|
if (op->op == SETOP_EXCEPT || dLeftGroups <= dRightGroups)
|
|
|
|
|
{
|
Make the upper part of the planner work by generating and comparing Paths.
I've been saying we needed to do this for more than five years, and here it
finally is. This patch removes the ever-growing tangle of spaghetti logic
that grouping_planner() used to use to try to identify the best plan for
post-scan/join query steps. Now, there is (nearly) independent
consideration of each execution step, and entirely separate construction of
Paths to represent each of the possible ways to do that step. We choose
the best Path or set of Paths using the same add_path() logic that's been
used inside query_planner() for years.
In addition, this patch removes the old restriction that subquery_planner()
could return only a single Plan. It now returns a RelOptInfo containing a
set of Paths, just as query_planner() does, and the parent query level can
use each of those Paths as the basis of a SubqueryScanPath at its level.
This allows finding some optimizations that we missed before, wherein a
subquery was capable of returning presorted data and thereby avoiding a
sort in the parent level, making the overall cost cheaper even though
delivering sorted output was not the cheapest plan for the subquery in
isolation. (A couple of regression test outputs change in consequence of
that. However, there is very little change in visible planner behavior
overall, because the point of this patch is not to get immediate planning
benefits but to create the infrastructure for future improvements.)
There is a great deal left to do here. This patch unblocks a lot of
planner work that was basically impractical in the old code structure,
such as allowing FDWs to implement remote aggregation, or rewriting
plan_set_operations() to allow consideration of multiple implementation
orders for set operations. (The latter will likely require a full
rewrite of plan_set_operations(); what I've done here is only to fix it
to return Paths not Plans.) I have also left unfinished some localized
refactoring in createplan.c and planner.c, because it was not necessary
to get this patch to a working state.
Thanks to Robert Haas, David Rowley, and Amit Kapila for review.
9 years ago
|
|
|
pathlist = list_make2(lpath, rpath);
|
|
|
|
|
tlist_list = list_make2(lpath_tlist, rpath_tlist);
|
|
|
|
|
firstFlag = 0;
|
|
|
|
|
}
|
|
|
|
|
else
|
|
|
|
|
{
|
Make the upper part of the planner work by generating and comparing Paths.
I've been saying we needed to do this for more than five years, and here it
finally is. This patch removes the ever-growing tangle of spaghetti logic
that grouping_planner() used to use to try to identify the best plan for
post-scan/join query steps. Now, there is (nearly) independent
consideration of each execution step, and entirely separate construction of
Paths to represent each of the possible ways to do that step. We choose
the best Path or set of Paths using the same add_path() logic that's been
used inside query_planner() for years.
In addition, this patch removes the old restriction that subquery_planner()
could return only a single Plan. It now returns a RelOptInfo containing a
set of Paths, just as query_planner() does, and the parent query level can
use each of those Paths as the basis of a SubqueryScanPath at its level.
This allows finding some optimizations that we missed before, wherein a
subquery was capable of returning presorted data and thereby avoiding a
sort in the parent level, making the overall cost cheaper even though
delivering sorted output was not the cheapest plan for the subquery in
isolation. (A couple of regression test outputs change in consequence of
that. However, there is very little change in visible planner behavior
overall, because the point of this patch is not to get immediate planning
benefits but to create the infrastructure for future improvements.)
There is a great deal left to do here. This patch unblocks a lot of
planner work that was basically impractical in the old code structure,
such as allowing FDWs to implement remote aggregation, or rewriting
plan_set_operations() to allow consideration of multiple implementation
orders for set operations. (The latter will likely require a full
rewrite of plan_set_operations(); what I've done here is only to fix it
to return Paths not Plans.) I have also left unfinished some localized
refactoring in createplan.c and planner.c, because it was not necessary
to get this patch to a working state.
Thanks to Robert Haas, David Rowley, and Amit Kapila for review.
9 years ago
|
|
|
pathlist = list_make2(rpath, lpath);
|
|
|
|
|
tlist_list = list_make2(rpath_tlist, lpath_tlist);
|
|
|
|
|
firstFlag = 1;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Generate tlist for Append plan node.
|
|
|
|
|
*
|
|
|
|
|
* The tlist for an Append plan isn't important as far as the Append is
|
|
|
|
|
* concerned, but we must make it look real anyway for the benefit of the
|
|
|
|
|
* next plan level up. In fact, it has to be real enough that the flag
|
|
|
|
|
* column is shown as a variable not a constant, else setrefs.c will get
|
|
|
|
|
* confused.
|
|
|
|
|
*/
|
|
|
|
|
tlist = generate_append_tlist(op->colTypes, op->colCollations, true,
|
Make the upper part of the planner work by generating and comparing Paths.
I've been saying we needed to do this for more than five years, and here it
finally is. This patch removes the ever-growing tangle of spaghetti logic
that grouping_planner() used to use to try to identify the best plan for
post-scan/join query steps. Now, there is (nearly) independent
consideration of each execution step, and entirely separate construction of
Paths to represent each of the possible ways to do that step. We choose
the best Path or set of Paths using the same add_path() logic that's been
used inside query_planner() for years.
In addition, this patch removes the old restriction that subquery_planner()
could return only a single Plan. It now returns a RelOptInfo containing a
set of Paths, just as query_planner() does, and the parent query level can
use each of those Paths as the basis of a SubqueryScanPath at its level.
This allows finding some optimizations that we missed before, wherein a
subquery was capable of returning presorted data and thereby avoiding a
sort in the parent level, making the overall cost cheaper even though
delivering sorted output was not the cheapest plan for the subquery in
isolation. (A couple of regression test outputs change in consequence of
that. However, there is very little change in visible planner behavior
overall, because the point of this patch is not to get immediate planning
benefits but to create the infrastructure for future improvements.)
There is a great deal left to do here. This patch unblocks a lot of
planner work that was basically impractical in the old code structure,
such as allowing FDWs to implement remote aggregation, or rewriting
plan_set_operations() to allow consideration of multiple implementation
orders for set operations. (The latter will likely require a full
rewrite of plan_set_operations(); what I've done here is only to fix it
to return Paths not Plans.) I have also left unfinished some localized
refactoring in createplan.c and planner.c, because it was not necessary
to get this patch to a working state.
Thanks to Robert Haas, David Rowley, and Amit Kapila for review.
9 years ago
|
|
|
tlist_list, refnames_tlist);
|
|
|
|
|
|
|
|
|
|
*pTargetList = tlist;
|
|
|
|
|
|
|
|
|
|
/* Build result relation. */
|
|
|
|
|
result_rel = fetch_upper_rel(root, UPPERREL_SETOP,
|
|
|
|
|
bms_union(lrel->relids, rrel->relids));
|
|
|
|
|
result_rel->reltarget = create_pathtarget(root, tlist);;
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Append the child results together.
|
|
|
|
|
*/
|
Support partition pruning at execution time
Existing partition pruning is only able to work at plan time, for query
quals that appear in the parsed query. This is good but limiting, as
there can be parameters that appear later that can be usefully used to
further prune partitions.
This commit adds support for pruning subnodes of Append which cannot
possibly contain any matching tuples, during execution, by evaluating
Params to determine the minimum set of subnodes that can possibly match.
We support more than just simple Params in WHERE clauses. Support
additionally includes:
1. Parameterized Nested Loop Joins: The parameter from the outer side of the
join can be used to determine the minimum set of inner side partitions to
scan.
2. Initplans: Once an initplan has been executed we can then determine which
partitions match the value from the initplan.
Partition pruning is performed in two ways. When Params external to the plan
are found to match the partition key we attempt to prune away unneeded Append
subplans during the initialization of the executor. This allows us to bypass
the initialization of non-matching subplans meaning they won't appear in the
EXPLAIN or EXPLAIN ANALYZE output.
For parameters whose value is only known during the actual execution
then the pruning of these subplans must wait. Subplans which are
eliminated during this stage of pruning are still visible in the EXPLAIN
output. In order to determine if pruning has actually taken place, the
EXPLAIN ANALYZE must be viewed. If a certain Append subplan was never
executed due to the elimination of the partition then the execution
timing area will state "(never executed)". Whereas, if, for example in
the case of parameterized nested loops, the number of loops stated in
the EXPLAIN ANALYZE output for certain subplans may appear lower than
others due to the subplan having been scanned fewer times. This is due
to the list of matching subnodes having to be evaluated whenever a
parameter which was found to match the partition key changes.
This commit required some additional infrastructure that permits the
building of a data structure which is able to perform the translation of
the matching partition IDs, as returned by get_matching_partitions, into
the list index of a subpaths list, as exist in node types such as
Append, MergeAppend and ModifyTable. This allows us to translate a list
of clauses into a Bitmapset of all the subpath indexes which must be
included to satisfy the clause list.
Author: David Rowley, based on an earlier effort by Beena Emerson
Reviewers: Amit Langote, Robert Haas, Amul Sul, Rajkumar Raghuwanshi,
Jesper Pedersen
Discussion: https://postgr.es/m/CAOG9ApE16ac-_VVZVvv0gePSgkg_BwYEV1NBqZFqDR2bBE0X0A@mail.gmail.com
7 years ago
|
|
|
path = (Path *) create_append_path(root, result_rel, pathlist, NIL,
|
Support Parallel Append plan nodes.
When we create an Append node, we can spread out the workers over the
subplans instead of piling on to each subplan one at a time, which
should typically be a bit more efficient, both because the startup
cost of any plan executed entirely by one worker is paid only once and
also because of reduced contention. We can also construct Append
plans using a mix of partial and non-partial subplans, which may allow
for parallelism in places that otherwise couldn't support it.
Unfortunately, this patch doesn't handle the important case of
parallelizing UNION ALL by running each branch in a separate worker;
the executor infrastructure is added here, but more planner work is
needed.
Amit Khandekar, Robert Haas, Amul Sul, reviewed and tested by
Ashutosh Bapat, Amit Langote, Rafia Sabih, Amit Kapila, and
Rajkumar Raghuwanshi.
Discussion: http://postgr.es/m/CAJ3gD9dy0K_E8r727heqXoBmWZ83HwLFwdcaSSmBQ1+S+vRuUQ@mail.gmail.com
8 years ago
|
|
|
NULL, 0, false, NIL, -1);
|
Make the upper part of the planner work by generating and comparing Paths.
I've been saying we needed to do this for more than five years, and here it
finally is. This patch removes the ever-growing tangle of spaghetti logic
that grouping_planner() used to use to try to identify the best plan for
post-scan/join query steps. Now, there is (nearly) independent
consideration of each execution step, and entirely separate construction of
Paths to represent each of the possible ways to do that step. We choose
the best Path or set of Paths using the same add_path() logic that's been
used inside query_planner() for years.
In addition, this patch removes the old restriction that subquery_planner()
could return only a single Plan. It now returns a RelOptInfo containing a
set of Paths, just as query_planner() does, and the parent query level can
use each of those Paths as the basis of a SubqueryScanPath at its level.
This allows finding some optimizations that we missed before, wherein a
subquery was capable of returning presorted data and thereby avoiding a
sort in the parent level, making the overall cost cheaper even though
delivering sorted output was not the cheapest plan for the subquery in
isolation. (A couple of regression test outputs change in consequence of
that. However, there is very little change in visible planner behavior
overall, because the point of this patch is not to get immediate planning
benefits but to create the infrastructure for future improvements.)
There is a great deal left to do here. This patch unblocks a lot of
planner work that was basically impractical in the old code structure,
such as allowing FDWs to implement remote aggregation, or rewriting
plan_set_operations() to allow consideration of multiple implementation
orders for set operations. (The latter will likely require a full
rewrite of plan_set_operations(); what I've done here is only to fix it
to return Paths not Plans.) I have also left unfinished some localized
refactoring in createplan.c and planner.c, because it was not necessary
to get this patch to a working state.
Thanks to Robert Haas, David Rowley, and Amit Kapila for review.
9 years ago
|
|
|
|
|
|
|
|
/* Identify the grouping semantics */
|
|
|
|
|
groupList = generate_setop_grouplist(op, tlist);
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Estimate number of distinct groups that we'll need hashtable entries
|
|
|
|
|
* for; this is the size of the left-hand input for EXCEPT, or the smaller
|
|
|
|
|
* input for INTERSECT. Also estimate the number of eventual output rows.
|
|
|
|
|
* In non-ALL cases, we estimate each group produces one output row; in
|
|
|
|
|
* ALL cases use the relevant relation size. These are worst-case
|
|
|
|
|
* estimates, of course, but we need to be conservative.
|
|
|
|
|
*/
|
|
|
|
|
if (op->op == SETOP_EXCEPT)
|
|
|
|
|
{
|
|
|
|
|
dNumGroups = dLeftGroups;
|
Make the upper part of the planner work by generating and comparing Paths.
I've been saying we needed to do this for more than five years, and here it
finally is. This patch removes the ever-growing tangle of spaghetti logic
that grouping_planner() used to use to try to identify the best plan for
post-scan/join query steps. Now, there is (nearly) independent
consideration of each execution step, and entirely separate construction of
Paths to represent each of the possible ways to do that step. We choose
the best Path or set of Paths using the same add_path() logic that's been
used inside query_planner() for years.
In addition, this patch removes the old restriction that subquery_planner()
could return only a single Plan. It now returns a RelOptInfo containing a
set of Paths, just as query_planner() does, and the parent query level can
use each of those Paths as the basis of a SubqueryScanPath at its level.
This allows finding some optimizations that we missed before, wherein a
subquery was capable of returning presorted data and thereby avoiding a
sort in the parent level, making the overall cost cheaper even though
delivering sorted output was not the cheapest plan for the subquery in
isolation. (A couple of regression test outputs change in consequence of
that. However, there is very little change in visible planner behavior
overall, because the point of this patch is not to get immediate planning
benefits but to create the infrastructure for future improvements.)
There is a great deal left to do here. This patch unblocks a lot of
planner work that was basically impractical in the old code structure,
such as allowing FDWs to implement remote aggregation, or rewriting
plan_set_operations() to allow consideration of multiple implementation
orders for set operations. (The latter will likely require a full
rewrite of plan_set_operations(); what I've done here is only to fix it
to return Paths not Plans.) I have also left unfinished some localized
refactoring in createplan.c and planner.c, because it was not necessary
to get this patch to a working state.
Thanks to Robert Haas, David Rowley, and Amit Kapila for review.
9 years ago
|
|
|
dNumOutputRows = op->all ? lpath->rows : dNumGroups;
|
|
|
|
|
}
|
|
|
|
|
else
|
|
|
|
|
{
|
|
|
|
|
dNumGroups = Min(dLeftGroups, dRightGroups);
|
Make the upper part of the planner work by generating and comparing Paths.
I've been saying we needed to do this for more than five years, and here it
finally is. This patch removes the ever-growing tangle of spaghetti logic
that grouping_planner() used to use to try to identify the best plan for
post-scan/join query steps. Now, there is (nearly) independent
consideration of each execution step, and entirely separate construction of
Paths to represent each of the possible ways to do that step. We choose
the best Path or set of Paths using the same add_path() logic that's been
used inside query_planner() for years.
In addition, this patch removes the old restriction that subquery_planner()
could return only a single Plan. It now returns a RelOptInfo containing a
set of Paths, just as query_planner() does, and the parent query level can
use each of those Paths as the basis of a SubqueryScanPath at its level.
This allows finding some optimizations that we missed before, wherein a
subquery was capable of returning presorted data and thereby avoiding a
sort in the parent level, making the overall cost cheaper even though
delivering sorted output was not the cheapest plan for the subquery in
isolation. (A couple of regression test outputs change in consequence of
that. However, there is very little change in visible planner behavior
overall, because the point of this patch is not to get immediate planning
benefits but to create the infrastructure for future improvements.)
There is a great deal left to do here. This patch unblocks a lot of
planner work that was basically impractical in the old code structure,
such as allowing FDWs to implement remote aggregation, or rewriting
plan_set_operations() to allow consideration of multiple implementation
orders for set operations. (The latter will likely require a full
rewrite of plan_set_operations(); what I've done here is only to fix it
to return Paths not Plans.) I have also left unfinished some localized
refactoring in createplan.c and planner.c, because it was not necessary
to get this patch to a working state.
Thanks to Robert Haas, David Rowley, and Amit Kapila for review.
9 years ago
|
|
|
dNumOutputRows = op->all ? Min(lpath->rows, rpath->rows) : dNumGroups;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Decide whether to hash or sort, and add a sort node if needed.
|
|
|
|
|
*/
|
Make the upper part of the planner work by generating and comparing Paths.
I've been saying we needed to do this for more than five years, and here it
finally is. This patch removes the ever-growing tangle of spaghetti logic
that grouping_planner() used to use to try to identify the best plan for
post-scan/join query steps. Now, there is (nearly) independent
consideration of each execution step, and entirely separate construction of
Paths to represent each of the possible ways to do that step. We choose
the best Path or set of Paths using the same add_path() logic that's been
used inside query_planner() for years.
In addition, this patch removes the old restriction that subquery_planner()
could return only a single Plan. It now returns a RelOptInfo containing a
set of Paths, just as query_planner() does, and the parent query level can
use each of those Paths as the basis of a SubqueryScanPath at its level.
This allows finding some optimizations that we missed before, wherein a
subquery was capable of returning presorted data and thereby avoiding a
sort in the parent level, making the overall cost cheaper even though
delivering sorted output was not the cheapest plan for the subquery in
isolation. (A couple of regression test outputs change in consequence of
that. However, there is very little change in visible planner behavior
overall, because the point of this patch is not to get immediate planning
benefits but to create the infrastructure for future improvements.)
There is a great deal left to do here. This patch unblocks a lot of
planner work that was basically impractical in the old code structure,
such as allowing FDWs to implement remote aggregation, or rewriting
plan_set_operations() to allow consideration of multiple implementation
orders for set operations. (The latter will likely require a full
rewrite of plan_set_operations(); what I've done here is only to fix it
to return Paths not Plans.) I have also left unfinished some localized
refactoring in createplan.c and planner.c, because it was not necessary
to get this patch to a working state.
Thanks to Robert Haas, David Rowley, and Amit Kapila for review.
9 years ago
|
|
|
use_hash = choose_hashed_setop(root, groupList, path,
|
|
|
|
|
dNumGroups, dNumOutputRows,
|
Phase 3 of pgindent updates.
Don't move parenthesized lines to the left, even if that means they
flow past the right margin.
By default, BSD indent lines up statement continuation lines that are
within parentheses so that they start just to the right of the preceding
left parenthesis. However, traditionally, if that resulted in the
continuation line extending to the right of the desired right margin,
then indent would push it left just far enough to not overrun the margin,
if it could do so without making the continuation line start to the left of
the current statement indent. That makes for a weird mix of indentations
unless one has been completely rigid about never violating the 80-column
limit.
This behavior has been pretty universally panned by Postgres developers.
Hence, disable it with indent's new -lpl switch, so that parenthesized
lines are always lined up with the preceding left paren.
This patch is much less interesting than the first round of indent
changes, but also bulkier, so I thought it best to separate the effects.
Discussion: https://postgr.es/m/E1dAmxK-0006EE-1r@gemulon.postgresql.org
Discussion: https://postgr.es/m/30527.1495162840@sss.pgh.pa.us
8 years ago
|
|
|
(op->op == SETOP_INTERSECT) ? "INTERSECT" : "EXCEPT");
|
|
|
|
|
|
|
|
|
|
if (groupList && !use_hash)
|
Make the upper part of the planner work by generating and comparing Paths.
I've been saying we needed to do this for more than five years, and here it
finally is. This patch removes the ever-growing tangle of spaghetti logic
that grouping_planner() used to use to try to identify the best plan for
post-scan/join query steps. Now, there is (nearly) independent
consideration of each execution step, and entirely separate construction of
Paths to represent each of the possible ways to do that step. We choose
the best Path or set of Paths using the same add_path() logic that's been
used inside query_planner() for years.
In addition, this patch removes the old restriction that subquery_planner()
could return only a single Plan. It now returns a RelOptInfo containing a
set of Paths, just as query_planner() does, and the parent query level can
use each of those Paths as the basis of a SubqueryScanPath at its level.
This allows finding some optimizations that we missed before, wherein a
subquery was capable of returning presorted data and thereby avoiding a
sort in the parent level, making the overall cost cheaper even though
delivering sorted output was not the cheapest plan for the subquery in
isolation. (A couple of regression test outputs change in consequence of
that. However, there is very little change in visible planner behavior
overall, because the point of this patch is not to get immediate planning
benefits but to create the infrastructure for future improvements.)
There is a great deal left to do here. This patch unblocks a lot of
planner work that was basically impractical in the old code structure,
such as allowing FDWs to implement remote aggregation, or rewriting
plan_set_operations() to allow consideration of multiple implementation
orders for set operations. (The latter will likely require a full
rewrite of plan_set_operations(); what I've done here is only to fix it
to return Paths not Plans.) I have also left unfinished some localized
refactoring in createplan.c and planner.c, because it was not necessary
to get this patch to a working state.
Thanks to Robert Haas, David Rowley, and Amit Kapila for review.
9 years ago
|
|
|
path = (Path *) create_sort_path(root,
|
|
|
|
|
result_rel,
|
|
|
|
|
path,
|
|
|
|
|
make_pathkeys_for_sortclauses(root,
|
Phase 3 of pgindent updates.
Don't move parenthesized lines to the left, even if that means they
flow past the right margin.
By default, BSD indent lines up statement continuation lines that are
within parentheses so that they start just to the right of the preceding
left parenthesis. However, traditionally, if that resulted in the
continuation line extending to the right of the desired right margin,
then indent would push it left just far enough to not overrun the margin,
if it could do so without making the continuation line start to the left of
the current statement indent. That makes for a weird mix of indentations
unless one has been completely rigid about never violating the 80-column
limit.
This behavior has been pretty universally panned by Postgres developers.
Hence, disable it with indent's new -lpl switch, so that parenthesized
lines are always lined up with the preceding left paren.
This patch is much less interesting than the first round of indent
changes, but also bulkier, so I thought it best to separate the effects.
Discussion: https://postgr.es/m/E1dAmxK-0006EE-1r@gemulon.postgresql.org
Discussion: https://postgr.es/m/30527.1495162840@sss.pgh.pa.us
8 years ago
|
|
|
groupList,
|
Make the upper part of the planner work by generating and comparing Paths.
I've been saying we needed to do this for more than five years, and here it
finally is. This patch removes the ever-growing tangle of spaghetti logic
that grouping_planner() used to use to try to identify the best plan for
post-scan/join query steps. Now, there is (nearly) independent
consideration of each execution step, and entirely separate construction of
Paths to represent each of the possible ways to do that step. We choose
the best Path or set of Paths using the same add_path() logic that's been
used inside query_planner() for years.
In addition, this patch removes the old restriction that subquery_planner()
could return only a single Plan. It now returns a RelOptInfo containing a
set of Paths, just as query_planner() does, and the parent query level can
use each of those Paths as the basis of a SubqueryScanPath at its level.
This allows finding some optimizations that we missed before, wherein a
subquery was capable of returning presorted data and thereby avoiding a
sort in the parent level, making the overall cost cheaper even though
delivering sorted output was not the cheapest plan for the subquery in
isolation. (A couple of regression test outputs change in consequence of
that. However, there is very little change in visible planner behavior
overall, because the point of this patch is not to get immediate planning
benefits but to create the infrastructure for future improvements.)
There is a great deal left to do here. This patch unblocks a lot of
planner work that was basically impractical in the old code structure,
such as allowing FDWs to implement remote aggregation, or rewriting
plan_set_operations() to allow consideration of multiple implementation
orders for set operations. (The latter will likely require a full
rewrite of plan_set_operations(); what I've done here is only to fix it
to return Paths not Plans.) I have also left unfinished some localized
refactoring in createplan.c and planner.c, because it was not necessary
to get this patch to a working state.
Thanks to Robert Haas, David Rowley, and Amit Kapila for review.
9 years ago
|
|
|
tlist),
|
|
|
|
|
-1.0);
|
|
|
|
|
|
|
|
|
|
/*
|
Make the upper part of the planner work by generating and comparing Paths.
I've been saying we needed to do this for more than five years, and here it
finally is. This patch removes the ever-growing tangle of spaghetti logic
that grouping_planner() used to use to try to identify the best plan for
post-scan/join query steps. Now, there is (nearly) independent
consideration of each execution step, and entirely separate construction of
Paths to represent each of the possible ways to do that step. We choose
the best Path or set of Paths using the same add_path() logic that's been
used inside query_planner() for years.
In addition, this patch removes the old restriction that subquery_planner()
could return only a single Plan. It now returns a RelOptInfo containing a
set of Paths, just as query_planner() does, and the parent query level can
use each of those Paths as the basis of a SubqueryScanPath at its level.
This allows finding some optimizations that we missed before, wherein a
subquery was capable of returning presorted data and thereby avoiding a
sort in the parent level, making the overall cost cheaper even though
delivering sorted output was not the cheapest plan for the subquery in
isolation. (A couple of regression test outputs change in consequence of
that. However, there is very little change in visible planner behavior
overall, because the point of this patch is not to get immediate planning
benefits but to create the infrastructure for future improvements.)
There is a great deal left to do here. This patch unblocks a lot of
planner work that was basically impractical in the old code structure,
such as allowing FDWs to implement remote aggregation, or rewriting
plan_set_operations() to allow consideration of multiple implementation
orders for set operations. (The latter will likely require a full
rewrite of plan_set_operations(); what I've done here is only to fix it
to return Paths not Plans.) I have also left unfinished some localized
refactoring in createplan.c and planner.c, because it was not necessary
to get this patch to a working state.
Thanks to Robert Haas, David Rowley, and Amit Kapila for review.
9 years ago
|
|
|
* Finally, add a SetOp path node to generate the correct output.
|
|
|
|
|
*/
|
|
|
|
|
switch (op->op)
|
|
|
|
|
{
|
|
|
|
|
case SETOP_INTERSECT:
|
|
|
|
|
cmd = op->all ? SETOPCMD_INTERSECT_ALL : SETOPCMD_INTERSECT;
|
|
|
|
|
break;
|
|
|
|
|
case SETOP_EXCEPT:
|
|
|
|
|
cmd = op->all ? SETOPCMD_EXCEPT_ALL : SETOPCMD_EXCEPT;
|
|
|
|
|
break;
|
|
|
|
|
default:
|
|
|
|
|
elog(ERROR, "unrecognized set op: %d", (int) op->op);
|
|
|
|
|
cmd = SETOPCMD_INTERSECT; /* keep compiler quiet */
|
|
|
|
|
break;
|
|
|
|
|
}
|
Make the upper part of the planner work by generating and comparing Paths.
I've been saying we needed to do this for more than five years, and here it
finally is. This patch removes the ever-growing tangle of spaghetti logic
that grouping_planner() used to use to try to identify the best plan for
post-scan/join query steps. Now, there is (nearly) independent
consideration of each execution step, and entirely separate construction of
Paths to represent each of the possible ways to do that step. We choose
the best Path or set of Paths using the same add_path() logic that's been
used inside query_planner() for years.
In addition, this patch removes the old restriction that subquery_planner()
could return only a single Plan. It now returns a RelOptInfo containing a
set of Paths, just as query_planner() does, and the parent query level can
use each of those Paths as the basis of a SubqueryScanPath at its level.
This allows finding some optimizations that we missed before, wherein a
subquery was capable of returning presorted data and thereby avoiding a
sort in the parent level, making the overall cost cheaper even though
delivering sorted output was not the cheapest plan for the subquery in
isolation. (A couple of regression test outputs change in consequence of
that. However, there is very little change in visible planner behavior
overall, because the point of this patch is not to get immediate planning
benefits but to create the infrastructure for future improvements.)
There is a great deal left to do here. This patch unblocks a lot of
planner work that was basically impractical in the old code structure,
such as allowing FDWs to implement remote aggregation, or rewriting
plan_set_operations() to allow consideration of multiple implementation
orders for set operations. (The latter will likely require a full
rewrite of plan_set_operations(); what I've done here is only to fix it
to return Paths not Plans.) I have also left unfinished some localized
refactoring in createplan.c and planner.c, because it was not necessary
to get this patch to a working state.
Thanks to Robert Haas, David Rowley, and Amit Kapila for review.
9 years ago
|
|
|
path = (Path *) create_setop_path(root,
|
|
|
|
|
result_rel,
|
|
|
|
|
path,
|
|
|
|
|
cmd,
|
|
|
|
|
use_hash ? SETOP_HASHED : SETOP_SORTED,
|
|
|
|
|
groupList,
|
|
|
|
|
list_length(op->colTypes) + 1,
|
|
|
|
|
use_hash ? firstFlag : -1,
|
|
|
|
|
dNumGroups,
|
|
|
|
|
dNumOutputRows);
|
|
|
|
|
|
|
|
|
|
result_rel->rows = path->rows;
|
|
|
|
|
add_path(result_rel, path);
|
|
|
|
|
return result_rel;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Pull up children of a UNION node that are identically-propertied UNIONs.
|
|
|
|
|
*
|
|
|
|
|
* NOTE: we can also pull a UNION ALL up into a UNION, since the distinct
|
|
|
|
|
* output rows will be lost anyway.
|
|
|
|
|
*
|
|
|
|
|
* NOTE: currently, we ignore collations while determining if a child has
|
|
|
|
|
* the same properties. This is semantically sound only so long as all
|
|
|
|
|
* collations have the same notion of equality. It is valid from an
|
|
|
|
|
* implementation standpoint because we don't care about the ordering of
|
|
|
|
|
* a UNION child's result: UNION ALL results are always unordered, and
|
|
|
|
|
* generate_union_paths will force a fresh sort if the top level is a UNION.
|
|
|
|
|
*/
|
|
|
|
|
static List *
|
|
|
|
|
plan_union_children(PlannerInfo *root,
|
|
|
|
|
SetOperationStmt *top_union,
|
|
|
|
|
List *refnames_tlist,
|
|
|
|
|
List **tlist_list)
|
|
|
|
|
{
|
|
|
|
|
List *pending_rels = list_make1(top_union);
|
|
|
|
|
List *result = NIL;
|
Make the upper part of the planner work by generating and comparing Paths.
I've been saying we needed to do this for more than five years, and here it
finally is. This patch removes the ever-growing tangle of spaghetti logic
that grouping_planner() used to use to try to identify the best plan for
post-scan/join query steps. Now, there is (nearly) independent
consideration of each execution step, and entirely separate construction of
Paths to represent each of the possible ways to do that step. We choose
the best Path or set of Paths using the same add_path() logic that's been
used inside query_planner() for years.
In addition, this patch removes the old restriction that subquery_planner()
could return only a single Plan. It now returns a RelOptInfo containing a
set of Paths, just as query_planner() does, and the parent query level can
use each of those Paths as the basis of a SubqueryScanPath at its level.
This allows finding some optimizations that we missed before, wherein a
subquery was capable of returning presorted data and thereby avoiding a
sort in the parent level, making the overall cost cheaper even though
delivering sorted output was not the cheapest plan for the subquery in
isolation. (A couple of regression test outputs change in consequence of
that. However, there is very little change in visible planner behavior
overall, because the point of this patch is not to get immediate planning
benefits but to create the infrastructure for future improvements.)
There is a great deal left to do here. This patch unblocks a lot of
planner work that was basically impractical in the old code structure,
such as allowing FDWs to implement remote aggregation, or rewriting
plan_set_operations() to allow consideration of multiple implementation
orders for set operations. (The latter will likely require a full
rewrite of plan_set_operations(); what I've done here is only to fix it
to return Paths not Plans.) I have also left unfinished some localized
refactoring in createplan.c and planner.c, because it was not necessary
to get this patch to a working state.
Thanks to Robert Haas, David Rowley, and Amit Kapila for review.
9 years ago
|
|
|
List *child_tlist;
|
|
|
|
|
|
|
|
|
|
*tlist_list = NIL;
|
|
|
|
|
|
|
|
|
|
while (pending_rels != NIL)
|
|
|
|
|
{
|
|
|
|
|
Node *setOp = linitial(pending_rels);
|
|
|
|
|
|
|
|
|
|
pending_rels = list_delete_first(pending_rels);
|
|
|
|
|
|
|
|
|
|
if (IsA(setOp, SetOperationStmt))
|
|
|
|
|
{
|
|
|
|
|
SetOperationStmt *op = (SetOperationStmt *) setOp;
|
|
|
|
|
|
|
|
|
|
if (op->op == top_union->op &&
|
|
|
|
|
(op->all == top_union->all || op->all) &&
|
|
|
|
|
equal(op->colTypes, top_union->colTypes))
|
|
|
|
|
{
|
|
|
|
|
/* Same UNION, so fold children into parent */
|
|
|
|
|
pending_rels = lcons(op->rarg, pending_rels);
|
|
|
|
|
pending_rels = lcons(op->larg, pending_rels);
|
|
|
|
|
continue;
|
|
|
|
|
}
|
|
|
|
|
}
|
Make the upper part of the planner work by generating and comparing Paths.
I've been saying we needed to do this for more than five years, and here it
finally is. This patch removes the ever-growing tangle of spaghetti logic
that grouping_planner() used to use to try to identify the best plan for
post-scan/join query steps. Now, there is (nearly) independent
consideration of each execution step, and entirely separate construction of
Paths to represent each of the possible ways to do that step. We choose
the best Path or set of Paths using the same add_path() logic that's been
used inside query_planner() for years.
In addition, this patch removes the old restriction that subquery_planner()
could return only a single Plan. It now returns a RelOptInfo containing a
set of Paths, just as query_planner() does, and the parent query level can
use each of those Paths as the basis of a SubqueryScanPath at its level.
This allows finding some optimizations that we missed before, wherein a
subquery was capable of returning presorted data and thereby avoiding a
sort in the parent level, making the overall cost cheaper even though
delivering sorted output was not the cheapest plan for the subquery in
isolation. (A couple of regression test outputs change in consequence of
that. However, there is very little change in visible planner behavior
overall, because the point of this patch is not to get immediate planning
benefits but to create the infrastructure for future improvements.)
There is a great deal left to do here. This patch unblocks a lot of
planner work that was basically impractical in the old code structure,
such as allowing FDWs to implement remote aggregation, or rewriting
plan_set_operations() to allow consideration of multiple implementation
orders for set operations. (The latter will likely require a full
rewrite of plan_set_operations(); what I've done here is only to fix it
to return Paths not Plans.) I have also left unfinished some localized
refactoring in createplan.c and planner.c, because it was not necessary
to get this patch to a working state.
Thanks to Robert Haas, David Rowley, and Amit Kapila for review.
9 years ago
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Not same, so plan this child separately.
|
|
|
|
|
*
|
|
|
|
|
* Note we disallow any resjunk columns in child results. This is
|
|
|
|
|
* necessary since the Append node that implements the union won't do
|
|
|
|
|
* any projection, and upper levels will get confused if some of our
|
|
|
|
|
* output tuples have junk and some don't. This case only arises when
|
|
|
|
|
* we have an EXCEPT or INTERSECT as child, else there won't be
|
|
|
|
|
* resjunk anyway.
|
|
|
|
|
*/
|
|
|
|
|
result = lappend(result, recurse_set_operations(setOp, root,
|
|
|
|
|
top_union->colTypes,
|
|
|
|
|
top_union->colCollations,
|
|
|
|
|
false, -1,
|
Make the upper part of the planner work by generating and comparing Paths.
I've been saying we needed to do this for more than five years, and here it
finally is. This patch removes the ever-growing tangle of spaghetti logic
that grouping_planner() used to use to try to identify the best plan for
post-scan/join query steps. Now, there is (nearly) independent
consideration of each execution step, and entirely separate construction of
Paths to represent each of the possible ways to do that step. We choose
the best Path or set of Paths using the same add_path() logic that's been
used inside query_planner() for years.
In addition, this patch removes the old restriction that subquery_planner()
could return only a single Plan. It now returns a RelOptInfo containing a
set of Paths, just as query_planner() does, and the parent query level can
use each of those Paths as the basis of a SubqueryScanPath at its level.
This allows finding some optimizations that we missed before, wherein a
subquery was capable of returning presorted data and thereby avoiding a
sort in the parent level, making the overall cost cheaper even though
delivering sorted output was not the cheapest plan for the subquery in
isolation. (A couple of regression test outputs change in consequence of
that. However, there is very little change in visible planner behavior
overall, because the point of this patch is not to get immediate planning
benefits but to create the infrastructure for future improvements.)
There is a great deal left to do here. This patch unblocks a lot of
planner work that was basically impractical in the old code structure,
such as allowing FDWs to implement remote aggregation, or rewriting
plan_set_operations() to allow consideration of multiple implementation
orders for set operations. (The latter will likely require a full
rewrite of plan_set_operations(); what I've done here is only to fix it
to return Paths not Plans.) I have also left unfinished some localized
refactoring in createplan.c and planner.c, because it was not necessary
to get this patch to a working state.
Thanks to Robert Haas, David Rowley, and Amit Kapila for review.
9 years ago
|
|
|
refnames_tlist,
|
|
|
|
|
&child_tlist,
|
|
|
|
|
NULL));
|
|
|
|
|
*tlist_list = lappend(*tlist_list, child_tlist);
|
|
|
|
|
}
|
|
|
|
|
|
Make the upper part of the planner work by generating and comparing Paths.
I've been saying we needed to do this for more than five years, and here it
finally is. This patch removes the ever-growing tangle of spaghetti logic
that grouping_planner() used to use to try to identify the best plan for
post-scan/join query steps. Now, there is (nearly) independent
consideration of each execution step, and entirely separate construction of
Paths to represent each of the possible ways to do that step. We choose
the best Path or set of Paths using the same add_path() logic that's been
used inside query_planner() for years.
In addition, this patch removes the old restriction that subquery_planner()
could return only a single Plan. It now returns a RelOptInfo containing a
set of Paths, just as query_planner() does, and the parent query level can
use each of those Paths as the basis of a SubqueryScanPath at its level.
This allows finding some optimizations that we missed before, wherein a
subquery was capable of returning presorted data and thereby avoiding a
sort in the parent level, making the overall cost cheaper even though
delivering sorted output was not the cheapest plan for the subquery in
isolation. (A couple of regression test outputs change in consequence of
that. However, there is very little change in visible planner behavior
overall, because the point of this patch is not to get immediate planning
benefits but to create the infrastructure for future improvements.)
There is a great deal left to do here. This patch unblocks a lot of
planner work that was basically impractical in the old code structure,
such as allowing FDWs to implement remote aggregation, or rewriting
plan_set_operations() to allow consideration of multiple implementation
orders for set operations. (The latter will likely require a full
rewrite of plan_set_operations(); what I've done here is only to fix it
to return Paths not Plans.) I have also left unfinished some localized
refactoring in createplan.c and planner.c, because it was not necessary
to get this patch to a working state.
Thanks to Robert Haas, David Rowley, and Amit Kapila for review.
9 years ago
|
|
|
return result;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/*
|
Make the upper part of the planner work by generating and comparing Paths.
I've been saying we needed to do this for more than five years, and here it
finally is. This patch removes the ever-growing tangle of spaghetti logic
that grouping_planner() used to use to try to identify the best plan for
post-scan/join query steps. Now, there is (nearly) independent
consideration of each execution step, and entirely separate construction of
Paths to represent each of the possible ways to do that step. We choose
the best Path or set of Paths using the same add_path() logic that's been
used inside query_planner() for years.
In addition, this patch removes the old restriction that subquery_planner()
could return only a single Plan. It now returns a RelOptInfo containing a
set of Paths, just as query_planner() does, and the parent query level can
use each of those Paths as the basis of a SubqueryScanPath at its level.
This allows finding some optimizations that we missed before, wherein a
subquery was capable of returning presorted data and thereby avoiding a
sort in the parent level, making the overall cost cheaper even though
delivering sorted output was not the cheapest plan for the subquery in
isolation. (A couple of regression test outputs change in consequence of
that. However, there is very little change in visible planner behavior
overall, because the point of this patch is not to get immediate planning
benefits but to create the infrastructure for future improvements.)
There is a great deal left to do here. This patch unblocks a lot of
planner work that was basically impractical in the old code structure,
such as allowing FDWs to implement remote aggregation, or rewriting
plan_set_operations() to allow consideration of multiple implementation
orders for set operations. (The latter will likely require a full
rewrite of plan_set_operations(); what I've done here is only to fix it
to return Paths not Plans.) I have also left unfinished some localized
refactoring in createplan.c and planner.c, because it was not necessary
to get this patch to a working state.
Thanks to Robert Haas, David Rowley, and Amit Kapila for review.
9 years ago
|
|
|
* Add nodes to the given path tree to unique-ify the result of a UNION.
|
|
|
|
|
*/
|
Make the upper part of the planner work by generating and comparing Paths.
I've been saying we needed to do this for more than five years, and here it
finally is. This patch removes the ever-growing tangle of spaghetti logic
that grouping_planner() used to use to try to identify the best plan for
post-scan/join query steps. Now, there is (nearly) independent
consideration of each execution step, and entirely separate construction of
Paths to represent each of the possible ways to do that step. We choose
the best Path or set of Paths using the same add_path() logic that's been
used inside query_planner() for years.
In addition, this patch removes the old restriction that subquery_planner()
could return only a single Plan. It now returns a RelOptInfo containing a
set of Paths, just as query_planner() does, and the parent query level can
use each of those Paths as the basis of a SubqueryScanPath at its level.
This allows finding some optimizations that we missed before, wherein a
subquery was capable of returning presorted data and thereby avoiding a
sort in the parent level, making the overall cost cheaper even though
delivering sorted output was not the cheapest plan for the subquery in
isolation. (A couple of regression test outputs change in consequence of
that. However, there is very little change in visible planner behavior
overall, because the point of this patch is not to get immediate planning
benefits but to create the infrastructure for future improvements.)
There is a great deal left to do here. This patch unblocks a lot of
planner work that was basically impractical in the old code structure,
such as allowing FDWs to implement remote aggregation, or rewriting
plan_set_operations() to allow consideration of multiple implementation
orders for set operations. (The latter will likely require a full
rewrite of plan_set_operations(); what I've done here is only to fix it
to return Paths not Plans.) I have also left unfinished some localized
refactoring in createplan.c and planner.c, because it was not necessary
to get this patch to a working state.
Thanks to Robert Haas, David Rowley, and Amit Kapila for review.
9 years ago
|
|
|
static Path *
|
|
|
|
|
make_union_unique(SetOperationStmt *op, Path *path, List *tlist,
|
|
|
|
|
PlannerInfo *root)
|
|
|
|
|
{
|
Make the upper part of the planner work by generating and comparing Paths.
I've been saying we needed to do this for more than five years, and here it
finally is. This patch removes the ever-growing tangle of spaghetti logic
that grouping_planner() used to use to try to identify the best plan for
post-scan/join query steps. Now, there is (nearly) independent
consideration of each execution step, and entirely separate construction of
Paths to represent each of the possible ways to do that step. We choose
the best Path or set of Paths using the same add_path() logic that's been
used inside query_planner() for years.
In addition, this patch removes the old restriction that subquery_planner()
could return only a single Plan. It now returns a RelOptInfo containing a
set of Paths, just as query_planner() does, and the parent query level can
use each of those Paths as the basis of a SubqueryScanPath at its level.
This allows finding some optimizations that we missed before, wherein a
subquery was capable of returning presorted data and thereby avoiding a
sort in the parent level, making the overall cost cheaper even though
delivering sorted output was not the cheapest plan for the subquery in
isolation. (A couple of regression test outputs change in consequence of
that. However, there is very little change in visible planner behavior
overall, because the point of this patch is not to get immediate planning
benefits but to create the infrastructure for future improvements.)
There is a great deal left to do here. This patch unblocks a lot of
planner work that was basically impractical in the old code structure,
such as allowing FDWs to implement remote aggregation, or rewriting
plan_set_operations() to allow consideration of multiple implementation
orders for set operations. (The latter will likely require a full
rewrite of plan_set_operations(); what I've done here is only to fix it
to return Paths not Plans.) I have also left unfinished some localized
refactoring in createplan.c and planner.c, because it was not necessary
to get this patch to a working state.
Thanks to Robert Haas, David Rowley, and Amit Kapila for review.
9 years ago
|
|
|
RelOptInfo *result_rel = fetch_upper_rel(root, UPPERREL_SETOP, NULL);
|
|
|
|
|
List *groupList;
|
|
|
|
|
double dNumGroups;
|
|
|
|
|
|
|
|
|
|
/* Identify the grouping semantics */
|
Make the upper part of the planner work by generating and comparing Paths.
I've been saying we needed to do this for more than five years, and here it
finally is. This patch removes the ever-growing tangle of spaghetti logic
that grouping_planner() used to use to try to identify the best plan for
post-scan/join query steps. Now, there is (nearly) independent
consideration of each execution step, and entirely separate construction of
Paths to represent each of the possible ways to do that step. We choose
the best Path or set of Paths using the same add_path() logic that's been
used inside query_planner() for years.
In addition, this patch removes the old restriction that subquery_planner()
could return only a single Plan. It now returns a RelOptInfo containing a
set of Paths, just as query_planner() does, and the parent query level can
use each of those Paths as the basis of a SubqueryScanPath at its level.
This allows finding some optimizations that we missed before, wherein a
subquery was capable of returning presorted data and thereby avoiding a
sort in the parent level, making the overall cost cheaper even though
delivering sorted output was not the cheapest plan for the subquery in
isolation. (A couple of regression test outputs change in consequence of
that. However, there is very little change in visible planner behavior
overall, because the point of this patch is not to get immediate planning
benefits but to create the infrastructure for future improvements.)
There is a great deal left to do here. This patch unblocks a lot of
planner work that was basically impractical in the old code structure,
such as allowing FDWs to implement remote aggregation, or rewriting
plan_set_operations() to allow consideration of multiple implementation
orders for set operations. (The latter will likely require a full
rewrite of plan_set_operations(); what I've done here is only to fix it
to return Paths not Plans.) I have also left unfinished some localized
refactoring in createplan.c and planner.c, because it was not necessary
to get this patch to a working state.
Thanks to Robert Haas, David Rowley, and Amit Kapila for review.
9 years ago
|
|
|
groupList = generate_setop_grouplist(op, tlist);
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* XXX for the moment, take the number of distinct groups as equal to the
|
|
|
|
|
* total input size, ie, the worst case. This is too conservative, but we
|
|
|
|
|
* don't want to risk having the hashtable overrun memory; also, it's not
|
|
|
|
|
* clear how to get a decent estimate of the true size. One should note
|
|
|
|
|
* as well the propensity of novices to write UNION rather than UNION ALL
|
|
|
|
|
* even when they don't expect any duplicates...
|
|
|
|
|
*/
|
Make the upper part of the planner work by generating and comparing Paths.
I've been saying we needed to do this for more than five years, and here it
finally is. This patch removes the ever-growing tangle of spaghetti logic
that grouping_planner() used to use to try to identify the best plan for
post-scan/join query steps. Now, there is (nearly) independent
consideration of each execution step, and entirely separate construction of
Paths to represent each of the possible ways to do that step. We choose
the best Path or set of Paths using the same add_path() logic that's been
used inside query_planner() for years.
In addition, this patch removes the old restriction that subquery_planner()
could return only a single Plan. It now returns a RelOptInfo containing a
set of Paths, just as query_planner() does, and the parent query level can
use each of those Paths as the basis of a SubqueryScanPath at its level.
This allows finding some optimizations that we missed before, wherein a
subquery was capable of returning presorted data and thereby avoiding a
sort in the parent level, making the overall cost cheaper even though
delivering sorted output was not the cheapest plan for the subquery in
isolation. (A couple of regression test outputs change in consequence of
that. However, there is very little change in visible planner behavior
overall, because the point of this patch is not to get immediate planning
benefits but to create the infrastructure for future improvements.)
There is a great deal left to do here. This patch unblocks a lot of
planner work that was basically impractical in the old code structure,
such as allowing FDWs to implement remote aggregation, or rewriting
plan_set_operations() to allow consideration of multiple implementation
orders for set operations. (The latter will likely require a full
rewrite of plan_set_operations(); what I've done here is only to fix it
to return Paths not Plans.) I have also left unfinished some localized
refactoring in createplan.c and planner.c, because it was not necessary
to get this patch to a working state.
Thanks to Robert Haas, David Rowley, and Amit Kapila for review.
9 years ago
|
|
|
dNumGroups = path->rows;
|
|
|
|
|
|
|
|
|
|
/* Decide whether to hash or sort */
|
Make the upper part of the planner work by generating and comparing Paths.
I've been saying we needed to do this for more than five years, and here it
finally is. This patch removes the ever-growing tangle of spaghetti logic
that grouping_planner() used to use to try to identify the best plan for
post-scan/join query steps. Now, there is (nearly) independent
consideration of each execution step, and entirely separate construction of
Paths to represent each of the possible ways to do that step. We choose
the best Path or set of Paths using the same add_path() logic that's been
used inside query_planner() for years.
In addition, this patch removes the old restriction that subquery_planner()
could return only a single Plan. It now returns a RelOptInfo containing a
set of Paths, just as query_planner() does, and the parent query level can
use each of those Paths as the basis of a SubqueryScanPath at its level.
This allows finding some optimizations that we missed before, wherein a
subquery was capable of returning presorted data and thereby avoiding a
sort in the parent level, making the overall cost cheaper even though
delivering sorted output was not the cheapest plan for the subquery in
isolation. (A couple of regression test outputs change in consequence of
that. However, there is very little change in visible planner behavior
overall, because the point of this patch is not to get immediate planning
benefits but to create the infrastructure for future improvements.)
There is a great deal left to do here. This patch unblocks a lot of
planner work that was basically impractical in the old code structure,
such as allowing FDWs to implement remote aggregation, or rewriting
plan_set_operations() to allow consideration of multiple implementation
orders for set operations. (The latter will likely require a full
rewrite of plan_set_operations(); what I've done here is only to fix it
to return Paths not Plans.) I have also left unfinished some localized
refactoring in createplan.c and planner.c, because it was not necessary
to get this patch to a working state.
Thanks to Robert Haas, David Rowley, and Amit Kapila for review.
9 years ago
|
|
|
if (choose_hashed_setop(root, groupList, path,
|
|
|
|
|
dNumGroups, dNumGroups,
|
|
|
|
|
"UNION"))
|
|
|
|
|
{
|
|
|
|
|
/* Hashed aggregate plan --- no sort needed */
|
Make the upper part of the planner work by generating and comparing Paths.
I've been saying we needed to do this for more than five years, and here it
finally is. This patch removes the ever-growing tangle of spaghetti logic
that grouping_planner() used to use to try to identify the best plan for
post-scan/join query steps. Now, there is (nearly) independent
consideration of each execution step, and entirely separate construction of
Paths to represent each of the possible ways to do that step. We choose
the best Path or set of Paths using the same add_path() logic that's been
used inside query_planner() for years.
In addition, this patch removes the old restriction that subquery_planner()
could return only a single Plan. It now returns a RelOptInfo containing a
set of Paths, just as query_planner() does, and the parent query level can
use each of those Paths as the basis of a SubqueryScanPath at its level.
This allows finding some optimizations that we missed before, wherein a
subquery was capable of returning presorted data and thereby avoiding a
sort in the parent level, making the overall cost cheaper even though
delivering sorted output was not the cheapest plan for the subquery in
isolation. (A couple of regression test outputs change in consequence of
that. However, there is very little change in visible planner behavior
overall, because the point of this patch is not to get immediate planning
benefits but to create the infrastructure for future improvements.)
There is a great deal left to do here. This patch unblocks a lot of
planner work that was basically impractical in the old code structure,
such as allowing FDWs to implement remote aggregation, or rewriting
plan_set_operations() to allow consideration of multiple implementation
orders for set operations. (The latter will likely require a full
rewrite of plan_set_operations(); what I've done here is only to fix it
to return Paths not Plans.) I have also left unfinished some localized
refactoring in createplan.c and planner.c, because it was not necessary
to get this patch to a working state.
Thanks to Robert Haas, David Rowley, and Amit Kapila for review.
9 years ago
|
|
|
path = (Path *) create_agg_path(root,
|
|
|
|
|
result_rel,
|
|
|
|
|
path,
|
|
|
|
|
create_pathtarget(root, tlist),
|
|
|
|
|
AGG_HASHED,
|
|
|
|
|
AGGSPLIT_SIMPLE,
|
Make the upper part of the planner work by generating and comparing Paths.
I've been saying we needed to do this for more than five years, and here it
finally is. This patch removes the ever-growing tangle of spaghetti logic
that grouping_planner() used to use to try to identify the best plan for
post-scan/join query steps. Now, there is (nearly) independent
consideration of each execution step, and entirely separate construction of
Paths to represent each of the possible ways to do that step. We choose
the best Path or set of Paths using the same add_path() logic that's been
used inside query_planner() for years.
In addition, this patch removes the old restriction that subquery_planner()
could return only a single Plan. It now returns a RelOptInfo containing a
set of Paths, just as query_planner() does, and the parent query level can
use each of those Paths as the basis of a SubqueryScanPath at its level.
This allows finding some optimizations that we missed before, wherein a
subquery was capable of returning presorted data and thereby avoiding a
sort in the parent level, making the overall cost cheaper even though
delivering sorted output was not the cheapest plan for the subquery in
isolation. (A couple of regression test outputs change in consequence of
that. However, there is very little change in visible planner behavior
overall, because the point of this patch is not to get immediate planning
benefits but to create the infrastructure for future improvements.)
There is a great deal left to do here. This patch unblocks a lot of
planner work that was basically impractical in the old code structure,
such as allowing FDWs to implement remote aggregation, or rewriting
plan_set_operations() to allow consideration of multiple implementation
orders for set operations. (The latter will likely require a full
rewrite of plan_set_operations(); what I've done here is only to fix it
to return Paths not Plans.) I have also left unfinished some localized
refactoring in createplan.c and planner.c, because it was not necessary
to get this patch to a working state.
Thanks to Robert Haas, David Rowley, and Amit Kapila for review.
9 years ago
|
|
|
groupList,
|
|
|
|
|
NIL,
|
|
|
|
|
NULL,
|
|
|
|
|
dNumGroups);
|
|
|
|
|
}
|
|
|
|
|
else
|
|
|
|
|
{
|
|
|
|
|
/* Sort and Unique */
|
|
|
|
|
if (groupList)
|
|
|
|
|
path = (Path *)
|
|
|
|
|
create_sort_path(root,
|
|
|
|
|
result_rel,
|
|
|
|
|
path,
|
|
|
|
|
make_pathkeys_for_sortclauses(root,
|
|
|
|
|
groupList,
|
|
|
|
|
tlist),
|
|
|
|
|
-1.0);
|
Make the upper part of the planner work by generating and comparing Paths.
I've been saying we needed to do this for more than five years, and here it
finally is. This patch removes the ever-growing tangle of spaghetti logic
that grouping_planner() used to use to try to identify the best plan for
post-scan/join query steps. Now, there is (nearly) independent
consideration of each execution step, and entirely separate construction of
Paths to represent each of the possible ways to do that step. We choose
the best Path or set of Paths using the same add_path() logic that's been
used inside query_planner() for years.
In addition, this patch removes the old restriction that subquery_planner()
could return only a single Plan. It now returns a RelOptInfo containing a
set of Paths, just as query_planner() does, and the parent query level can
use each of those Paths as the basis of a SubqueryScanPath at its level.
This allows finding some optimizations that we missed before, wherein a
subquery was capable of returning presorted data and thereby avoiding a
sort in the parent level, making the overall cost cheaper even though
delivering sorted output was not the cheapest plan for the subquery in
isolation. (A couple of regression test outputs change in consequence of
that. However, there is very little change in visible planner behavior
overall, because the point of this patch is not to get immediate planning
benefits but to create the infrastructure for future improvements.)
There is a great deal left to do here. This patch unblocks a lot of
planner work that was basically impractical in the old code structure,
such as allowing FDWs to implement remote aggregation, or rewriting
plan_set_operations() to allow consideration of multiple implementation
orders for set operations. (The latter will likely require a full
rewrite of plan_set_operations(); what I've done here is only to fix it
to return Paths not Plans.) I have also left unfinished some localized
refactoring in createplan.c and planner.c, because it was not necessary
to get this patch to a working state.
Thanks to Robert Haas, David Rowley, and Amit Kapila for review.
9 years ago
|
|
|
path = (Path *) create_upper_unique_path(root,
|
|
|
|
|
result_rel,
|
|
|
|
|
path,
|
|
|
|
|
list_length(path->pathkeys),
|
|
|
|
|
dNumGroups);
|
|
|
|
|
}
|
|
|
|
|
|
Make the upper part of the planner work by generating and comparing Paths.
I've been saying we needed to do this for more than five years, and here it
finally is. This patch removes the ever-growing tangle of spaghetti logic
that grouping_planner() used to use to try to identify the best plan for
post-scan/join query steps. Now, there is (nearly) independent
consideration of each execution step, and entirely separate construction of
Paths to represent each of the possible ways to do that step. We choose
the best Path or set of Paths using the same add_path() logic that's been
used inside query_planner() for years.
In addition, this patch removes the old restriction that subquery_planner()
could return only a single Plan. It now returns a RelOptInfo containing a
set of Paths, just as query_planner() does, and the parent query level can
use each of those Paths as the basis of a SubqueryScanPath at its level.
This allows finding some optimizations that we missed before, wherein a
subquery was capable of returning presorted data and thereby avoiding a
sort in the parent level, making the overall cost cheaper even though
delivering sorted output was not the cheapest plan for the subquery in
isolation. (A couple of regression test outputs change in consequence of
that. However, there is very little change in visible planner behavior
overall, because the point of this patch is not to get immediate planning
benefits but to create the infrastructure for future improvements.)
There is a great deal left to do here. This patch unblocks a lot of
planner work that was basically impractical in the old code structure,
such as allowing FDWs to implement remote aggregation, or rewriting
plan_set_operations() to allow consideration of multiple implementation
orders for set operations. (The latter will likely require a full
rewrite of plan_set_operations(); what I've done here is only to fix it
to return Paths not Plans.) I have also left unfinished some localized
refactoring in createplan.c and planner.c, because it was not necessary
to get this patch to a working state.
Thanks to Robert Haas, David Rowley, and Amit Kapila for review.
9 years ago
|
|
|
return path;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* postprocess_setop_rel - perform steps required after adding paths
|
|
|
|
|
*/
|
|
|
|
|
static void
|
|
|
|
|
postprocess_setop_rel(PlannerInfo *root, RelOptInfo *rel)
|
|
|
|
|
{
|
|
|
|
|
/*
|
|
|
|
|
* We don't currently worry about allowing FDWs to contribute paths to
|
|
|
|
|
* this relation, but give extensions a chance.
|
|
|
|
|
*/
|
|
|
|
|
if (create_upper_paths_hook)
|
|
|
|
|
(*create_upper_paths_hook) (root, UPPERREL_SETOP,
|
|
|
|
|
NULL, rel, NULL);
|
|
|
|
|
|
|
|
|
|
/* Select cheapest path */
|
|
|
|
|
set_cheapest(rel);
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* choose_hashed_setop - should we use hashing for a set operation?
|
|
|
|
|
*/
|
|
|
|
|
static bool
|
|
|
|
|
choose_hashed_setop(PlannerInfo *root, List *groupClauses,
|
Make the upper part of the planner work by generating and comparing Paths.
I've been saying we needed to do this for more than five years, and here it
finally is. This patch removes the ever-growing tangle of spaghetti logic
that grouping_planner() used to use to try to identify the best plan for
post-scan/join query steps. Now, there is (nearly) independent
consideration of each execution step, and entirely separate construction of
Paths to represent each of the possible ways to do that step. We choose
the best Path or set of Paths using the same add_path() logic that's been
used inside query_planner() for years.
In addition, this patch removes the old restriction that subquery_planner()
could return only a single Plan. It now returns a RelOptInfo containing a
set of Paths, just as query_planner() does, and the parent query level can
use each of those Paths as the basis of a SubqueryScanPath at its level.
This allows finding some optimizations that we missed before, wherein a
subquery was capable of returning presorted data and thereby avoiding a
sort in the parent level, making the overall cost cheaper even though
delivering sorted output was not the cheapest plan for the subquery in
isolation. (A couple of regression test outputs change in consequence of
that. However, there is very little change in visible planner behavior
overall, because the point of this patch is not to get immediate planning
benefits but to create the infrastructure for future improvements.)
There is a great deal left to do here. This patch unblocks a lot of
planner work that was basically impractical in the old code structure,
such as allowing FDWs to implement remote aggregation, or rewriting
plan_set_operations() to allow consideration of multiple implementation
orders for set operations. (The latter will likely require a full
rewrite of plan_set_operations(); what I've done here is only to fix it
to return Paths not Plans.) I have also left unfinished some localized
refactoring in createplan.c and planner.c, because it was not necessary
to get this patch to a working state.
Thanks to Robert Haas, David Rowley, and Amit Kapila for review.
9 years ago
|
|
|
Path *input_path,
|
|
|
|
|
double dNumGroups, double dNumOutputRows,
|
|
|
|
|
const char *construct)
|
|
|
|
|
{
|
|
|
|
|
int numGroupCols = list_length(groupClauses);
|
|
|
|
|
bool can_sort;
|
|
|
|
|
bool can_hash;
|
|
|
|
|
Size hashentrysize;
|
|
|
|
|
Path hashed_p;
|
|
|
|
|
Path sorted_p;
|
Make the upper part of the planner work by generating and comparing Paths.
I've been saying we needed to do this for more than five years, and here it
finally is. This patch removes the ever-growing tangle of spaghetti logic
that grouping_planner() used to use to try to identify the best plan for
post-scan/join query steps. Now, there is (nearly) independent
consideration of each execution step, and entirely separate construction of
Paths to represent each of the possible ways to do that step. We choose
the best Path or set of Paths using the same add_path() logic that's been
used inside query_planner() for years.
In addition, this patch removes the old restriction that subquery_planner()
could return only a single Plan. It now returns a RelOptInfo containing a
set of Paths, just as query_planner() does, and the parent query level can
use each of those Paths as the basis of a SubqueryScanPath at its level.
This allows finding some optimizations that we missed before, wherein a
subquery was capable of returning presorted data and thereby avoiding a
sort in the parent level, making the overall cost cheaper even though
delivering sorted output was not the cheapest plan for the subquery in
isolation. (A couple of regression test outputs change in consequence of
that. However, there is very little change in visible planner behavior
overall, because the point of this patch is not to get immediate planning
benefits but to create the infrastructure for future improvements.)
There is a great deal left to do here. This patch unblocks a lot of
planner work that was basically impractical in the old code structure,
such as allowing FDWs to implement remote aggregation, or rewriting
plan_set_operations() to allow consideration of multiple implementation
orders for set operations. (The latter will likely require a full
rewrite of plan_set_operations(); what I've done here is only to fix it
to return Paths not Plans.) I have also left unfinished some localized
refactoring in createplan.c and planner.c, because it was not necessary
to get this patch to a working state.
Thanks to Robert Haas, David Rowley, and Amit Kapila for review.
9 years ago
|
|
|
double tuple_fraction;
|
|
|
|
|
|
|
|
|
|
/* Check whether the operators support sorting or hashing */
|
|
|
|
|
can_sort = grouping_is_sortable(groupClauses);
|
|
|
|
|
can_hash = grouping_is_hashable(groupClauses);
|
|
|
|
|
if (can_hash && can_sort)
|
|
|
|
|
{
|
|
|
|
|
/* we have a meaningful choice to make, continue ... */
|
|
|
|
|
}
|
|
|
|
|
else if (can_hash)
|
|
|
|
|
return true;
|
|
|
|
|
else if (can_sort)
|
|
|
|
|
return false;
|
|
|
|
|
else
|
|
|
|
|
ereport(ERROR,
|
|
|
|
|
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
|
|
|
|
|
/* translator: %s is UNION, INTERSECT, or EXCEPT */
|
|
|
|
|
errmsg("could not implement %s", construct),
|
|
|
|
|
errdetail("Some of the datatypes only support hashing, while others only support sorting.")));
|
|
|
|
|
|
|
|
|
|
/* Prefer sorting when enable_hashagg is off */
|
|
|
|
|
if (!enable_hashagg)
|
|
|
|
|
return false;
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Don't do it if it doesn't look like the hashtable will fit into
|
|
|
|
|
* work_mem.
|
|
|
|
|
*/
|
Make the upper part of the planner work by generating and comparing Paths.
I've been saying we needed to do this for more than five years, and here it
finally is. This patch removes the ever-growing tangle of spaghetti logic
that grouping_planner() used to use to try to identify the best plan for
post-scan/join query steps. Now, there is (nearly) independent
consideration of each execution step, and entirely separate construction of
Paths to represent each of the possible ways to do that step. We choose
the best Path or set of Paths using the same add_path() logic that's been
used inside query_planner() for years.
In addition, this patch removes the old restriction that subquery_planner()
could return only a single Plan. It now returns a RelOptInfo containing a
set of Paths, just as query_planner() does, and the parent query level can
use each of those Paths as the basis of a SubqueryScanPath at its level.
This allows finding some optimizations that we missed before, wherein a
subquery was capable of returning presorted data and thereby avoiding a
sort in the parent level, making the overall cost cheaper even though
delivering sorted output was not the cheapest plan for the subquery in
isolation. (A couple of regression test outputs change in consequence of
that. However, there is very little change in visible planner behavior
overall, because the point of this patch is not to get immediate planning
benefits but to create the infrastructure for future improvements.)
There is a great deal left to do here. This patch unblocks a lot of
planner work that was basically impractical in the old code structure,
such as allowing FDWs to implement remote aggregation, or rewriting
plan_set_operations() to allow consideration of multiple implementation
orders for set operations. (The latter will likely require a full
rewrite of plan_set_operations(); what I've done here is only to fix it
to return Paths not Plans.) I have also left unfinished some localized
refactoring in createplan.c and planner.c, because it was not necessary
to get this patch to a working state.
Thanks to Robert Haas, David Rowley, and Amit Kapila for review.
9 years ago
|
|
|
hashentrysize = MAXALIGN(input_path->pathtarget->width) + MAXALIGN(SizeofMinimalTupleHeader);
|
|
|
|
|
|
|
|
|
|
if (hashentrysize * dNumGroups > work_mem * 1024L)
|
|
|
|
|
return false;
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* See if the estimated cost is no more than doing it the other way.
|
|
|
|
|
*
|
|
|
|
|
* We need to consider input_plan + hashagg versus input_plan + sort +
|
|
|
|
|
* group. Note that the actual result plan might involve a SetOp or
|
|
|
|
|
* Unique node, not Agg or Group, but the cost estimates for Agg and Group
|
|
|
|
|
* should be close enough for our purposes here.
|
|
|
|
|
*
|
|
|
|
|
* These path variables are dummies that just hold cost fields; we don't
|
|
|
|
|
* make actual Paths for these steps.
|
|
|
|
|
*/
|
|
|
|
|
cost_agg(&hashed_p, root, AGG_HASHED, NULL,
|
|
|
|
|
numGroupCols, dNumGroups,
|
|
|
|
|
NIL,
|
Make the upper part of the planner work by generating and comparing Paths.
I've been saying we needed to do this for more than five years, and here it
finally is. This patch removes the ever-growing tangle of spaghetti logic
that grouping_planner() used to use to try to identify the best plan for
post-scan/join query steps. Now, there is (nearly) independent
consideration of each execution step, and entirely separate construction of
Paths to represent each of the possible ways to do that step. We choose
the best Path or set of Paths using the same add_path() logic that's been
used inside query_planner() for years.
In addition, this patch removes the old restriction that subquery_planner()
could return only a single Plan. It now returns a RelOptInfo containing a
set of Paths, just as query_planner() does, and the parent query level can
use each of those Paths as the basis of a SubqueryScanPath at its level.
This allows finding some optimizations that we missed before, wherein a
subquery was capable of returning presorted data and thereby avoiding a
sort in the parent level, making the overall cost cheaper even though
delivering sorted output was not the cheapest plan for the subquery in
isolation. (A couple of regression test outputs change in consequence of
that. However, there is very little change in visible planner behavior
overall, because the point of this patch is not to get immediate planning
benefits but to create the infrastructure for future improvements.)
There is a great deal left to do here. This patch unblocks a lot of
planner work that was basically impractical in the old code structure,
such as allowing FDWs to implement remote aggregation, or rewriting
plan_set_operations() to allow consideration of multiple implementation
orders for set operations. (The latter will likely require a full
rewrite of plan_set_operations(); what I've done here is only to fix it
to return Paths not Plans.) I have also left unfinished some localized
refactoring in createplan.c and planner.c, because it was not necessary
to get this patch to a working state.
Thanks to Robert Haas, David Rowley, and Amit Kapila for review.
9 years ago
|
|
|
input_path->startup_cost, input_path->total_cost,
|
|
|
|
|
input_path->rows);
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Now for the sorted case. Note that the input is *always* unsorted,
|
|
|
|
|
* since it was made by appending unrelated sub-relations together.
|
|
|
|
|
*/
|
Make the upper part of the planner work by generating and comparing Paths.
I've been saying we needed to do this for more than five years, and here it
finally is. This patch removes the ever-growing tangle of spaghetti logic
that grouping_planner() used to use to try to identify the best plan for
post-scan/join query steps. Now, there is (nearly) independent
consideration of each execution step, and entirely separate construction of
Paths to represent each of the possible ways to do that step. We choose
the best Path or set of Paths using the same add_path() logic that's been
used inside query_planner() for years.
In addition, this patch removes the old restriction that subquery_planner()
could return only a single Plan. It now returns a RelOptInfo containing a
set of Paths, just as query_planner() does, and the parent query level can
use each of those Paths as the basis of a SubqueryScanPath at its level.
This allows finding some optimizations that we missed before, wherein a
subquery was capable of returning presorted data and thereby avoiding a
sort in the parent level, making the overall cost cheaper even though
delivering sorted output was not the cheapest plan for the subquery in
isolation. (A couple of regression test outputs change in consequence of
that. However, there is very little change in visible planner behavior
overall, because the point of this patch is not to get immediate planning
benefits but to create the infrastructure for future improvements.)
There is a great deal left to do here. This patch unblocks a lot of
planner work that was basically impractical in the old code structure,
such as allowing FDWs to implement remote aggregation, or rewriting
plan_set_operations() to allow consideration of multiple implementation
orders for set operations. (The latter will likely require a full
rewrite of plan_set_operations(); what I've done here is only to fix it
to return Paths not Plans.) I have also left unfinished some localized
refactoring in createplan.c and planner.c, because it was not necessary
to get this patch to a working state.
Thanks to Robert Haas, David Rowley, and Amit Kapila for review.
9 years ago
|
|
|
sorted_p.startup_cost = input_path->startup_cost;
|
|
|
|
|
sorted_p.total_cost = input_path->total_cost;
|
|
|
|
|
/* XXX cost_sort doesn't actually look at pathkeys, so just pass NIL */
|
|
|
|
|
cost_sort(&sorted_p, root, NIL, sorted_p.total_cost,
|
Make the upper part of the planner work by generating and comparing Paths.
I've been saying we needed to do this for more than five years, and here it
finally is. This patch removes the ever-growing tangle of spaghetti logic
that grouping_planner() used to use to try to identify the best plan for
post-scan/join query steps. Now, there is (nearly) independent
consideration of each execution step, and entirely separate construction of
Paths to represent each of the possible ways to do that step. We choose
the best Path or set of Paths using the same add_path() logic that's been
used inside query_planner() for years.
In addition, this patch removes the old restriction that subquery_planner()
could return only a single Plan. It now returns a RelOptInfo containing a
set of Paths, just as query_planner() does, and the parent query level can
use each of those Paths as the basis of a SubqueryScanPath at its level.
This allows finding some optimizations that we missed before, wherein a
subquery was capable of returning presorted data and thereby avoiding a
sort in the parent level, making the overall cost cheaper even though
delivering sorted output was not the cheapest plan for the subquery in
isolation. (A couple of regression test outputs change in consequence of
that. However, there is very little change in visible planner behavior
overall, because the point of this patch is not to get immediate planning
benefits but to create the infrastructure for future improvements.)
There is a great deal left to do here. This patch unblocks a lot of
planner work that was basically impractical in the old code structure,
such as allowing FDWs to implement remote aggregation, or rewriting
plan_set_operations() to allow consideration of multiple implementation
orders for set operations. (The latter will likely require a full
rewrite of plan_set_operations(); what I've done here is only to fix it
to return Paths not Plans.) I have also left unfinished some localized
refactoring in createplan.c and planner.c, because it was not necessary
to get this patch to a working state.
Thanks to Robert Haas, David Rowley, and Amit Kapila for review.
9 years ago
|
|
|
input_path->rows, input_path->pathtarget->width,
|
|
|
|
|
0.0, work_mem, -1.0);
|
|
|
|
|
cost_group(&sorted_p, root, numGroupCols, dNumGroups,
|
|
|
|
|
NIL,
|
|
|
|
|
sorted_p.startup_cost, sorted_p.total_cost,
|
Make the upper part of the planner work by generating and comparing Paths.
I've been saying we needed to do this for more than five years, and here it
finally is. This patch removes the ever-growing tangle of spaghetti logic
that grouping_planner() used to use to try to identify the best plan for
post-scan/join query steps. Now, there is (nearly) independent
consideration of each execution step, and entirely separate construction of
Paths to represent each of the possible ways to do that step. We choose
the best Path or set of Paths using the same add_path() logic that's been
used inside query_planner() for years.
In addition, this patch removes the old restriction that subquery_planner()
could return only a single Plan. It now returns a RelOptInfo containing a
set of Paths, just as query_planner() does, and the parent query level can
use each of those Paths as the basis of a SubqueryScanPath at its level.
This allows finding some optimizations that we missed before, wherein a
subquery was capable of returning presorted data and thereby avoiding a
sort in the parent level, making the overall cost cheaper even though
delivering sorted output was not the cheapest plan for the subquery in
isolation. (A couple of regression test outputs change in consequence of
that. However, there is very little change in visible planner behavior
overall, because the point of this patch is not to get immediate planning
benefits but to create the infrastructure for future improvements.)
There is a great deal left to do here. This patch unblocks a lot of
planner work that was basically impractical in the old code structure,
such as allowing FDWs to implement remote aggregation, or rewriting
plan_set_operations() to allow consideration of multiple implementation
orders for set operations. (The latter will likely require a full
rewrite of plan_set_operations(); what I've done here is only to fix it
to return Paths not Plans.) I have also left unfinished some localized
refactoring in createplan.c and planner.c, because it was not necessary
to get this patch to a working state.
Thanks to Robert Haas, David Rowley, and Amit Kapila for review.
9 years ago
|
|
|
input_path->rows);
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Now make the decision using the top-level tuple fraction. First we
|
|
|
|
|
* have to convert an absolute count (LIMIT) into fractional form.
|
|
|
|
|
*/
|
Make the upper part of the planner work by generating and comparing Paths.
I've been saying we needed to do this for more than five years, and here it
finally is. This patch removes the ever-growing tangle of spaghetti logic
that grouping_planner() used to use to try to identify the best plan for
post-scan/join query steps. Now, there is (nearly) independent
consideration of each execution step, and entirely separate construction of
Paths to represent each of the possible ways to do that step. We choose
the best Path or set of Paths using the same add_path() logic that's been
used inside query_planner() for years.
In addition, this patch removes the old restriction that subquery_planner()
could return only a single Plan. It now returns a RelOptInfo containing a
set of Paths, just as query_planner() does, and the parent query level can
use each of those Paths as the basis of a SubqueryScanPath at its level.
This allows finding some optimizations that we missed before, wherein a
subquery was capable of returning presorted data and thereby avoiding a
sort in the parent level, making the overall cost cheaper even though
delivering sorted output was not the cheapest plan for the subquery in
isolation. (A couple of regression test outputs change in consequence of
that. However, there is very little change in visible planner behavior
overall, because the point of this patch is not to get immediate planning
benefits but to create the infrastructure for future improvements.)
There is a great deal left to do here. This patch unblocks a lot of
planner work that was basically impractical in the old code structure,
such as allowing FDWs to implement remote aggregation, or rewriting
plan_set_operations() to allow consideration of multiple implementation
orders for set operations. (The latter will likely require a full
rewrite of plan_set_operations(); what I've done here is only to fix it
to return Paths not Plans.) I have also left unfinished some localized
refactoring in createplan.c and planner.c, because it was not necessary
to get this patch to a working state.
Thanks to Robert Haas, David Rowley, and Amit Kapila for review.
9 years ago
|
|
|
tuple_fraction = root->tuple_fraction;
|
|
|
|
|
if (tuple_fraction >= 1.0)
|
|
|
|
|
tuple_fraction /= dNumOutputRows;
|
|
|
|
|
|
|
|
|
|
if (compare_fractional_path_costs(&hashed_p, &sorted_p,
|
|
|
|
|
tuple_fraction) < 0)
|
|
|
|
|
{
|
|
|
|
|
/* Hashed is cheaper, so use it */
|
|
|
|
|
return true;
|
|
|
|
|
}
|
|
|
|
|
return false;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Generate targetlist for a set-operation plan node
|
|
|
|
|
*
|
|
|
|
|
* colTypes: OID list of set-op's result column datatypes
|
|
|
|
|
* colCollations: OID list of set-op's result column collations
|
|
|
|
|
* flag: -1 if no flag column needed, 0 or 1 to create a const flag column
|
|
|
|
|
* varno: varno to use in generated Vars
|
|
|
|
|
* hack_constants: true to copy up constants (see comments in code)
|
|
|
|
|
* input_tlist: targetlist of this node's input node
|
|
|
|
|
* refnames_tlist: targetlist to take column names from
|
|
|
|
|
*/
|
|
|
|
|
static List *
|
|
|
|
|
generate_setop_tlist(List *colTypes, List *colCollations,
|
|
|
|
|
int flag,
|
|
|
|
|
Index varno,
|
|
|
|
|
bool hack_constants,
|
|
|
|
|
List *input_tlist,
|
|
|
|
|
List *refnames_tlist)
|
|
|
|
|
{
|
|
|
|
|
List *tlist = NIL;
|
|
|
|
|
int resno = 1;
|
|
|
|
|
ListCell *ctlc,
|
|
|
|
|
*cclc,
|
|
|
|
|
*itlc,
|
|
|
|
|
*rtlc;
|
|
|
|
|
TargetEntry *tle;
|
|
|
|
|
Node *expr;
|
|
|
|
|
|
|
|
|
|
/* there's no forfour() so we must chase one list manually */
|
|
|
|
|
rtlc = list_head(refnames_tlist);
|
|
|
|
|
forthree(ctlc, colTypes, cclc, colCollations, itlc, input_tlist)
|
|
|
|
|
{
|
|
|
|
|
Oid colType = lfirst_oid(ctlc);
|
|
|
|
|
Oid colColl = lfirst_oid(cclc);
|
|
|
|
|
TargetEntry *inputtle = (TargetEntry *) lfirst(itlc);
|
|
|
|
|
TargetEntry *reftle = (TargetEntry *) lfirst(rtlc);
|
|
|
|
|
|
|
|
|
|
rtlc = lnext(rtlc);
|
|
|
|
|
|
|
|
|
|
Assert(inputtle->resno == resno);
|
|
|
|
|
Assert(reftle->resno == resno);
|
|
|
|
|
Assert(!inputtle->resjunk);
|
|
|
|
|
Assert(!reftle->resjunk);
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Generate columns referencing input columns and having appropriate
|
|
|
|
|
* data types and column names. Insert datatype coercions where
|
|
|
|
|
* necessary.
|
|
|
|
|
*
|
|
|
|
|
* HACK: constants in the input's targetlist are copied up as-is
|
|
|
|
|
* rather than being referenced as subquery outputs. This is mainly
|
|
|
|
|
* to ensure that when we try to coerce them to the output column's
|
|
|
|
|
* datatype, the right things happen for UNKNOWN constants. But do
|
|
|
|
|
* this only at the first level of subquery-scan plans; we don't want
|
|
|
|
|
* phony constants appearing in the output tlists of upper-level
|
|
|
|
|
* nodes!
|
|
|
|
|
*/
|
|
|
|
|
if (hack_constants && inputtle->expr && IsA(inputtle->expr, Const))
|
|
|
|
|
expr = (Node *) inputtle->expr;
|
|
|
|
|
else
|
|
|
|
|
expr = (Node *) makeVar(varno,
|
|
|
|
|
inputtle->resno,
|
|
|
|
|
exprType((Node *) inputtle->expr),
|
|
|
|
|
exprTypmod((Node *) inputtle->expr),
|
|
|
|
|
exprCollation((Node *) inputtle->expr),
|
|
|
|
|
0);
|
|
|
|
|
|
|
|
|
|
if (exprType(expr) != colType)
|
|
|
|
|
{
|
|
|
|
|
/*
|
|
|
|
|
* Note: it's not really cool to be applying coerce_to_common_type
|
|
|
|
|
* here; one notable point is that assign_expr_collations never
|
|
|
|
|
* gets run on any generated nodes. For the moment that's not a
|
|
|
|
|
* problem because we force the correct exposed collation below.
|
|
|
|
|
* It would likely be best to make the parser generate the correct
|
|
|
|
|
* output tlist for every set-op to begin with, though.
|
|
|
|
|
*/
|
|
|
|
|
expr = coerce_to_common_type(NULL, /* no UNKNOWNs here */
|
|
|
|
|
expr,
|
|
|
|
|
colType,
|
|
|
|
|
"UNION/INTERSECT/EXCEPT");
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Ensure the tlist entry's exposed collation matches the set-op. This
|
|
|
|
|
* is necessary because plan_set_operations() reports the result
|
|
|
|
|
* ordering as a list of SortGroupClauses, which don't carry collation
|
|
|
|
|
* themselves but just refer to tlist entries. If we don't show the
|
|
|
|
|
* right collation then planner.c might do the wrong thing in
|
|
|
|
|
* higher-level queries.
|
|
|
|
|
*
|
|
|
|
|
* Note we use RelabelType, not CollateExpr, since this expression
|
|
|
|
|
* will reach the executor without any further processing.
|
|
|
|
|
*/
|
|
|
|
|
if (exprCollation(expr) != colColl)
|
|
|
|
|
{
|
|
|
|
|
expr = (Node *) makeRelabelType((Expr *) expr,
|
|
|
|
|
exprType(expr),
|
|
|
|
|
exprTypmod(expr),
|
|
|
|
|
colColl,
|
|
|
|
|
COERCE_IMPLICIT_CAST);
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
tle = makeTargetEntry((Expr *) expr,
|
|
|
|
|
(AttrNumber) resno++,
|
|
|
|
|
pstrdup(reftle->resname),
|
|
|
|
|
false);
|
Make the upper part of the planner work by generating and comparing Paths.
I've been saying we needed to do this for more than five years, and here it
finally is. This patch removes the ever-growing tangle of spaghetti logic
that grouping_planner() used to use to try to identify the best plan for
post-scan/join query steps. Now, there is (nearly) independent
consideration of each execution step, and entirely separate construction of
Paths to represent each of the possible ways to do that step. We choose
the best Path or set of Paths using the same add_path() logic that's been
used inside query_planner() for years.
In addition, this patch removes the old restriction that subquery_planner()
could return only a single Plan. It now returns a RelOptInfo containing a
set of Paths, just as query_planner() does, and the parent query level can
use each of those Paths as the basis of a SubqueryScanPath at its level.
This allows finding some optimizations that we missed before, wherein a
subquery was capable of returning presorted data and thereby avoiding a
sort in the parent level, making the overall cost cheaper even though
delivering sorted output was not the cheapest plan for the subquery in
isolation. (A couple of regression test outputs change in consequence of
that. However, there is very little change in visible planner behavior
overall, because the point of this patch is not to get immediate planning
benefits but to create the infrastructure for future improvements.)
There is a great deal left to do here. This patch unblocks a lot of
planner work that was basically impractical in the old code structure,
such as allowing FDWs to implement remote aggregation, or rewriting
plan_set_operations() to allow consideration of multiple implementation
orders for set operations. (The latter will likely require a full
rewrite of plan_set_operations(); what I've done here is only to fix it
to return Paths not Plans.) I have also left unfinished some localized
refactoring in createplan.c and planner.c, because it was not necessary
to get this patch to a working state.
Thanks to Robert Haas, David Rowley, and Amit Kapila for review.
9 years ago
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* By convention, all non-resjunk columns in a setop tree have
|
|
|
|
|
* ressortgroupref equal to their resno. In some cases the ref isn't
|
|
|
|
|
* needed, but this is a cleaner way than modifying the tlist later.
|
|
|
|
|
*/
|
|
|
|
|
tle->ressortgroupref = tle->resno;
|
|
|
|
|
|
|
|
|
|
tlist = lappend(tlist, tle);
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
if (flag >= 0)
|
|
|
|
|
{
|
|
|
|
|
/* Add a resjunk flag column */
|
|
|
|
|
/* flag value is the given constant */
|
|
|
|
|
expr = (Node *) makeConst(INT4OID,
|
|
|
|
|
-1,
|
|
|
|
|
InvalidOid,
|
|
|
|
|
sizeof(int32),
|
|
|
|
|
Int32GetDatum(flag),
|
|
|
|
|
false,
|
|
|
|
|
true);
|
|
|
|
|
tle = makeTargetEntry((Expr *) expr,
|
|
|
|
|
(AttrNumber) resno++,
|
|
|
|
|
pstrdup("flag"),
|
|
|
|
|
true);
|
|
|
|
|
tlist = lappend(tlist, tle);
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
return tlist;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Generate targetlist for a set-operation Append node
|
|
|
|
|
*
|
|
|
|
|
* colTypes: OID list of set-op's result column datatypes
|
|
|
|
|
* colCollations: OID list of set-op's result column collations
|
|
|
|
|
* flag: true to create a flag column copied up from subplans
|
Make the upper part of the planner work by generating and comparing Paths.
I've been saying we needed to do this for more than five years, and here it
finally is. This patch removes the ever-growing tangle of spaghetti logic
that grouping_planner() used to use to try to identify the best plan for
post-scan/join query steps. Now, there is (nearly) independent
consideration of each execution step, and entirely separate construction of
Paths to represent each of the possible ways to do that step. We choose
the best Path or set of Paths using the same add_path() logic that's been
used inside query_planner() for years.
In addition, this patch removes the old restriction that subquery_planner()
could return only a single Plan. It now returns a RelOptInfo containing a
set of Paths, just as query_planner() does, and the parent query level can
use each of those Paths as the basis of a SubqueryScanPath at its level.
This allows finding some optimizations that we missed before, wherein a
subquery was capable of returning presorted data and thereby avoiding a
sort in the parent level, making the overall cost cheaper even though
delivering sorted output was not the cheapest plan for the subquery in
isolation. (A couple of regression test outputs change in consequence of
that. However, there is very little change in visible planner behavior
overall, because the point of this patch is not to get immediate planning
benefits but to create the infrastructure for future improvements.)
There is a great deal left to do here. This patch unblocks a lot of
planner work that was basically impractical in the old code structure,
such as allowing FDWs to implement remote aggregation, or rewriting
plan_set_operations() to allow consideration of multiple implementation
orders for set operations. (The latter will likely require a full
rewrite of plan_set_operations(); what I've done here is only to fix it
to return Paths not Plans.) I have also left unfinished some localized
refactoring in createplan.c and planner.c, because it was not necessary
to get this patch to a working state.
Thanks to Robert Haas, David Rowley, and Amit Kapila for review.
9 years ago
|
|
|
* input_tlists: list of tlists for sub-plans of the Append
|
|
|
|
|
* refnames_tlist: targetlist to take column names from
|
|
|
|
|
*
|
|
|
|
|
* The entries in the Append's targetlist should always be simple Vars;
|
|
|
|
|
* we just have to make sure they have the right datatypes/typmods/collations.
|
|
|
|
|
* The Vars are always generated with varno 0.
|
Make the upper part of the planner work by generating and comparing Paths.
I've been saying we needed to do this for more than five years, and here it
finally is. This patch removes the ever-growing tangle of spaghetti logic
that grouping_planner() used to use to try to identify the best plan for
post-scan/join query steps. Now, there is (nearly) independent
consideration of each execution step, and entirely separate construction of
Paths to represent each of the possible ways to do that step. We choose
the best Path or set of Paths using the same add_path() logic that's been
used inside query_planner() for years.
In addition, this patch removes the old restriction that subquery_planner()
could return only a single Plan. It now returns a RelOptInfo containing a
set of Paths, just as query_planner() does, and the parent query level can
use each of those Paths as the basis of a SubqueryScanPath at its level.
This allows finding some optimizations that we missed before, wherein a
subquery was capable of returning presorted data and thereby avoiding a
sort in the parent level, making the overall cost cheaper even though
delivering sorted output was not the cheapest plan for the subquery in
isolation. (A couple of regression test outputs change in consequence of
that. However, there is very little change in visible planner behavior
overall, because the point of this patch is not to get immediate planning
benefits but to create the infrastructure for future improvements.)
There is a great deal left to do here. This patch unblocks a lot of
planner work that was basically impractical in the old code structure,
such as allowing FDWs to implement remote aggregation, or rewriting
plan_set_operations() to allow consideration of multiple implementation
orders for set operations. (The latter will likely require a full
rewrite of plan_set_operations(); what I've done here is only to fix it
to return Paths not Plans.) I have also left unfinished some localized
refactoring in createplan.c and planner.c, because it was not necessary
to get this patch to a working state.
Thanks to Robert Haas, David Rowley, and Amit Kapila for review.
9 years ago
|
|
|
*
|
|
|
|
|
* XXX a problem with the varno-zero approach is that set_pathtarget_cost_width
|
|
|
|
|
* cannot figure out a realistic width for the tlist we make here. But we
|
|
|
|
|
* ought to refactor this code to produce a PathTarget directly, anyway.
|
|
|
|
|
*/
|
|
|
|
|
static List *
|
|
|
|
|
generate_append_tlist(List *colTypes, List *colCollations,
|
|
|
|
|
bool flag,
|
Make the upper part of the planner work by generating and comparing Paths.
I've been saying we needed to do this for more than five years, and here it
finally is. This patch removes the ever-growing tangle of spaghetti logic
that grouping_planner() used to use to try to identify the best plan for
post-scan/join query steps. Now, there is (nearly) independent
consideration of each execution step, and entirely separate construction of
Paths to represent each of the possible ways to do that step. We choose
the best Path or set of Paths using the same add_path() logic that's been
used inside query_planner() for years.
In addition, this patch removes the old restriction that subquery_planner()
could return only a single Plan. It now returns a RelOptInfo containing a
set of Paths, just as query_planner() does, and the parent query level can
use each of those Paths as the basis of a SubqueryScanPath at its level.
This allows finding some optimizations that we missed before, wherein a
subquery was capable of returning presorted data and thereby avoiding a
sort in the parent level, making the overall cost cheaper even though
delivering sorted output was not the cheapest plan for the subquery in
isolation. (A couple of regression test outputs change in consequence of
that. However, there is very little change in visible planner behavior
overall, because the point of this patch is not to get immediate planning
benefits but to create the infrastructure for future improvements.)
There is a great deal left to do here. This patch unblocks a lot of
planner work that was basically impractical in the old code structure,
such as allowing FDWs to implement remote aggregation, or rewriting
plan_set_operations() to allow consideration of multiple implementation
orders for set operations. (The latter will likely require a full
rewrite of plan_set_operations(); what I've done here is only to fix it
to return Paths not Plans.) I have also left unfinished some localized
refactoring in createplan.c and planner.c, because it was not necessary
to get this patch to a working state.
Thanks to Robert Haas, David Rowley, and Amit Kapila for review.
9 years ago
|
|
|
List *input_tlists,
|
|
|
|
|
List *refnames_tlist)
|
|
|
|
|
{
|
|
|
|
|
List *tlist = NIL;
|
|
|
|
|
int resno = 1;
|
|
|
|
|
ListCell *curColType;
|
|
|
|
|
ListCell *curColCollation;
|
|
|
|
|
ListCell *ref_tl_item;
|
|
|
|
|
int colindex;
|
|
|
|
|
TargetEntry *tle;
|
|
|
|
|
Node *expr;
|
Make the upper part of the planner work by generating and comparing Paths.
I've been saying we needed to do this for more than five years, and here it
finally is. This patch removes the ever-growing tangle of spaghetti logic
that grouping_planner() used to use to try to identify the best plan for
post-scan/join query steps. Now, there is (nearly) independent
consideration of each execution step, and entirely separate construction of
Paths to represent each of the possible ways to do that step. We choose
the best Path or set of Paths using the same add_path() logic that's been
used inside query_planner() for years.
In addition, this patch removes the old restriction that subquery_planner()
could return only a single Plan. It now returns a RelOptInfo containing a
set of Paths, just as query_planner() does, and the parent query level can
use each of those Paths as the basis of a SubqueryScanPath at its level.
This allows finding some optimizations that we missed before, wherein a
subquery was capable of returning presorted data and thereby avoiding a
sort in the parent level, making the overall cost cheaper even though
delivering sorted output was not the cheapest plan for the subquery in
isolation. (A couple of regression test outputs change in consequence of
that. However, there is very little change in visible planner behavior
overall, because the point of this patch is not to get immediate planning
benefits but to create the infrastructure for future improvements.)
There is a great deal left to do here. This patch unblocks a lot of
planner work that was basically impractical in the old code structure,
such as allowing FDWs to implement remote aggregation, or rewriting
plan_set_operations() to allow consideration of multiple implementation
orders for set operations. (The latter will likely require a full
rewrite of plan_set_operations(); what I've done here is only to fix it
to return Paths not Plans.) I have also left unfinished some localized
refactoring in createplan.c and planner.c, because it was not necessary
to get this patch to a working state.
Thanks to Robert Haas, David Rowley, and Amit Kapila for review.
9 years ago
|
|
|
ListCell *tlistl;
|
|
|
|
|
int32 *colTypmods;
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* First extract typmods to use.
|
|
|
|
|
*
|
|
|
|
|
* If the inputs all agree on type and typmod of a particular column, use
|
|
|
|
|
* that typmod; else use -1.
|
|
|
|
|
*/
|
|
|
|
|
colTypmods = (int32 *) palloc(list_length(colTypes) * sizeof(int32));
|
|
|
|
|
|
Make the upper part of the planner work by generating and comparing Paths.
I've been saying we needed to do this for more than five years, and here it
finally is. This patch removes the ever-growing tangle of spaghetti logic
that grouping_planner() used to use to try to identify the best plan for
post-scan/join query steps. Now, there is (nearly) independent
consideration of each execution step, and entirely separate construction of
Paths to represent each of the possible ways to do that step. We choose
the best Path or set of Paths using the same add_path() logic that's been
used inside query_planner() for years.
In addition, this patch removes the old restriction that subquery_planner()
could return only a single Plan. It now returns a RelOptInfo containing a
set of Paths, just as query_planner() does, and the parent query level can
use each of those Paths as the basis of a SubqueryScanPath at its level.
This allows finding some optimizations that we missed before, wherein a
subquery was capable of returning presorted data and thereby avoiding a
sort in the parent level, making the overall cost cheaper even though
delivering sorted output was not the cheapest plan for the subquery in
isolation. (A couple of regression test outputs change in consequence of
that. However, there is very little change in visible planner behavior
overall, because the point of this patch is not to get immediate planning
benefits but to create the infrastructure for future improvements.)
There is a great deal left to do here. This patch unblocks a lot of
planner work that was basically impractical in the old code structure,
such as allowing FDWs to implement remote aggregation, or rewriting
plan_set_operations() to allow consideration of multiple implementation
orders for set operations. (The latter will likely require a full
rewrite of plan_set_operations(); what I've done here is only to fix it
to return Paths not Plans.) I have also left unfinished some localized
refactoring in createplan.c and planner.c, because it was not necessary
to get this patch to a working state.
Thanks to Robert Haas, David Rowley, and Amit Kapila for review.
9 years ago
|
|
|
foreach(tlistl, input_tlists)
|
|
|
|
|
{
|
Make the upper part of the planner work by generating and comparing Paths.
I've been saying we needed to do this for more than five years, and here it
finally is. This patch removes the ever-growing tangle of spaghetti logic
that grouping_planner() used to use to try to identify the best plan for
post-scan/join query steps. Now, there is (nearly) independent
consideration of each execution step, and entirely separate construction of
Paths to represent each of the possible ways to do that step. We choose
the best Path or set of Paths using the same add_path() logic that's been
used inside query_planner() for years.
In addition, this patch removes the old restriction that subquery_planner()
could return only a single Plan. It now returns a RelOptInfo containing a
set of Paths, just as query_planner() does, and the parent query level can
use each of those Paths as the basis of a SubqueryScanPath at its level.
This allows finding some optimizations that we missed before, wherein a
subquery was capable of returning presorted data and thereby avoiding a
sort in the parent level, making the overall cost cheaper even though
delivering sorted output was not the cheapest plan for the subquery in
isolation. (A couple of regression test outputs change in consequence of
that. However, there is very little change in visible planner behavior
overall, because the point of this patch is not to get immediate planning
benefits but to create the infrastructure for future improvements.)
There is a great deal left to do here. This patch unblocks a lot of
planner work that was basically impractical in the old code structure,
such as allowing FDWs to implement remote aggregation, or rewriting
plan_set_operations() to allow consideration of multiple implementation
orders for set operations. (The latter will likely require a full
rewrite of plan_set_operations(); what I've done here is only to fix it
to return Paths not Plans.) I have also left unfinished some localized
refactoring in createplan.c and planner.c, because it was not necessary
to get this patch to a working state.
Thanks to Robert Haas, David Rowley, and Amit Kapila for review.
9 years ago
|
|
|
List *subtlist = (List *) lfirst(tlistl);
|
|
|
|
|
ListCell *subtlistl;
|
|
|
|
|
|
|
|
|
|
curColType = list_head(colTypes);
|
|
|
|
|
colindex = 0;
|
Make the upper part of the planner work by generating and comparing Paths.
I've been saying we needed to do this for more than five years, and here it
finally is. This patch removes the ever-growing tangle of spaghetti logic
that grouping_planner() used to use to try to identify the best plan for
post-scan/join query steps. Now, there is (nearly) independent
consideration of each execution step, and entirely separate construction of
Paths to represent each of the possible ways to do that step. We choose
the best Path or set of Paths using the same add_path() logic that's been
used inside query_planner() for years.
In addition, this patch removes the old restriction that subquery_planner()
could return only a single Plan. It now returns a RelOptInfo containing a
set of Paths, just as query_planner() does, and the parent query level can
use each of those Paths as the basis of a SubqueryScanPath at its level.
This allows finding some optimizations that we missed before, wherein a
subquery was capable of returning presorted data and thereby avoiding a
sort in the parent level, making the overall cost cheaper even though
delivering sorted output was not the cheapest plan for the subquery in
isolation. (A couple of regression test outputs change in consequence of
that. However, there is very little change in visible planner behavior
overall, because the point of this patch is not to get immediate planning
benefits but to create the infrastructure for future improvements.)
There is a great deal left to do here. This patch unblocks a lot of
planner work that was basically impractical in the old code structure,
such as allowing FDWs to implement remote aggregation, or rewriting
plan_set_operations() to allow consideration of multiple implementation
orders for set operations. (The latter will likely require a full
rewrite of plan_set_operations(); what I've done here is only to fix it
to return Paths not Plans.) I have also left unfinished some localized
refactoring in createplan.c and planner.c, because it was not necessary
to get this patch to a working state.
Thanks to Robert Haas, David Rowley, and Amit Kapila for review.
9 years ago
|
|
|
foreach(subtlistl, subtlist)
|
|
|
|
|
{
|
Make the upper part of the planner work by generating and comparing Paths.
I've been saying we needed to do this for more than five years, and here it
finally is. This patch removes the ever-growing tangle of spaghetti logic
that grouping_planner() used to use to try to identify the best plan for
post-scan/join query steps. Now, there is (nearly) independent
consideration of each execution step, and entirely separate construction of
Paths to represent each of the possible ways to do that step. We choose
the best Path or set of Paths using the same add_path() logic that's been
used inside query_planner() for years.
In addition, this patch removes the old restriction that subquery_planner()
could return only a single Plan. It now returns a RelOptInfo containing a
set of Paths, just as query_planner() does, and the parent query level can
use each of those Paths as the basis of a SubqueryScanPath at its level.
This allows finding some optimizations that we missed before, wherein a
subquery was capable of returning presorted data and thereby avoiding a
sort in the parent level, making the overall cost cheaper even though
delivering sorted output was not the cheapest plan for the subquery in
isolation. (A couple of regression test outputs change in consequence of
that. However, there is very little change in visible planner behavior
overall, because the point of this patch is not to get immediate planning
benefits but to create the infrastructure for future improvements.)
There is a great deal left to do here. This patch unblocks a lot of
planner work that was basically impractical in the old code structure,
such as allowing FDWs to implement remote aggregation, or rewriting
plan_set_operations() to allow consideration of multiple implementation
orders for set operations. (The latter will likely require a full
rewrite of plan_set_operations(); what I've done here is only to fix it
to return Paths not Plans.) I have also left unfinished some localized
refactoring in createplan.c and planner.c, because it was not necessary
to get this patch to a working state.
Thanks to Robert Haas, David Rowley, and Amit Kapila for review.
9 years ago
|
|
|
TargetEntry *subtle = (TargetEntry *) lfirst(subtlistl);
|
|
|
|
|
|
|
|
|
|
if (subtle->resjunk)
|
|
|
|
|
continue;
|
|
|
|
|
Assert(curColType != NULL);
|
|
|
|
|
if (exprType((Node *) subtle->expr) == lfirst_oid(curColType))
|
|
|
|
|
{
|
|
|
|
|
/* If first subplan, copy the typmod; else compare */
|
|
|
|
|
int32 subtypmod = exprTypmod((Node *) subtle->expr);
|
|
|
|
|
|
Make the upper part of the planner work by generating and comparing Paths.
I've been saying we needed to do this for more than five years, and here it
finally is. This patch removes the ever-growing tangle of spaghetti logic
that grouping_planner() used to use to try to identify the best plan for
post-scan/join query steps. Now, there is (nearly) independent
consideration of each execution step, and entirely separate construction of
Paths to represent each of the possible ways to do that step. We choose
the best Path or set of Paths using the same add_path() logic that's been
used inside query_planner() for years.
In addition, this patch removes the old restriction that subquery_planner()
could return only a single Plan. It now returns a RelOptInfo containing a
set of Paths, just as query_planner() does, and the parent query level can
use each of those Paths as the basis of a SubqueryScanPath at its level.
This allows finding some optimizations that we missed before, wherein a
subquery was capable of returning presorted data and thereby avoiding a
sort in the parent level, making the overall cost cheaper even though
delivering sorted output was not the cheapest plan for the subquery in
isolation. (A couple of regression test outputs change in consequence of
that. However, there is very little change in visible planner behavior
overall, because the point of this patch is not to get immediate planning
benefits but to create the infrastructure for future improvements.)
There is a great deal left to do here. This patch unblocks a lot of
planner work that was basically impractical in the old code structure,
such as allowing FDWs to implement remote aggregation, or rewriting
plan_set_operations() to allow consideration of multiple implementation
orders for set operations. (The latter will likely require a full
rewrite of plan_set_operations(); what I've done here is only to fix it
to return Paths not Plans.) I have also left unfinished some localized
refactoring in createplan.c and planner.c, because it was not necessary
to get this patch to a working state.
Thanks to Robert Haas, David Rowley, and Amit Kapila for review.
9 years ago
|
|
|
if (tlistl == list_head(input_tlists))
|
|
|
|
|
colTypmods[colindex] = subtypmod;
|
|
|
|
|
else if (subtypmod != colTypmods[colindex])
|
|
|
|
|
colTypmods[colindex] = -1;
|
|
|
|
|
}
|
|
|
|
|
else
|
|
|
|
|
{
|
|
|
|
|
/* types disagree, so force typmod to -1 */
|
|
|
|
|
colTypmods[colindex] = -1;
|
|
|
|
|
}
|
|
|
|
|
curColType = lnext(curColType);
|
|
|
|
|
colindex++;
|
|
|
|
|
}
|
|
|
|
|
Assert(curColType == NULL);
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Now we can build the tlist for the Append.
|
|
|
|
|
*/
|
|
|
|
|
colindex = 0;
|
|
|
|
|
forthree(curColType, colTypes, curColCollation, colCollations,
|
|
|
|
|
ref_tl_item, refnames_tlist)
|
|
|
|
|
{
|
|
|
|
|
Oid colType = lfirst_oid(curColType);
|
|
|
|
|
int32 colTypmod = colTypmods[colindex++];
|
|
|
|
|
Oid colColl = lfirst_oid(curColCollation);
|
|
|
|
|
TargetEntry *reftle = (TargetEntry *) lfirst(ref_tl_item);
|
|
|
|
|
|
|
|
|
|
Assert(reftle->resno == resno);
|
|
|
|
|
Assert(!reftle->resjunk);
|
|
|
|
|
expr = (Node *) makeVar(0,
|
|
|
|
|
resno,
|
|
|
|
|
colType,
|
|
|
|
|
colTypmod,
|
|
|
|
|
colColl,
|
|
|
|
|
0);
|
|
|
|
|
tle = makeTargetEntry((Expr *) expr,
|
|
|
|
|
(AttrNumber) resno++,
|
|
|
|
|
pstrdup(reftle->resname),
|
|
|
|
|
false);
|
Make the upper part of the planner work by generating and comparing Paths.
I've been saying we needed to do this for more than five years, and here it
finally is. This patch removes the ever-growing tangle of spaghetti logic
that grouping_planner() used to use to try to identify the best plan for
post-scan/join query steps. Now, there is (nearly) independent
consideration of each execution step, and entirely separate construction of
Paths to represent each of the possible ways to do that step. We choose
the best Path or set of Paths using the same add_path() logic that's been
used inside query_planner() for years.
In addition, this patch removes the old restriction that subquery_planner()
could return only a single Plan. It now returns a RelOptInfo containing a
set of Paths, just as query_planner() does, and the parent query level can
use each of those Paths as the basis of a SubqueryScanPath at its level.
This allows finding some optimizations that we missed before, wherein a
subquery was capable of returning presorted data and thereby avoiding a
sort in the parent level, making the overall cost cheaper even though
delivering sorted output was not the cheapest plan for the subquery in
isolation. (A couple of regression test outputs change in consequence of
that. However, there is very little change in visible planner behavior
overall, because the point of this patch is not to get immediate planning
benefits but to create the infrastructure for future improvements.)
There is a great deal left to do here. This patch unblocks a lot of
planner work that was basically impractical in the old code structure,
such as allowing FDWs to implement remote aggregation, or rewriting
plan_set_operations() to allow consideration of multiple implementation
orders for set operations. (The latter will likely require a full
rewrite of plan_set_operations(); what I've done here is only to fix it
to return Paths not Plans.) I have also left unfinished some localized
refactoring in createplan.c and planner.c, because it was not necessary
to get this patch to a working state.
Thanks to Robert Haas, David Rowley, and Amit Kapila for review.
9 years ago
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* By convention, all non-resjunk columns in a setop tree have
|
|
|
|
|
* ressortgroupref equal to their resno. In some cases the ref isn't
|
|
|
|
|
* needed, but this is a cleaner way than modifying the tlist later.
|
|
|
|
|
*/
|
|
|
|
|
tle->ressortgroupref = tle->resno;
|
|
|
|
|
|
|
|
|
|
tlist = lappend(tlist, tle);
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
if (flag)
|
|
|
|
|
{
|
|
|
|
|
/* Add a resjunk flag column */
|
|
|
|
|
/* flag value is shown as copied up from subplan */
|
|
|
|
|
expr = (Node *) makeVar(0,
|
|
|
|
|
resno,
|
|
|
|
|
INT4OID,
|
|
|
|
|
-1,
|
|
|
|
|
InvalidOid,
|
|
|
|
|
0);
|
|
|
|
|
tle = makeTargetEntry((Expr *) expr,
|
|
|
|
|
(AttrNumber) resno++,
|
|
|
|
|
pstrdup("flag"),
|
|
|
|
|
true);
|
|
|
|
|
tlist = lappend(tlist, tle);
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
pfree(colTypmods);
|
|
|
|
|
|
|
|
|
|
return tlist;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* generate_setop_grouplist
|
|
|
|
|
* Build a SortGroupClause list defining the sort/grouping properties
|
|
|
|
|
* of the setop's output columns.
|
|
|
|
|
*
|
|
|
|
|
* Parse analysis already determined the properties and built a suitable
|
|
|
|
|
* list, except that the entries do not have sortgrouprefs set because
|
|
|
|
|
* the parser output representation doesn't include a tlist for each
|
|
|
|
|
* setop. So what we need to do here is copy that list and install
|
Make the upper part of the planner work by generating and comparing Paths.
I've been saying we needed to do this for more than five years, and here it
finally is. This patch removes the ever-growing tangle of spaghetti logic
that grouping_planner() used to use to try to identify the best plan for
post-scan/join query steps. Now, there is (nearly) independent
consideration of each execution step, and entirely separate construction of
Paths to represent each of the possible ways to do that step. We choose
the best Path or set of Paths using the same add_path() logic that's been
used inside query_planner() for years.
In addition, this patch removes the old restriction that subquery_planner()
could return only a single Plan. It now returns a RelOptInfo containing a
set of Paths, just as query_planner() does, and the parent query level can
use each of those Paths as the basis of a SubqueryScanPath at its level.
This allows finding some optimizations that we missed before, wherein a
subquery was capable of returning presorted data and thereby avoiding a
sort in the parent level, making the overall cost cheaper even though
delivering sorted output was not the cheapest plan for the subquery in
isolation. (A couple of regression test outputs change in consequence of
that. However, there is very little change in visible planner behavior
overall, because the point of this patch is not to get immediate planning
benefits but to create the infrastructure for future improvements.)
There is a great deal left to do here. This patch unblocks a lot of
planner work that was basically impractical in the old code structure,
such as allowing FDWs to implement remote aggregation, or rewriting
plan_set_operations() to allow consideration of multiple implementation
orders for set operations. (The latter will likely require a full
rewrite of plan_set_operations(); what I've done here is only to fix it
to return Paths not Plans.) I have also left unfinished some localized
refactoring in createplan.c and planner.c, because it was not necessary
to get this patch to a working state.
Thanks to Robert Haas, David Rowley, and Amit Kapila for review.
9 years ago
|
|
|
* proper sortgrouprefs into it (copying those from the targetlist).
|
|
|
|
|
*/
|
|
|
|
|
static List *
|
|
|
|
|
generate_setop_grouplist(SetOperationStmt *op, List *targetlist)
|
|
|
|
|
{
|
|
|
|
|
List *grouplist = copyObject(op->groupClauses);
|
|
|
|
|
ListCell *lg;
|
|
|
|
|
ListCell *lt;
|
|
|
|
|
|
|
|
|
|
lg = list_head(grouplist);
|
|
|
|
|
foreach(lt, targetlist)
|
|
|
|
|
{
|
|
|
|
|
TargetEntry *tle = (TargetEntry *) lfirst(lt);
|
|
|
|
|
SortGroupClause *sgc;
|
|
|
|
|
|
|
|
|
|
if (tle->resjunk)
|
Make the upper part of the planner work by generating and comparing Paths.
I've been saying we needed to do this for more than five years, and here it
finally is. This patch removes the ever-growing tangle of spaghetti logic
that grouping_planner() used to use to try to identify the best plan for
post-scan/join query steps. Now, there is (nearly) independent
consideration of each execution step, and entirely separate construction of
Paths to represent each of the possible ways to do that step. We choose
the best Path or set of Paths using the same add_path() logic that's been
used inside query_planner() for years.
In addition, this patch removes the old restriction that subquery_planner()
could return only a single Plan. It now returns a RelOptInfo containing a
set of Paths, just as query_planner() does, and the parent query level can
use each of those Paths as the basis of a SubqueryScanPath at its level.
This allows finding some optimizations that we missed before, wherein a
subquery was capable of returning presorted data and thereby avoiding a
sort in the parent level, making the overall cost cheaper even though
delivering sorted output was not the cheapest plan for the subquery in
isolation. (A couple of regression test outputs change in consequence of
that. However, there is very little change in visible planner behavior
overall, because the point of this patch is not to get immediate planning
benefits but to create the infrastructure for future improvements.)
There is a great deal left to do here. This patch unblocks a lot of
planner work that was basically impractical in the old code structure,
such as allowing FDWs to implement remote aggregation, or rewriting
plan_set_operations() to allow consideration of multiple implementation
orders for set operations. (The latter will likely require a full
rewrite of plan_set_operations(); what I've done here is only to fix it
to return Paths not Plans.) I have also left unfinished some localized
refactoring in createplan.c and planner.c, because it was not necessary
to get this patch to a working state.
Thanks to Robert Haas, David Rowley, and Amit Kapila for review.
9 years ago
|
|
|
{
|
|
|
|
|
/* resjunk columns should not have sortgrouprefs */
|
|
|
|
|
Assert(tle->ressortgroupref == 0);
|
|
|
|
|
continue; /* ignore resjunk columns */
|
Make the upper part of the planner work by generating and comparing Paths.
I've been saying we needed to do this for more than five years, and here it
finally is. This patch removes the ever-growing tangle of spaghetti logic
that grouping_planner() used to use to try to identify the best plan for
post-scan/join query steps. Now, there is (nearly) independent
consideration of each execution step, and entirely separate construction of
Paths to represent each of the possible ways to do that step. We choose
the best Path or set of Paths using the same add_path() logic that's been
used inside query_planner() for years.
In addition, this patch removes the old restriction that subquery_planner()
could return only a single Plan. It now returns a RelOptInfo containing a
set of Paths, just as query_planner() does, and the parent query level can
use each of those Paths as the basis of a SubqueryScanPath at its level.
This allows finding some optimizations that we missed before, wherein a
subquery was capable of returning presorted data and thereby avoiding a
sort in the parent level, making the overall cost cheaper even though
delivering sorted output was not the cheapest plan for the subquery in
isolation. (A couple of regression test outputs change in consequence of
that. However, there is very little change in visible planner behavior
overall, because the point of this patch is not to get immediate planning
benefits but to create the infrastructure for future improvements.)
There is a great deal left to do here. This patch unblocks a lot of
planner work that was basically impractical in the old code structure,
such as allowing FDWs to implement remote aggregation, or rewriting
plan_set_operations() to allow consideration of multiple implementation
orders for set operations. (The latter will likely require a full
rewrite of plan_set_operations(); what I've done here is only to fix it
to return Paths not Plans.) I have also left unfinished some localized
refactoring in createplan.c and planner.c, because it was not necessary
to get this patch to a working state.
Thanks to Robert Haas, David Rowley, and Amit Kapila for review.
9 years ago
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/* non-resjunk columns should have sortgroupref = resno */
|
|
|
|
|
Assert(tle->ressortgroupref == tle->resno);
|
|
|
|
|
|
|
|
|
|
/* non-resjunk columns should have grouping clauses */
|
|
|
|
|
Assert(lg != NULL);
|
|
|
|
|
sgc = (SortGroupClause *) lfirst(lg);
|
|
|
|
|
lg = lnext(lg);
|
|
|
|
|
Assert(sgc->tleSortGroupRef == 0);
|
|
|
|
|
|
Make the upper part of the planner work by generating and comparing Paths.
I've been saying we needed to do this for more than five years, and here it
finally is. This patch removes the ever-growing tangle of spaghetti logic
that grouping_planner() used to use to try to identify the best plan for
post-scan/join query steps. Now, there is (nearly) independent
consideration of each execution step, and entirely separate construction of
Paths to represent each of the possible ways to do that step. We choose
the best Path or set of Paths using the same add_path() logic that's been
used inside query_planner() for years.
In addition, this patch removes the old restriction that subquery_planner()
could return only a single Plan. It now returns a RelOptInfo containing a
set of Paths, just as query_planner() does, and the parent query level can
use each of those Paths as the basis of a SubqueryScanPath at its level.
This allows finding some optimizations that we missed before, wherein a
subquery was capable of returning presorted data and thereby avoiding a
sort in the parent level, making the overall cost cheaper even though
delivering sorted output was not the cheapest plan for the subquery in
isolation. (A couple of regression test outputs change in consequence of
that. However, there is very little change in visible planner behavior
overall, because the point of this patch is not to get immediate planning
benefits but to create the infrastructure for future improvements.)
There is a great deal left to do here. This patch unblocks a lot of
planner work that was basically impractical in the old code structure,
such as allowing FDWs to implement remote aggregation, or rewriting
plan_set_operations() to allow consideration of multiple implementation
orders for set operations. (The latter will likely require a full
rewrite of plan_set_operations(); what I've done here is only to fix it
to return Paths not Plans.) I have also left unfinished some localized
refactoring in createplan.c and planner.c, because it was not necessary
to get this patch to a working state.
Thanks to Robert Haas, David Rowley, and Amit Kapila for review.
9 years ago
|
|
|
sgc->tleSortGroupRef = tle->ressortgroupref;
|
|
|
|
|
}
|
|
|
|
|
Assert(lg == NULL);
|
|
|
|
|
return grouplist;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* expand_inherited_tables
|
|
|
|
|
* Expand each rangetable entry that represents an inheritance set
|
|
|
|
|
* into an "append relation". At the conclusion of this process,
|
|
|
|
|
* the "inh" flag is set in all and only those RTEs that are append
|
|
|
|
|
* relation parents.
|
|
|
|
|
*/
|
|
|
|
|
void
|
|
|
|
|
expand_inherited_tables(PlannerInfo *root)
|
|
|
|
|
{
|
|
|
|
|
Index nrtes;
|
|
|
|
|
Index rti;
|
|
|
|
|
ListCell *rl;
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* expand_inherited_rtentry may add RTEs to parse->rtable. The function is
|
|
|
|
|
* expected to recursively handle any RTEs that it creates with inh=true.
|
|
|
|
|
* So just scan as far as the original end of the rtable list.
|
|
|
|
|
*/
|
|
|
|
|
nrtes = list_length(root->parse->rtable);
|
|
|
|
|
rl = list_head(root->parse->rtable);
|
|
|
|
|
for (rti = 1; rti <= nrtes; rti++)
|
|
|
|
|
{
|
|
|
|
|
RangeTblEntry *rte = (RangeTblEntry *) lfirst(rl);
|
|
|
|
|
|
|
|
|
|
expand_inherited_rtentry(root, rte, rti);
|
|
|
|
|
rl = lnext(rl);
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* expand_inherited_rtentry
|
|
|
|
|
* Check whether a rangetable entry represents an inheritance set.
|
|
|
|
|
* If so, add entries for all the child tables to the query's
|
|
|
|
|
* rangetable, and build AppendRelInfo nodes for all the child tables
|
|
|
|
|
* and add them to root->append_rel_list. If not, clear the entry's
|
|
|
|
|
* "inh" flag to prevent later code from looking for AppendRelInfos.
|
|
|
|
|
*
|
|
|
|
|
* Note that the original RTE is considered to represent the whole
|
|
|
|
|
* inheritance set. The first of the generated RTEs is an RTE for the same
|
|
|
|
|
* table, but with inh = false, to represent the parent table in its role
|
|
|
|
|
* as a simple member of the inheritance set.
|
|
|
|
|
*
|
|
|
|
|
* A childless table is never considered to be an inheritance set. For
|
|
|
|
|
* regular inheritance, a parent RTE must always have at least two associated
|
|
|
|
|
* AppendRelInfos: one corresponding to the parent table as a simple member of
|
|
|
|
|
* inheritance set and one or more corresponding to the actual children.
|
|
|
|
|
* Since a partitioned table is not scanned, it might have only one associated
|
|
|
|
|
* AppendRelInfo.
|
|
|
|
|
*/
|
|
|
|
|
static void
|
|
|
|
|
expand_inherited_rtentry(PlannerInfo *root, RangeTblEntry *rte, Index rti)
|
|
|
|
|
{
|
|
|
|
|
Query *parse = root->parse;
|
|
|
|
|
Oid parentOID;
|
Re-implement EvalPlanQual processing to improve its performance and eliminate
a lot of strange behaviors that occurred in join cases. We now identify the
"current" row for every joined relation in UPDATE, DELETE, and SELECT FOR
UPDATE/SHARE queries. If an EvalPlanQual recheck is necessary, we jam the
appropriate row into each scan node in the rechecking plan, forcing it to emit
only that one row. The former behavior could rescan the whole of each joined
relation for each recheck, which was terrible for performance, and what's much
worse could result in duplicated output tuples.
Also, the original implementation of EvalPlanQual could not re-use the recheck
execution tree --- it had to go through a full executor init and shutdown for
every row to be tested. To avoid this overhead, I've associated a special
runtime Param with each LockRows or ModifyTable plan node, and arranged to
make every scan node below such a node depend on that Param. Thus, by
signaling a change in that Param, the EPQ machinery can just rescan the
already-built test plan.
This patch also adds a prohibition on set-returning functions in the
targetlist of SELECT FOR UPDATE/SHARE. This is needed to avoid the
duplicate-output-tuple problem. It seems fairly reasonable since the
other restrictions on SELECT FOR UPDATE are meant to ensure that there
is a unique correspondence between source tuples and result tuples,
which an output SRF destroys as much as anything else does.
16 years ago
|
|
|
PlanRowMark *oldrc;
|
|
|
|
|
Relation oldrelation;
|
|
|
|
|
LOCKMODE lockmode;
|
|
|
|
|
List *inhOIDs;
|
|
|
|
|
ListCell *l;
|
|
|
|
|
|
|
|
|
|
/* Does RT entry allow inheritance? */
|
|
|
|
|
if (!rte->inh)
|
|
|
|
|
return;
|
|
|
|
|
/* Ignore any already-expanded UNION ALL nodes */
|
|
|
|
|
if (rte->rtekind != RTE_RELATION)
|
|
|
|
|
{
|
|
|
|
|
Assert(rte->rtekind == RTE_SUBQUERY);
|
|
|
|
|
return;
|
|
|
|
|
}
|
|
|
|
|
/* Fast path for common case of childless table */
|
|
|
|
|
parentOID = rte->relid;
|
|
|
|
|
if (!has_subclass(parentOID))
|
|
|
|
|
{
|
|
|
|
|
/* Clear flag before returning */
|
|
|
|
|
rte->inh = false;
|
|
|
|
|
return;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* The rewriter should already have obtained an appropriate lock on each
|
|
|
|
|
* relation named in the query. However, for each child relation we add
|
|
|
|
|
* to the query, we must obtain an appropriate lock, because this will be
|
|
|
|
|
* the first use of those relations in the parse/rewrite/plan pipeline.
|
|
|
|
|
*
|
|
|
|
|
* If the parent relation is the query's result relation, then we need
|
|
|
|
|
* RowExclusiveLock. Otherwise, if it's accessed FOR UPDATE/SHARE, we
|
|
|
|
|
* need RowShareLock; otherwise AccessShareLock. We can't just grab
|
|
|
|
|
* AccessShareLock because then the executor would be trying to upgrade
|
|
|
|
|
* the lock, leading to possible deadlocks. (This code should match the
|
|
|
|
|
* parser and rewriter.)
|
|
|
|
|
*/
|
Re-implement EvalPlanQual processing to improve its performance and eliminate
a lot of strange behaviors that occurred in join cases. We now identify the
"current" row for every joined relation in UPDATE, DELETE, and SELECT FOR
UPDATE/SHARE queries. If an EvalPlanQual recheck is necessary, we jam the
appropriate row into each scan node in the rechecking plan, forcing it to emit
only that one row. The former behavior could rescan the whole of each joined
relation for each recheck, which was terrible for performance, and what's much
worse could result in duplicated output tuples.
Also, the original implementation of EvalPlanQual could not re-use the recheck
execution tree --- it had to go through a full executor init and shutdown for
every row to be tested. To avoid this overhead, I've associated a special
runtime Param with each LockRows or ModifyTable plan node, and arranged to
make every scan node below such a node depend on that Param. Thus, by
signaling a change in that Param, the EPQ machinery can just rescan the
already-built test plan.
This patch also adds a prohibition on set-returning functions in the
targetlist of SELECT FOR UPDATE/SHARE. This is needed to avoid the
duplicate-output-tuple problem. It seems fairly reasonable since the
other restrictions on SELECT FOR UPDATE are meant to ensure that there
is a unique correspondence between source tuples and result tuples,
which an output SRF destroys as much as anything else does.
16 years ago
|
|
|
oldrc = get_plan_rowmark(root->rowMarks, rti);
|
|
|
|
|
if (rti == parse->resultRelation)
|
|
|
|
|
lockmode = RowExclusiveLock;
|
Re-implement EvalPlanQual processing to improve its performance and eliminate
a lot of strange behaviors that occurred in join cases. We now identify the
"current" row for every joined relation in UPDATE, DELETE, and SELECT FOR
UPDATE/SHARE queries. If an EvalPlanQual recheck is necessary, we jam the
appropriate row into each scan node in the rechecking plan, forcing it to emit
only that one row. The former behavior could rescan the whole of each joined
relation for each recheck, which was terrible for performance, and what's much
worse could result in duplicated output tuples.
Also, the original implementation of EvalPlanQual could not re-use the recheck
execution tree --- it had to go through a full executor init and shutdown for
every row to be tested. To avoid this overhead, I've associated a special
runtime Param with each LockRows or ModifyTable plan node, and arranged to
make every scan node below such a node depend on that Param. Thus, by
signaling a change in that Param, the EPQ machinery can just rescan the
already-built test plan.
This patch also adds a prohibition on set-returning functions in the
targetlist of SELECT FOR UPDATE/SHARE. This is needed to avoid the
duplicate-output-tuple problem. It seems fairly reasonable since the
other restrictions on SELECT FOR UPDATE are meant to ensure that there
is a unique correspondence between source tuples and result tuples,
which an output SRF destroys as much as anything else does.
16 years ago
|
|
|
else if (oldrc && RowMarkRequiresRowShareLock(oldrc->markType))
|
|
|
|
|
lockmode = RowShareLock;
|
|
|
|
|
else
|
|
|
|
|
lockmode = AccessShareLock;
|
|
|
|
|
|
|
|
|
|
/* Scan for all members of inheritance set, acquire needed locks */
|
|
|
|
|
inhOIDs = find_all_inheritors(parentOID, lockmode, NULL);
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Check that there's at least one descendant, else treat as no-child
|
|
|
|
|
* case. This could happen despite above has_subclass() check, if table
|
|
|
|
|
* once had a child but no longer does.
|
|
|
|
|
*/
|
|
|
|
|
if (list_length(inhOIDs) < 2)
|
|
|
|
|
{
|
|
|
|
|
/* Clear flag before returning */
|
|
|
|
|
rte->inh = false;
|
|
|
|
|
return;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* If parent relation is selected FOR UPDATE/SHARE, we need to mark its
|
|
|
|
|
* PlanRowMark as isParent = true, and generate a new PlanRowMark for each
|
|
|
|
|
* child.
|
|
|
|
|
*/
|
|
|
|
|
if (oldrc)
|
|
|
|
|
oldrc->isParent = true;
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Must open the parent relation to examine its tupdesc. We need not lock
|
|
|
|
|
* it; we assume the rewriter already did.
|
|
|
|
|
*/
|
|
|
|
|
oldrelation = heap_open(parentOID, NoLock);
|
|
|
|
|
|
|
|
|
|
/* Scan the inheritance set and expand it */
|
|
|
|
|
if (RelationGetPartitionDesc(oldrelation) != NULL)
|
|
|
|
|
{
|
|
|
|
|
Assert(rte->relkind == RELKIND_PARTITIONED_TABLE);
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* If this table has partitions, recursively expand them in the order
|
Allow UPDATE to move rows between partitions.
When an UPDATE causes a row to no longer match the partition
constraint, try to move it to a different partition where it does
match the partition constraint. In essence, the UPDATE is split into
a DELETE from the old partition and an INSERT into the new one. This
can lead to surprising behavior in concurrency scenarios because
EvalPlanQual rechecks won't work as they normally did; the known
problems are documented. (There is a pending patch to improve the
situation further, but it needs more review.)
Amit Khandekar, reviewed and tested by Amit Langote, David Rowley,
Rajkumar Raghuwanshi, Dilip Kumar, Amul Sul, Thomas Munro, Álvaro
Herrera, Amit Kapila, and me. A few final revisions by me.
Discussion: http://postgr.es/m/CAJ3gD9do9o2ccQ7j7+tSgiE1REY65XRiMb=yJO3u3QhyP8EEPQ@mail.gmail.com
8 years ago
|
|
|
* in which they appear in the PartitionDesc. While at it, also
|
|
|
|
|
* extract the partition key columns of all the partitioned tables.
|
|
|
|
|
*/
|
|
|
|
|
expand_partitioned_rtentry(root, rte, rti, oldrelation, oldrc,
|
Faster partition pruning
Add a new module backend/partitioning/partprune.c, implementing a more
sophisticated algorithm for partition pruning. The new module uses each
partition's "boundinfo" for pruning instead of constraint exclusion,
based on an idea proposed by Robert Haas of a "pruning program": a list
of steps generated from the query quals which are run iteratively to
obtain a list of partitions that must be scanned in order to satisfy
those quals.
At present, this targets planner-time partition pruning, but there exist
further patches to apply partition pruning at execution time as well.
This commit also moves some definitions from include/catalog/partition.h
to a new file include/partitioning/partbounds.h, in an attempt to
rationalize partitioning related code.
Authors: Amit Langote, David Rowley, Dilip Kumar
Reviewers: Robert Haas, Kyotaro Horiguchi, Ashutosh Bapat, Jesper Pedersen.
Discussion: https://postgr.es/m/098b9c71-1915-1a2a-8d52-1a7a50ce79e8@lab.ntt.co.jp
7 years ago
|
|
|
lockmode, &root->append_rel_list);
|
|
|
|
|
}
|
|
|
|
|
else
|
|
|
|
|
{
|
|
|
|
|
List *appinfos = NIL;
|
|
|
|
|
RangeTblEntry *childrte;
|
|
|
|
|
Index childRTindex;
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* This table has no partitions. Expand any plain inheritance
|
|
|
|
|
* children in the order the OIDs were returned by
|
|
|
|
|
* find_all_inheritors.
|
|
|
|
|
*/
|
|
|
|
|
foreach(l, inhOIDs)
|
|
|
|
|
{
|
|
|
|
|
Oid childOID = lfirst_oid(l);
|
|
|
|
|
Relation newrelation;
|
|
|
|
|
|
|
|
|
|
/* Open rel if needed; we already have required locks */
|
|
|
|
|
if (childOID != parentOID)
|
|
|
|
|
newrelation = heap_open(childOID, NoLock);
|
|
|
|
|
else
|
|
|
|
|
newrelation = oldrelation;
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* It is possible that the parent table has children that are temp
|
|
|
|
|
* tables of other backends. We cannot safely access such tables
|
|
|
|
|
* (because of buffering issues), and the best thing to do seems
|
|
|
|
|
* to be to silently ignore them.
|
|
|
|
|
*/
|
|
|
|
|
if (childOID != parentOID && RELATION_IS_OTHER_TEMP(newrelation))
|
|
|
|
|
{
|
|
|
|
|
heap_close(newrelation, lockmode);
|
|
|
|
|
continue;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
expand_single_inheritance_child(root, rte, rti, oldrelation, oldrc,
|
|
|
|
|
newrelation,
|
|
|
|
|
&appinfos, &childrte,
|
|
|
|
|
&childRTindex);
|
Improve representation of PlanRowMark.
This patch fixes two inadequacies of the PlanRowMark representation.
First, that the original LockingClauseStrength isn't stored (and cannot be
inferred for foreign tables, which always get ROW_MARK_COPY). Since some
PlanRowMarks are created out of whole cloth and don't actually have an
ancestral RowMarkClause, this requires adding a dummy LCS_NONE value to
enum LockingClauseStrength, which is fairly annoying but the alternatives
seem worse. This fix allows getting rid of the use of get_parse_rowmark()
in FDWs (as per the discussion around commits 462bd95705a0c23b and
8ec8760fc87ecde0), and it simplifies some things elsewhere.
Second, that the representation assumed that all child tables in an
inheritance hierarchy would use the same RowMarkType. That's true today
but will soon not be true. We add an "allMarkTypes" field that identifies
the union of mark types used in all a parent table's children, and use
that where appropriate (currently, only in preprocess_targetlist()).
In passing fix a couple of minor infelicities left over from the SKIP
LOCKED patch, notably that _outPlanRowMark still thought waitPolicy
is a bool.
Catversion bump is required because the numeric values of enum
LockingClauseStrength can appear in on-disk rules.
Extracted from a much larger patch to support foreign table inheritance;
it seemed worth breaking this out, since it's a separable concern.
Shigeru Hanada and Etsuro Fujita, somewhat modified by me
10 years ago
|
|
|
|
|
|
|
|
/* Close child relations, but keep locks */
|
|
|
|
|
if (childOID != parentOID)
|
|
|
|
|
heap_close(newrelation, NoLock);
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* If all the children were temp tables, pretend it's a
|
|
|
|
|
* non-inheritance situation; we don't need Append node in that case.
|
|
|
|
|
* The duplicate RTE we added for the parent table is harmless, so we
|
|
|
|
|
* don't bother to get rid of it; ditto for the useless PlanRowMark
|
|
|
|
|
* node.
|
|
|
|
|
*/
|
|
|
|
|
if (list_length(appinfos) < 2)
|
|
|
|
|
rte->inh = false;
|
|
|
|
|
else
|
|
|
|
|
root->append_rel_list = list_concat(root->append_rel_list,
|
|
|
|
|
appinfos);
|
|
|
|
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
heap_close(oldrelation, NoLock);
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* expand_partitioned_rtentry
|
|
|
|
|
* Recursively expand an RTE for a partitioned table.
|
|
|
|
|
*
|
|
|
|
|
* Note that RelationGetPartitionDispatchInfo will expand partitions in the
|
|
|
|
|
* same order as this code.
|
|
|
|
|
*/
|
|
|
|
|
static void
|
|
|
|
|
expand_partitioned_rtentry(PlannerInfo *root, RangeTblEntry *parentrte,
|
|
|
|
|
Index parentRTindex, Relation parentrel,
|
|
|
|
|
PlanRowMark *top_parentrc, LOCKMODE lockmode,
|
Faster partition pruning
Add a new module backend/partitioning/partprune.c, implementing a more
sophisticated algorithm for partition pruning. The new module uses each
partition's "boundinfo" for pruning instead of constraint exclusion,
based on an idea proposed by Robert Haas of a "pruning program": a list
of steps generated from the query quals which are run iteratively to
obtain a list of partitions that must be scanned in order to satisfy
those quals.
At present, this targets planner-time partition pruning, but there exist
further patches to apply partition pruning at execution time as well.
This commit also moves some definitions from include/catalog/partition.h
to a new file include/partitioning/partbounds.h, in an attempt to
rationalize partitioning related code.
Authors: Amit Langote, David Rowley, Dilip Kumar
Reviewers: Robert Haas, Kyotaro Horiguchi, Ashutosh Bapat, Jesper Pedersen.
Discussion: https://postgr.es/m/098b9c71-1915-1a2a-8d52-1a7a50ce79e8@lab.ntt.co.jp
7 years ago
|
|
|
List **appinfos)
|
|
|
|
|
{
|
|
|
|
|
int i;
|
|
|
|
|
RangeTblEntry *childrte;
|
|
|
|
|
Index childRTindex;
|
|
|
|
|
PartitionDesc partdesc = RelationGetPartitionDesc(parentrel);
|
|
|
|
|
|
|
|
|
|
check_stack_depth();
|
|
|
|
|
|
|
|
|
|
/* A partitioned table should always have a partition descriptor. */
|
|
|
|
|
Assert(partdesc);
|
|
|
|
|
|
|
|
|
|
Assert(parentrte->inh);
|
|
|
|
|
|
Allow UPDATE to move rows between partitions.
When an UPDATE causes a row to no longer match the partition
constraint, try to move it to a different partition where it does
match the partition constraint. In essence, the UPDATE is split into
a DELETE from the old partition and an INSERT into the new one. This
can lead to surprising behavior in concurrency scenarios because
EvalPlanQual rechecks won't work as they normally did; the known
problems are documented. (There is a pending patch to improve the
situation further, but it needs more review.)
Amit Khandekar, reviewed and tested by Amit Langote, David Rowley,
Rajkumar Raghuwanshi, Dilip Kumar, Amul Sul, Thomas Munro, Álvaro
Herrera, Amit Kapila, and me. A few final revisions by me.
Discussion: http://postgr.es/m/CAJ3gD9do9o2ccQ7j7+tSgiE1REY65XRiMb=yJO3u3QhyP8EEPQ@mail.gmail.com
8 years ago
|
|
|
/*
|
|
|
|
|
* Note down whether any partition key cols are being updated. Though it's
|
|
|
|
|
* the root partitioned table's updatedCols we are interested in, we
|
|
|
|
|
* instead use parentrte to get the updatedCols. This is convenient
|
|
|
|
|
* because parentrte already has the root partrel's updatedCols translated
|
|
|
|
|
* to match the attribute ordering of parentrel.
|
Allow UPDATE to move rows between partitions.
When an UPDATE causes a row to no longer match the partition
constraint, try to move it to a different partition where it does
match the partition constraint. In essence, the UPDATE is split into
a DELETE from the old partition and an INSERT into the new one. This
can lead to surprising behavior in concurrency scenarios because
EvalPlanQual rechecks won't work as they normally did; the known
problems are documented. (There is a pending patch to improve the
situation further, but it needs more review.)
Amit Khandekar, reviewed and tested by Amit Langote, David Rowley,
Rajkumar Raghuwanshi, Dilip Kumar, Amul Sul, Thomas Munro, Álvaro
Herrera, Amit Kapila, and me. A few final revisions by me.
Discussion: http://postgr.es/m/CAJ3gD9do9o2ccQ7j7+tSgiE1REY65XRiMb=yJO3u3QhyP8EEPQ@mail.gmail.com
8 years ago
|
|
|
*/
|
Faster partition pruning
Add a new module backend/partitioning/partprune.c, implementing a more
sophisticated algorithm for partition pruning. The new module uses each
partition's "boundinfo" for pruning instead of constraint exclusion,
based on an idea proposed by Robert Haas of a "pruning program": a list
of steps generated from the query quals which are run iteratively to
obtain a list of partitions that must be scanned in order to satisfy
those quals.
At present, this targets planner-time partition pruning, but there exist
further patches to apply partition pruning at execution time as well.
This commit also moves some definitions from include/catalog/partition.h
to a new file include/partitioning/partbounds.h, in an attempt to
rationalize partitioning related code.
Authors: Amit Langote, David Rowley, Dilip Kumar
Reviewers: Robert Haas, Kyotaro Horiguchi, Ashutosh Bapat, Jesper Pedersen.
Discussion: https://postgr.es/m/098b9c71-1915-1a2a-8d52-1a7a50ce79e8@lab.ntt.co.jp
7 years ago
|
|
|
if (!root->partColsUpdated)
|
|
|
|
|
root->partColsUpdated =
|
Allow UPDATE to move rows between partitions.
When an UPDATE causes a row to no longer match the partition
constraint, try to move it to a different partition where it does
match the partition constraint. In essence, the UPDATE is split into
a DELETE from the old partition and an INSERT into the new one. This
can lead to surprising behavior in concurrency scenarios because
EvalPlanQual rechecks won't work as they normally did; the known
problems are documented. (There is a pending patch to improve the
situation further, but it needs more review.)
Amit Khandekar, reviewed and tested by Amit Langote, David Rowley,
Rajkumar Raghuwanshi, Dilip Kumar, Amul Sul, Thomas Munro, Álvaro
Herrera, Amit Kapila, and me. A few final revisions by me.
Discussion: http://postgr.es/m/CAJ3gD9do9o2ccQ7j7+tSgiE1REY65XRiMb=yJO3u3QhyP8EEPQ@mail.gmail.com
8 years ago
|
|
|
has_partition_attrs(parentrel, parentrte->updatedCols, NULL);
|
|
|
|
|
|
|
|
|
|
/* First expand the partitioned table itself. */
|
|
|
|
|
expand_single_inheritance_child(root, parentrte, parentRTindex, parentrel,
|
|
|
|
|
top_parentrc, parentrel,
|
|
|
|
|
appinfos, &childrte, &childRTindex);
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* If the partitioned table has no partitions, treat this as the
|
|
|
|
|
* non-inheritance case.
|
|
|
|
|
*/
|
|
|
|
|
if (partdesc->nparts == 0)
|
|
|
|
|
{
|
|
|
|
|
parentrte->inh = false;
|
|
|
|
|
return;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
for (i = 0; i < partdesc->nparts; i++)
|
|
|
|
|
{
|
|
|
|
|
Oid childOID = partdesc->oids[i];
|
|
|
|
|
Relation childrel;
|
|
|
|
|
|
|
|
|
|
/* Open rel; we already have required locks */
|
|
|
|
|
childrel = heap_open(childOID, NoLock);
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Temporary partitions belonging to other sessions should have been
|
|
|
|
|
* disallowed at definition, but for paranoia's sake, let's double
|
|
|
|
|
* check.
|
|
|
|
|
*/
|
|
|
|
|
if (RELATION_IS_OTHER_TEMP(childrel))
|
|
|
|
|
elog(ERROR, "temporary relation from another session found as partition");
|
|
|
|
|
|
|
|
|
|
expand_single_inheritance_child(root, parentrte, parentRTindex,
|
|
|
|
|
parentrel, top_parentrc, childrel,
|
|
|
|
|
appinfos, &childrte, &childRTindex);
|
|
|
|
|
|
|
|
|
|
/* If this child is itself partitioned, recurse */
|
|
|
|
|
if (childrel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
|
|
|
|
|
expand_partitioned_rtentry(root, childrte, childRTindex,
|
|
|
|
|
childrel, top_parentrc, lockmode,
|
Faster partition pruning
Add a new module backend/partitioning/partprune.c, implementing a more
sophisticated algorithm for partition pruning. The new module uses each
partition's "boundinfo" for pruning instead of constraint exclusion,
based on an idea proposed by Robert Haas of a "pruning program": a list
of steps generated from the query quals which are run iteratively to
obtain a list of partitions that must be scanned in order to satisfy
those quals.
At present, this targets planner-time partition pruning, but there exist
further patches to apply partition pruning at execution time as well.
This commit also moves some definitions from include/catalog/partition.h
to a new file include/partitioning/partbounds.h, in an attempt to
rationalize partitioning related code.
Authors: Amit Langote, David Rowley, Dilip Kumar
Reviewers: Robert Haas, Kyotaro Horiguchi, Ashutosh Bapat, Jesper Pedersen.
Discussion: https://postgr.es/m/098b9c71-1915-1a2a-8d52-1a7a50ce79e8@lab.ntt.co.jp
7 years ago
|
|
|
appinfos);
|
|
|
|
|
|
|
|
|
|
/* Close child relation, but keep locks */
|
|
|
|
|
heap_close(childrel, NoLock);
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* expand_single_inheritance_child
|
|
|
|
|
* Build a RangeTblEntry and an AppendRelInfo, if appropriate, plus
|
|
|
|
|
* maybe a PlanRowMark.
|
|
|
|
|
*
|
|
|
|
|
* We now expand the partition hierarchy level by level, creating a
|
|
|
|
|
* corresponding hierarchy of AppendRelInfos and RelOptInfos, where each
|
|
|
|
|
* partitioned descendant acts as a parent of its immediate partitions.
|
|
|
|
|
* (This is a difference from what older versions of PostgreSQL did and what
|
|
|
|
|
* is still done in the case of table inheritance for unpartitioned tables,
|
|
|
|
|
* where the hierarchy is flattened during RTE expansion.)
|
|
|
|
|
*
|
|
|
|
|
* PlanRowMarks still carry the top-parent's RTI, and the top-parent's
|
|
|
|
|
* allMarkTypes field still accumulates values from all descendents.
|
|
|
|
|
*
|
|
|
|
|
* "parentrte" and "parentRTindex" are immediate parent's RTE and
|
|
|
|
|
* RTI. "top_parentrc" is top parent's PlanRowMark.
|
|
|
|
|
*
|
|
|
|
|
* The child RangeTblEntry and its RTI are returned in "childrte_p" and
|
|
|
|
|
* "childRTindex_p" resp.
|
|
|
|
|
*/
|
|
|
|
|
static void
|
|
|
|
|
expand_single_inheritance_child(PlannerInfo *root, RangeTblEntry *parentrte,
|
|
|
|
|
Index parentRTindex, Relation parentrel,
|
|
|
|
|
PlanRowMark *top_parentrc, Relation childrel,
|
|
|
|
|
List **appinfos, RangeTblEntry **childrte_p,
|
|
|
|
|
Index *childRTindex_p)
|
|
|
|
|
{
|
|
|
|
|
Query *parse = root->parse;
|
|
|
|
|
Oid parentOID = RelationGetRelid(parentrel);
|
|
|
|
|
Oid childOID = RelationGetRelid(childrel);
|
|
|
|
|
RangeTblEntry *childrte;
|
|
|
|
|
Index childRTindex;
|
|
|
|
|
AppendRelInfo *appinfo;
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Build an RTE for the child, and attach to query's rangetable list. We
|
|
|
|
|
* copy most fields of the parent's RTE, but replace relation OID and
|
|
|
|
|
* relkind, and set inh = false. Also, set requiredPerms to zero since
|
|
|
|
|
* all required permissions checks are done on the original RTE. Likewise,
|
|
|
|
|
* set the child's securityQuals to empty, because we only want to apply
|
|
|
|
|
* the parent's RLS conditions regardless of what RLS properties
|
|
|
|
|
* individual children may have. (This is an intentional choice to make
|
|
|
|
|
* inherited RLS work like regular permissions checks.) The parent
|
|
|
|
|
* securityQuals will be propagated to children along with other base
|
|
|
|
|
* restriction clauses, so we don't need to do it here.
|
|
|
|
|
*/
|
|
|
|
|
childrte = copyObject(parentrte);
|
|
|
|
|
*childrte_p = childrte;
|
|
|
|
|
childrte->relid = childOID;
|
|
|
|
|
childrte->relkind = childrel->rd_rel->relkind;
|
|
|
|
|
/* A partitioned child will need to be expanded further. */
|
|
|
|
|
if (childOID != parentOID &&
|
|
|
|
|
childrte->relkind == RELKIND_PARTITIONED_TABLE)
|
|
|
|
|
childrte->inh = true;
|
|
|
|
|
else
|
|
|
|
|
childrte->inh = false;
|
|
|
|
|
childrte->requiredPerms = 0;
|
|
|
|
|
childrte->securityQuals = NIL;
|
|
|
|
|
parse->rtable = lappend(parse->rtable, childrte);
|
|
|
|
|
childRTindex = list_length(parse->rtable);
|
|
|
|
|
*childRTindex_p = childRTindex;
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* We need an AppendRelInfo if paths will be built for the child RTE. If
|
|
|
|
|
* childrte->inh is true, then we'll always need to generate append paths
|
|
|
|
|
* for it. If childrte->inh is false, we must scan it if it's not a
|
|
|
|
|
* partitioned table; but if it is a partitioned table, then it never has
|
|
|
|
|
* any data of its own and need not be scanned.
|
|
|
|
|
*/
|
|
|
|
|
if (childrte->relkind != RELKIND_PARTITIONED_TABLE || childrte->inh)
|
|
|
|
|
{
|
|
|
|
|
appinfo = makeNode(AppendRelInfo);
|
|
|
|
|
appinfo->parent_relid = parentRTindex;
|
|
|
|
|
appinfo->child_relid = childRTindex;
|
|
|
|
|
appinfo->parent_reltype = parentrel->rd_rel->reltype;
|
|
|
|
|
appinfo->child_reltype = childrel->rd_rel->reltype;
|
|
|
|
|
make_inh_translation_list(parentrel, childrel, childRTindex,
|
|
|
|
|
&appinfo->translated_vars);
|
|
|
|
|
appinfo->parent_reloid = parentOID;
|
|
|
|
|
*appinfos = lappend(*appinfos, appinfo);
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Translate the column permissions bitmaps to the child's attnums (we
|
|
|
|
|
* have to build the translated_vars list before we can do this). But
|
|
|
|
|
* if this is the parent table, leave copyObject's result alone.
|
|
|
|
|
*
|
|
|
|
|
* Note: we need to do this even though the executor won't run any
|
|
|
|
|
* permissions checks on the child RTE. The insertedCols/updatedCols
|
|
|
|
|
* bitmaps may be examined for trigger-firing purposes.
|
|
|
|
|
*/
|
|
|
|
|
if (childOID != parentOID)
|
|
|
|
|
{
|
|
|
|
|
childrte->selectedCols = translate_col_privs(parentrte->selectedCols,
|
|
|
|
|
appinfo->translated_vars);
|
|
|
|
|
childrte->insertedCols = translate_col_privs(parentrte->insertedCols,
|
|
|
|
|
appinfo->translated_vars);
|
|
|
|
|
childrte->updatedCols = translate_col_privs(parentrte->updatedCols,
|
|
|
|
|
appinfo->translated_vars);
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Build a PlanRowMark if parent is marked FOR UPDATE/SHARE.
|
|
|
|
|
*/
|
|
|
|
|
if (top_parentrc)
|
|
|
|
|
{
|
|
|
|
|
PlanRowMark *childrc = makeNode(PlanRowMark);
|
|
|
|
|
|
|
|
|
|
childrc->rti = childRTindex;
|
|
|
|
|
childrc->prti = top_parentrc->rti;
|
|
|
|
|
childrc->rowmarkId = top_parentrc->rowmarkId;
|
|
|
|
|
/* Reselect rowmark type, because relkind might not match parent */
|
|
|
|
|
childrc->markType = select_rowmark_type(childrte,
|
|
|
|
|
top_parentrc->strength);
|
|
|
|
|
childrc->allMarkTypes = (1 << childrc->markType);
|
|
|
|
|
childrc->strength = top_parentrc->strength;
|
|
|
|
|
childrc->waitPolicy = top_parentrc->waitPolicy;
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* We mark RowMarks for partitioned child tables as parent RowMarks so
|
|
|
|
|
* that the executor ignores them (except their existence means that
|
|
|
|
|
* the child tables be locked using appropriate mode).
|
|
|
|
|
*/
|
|
|
|
|
childrc->isParent = (childrte->relkind == RELKIND_PARTITIONED_TABLE);
|
|
|
|
|
|
|
|
|
|
/* Include child's rowmark type in top parent's allMarkTypes */
|
|
|
|
|
top_parentrc->allMarkTypes |= childrc->allMarkTypes;
|
|
|
|
|
|
|
|
|
|
root->rowMarks = lappend(root->rowMarks, childrc);
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* make_inh_translation_list
|
|
|
|
|
* Build the list of translations from parent Vars to child Vars for
|
|
|
|
|
* an inheritance child.
|
|
|
|
|
*
|
|
|
|
|
* For paranoia's sake, we match type/collation as well as attribute name.
|
|
|
|
|
*/
|
|
|
|
|
static void
|
|
|
|
|
make_inh_translation_list(Relation oldrelation, Relation newrelation,
|
|
|
|
|
Index newvarno,
|
|
|
|
|
List **translated_vars)
|
|
|
|
|
{
|
|
|
|
|
List *vars = NIL;
|
|
|
|
|
TupleDesc old_tupdesc = RelationGetDescr(oldrelation);
|
|
|
|
|
TupleDesc new_tupdesc = RelationGetDescr(newrelation);
|
|
|
|
|
int oldnatts = old_tupdesc->natts;
|
|
|
|
|
int newnatts = new_tupdesc->natts;
|
|
|
|
|
int old_attno;
|
|
|
|
|
|
|
|
|
|
for (old_attno = 0; old_attno < oldnatts; old_attno++)
|
|
|
|
|
{
|
|
|
|
|
Form_pg_attribute att;
|
|
|
|
|
char *attname;
|
|
|
|
|
Oid atttypid;
|
|
|
|
|
int32 atttypmod;
|
|
|
|
|
Oid attcollation;
|
|
|
|
|
int new_attno;
|
|
|
|
|
|
|
|
|
|
att = TupleDescAttr(old_tupdesc, old_attno);
|
|
|
|
|
if (att->attisdropped)
|
|
|
|
|
{
|
|
|
|
|
/* Just put NULL into this list entry */
|
|
|
|
|
vars = lappend(vars, NULL);
|
|
|
|
|
continue;
|
|
|
|
|
}
|
|
|
|
|
attname = NameStr(att->attname);
|
|
|
|
|
atttypid = att->atttypid;
|
|
|
|
|
atttypmod = att->atttypmod;
|
|
|
|
|
attcollation = att->attcollation;
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* When we are generating the "translation list" for the parent table
|
|
|
|
|
* of an inheritance set, no need to search for matches.
|
|
|
|
|
*/
|
|
|
|
|
if (oldrelation == newrelation)
|
|
|
|
|
{
|
|
|
|
|
vars = lappend(vars, makeVar(newvarno,
|
|
|
|
|
(AttrNumber) (old_attno + 1),
|
|
|
|
|
atttypid,
|
|
|
|
|
atttypmod,
|
|
|
|
|
attcollation,
|
|
|
|
|
0));
|
|
|
|
|
continue;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Otherwise we have to search for the matching column by name.
|
|
|
|
|
* There's no guarantee it'll have the same column position, because
|
|
|
|
|
* of cases like ALTER TABLE ADD COLUMN and multiple inheritance.
|
|
|
|
|
* However, in simple cases it will be the same column number, so try
|
|
|
|
|
* that before we go groveling through all the columns.
|
|
|
|
|
*
|
|
|
|
|
* Note: the test for (att = ...) != NULL cannot fail, it's just a
|
|
|
|
|
* notational device to include the assignment into the if-clause.
|
|
|
|
|
*/
|
|
|
|
|
if (old_attno < newnatts &&
|
|
|
|
|
(att = TupleDescAttr(new_tupdesc, old_attno)) != NULL &&
|
|
|
|
|
!att->attisdropped &&
|
|
|
|
|
strcmp(attname, NameStr(att->attname)) == 0)
|
|
|
|
|
new_attno = old_attno;
|
|
|
|
|
else
|
|
|
|
|
{
|
|
|
|
|
for (new_attno = 0; new_attno < newnatts; new_attno++)
|
|
|
|
|
{
|
|
|
|
|
att = TupleDescAttr(new_tupdesc, new_attno);
|
|
|
|
|
if (!att->attisdropped &&
|
|
|
|
|
strcmp(attname, NameStr(att->attname)) == 0)
|
|
|
|
|
break;
|
|
|
|
|
}
|
|
|
|
|
if (new_attno >= newnatts)
|
|
|
|
|
elog(ERROR, "could not find inherited attribute \"%s\" of relation \"%s\"",
|
|
|
|
|
attname, RelationGetRelationName(newrelation));
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/* Found it, check type and collation match */
|
|
|
|
|
if (atttypid != att->atttypid || atttypmod != att->atttypmod)
|
|
|
|
|
elog(ERROR, "attribute \"%s\" of relation \"%s\" does not match parent's type",
|
|
|
|
|
attname, RelationGetRelationName(newrelation));
|
|
|
|
|
if (attcollation != att->attcollation)
|
|
|
|
|
elog(ERROR, "attribute \"%s\" of relation \"%s\" does not match parent's collation",
|
|
|
|
|
attname, RelationGetRelationName(newrelation));
|
|
|
|
|
|
|
|
|
|
vars = lappend(vars, makeVar(newvarno,
|
|
|
|
|
(AttrNumber) (new_attno + 1),
|
|
|
|
|
atttypid,
|
|
|
|
|
atttypmod,
|
|
|
|
|
attcollation,
|
|
|
|
|
0));
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
*translated_vars = vars;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* translate_col_privs
|
|
|
|
|
* Translate a bitmapset representing per-column privileges from the
|
|
|
|
|
* parent rel's attribute numbering to the child's.
|
|
|
|
|
*
|
|
|
|
|
* The only surprise here is that we don't translate a parent whole-row
|
|
|
|
|
* reference into a child whole-row reference. That would mean requiring
|
|
|
|
|
* permissions on all child columns, which is overly strict, since the
|
|
|
|
|
* query is really only going to reference the inherited columns. Instead
|
|
|
|
|
* we set the per-column bits for all inherited columns.
|
|
|
|
|
*/
|
|
|
|
|
static Bitmapset *
|
|
|
|
|
translate_col_privs(const Bitmapset *parent_privs,
|
|
|
|
|
List *translated_vars)
|
|
|
|
|
{
|
|
|
|
|
Bitmapset *child_privs = NULL;
|
|
|
|
|
bool whole_row;
|
|
|
|
|
int attno;
|
|
|
|
|
ListCell *lc;
|
|
|
|
|
|
|
|
|
|
/* System attributes have the same numbers in all tables */
|
|
|
|
|
for (attno = FirstLowInvalidHeapAttributeNumber + 1; attno < 0; attno++)
|
|
|
|
|
{
|
|
|
|
|
if (bms_is_member(attno - FirstLowInvalidHeapAttributeNumber,
|
|
|
|
|
parent_privs))
|
|
|
|
|
child_privs = bms_add_member(child_privs,
|
Phase 3 of pgindent updates.
Don't move parenthesized lines to the left, even if that means they
flow past the right margin.
By default, BSD indent lines up statement continuation lines that are
within parentheses so that they start just to the right of the preceding
left parenthesis. However, traditionally, if that resulted in the
continuation line extending to the right of the desired right margin,
then indent would push it left just far enough to not overrun the margin,
if it could do so without making the continuation line start to the left of
the current statement indent. That makes for a weird mix of indentations
unless one has been completely rigid about never violating the 80-column
limit.
This behavior has been pretty universally panned by Postgres developers.
Hence, disable it with indent's new -lpl switch, so that parenthesized
lines are always lined up with the preceding left paren.
This patch is much less interesting than the first round of indent
changes, but also bulkier, so I thought it best to separate the effects.
Discussion: https://postgr.es/m/E1dAmxK-0006EE-1r@gemulon.postgresql.org
Discussion: https://postgr.es/m/30527.1495162840@sss.pgh.pa.us
8 years ago
|
|
|
attno - FirstLowInvalidHeapAttributeNumber);
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/* Check if parent has whole-row reference */
|
|
|
|
|
whole_row = bms_is_member(InvalidAttrNumber - FirstLowInvalidHeapAttributeNumber,
|
|
|
|
|
parent_privs);
|
|
|
|
|
|
|
|
|
|
/* And now translate the regular user attributes, using the vars list */
|
|
|
|
|
attno = InvalidAttrNumber;
|
|
|
|
|
foreach(lc, translated_vars)
|
|
|
|
|
{
|
Improve castNode notation by introducing list-extraction-specific variants.
This extends the castNode() notation introduced by commit 5bcab1114 to
provide, in one step, extraction of a list cell's pointer and coercion to
a concrete node type. For example, "lfirst_node(Foo, lc)" is the same
as "castNode(Foo, lfirst(lc))". Almost half of the uses of castNode
that have appeared so far include a list extraction call, so this is
pretty widely useful, and it saves a few more keystrokes compared to the
old way.
As with the previous patch, back-patch the addition of these macros to
pg_list.h, so that the notation will be available when back-patching.
Patch by me, after an idea of Andrew Gierth's.
Discussion: https://postgr.es/m/14197.1491841216@sss.pgh.pa.us
8 years ago
|
|
|
Var *var = lfirst_node(Var, lc);
|
|
|
|
|
|
|
|
|
|
attno++;
|
|
|
|
|
if (var == NULL) /* ignore dropped columns */
|
|
|
|
|
continue;
|
|
|
|
|
if (whole_row ||
|
|
|
|
|
bms_is_member(attno - FirstLowInvalidHeapAttributeNumber,
|
|
|
|
|
parent_privs))
|
|
|
|
|
child_privs = bms_add_member(child_privs,
|
Phase 3 of pgindent updates.
Don't move parenthesized lines to the left, even if that means they
flow past the right margin.
By default, BSD indent lines up statement continuation lines that are
within parentheses so that they start just to the right of the preceding
left parenthesis. However, traditionally, if that resulted in the
continuation line extending to the right of the desired right margin,
then indent would push it left just far enough to not overrun the margin,
if it could do so without making the continuation line start to the left of
the current statement indent. That makes for a weird mix of indentations
unless one has been completely rigid about never violating the 80-column
limit.
This behavior has been pretty universally panned by Postgres developers.
Hence, disable it with indent's new -lpl switch, so that parenthesized
lines are always lined up with the preceding left paren.
This patch is much less interesting than the first round of indent
changes, but also bulkier, so I thought it best to separate the effects.
Discussion: https://postgr.es/m/E1dAmxK-0006EE-1r@gemulon.postgresql.org
Discussion: https://postgr.es/m/30527.1495162840@sss.pgh.pa.us
8 years ago
|
|
|
var->varattno - FirstLowInvalidHeapAttributeNumber);
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
return child_privs;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* adjust_appendrel_attrs
|
Teach adjust_appendrel_attrs(_multilevel) to do multiple translations.
Currently, child relations are always base relations, so when we
translate parent relids to child relids, we only need to translate
a singler relid. However, the proposed partition-wise join feature
will create child joins, which will mean we need to translate a set
of parent relids to the corresponding child relids. This is
preliminary refactoring to make that possible.
Ashutosh Bapat. Review and testing of the larger patch set of which
this is a part by Amit Langote, Rajkumar Raghuwanshi, Rafia Sabih,
Thomas Munro, Dilip Kumar, and me. Some adjustments, mostly
cosmetic, by me.
Discussion: http://postgr.es/m/CA+TgmobQK80vtXjAsPZWWXd7c8u13G86gmuLupN+uUJjA+i4nA@mail.gmail.com
8 years ago
|
|
|
* Copy the specified query or expression and translate Vars referring to a
|
|
|
|
|
* parent rel to refer to the corresponding child rel instead. We also
|
|
|
|
|
* update rtindexes appearing outside Vars, such as resultRelation and
|
|
|
|
|
* jointree relids.
|
|
|
|
|
*
|
Improve RLS planning by marking individual quals with security levels.
In an RLS query, we must ensure that security filter quals are evaluated
before ordinary query quals, in case the latter contain "leaky" functions
that could expose the contents of sensitive rows. The original
implementation of RLS planning ensured this by pushing the scan of a
secured table into a sub-query that it marked as a security-barrier view.
Unfortunately this results in very inefficient plans in many cases, because
the sub-query cannot be flattened and gets planned independently of the
rest of the query.
To fix, drop the use of sub-queries to enforce RLS qual order, and instead
mark each qual (RestrictInfo) with a security_level field establishing its
priority for evaluation. Quals must be evaluated in security_level order,
except that "leakproof" quals can be allowed to go ahead of quals of lower
security_level, if it's helpful to do so. This has to be enforced within
the ordering of any one list of quals to be evaluated at a table scan node,
and we also have to ensure that quals are not chosen for early evaluation
(i.e., use as an index qual or TID scan qual) if they're not allowed to go
ahead of other quals at the scan node.
This is sufficient to fix the problem for RLS quals, since we only support
RLS policies on simple tables and thus RLS quals will always exist at the
table scan level only. Eventually these qual ordering rules should be
enforced for join quals as well, which would permit improving planning for
explicit security-barrier views; but that's a task for another patch.
Note that FDWs would need to be aware of these rules --- and not, for
example, send an insecure qual for remote execution --- but since we do
not yet allow RLS policies on foreign tables, the case doesn't arise.
This will need to be addressed before we can allow such policies.
Patch by me, reviewed by Stephen Frost and Dean Rasheed.
Discussion: https://postgr.es/m/8185.1477432701@sss.pgh.pa.us
9 years ago
|
|
|
* Note: this is only applied after conversion of sublinks to subplans,
|
|
|
|
|
* so we don't need to cope with recursion into sub-queries.
|
|
|
|
|
*
|
Fix subquery pullup to wrap a PlaceHolderVar around the entire RowExpr
that's generated for a whole-row Var referencing the subquery, when the
subquery is in the nullable side of an outer join. The previous coding
instead put PlaceHolderVars around the elements of the RowExpr. The effect
was that when the outer join made the subquery outputs go to null, the
whole-row Var produced ROW(NULL,NULL,...) rather than just NULL. There
are arguments afoot about whether those things ought to be semantically
indistinguishable, but for the moment they are not entirely so, and the
planner needs to take care that its machinations preserve the difference.
Per bug #5025.
Making this feasible required refactoring ResolveNew() to allow more caller
control over what is substituted for a Var. I chose to make ResolveNew()
a wrapper around a new general-purpose function replace_rte_variables().
I also fixed the ancient bogosity that ResolveNew might fail to set
a query's hasSubLinks field after inserting a SubLink in it. Although
all current callers make sure that happens anyway, we've had bugs of that
sort before, and it seemed like a good time to install a proper solution.
Back-patch to 8.4. The problem can be demonstrated clear back to 8.0,
but the fix would be too invasive in earlier branches; not to mention
that people may be depending on the subtly-incorrect behavior. The
8.4 series is new enough that fixing this probably won't cause complaints,
but it might in older branches. Also, 8.4 shows the incorrect behavior
in more cases than older branches do, because it is able to flatten
subqueries in more cases.
16 years ago
|
|
|
* Note: this is not hugely different from what pullup_replace_vars() does;
|
|
|
|
|
* maybe we should try to fold the two routines together.
|
|
|
|
|
*/
|
|
|
|
|
Node *
|
Teach adjust_appendrel_attrs(_multilevel) to do multiple translations.
Currently, child relations are always base relations, so when we
translate parent relids to child relids, we only need to translate
a singler relid. However, the proposed partition-wise join feature
will create child joins, which will mean we need to translate a set
of parent relids to the corresponding child relids. This is
preliminary refactoring to make that possible.
Ashutosh Bapat. Review and testing of the larger patch set of which
this is a part by Amit Langote, Rajkumar Raghuwanshi, Rafia Sabih,
Thomas Munro, Dilip Kumar, and me. Some adjustments, mostly
cosmetic, by me.
Discussion: http://postgr.es/m/CA+TgmobQK80vtXjAsPZWWXd7c8u13G86gmuLupN+uUJjA+i4nA@mail.gmail.com
8 years ago
|
|
|
adjust_appendrel_attrs(PlannerInfo *root, Node *node, int nappinfos,
|
|
|
|
|
AppendRelInfo **appinfos)
|
|
|
|
|
{
|
|
|
|
|
Node *result;
|
|
|
|
|
adjust_appendrel_attrs_context context;
|
|
|
|
|
|
|
|
|
|
context.root = root;
|
Teach adjust_appendrel_attrs(_multilevel) to do multiple translations.
Currently, child relations are always base relations, so when we
translate parent relids to child relids, we only need to translate
a singler relid. However, the proposed partition-wise join feature
will create child joins, which will mean we need to translate a set
of parent relids to the corresponding child relids. This is
preliminary refactoring to make that possible.
Ashutosh Bapat. Review and testing of the larger patch set of which
this is a part by Amit Langote, Rajkumar Raghuwanshi, Rafia Sabih,
Thomas Munro, Dilip Kumar, and me. Some adjustments, mostly
cosmetic, by me.
Discussion: http://postgr.es/m/CA+TgmobQK80vtXjAsPZWWXd7c8u13G86gmuLupN+uUJjA+i4nA@mail.gmail.com
8 years ago
|
|
|
context.nappinfos = nappinfos;
|
|
|
|
|
context.appinfos = appinfos;
|
|
|
|
|
|
|
|
|
|
/* If there's nothing to adjust, don't call this function. */
|
|
|
|
|
Assert(nappinfos >= 1 && appinfos != NULL);
|
|
|
|
|
|
|
|
|
|
/*
|
Improve RLS planning by marking individual quals with security levels.
In an RLS query, we must ensure that security filter quals are evaluated
before ordinary query quals, in case the latter contain "leaky" functions
that could expose the contents of sensitive rows. The original
implementation of RLS planning ensured this by pushing the scan of a
secured table into a sub-query that it marked as a security-barrier view.
Unfortunately this results in very inefficient plans in many cases, because
the sub-query cannot be flattened and gets planned independently of the
rest of the query.
To fix, drop the use of sub-queries to enforce RLS qual order, and instead
mark each qual (RestrictInfo) with a security_level field establishing its
priority for evaluation. Quals must be evaluated in security_level order,
except that "leakproof" quals can be allowed to go ahead of quals of lower
security_level, if it's helpful to do so. This has to be enforced within
the ordering of any one list of quals to be evaluated at a table scan node,
and we also have to ensure that quals are not chosen for early evaluation
(i.e., use as an index qual or TID scan qual) if they're not allowed to go
ahead of other quals at the scan node.
This is sufficient to fix the problem for RLS quals, since we only support
RLS policies on simple tables and thus RLS quals will always exist at the
table scan level only. Eventually these qual ordering rules should be
enforced for join quals as well, which would permit improving planning for
explicit security-barrier views; but that's a task for another patch.
Note that FDWs would need to be aware of these rules --- and not, for
example, send an insecure qual for remote execution --- but since we do
not yet allow RLS policies on foreign tables, the case doesn't arise.
This will need to be addressed before we can allow such policies.
Patch by me, reviewed by Stephen Frost and Dean Rasheed.
Discussion: https://postgr.es/m/8185.1477432701@sss.pgh.pa.us
9 years ago
|
|
|
* Must be prepared to start with a Query or a bare expression tree.
|
|
|
|
|
*/
|
|
|
|
|
if (node && IsA(node, Query))
|
|
|
|
|
{
|
|
|
|
|
Query *newnode;
|
Teach adjust_appendrel_attrs(_multilevel) to do multiple translations.
Currently, child relations are always base relations, so when we
translate parent relids to child relids, we only need to translate
a singler relid. However, the proposed partition-wise join feature
will create child joins, which will mean we need to translate a set
of parent relids to the corresponding child relids. This is
preliminary refactoring to make that possible.
Ashutosh Bapat. Review and testing of the larger patch set of which
this is a part by Amit Langote, Rajkumar Raghuwanshi, Rafia Sabih,
Thomas Munro, Dilip Kumar, and me. Some adjustments, mostly
cosmetic, by me.
Discussion: http://postgr.es/m/CA+TgmobQK80vtXjAsPZWWXd7c8u13G86gmuLupN+uUJjA+i4nA@mail.gmail.com
8 years ago
|
|
|
int cnt;
|
|
|
|
|
|
|
|
|
|
newnode = query_tree_mutator((Query *) node,
|
|
|
|
|
adjust_appendrel_attrs_mutator,
|
|
|
|
|
(void *) &context,
|
|
|
|
|
QTW_IGNORE_RC_SUBQUERIES);
|
Teach adjust_appendrel_attrs(_multilevel) to do multiple translations.
Currently, child relations are always base relations, so when we
translate parent relids to child relids, we only need to translate
a singler relid. However, the proposed partition-wise join feature
will create child joins, which will mean we need to translate a set
of parent relids to the corresponding child relids. This is
preliminary refactoring to make that possible.
Ashutosh Bapat. Review and testing of the larger patch set of which
this is a part by Amit Langote, Rajkumar Raghuwanshi, Rafia Sabih,
Thomas Munro, Dilip Kumar, and me. Some adjustments, mostly
cosmetic, by me.
Discussion: http://postgr.es/m/CA+TgmobQK80vtXjAsPZWWXd7c8u13G86gmuLupN+uUJjA+i4nA@mail.gmail.com
8 years ago
|
|
|
for (cnt = 0; cnt < nappinfos; cnt++)
|
|
|
|
|
{
|
Teach adjust_appendrel_attrs(_multilevel) to do multiple translations.
Currently, child relations are always base relations, so when we
translate parent relids to child relids, we only need to translate
a singler relid. However, the proposed partition-wise join feature
will create child joins, which will mean we need to translate a set
of parent relids to the corresponding child relids. This is
preliminary refactoring to make that possible.
Ashutosh Bapat. Review and testing of the larger patch set of which
this is a part by Amit Langote, Rajkumar Raghuwanshi, Rafia Sabih,
Thomas Munro, Dilip Kumar, and me. Some adjustments, mostly
cosmetic, by me.
Discussion: http://postgr.es/m/CA+TgmobQK80vtXjAsPZWWXd7c8u13G86gmuLupN+uUJjA+i4nA@mail.gmail.com
8 years ago
|
|
|
AppendRelInfo *appinfo = appinfos[cnt];
|
|
|
|
|
|
|
|
|
|
if (newnode->resultRelation == appinfo->parent_relid)
|
|
|
|
|
{
|
|
|
|
|
newnode->resultRelation = appinfo->child_relid;
|
|
|
|
|
/* Fix tlist resnos too, if it's inherited UPDATE */
|
|
|
|
|
if (newnode->commandType == CMD_UPDATE)
|
|
|
|
|
newnode->targetList =
|
|
|
|
|
adjust_inherited_tlist(newnode->targetList,
|
|
|
|
|
appinfo);
|
|
|
|
|
break;
|
|
|
|
|
}
|
|
|
|
|
}
|
Teach adjust_appendrel_attrs(_multilevel) to do multiple translations.
Currently, child relations are always base relations, so when we
translate parent relids to child relids, we only need to translate
a singler relid. However, the proposed partition-wise join feature
will create child joins, which will mean we need to translate a set
of parent relids to the corresponding child relids. This is
preliminary refactoring to make that possible.
Ashutosh Bapat. Review and testing of the larger patch set of which
this is a part by Amit Langote, Rajkumar Raghuwanshi, Rafia Sabih,
Thomas Munro, Dilip Kumar, and me. Some adjustments, mostly
cosmetic, by me.
Discussion: http://postgr.es/m/CA+TgmobQK80vtXjAsPZWWXd7c8u13G86gmuLupN+uUJjA+i4nA@mail.gmail.com
8 years ago
|
|
|
|
|
|
|
|
result = (Node *) newnode;
|
|
|
|
|
}
|
|
|
|
|
else
|
|
|
|
|
result = adjust_appendrel_attrs_mutator(node, &context);
|
|
|
|
|
|
|
|
|
|
return result;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
static Node *
|
|
|
|
|
adjust_appendrel_attrs_mutator(Node *node,
|
|
|
|
|
adjust_appendrel_attrs_context *context)
|
|
|
|
|
{
|
Teach adjust_appendrel_attrs(_multilevel) to do multiple translations.
Currently, child relations are always base relations, so when we
translate parent relids to child relids, we only need to translate
a singler relid. However, the proposed partition-wise join feature
will create child joins, which will mean we need to translate a set
of parent relids to the corresponding child relids. This is
preliminary refactoring to make that possible.
Ashutosh Bapat. Review and testing of the larger patch set of which
this is a part by Amit Langote, Rajkumar Raghuwanshi, Rafia Sabih,
Thomas Munro, Dilip Kumar, and me. Some adjustments, mostly
cosmetic, by me.
Discussion: http://postgr.es/m/CA+TgmobQK80vtXjAsPZWWXd7c8u13G86gmuLupN+uUJjA+i4nA@mail.gmail.com
8 years ago
|
|
|
AppendRelInfo **appinfos = context->appinfos;
|
|
|
|
|
int nappinfos = context->nappinfos;
|
|
|
|
|
int cnt;
|
|
|
|
|
|
|
|
|
|
if (node == NULL)
|
|
|
|
|
return NULL;
|
|
|
|
|
if (IsA(node, Var))
|
|
|
|
|
{
|
|
|
|
|
Var *var = (Var *) copyObject(node);
|
Teach adjust_appendrel_attrs(_multilevel) to do multiple translations.
Currently, child relations are always base relations, so when we
translate parent relids to child relids, we only need to translate
a singler relid. However, the proposed partition-wise join feature
will create child joins, which will mean we need to translate a set
of parent relids to the corresponding child relids. This is
preliminary refactoring to make that possible.
Ashutosh Bapat. Review and testing of the larger patch set of which
this is a part by Amit Langote, Rajkumar Raghuwanshi, Rafia Sabih,
Thomas Munro, Dilip Kumar, and me. Some adjustments, mostly
cosmetic, by me.
Discussion: http://postgr.es/m/CA+TgmobQK80vtXjAsPZWWXd7c8u13G86gmuLupN+uUJjA+i4nA@mail.gmail.com
8 years ago
|
|
|
AppendRelInfo *appinfo = NULL;
|
|
|
|
|
|
Teach adjust_appendrel_attrs(_multilevel) to do multiple translations.
Currently, child relations are always base relations, so when we
translate parent relids to child relids, we only need to translate
a singler relid. However, the proposed partition-wise join feature
will create child joins, which will mean we need to translate a set
of parent relids to the corresponding child relids. This is
preliminary refactoring to make that possible.
Ashutosh Bapat. Review and testing of the larger patch set of which
this is a part by Amit Langote, Rajkumar Raghuwanshi, Rafia Sabih,
Thomas Munro, Dilip Kumar, and me. Some adjustments, mostly
cosmetic, by me.
Discussion: http://postgr.es/m/CA+TgmobQK80vtXjAsPZWWXd7c8u13G86gmuLupN+uUJjA+i4nA@mail.gmail.com
8 years ago
|
|
|
for (cnt = 0; cnt < nappinfos; cnt++)
|
|
|
|
|
{
|
|
|
|
|
if (var->varno == appinfos[cnt]->parent_relid)
|
|
|
|
|
{
|
|
|
|
|
appinfo = appinfos[cnt];
|
|
|
|
|
break;
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
if (var->varlevelsup == 0 && appinfo)
|
|
|
|
|
{
|
|
|
|
|
var->varno = appinfo->child_relid;
|
|
|
|
|
var->varnoold = appinfo->child_relid;
|
|
|
|
|
if (var->varattno > 0)
|
|
|
|
|
{
|
|
|
|
|
Node *newnode;
|
|
|
|
|
|
|
|
|
|
if (var->varattno > list_length(appinfo->translated_vars))
|
|
|
|
|
elog(ERROR, "attribute %d of relation \"%s\" does not exist",
|
|
|
|
|
var->varattno, get_rel_name(appinfo->parent_reloid));
|
|
|
|
|
newnode = copyObject(list_nth(appinfo->translated_vars,
|
|
|
|
|
var->varattno - 1));
|
|
|
|
|
if (newnode == NULL)
|
|
|
|
|
elog(ERROR, "attribute %d of relation \"%s\" does not exist",
|
|
|
|
|
var->varattno, get_rel_name(appinfo->parent_reloid));
|
|
|
|
|
return newnode;
|
|
|
|
|
}
|
|
|
|
|
else if (var->varattno == 0)
|
|
|
|
|
{
|
|
|
|
|
/*
|
|
|
|
|
* Whole-row Var: if we are dealing with named rowtypes, we
|
|
|
|
|
* can use a whole-row Var for the child table plus a coercion
|
|
|
|
|
* step to convert the tuple layout to the parent's rowtype.
|
|
|
|
|
* Otherwise we have to generate a RowExpr.
|
|
|
|
|
*/
|
|
|
|
|
if (OidIsValid(appinfo->child_reltype))
|
|
|
|
|
{
|
|
|
|
|
Assert(var->vartype == appinfo->parent_reltype);
|
|
|
|
|
if (appinfo->parent_reltype != appinfo->child_reltype)
|
|
|
|
|
{
|
|
|
|
|
ConvertRowtypeExpr *r = makeNode(ConvertRowtypeExpr);
|
|
|
|
|
|
|
|
|
|
r->arg = (Expr *) var;
|
|
|
|
|
r->resulttype = appinfo->parent_reltype;
|
|
|
|
|
r->convertformat = COERCE_IMPLICIT_CAST;
|
|
|
|
|
r->location = -1;
|
|
|
|
|
/* Make sure the Var node has the right type ID, too */
|
|
|
|
|
var->vartype = appinfo->child_reltype;
|
|
|
|
|
return (Node *) r;
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
else
|
|
|
|
|
{
|
|
|
|
|
/*
|
|
|
|
|
* Build a RowExpr containing the translated variables.
|
|
|
|
|
*
|
|
|
|
|
* In practice var->vartype will always be RECORDOID here,
|
|
|
|
|
* so we need to come up with some suitable column names.
|
|
|
|
|
* We use the parent RTE's column names.
|
|
|
|
|
*
|
|
|
|
|
* Note: we can't get here for inheritance cases, so there
|
|
|
|
|
* is no need to worry that translated_vars might contain
|
|
|
|
|
* some dummy NULLs.
|
|
|
|
|
*/
|
|
|
|
|
RowExpr *rowexpr;
|
|
|
|
|
List *fields;
|
|
|
|
|
RangeTblEntry *rte;
|
|
|
|
|
|
|
|
|
|
rte = rt_fetch(appinfo->parent_relid,
|
|
|
|
|
context->root->parse->rtable);
|
|
|
|
|
fields = copyObject(appinfo->translated_vars);
|
|
|
|
|
rowexpr = makeNode(RowExpr);
|
|
|
|
|
rowexpr->args = fields;
|
|
|
|
|
rowexpr->row_typeid = var->vartype;
|
|
|
|
|
rowexpr->row_format = COERCE_IMPLICIT_CAST;
|
|
|
|
|
rowexpr->colnames = copyObject(rte->eref->colnames);
|
|
|
|
|
rowexpr->location = -1;
|
|
|
|
|
|
|
|
|
|
return (Node *) rowexpr;
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
/* system attributes don't need any other translation */
|
|
|
|
|
}
|
|
|
|
|
return (Node *) var;
|
|
|
|
|
}
|
|
|
|
|
if (IsA(node, CurrentOfExpr))
|
|
|
|
|
{
|
|
|
|
|
CurrentOfExpr *cexpr = (CurrentOfExpr *) copyObject(node);
|
|
|
|
|
|
Teach adjust_appendrel_attrs(_multilevel) to do multiple translations.
Currently, child relations are always base relations, so when we
translate parent relids to child relids, we only need to translate
a singler relid. However, the proposed partition-wise join feature
will create child joins, which will mean we need to translate a set
of parent relids to the corresponding child relids. This is
preliminary refactoring to make that possible.
Ashutosh Bapat. Review and testing of the larger patch set of which
this is a part by Amit Langote, Rajkumar Raghuwanshi, Rafia Sabih,
Thomas Munro, Dilip Kumar, and me. Some adjustments, mostly
cosmetic, by me.
Discussion: http://postgr.es/m/CA+TgmobQK80vtXjAsPZWWXd7c8u13G86gmuLupN+uUJjA+i4nA@mail.gmail.com
8 years ago
|
|
|
for (cnt = 0; cnt < nappinfos; cnt++)
|
|
|
|
|
{
|
|
|
|
|
AppendRelInfo *appinfo = appinfos[cnt];
|
|
|
|
|
|
|
|
|
|
if (cexpr->cvarno == appinfo->parent_relid)
|
|
|
|
|
{
|
|
|
|
|
cexpr->cvarno = appinfo->child_relid;
|
|
|
|
|
break;
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
return (Node *) cexpr;
|
|
|
|
|
}
|
|
|
|
|
if (IsA(node, RangeTblRef))
|
|
|
|
|
{
|
|
|
|
|
RangeTblRef *rtr = (RangeTblRef *) copyObject(node);
|
|
|
|
|
|
Teach adjust_appendrel_attrs(_multilevel) to do multiple translations.
Currently, child relations are always base relations, so when we
translate parent relids to child relids, we only need to translate
a singler relid. However, the proposed partition-wise join feature
will create child joins, which will mean we need to translate a set
of parent relids to the corresponding child relids. This is
preliminary refactoring to make that possible.
Ashutosh Bapat. Review and testing of the larger patch set of which
this is a part by Amit Langote, Rajkumar Raghuwanshi, Rafia Sabih,
Thomas Munro, Dilip Kumar, and me. Some adjustments, mostly
cosmetic, by me.
Discussion: http://postgr.es/m/CA+TgmobQK80vtXjAsPZWWXd7c8u13G86gmuLupN+uUJjA+i4nA@mail.gmail.com
8 years ago
|
|
|
for (cnt = 0; cnt < nappinfos; cnt++)
|
|
|
|
|
{
|
|
|
|
|
AppendRelInfo *appinfo = appinfos[cnt];
|
|
|
|
|
|
|
|
|
|
if (rtr->rtindex == appinfo->parent_relid)
|
|
|
|
|
{
|
|
|
|
|
rtr->rtindex = appinfo->child_relid;
|
|
|
|
|
break;
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
return (Node *) rtr;
|
|
|
|
|
}
|
|
|
|
|
if (IsA(node, JoinExpr))
|
|
|
|
|
{
|
|
|
|
|
/* Copy the JoinExpr node with correct mutation of subnodes */
|
|
|
|
|
JoinExpr *j;
|
Teach adjust_appendrel_attrs(_multilevel) to do multiple translations.
Currently, child relations are always base relations, so when we
translate parent relids to child relids, we only need to translate
a singler relid. However, the proposed partition-wise join feature
will create child joins, which will mean we need to translate a set
of parent relids to the corresponding child relids. This is
preliminary refactoring to make that possible.
Ashutosh Bapat. Review and testing of the larger patch set of which
this is a part by Amit Langote, Rajkumar Raghuwanshi, Rafia Sabih,
Thomas Munro, Dilip Kumar, and me. Some adjustments, mostly
cosmetic, by me.
Discussion: http://postgr.es/m/CA+TgmobQK80vtXjAsPZWWXd7c8u13G86gmuLupN+uUJjA+i4nA@mail.gmail.com
8 years ago
|
|
|
AppendRelInfo *appinfo;
|
|
|
|
|
|
|
|
|
|
j = (JoinExpr *) expression_tree_mutator(node,
|
Phase 3 of pgindent updates.
Don't move parenthesized lines to the left, even if that means they
flow past the right margin.
By default, BSD indent lines up statement continuation lines that are
within parentheses so that they start just to the right of the preceding
left parenthesis. However, traditionally, if that resulted in the
continuation line extending to the right of the desired right margin,
then indent would push it left just far enough to not overrun the margin,
if it could do so without making the continuation line start to the left of
the current statement indent. That makes for a weird mix of indentations
unless one has been completely rigid about never violating the 80-column
limit.
This behavior has been pretty universally panned by Postgres developers.
Hence, disable it with indent's new -lpl switch, so that parenthesized
lines are always lined up with the preceding left paren.
This patch is much less interesting than the first round of indent
changes, but also bulkier, so I thought it best to separate the effects.
Discussion: https://postgr.es/m/E1dAmxK-0006EE-1r@gemulon.postgresql.org
Discussion: https://postgr.es/m/30527.1495162840@sss.pgh.pa.us
8 years ago
|
|
|
adjust_appendrel_attrs_mutator,
|
|
|
|
|
(void *) context);
|
|
|
|
|
/* now fix JoinExpr's rtindex (probably never happens) */
|
Teach adjust_appendrel_attrs(_multilevel) to do multiple translations.
Currently, child relations are always base relations, so when we
translate parent relids to child relids, we only need to translate
a singler relid. However, the proposed partition-wise join feature
will create child joins, which will mean we need to translate a set
of parent relids to the corresponding child relids. This is
preliminary refactoring to make that possible.
Ashutosh Bapat. Review and testing of the larger patch set of which
this is a part by Amit Langote, Rajkumar Raghuwanshi, Rafia Sabih,
Thomas Munro, Dilip Kumar, and me. Some adjustments, mostly
cosmetic, by me.
Discussion: http://postgr.es/m/CA+TgmobQK80vtXjAsPZWWXd7c8u13G86gmuLupN+uUJjA+i4nA@mail.gmail.com
8 years ago
|
|
|
for (cnt = 0; cnt < nappinfos; cnt++)
|
|
|
|
|
{
|
|
|
|
|
appinfo = appinfos[cnt];
|
|
|
|
|
|
|
|
|
|
if (j->rtindex == appinfo->parent_relid)
|
|
|
|
|
{
|
|
|
|
|
j->rtindex = appinfo->child_relid;
|
|
|
|
|
break;
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
return (Node *) j;
|
|
|
|
|
}
|
|
|
|
|
if (IsA(node, PlaceHolderVar))
|
|
|
|
|
{
|
|
|
|
|
/* Copy the PlaceHolderVar node with correct mutation of subnodes */
|
|
|
|
|
PlaceHolderVar *phv;
|
|
|
|
|
|
|
|
|
|
phv = (PlaceHolderVar *) expression_tree_mutator(node,
|
Phase 3 of pgindent updates.
Don't move parenthesized lines to the left, even if that means they
flow past the right margin.
By default, BSD indent lines up statement continuation lines that are
within parentheses so that they start just to the right of the preceding
left parenthesis. However, traditionally, if that resulted in the
continuation line extending to the right of the desired right margin,
then indent would push it left just far enough to not overrun the margin,
if it could do so without making the continuation line start to the left of
the current statement indent. That makes for a weird mix of indentations
unless one has been completely rigid about never violating the 80-column
limit.
This behavior has been pretty universally panned by Postgres developers.
Hence, disable it with indent's new -lpl switch, so that parenthesized
lines are always lined up with the preceding left paren.
This patch is much less interesting than the first round of indent
changes, but also bulkier, so I thought it best to separate the effects.
Discussion: https://postgr.es/m/E1dAmxK-0006EE-1r@gemulon.postgresql.org
Discussion: https://postgr.es/m/30527.1495162840@sss.pgh.pa.us
8 years ago
|
|
|
adjust_appendrel_attrs_mutator,
|
|
|
|
|
(void *) context);
|
|
|
|
|
/* now fix PlaceHolderVar's relid sets */
|
Improve RLS planning by marking individual quals with security levels.
In an RLS query, we must ensure that security filter quals are evaluated
before ordinary query quals, in case the latter contain "leaky" functions
that could expose the contents of sensitive rows. The original
implementation of RLS planning ensured this by pushing the scan of a
secured table into a sub-query that it marked as a security-barrier view.
Unfortunately this results in very inefficient plans in many cases, because
the sub-query cannot be flattened and gets planned independently of the
rest of the query.
To fix, drop the use of sub-queries to enforce RLS qual order, and instead
mark each qual (RestrictInfo) with a security_level field establishing its
priority for evaluation. Quals must be evaluated in security_level order,
except that "leakproof" quals can be allowed to go ahead of quals of lower
security_level, if it's helpful to do so. This has to be enforced within
the ordering of any one list of quals to be evaluated at a table scan node,
and we also have to ensure that quals are not chosen for early evaluation
(i.e., use as an index qual or TID scan qual) if they're not allowed to go
ahead of other quals at the scan node.
This is sufficient to fix the problem for RLS quals, since we only support
RLS policies on simple tables and thus RLS quals will always exist at the
table scan level only. Eventually these qual ordering rules should be
enforced for join quals as well, which would permit improving planning for
explicit security-barrier views; but that's a task for another patch.
Note that FDWs would need to be aware of these rules --- and not, for
example, send an insecure qual for remote execution --- but since we do
not yet allow RLS policies on foreign tables, the case doesn't arise.
This will need to be addressed before we can allow such policies.
Patch by me, reviewed by Stephen Frost and Dean Rasheed.
Discussion: https://postgr.es/m/8185.1477432701@sss.pgh.pa.us
9 years ago
|
|
|
if (phv->phlevelsup == 0)
|
Teach adjust_appendrel_attrs(_multilevel) to do multiple translations.
Currently, child relations are always base relations, so when we
translate parent relids to child relids, we only need to translate
a singler relid. However, the proposed partition-wise join feature
will create child joins, which will mean we need to translate a set
of parent relids to the corresponding child relids. This is
preliminary refactoring to make that possible.
Ashutosh Bapat. Review and testing of the larger patch set of which
this is a part by Amit Langote, Rajkumar Raghuwanshi, Rafia Sabih,
Thomas Munro, Dilip Kumar, and me. Some adjustments, mostly
cosmetic, by me.
Discussion: http://postgr.es/m/CA+TgmobQK80vtXjAsPZWWXd7c8u13G86gmuLupN+uUJjA+i4nA@mail.gmail.com
8 years ago
|
|
|
phv->phrels = adjust_child_relids(phv->phrels, context->nappinfos,
|
|
|
|
|
context->appinfos);
|
|
|
|
|
return (Node *) phv;
|
|
|
|
|
}
|
|
|
|
|
/* Shouldn't need to handle planner auxiliary nodes here */
|
|
|
|
|
Assert(!IsA(node, SpecialJoinInfo));
|
|
|
|
|
Assert(!IsA(node, AppendRelInfo));
|
|
|
|
|
Assert(!IsA(node, PlaceHolderInfo));
|
|
|
|
|
Assert(!IsA(node, MinMaxAggInfo));
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* We have to process RestrictInfo nodes specially. (Note: although
|
|
|
|
|
* set_append_rel_pathlist will hide RestrictInfos in the parent's
|
|
|
|
|
* baserestrictinfo list from us, it doesn't hide those in joininfo.)
|
|
|
|
|
*/
|
|
|
|
|
if (IsA(node, RestrictInfo))
|
|
|
|
|
{
|
|
|
|
|
RestrictInfo *oldinfo = (RestrictInfo *) node;
|
|
|
|
|
RestrictInfo *newinfo = makeNode(RestrictInfo);
|
|
|
|
|
|
|
|
|
|
/* Copy all flat-copiable fields */
|
|
|
|
|
memcpy(newinfo, oldinfo, sizeof(RestrictInfo));
|
|
|
|
|
|
|
|
|
|
/* Recursively fix the clause itself */
|
|
|
|
|
newinfo->clause = (Expr *)
|
|
|
|
|
adjust_appendrel_attrs_mutator((Node *) oldinfo->clause, context);
|
|
|
|
|
|
|
|
|
|
/* and the modified version, if an OR clause */
|
|
|
|
|
newinfo->orclause = (Expr *)
|
|
|
|
|
adjust_appendrel_attrs_mutator((Node *) oldinfo->orclause, context);
|
|
|
|
|
|
|
|
|
|
/* adjust relid sets too */
|
Teach adjust_appendrel_attrs(_multilevel) to do multiple translations.
Currently, child relations are always base relations, so when we
translate parent relids to child relids, we only need to translate
a singler relid. However, the proposed partition-wise join feature
will create child joins, which will mean we need to translate a set
of parent relids to the corresponding child relids. This is
preliminary refactoring to make that possible.
Ashutosh Bapat. Review and testing of the larger patch set of which
this is a part by Amit Langote, Rajkumar Raghuwanshi, Rafia Sabih,
Thomas Munro, Dilip Kumar, and me. Some adjustments, mostly
cosmetic, by me.
Discussion: http://postgr.es/m/CA+TgmobQK80vtXjAsPZWWXd7c8u13G86gmuLupN+uUJjA+i4nA@mail.gmail.com
8 years ago
|
|
|
newinfo->clause_relids = adjust_child_relids(oldinfo->clause_relids,
|
|
|
|
|
context->nappinfos,
|
|
|
|
|
context->appinfos);
|
|
|
|
|
newinfo->required_relids = adjust_child_relids(oldinfo->required_relids,
|
|
|
|
|
context->nappinfos,
|
|
|
|
|
context->appinfos);
|
|
|
|
|
newinfo->outer_relids = adjust_child_relids(oldinfo->outer_relids,
|
|
|
|
|
context->nappinfos,
|
|
|
|
|
context->appinfos);
|
|
|
|
|
newinfo->nullable_relids = adjust_child_relids(oldinfo->nullable_relids,
|
|
|
|
|
context->nappinfos,
|
|
|
|
|
context->appinfos);
|
|
|
|
|
newinfo->left_relids = adjust_child_relids(oldinfo->left_relids,
|
|
|
|
|
context->nappinfos,
|
|
|
|
|
context->appinfos);
|
|
|
|
|
newinfo->right_relids = adjust_child_relids(oldinfo->right_relids,
|
|
|
|
|
context->nappinfos,
|
|
|
|
|
context->appinfos);
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Reset cached derivative fields, since these might need to have
|
|
|
|
|
* different values when considering the child relation. Note we
|
|
|
|
|
* don't reset left_ec/right_ec: each child variable is implicitly
|
|
|
|
|
* equivalent to its parent, so still a member of the same EC if any.
|
|
|
|
|
*/
|
|
|
|
|
newinfo->eval_cost.startup = -1;
|
|
|
|
|
newinfo->norm_selec = -1;
|
|
|
|
|
newinfo->outer_selec = -1;
|
|
|
|
|
newinfo->left_em = NULL;
|
|
|
|
|
newinfo->right_em = NULL;
|
|
|
|
|
newinfo->scansel_cache = NIL;
|
|
|
|
|
newinfo->left_bucketsize = -1;
|
|
|
|
|
newinfo->right_bucketsize = -1;
|
Avoid out-of-memory in a hash join with many duplicate inner keys.
The executor is capable of splitting buckets during a hash join if
too much memory is being used by a small number of buckets. However,
this only helps if a bucket's population is actually divisible; if
all the hash keys are alike, the tuples still end up in the same
new bucket. This can result in an OOM failure if there are enough
inner keys with identical hash values. The planner's cost estimates
will bias it against choosing a hash join in such situations, but not
by so much that it will never do so. To mitigate the OOM hazard,
explicitly estimate the hash bucket space needed by just the inner
side's most common value, and if that would exceed work_mem then
add disable_cost to the hash cost estimate.
This approach doesn't account for the possibility that two or more
common values would share the same hash value. On the other hand,
work_mem is normally a fairly conservative bound, so that eating
two or more times that much space is probably not going to kill us.
If we have no stats about the inner side, ignore this consideration.
There was some discussion of making a conservative assumption, but that
would effectively result in disabling hash join whenever we lack stats,
which seems like an overreaction given how seldom the problem manifests
in the field.
Per a complaint from David Hinkle. Although this could be viewed
as a bug fix, the lack of similar complaints weighs against back-
patching; indeed we waited for v11 because it seemed already rather
late in the v10 cycle to be making plan choice changes like this one.
Discussion: https://postgr.es/m/32013.1487271761@sss.pgh.pa.us
8 years ago
|
|
|
newinfo->left_mcvfreq = -1;
|
|
|
|
|
newinfo->right_mcvfreq = -1;
|
|
|
|
|
|
|
|
|
|
return (Node *) newinfo;
|
|
|
|
|
}
|
|
|
|
|
|
Improve RLS planning by marking individual quals with security levels.
In an RLS query, we must ensure that security filter quals are evaluated
before ordinary query quals, in case the latter contain "leaky" functions
that could expose the contents of sensitive rows. The original
implementation of RLS planning ensured this by pushing the scan of a
secured table into a sub-query that it marked as a security-barrier view.
Unfortunately this results in very inefficient plans in many cases, because
the sub-query cannot be flattened and gets planned independently of the
rest of the query.
To fix, drop the use of sub-queries to enforce RLS qual order, and instead
mark each qual (RestrictInfo) with a security_level field establishing its
priority for evaluation. Quals must be evaluated in security_level order,
except that "leakproof" quals can be allowed to go ahead of quals of lower
security_level, if it's helpful to do so. This has to be enforced within
the ordering of any one list of quals to be evaluated at a table scan node,
and we also have to ensure that quals are not chosen for early evaluation
(i.e., use as an index qual or TID scan qual) if they're not allowed to go
ahead of other quals at the scan node.
This is sufficient to fix the problem for RLS quals, since we only support
RLS policies on simple tables and thus RLS quals will always exist at the
table scan level only. Eventually these qual ordering rules should be
enforced for join quals as well, which would permit improving planning for
explicit security-barrier views; but that's a task for another patch.
Note that FDWs would need to be aware of these rules --- and not, for
example, send an insecure qual for remote execution --- but since we do
not yet allow RLS policies on foreign tables, the case doesn't arise.
This will need to be addressed before we can allow such policies.
Patch by me, reviewed by Stephen Frost and Dean Rasheed.
Discussion: https://postgr.es/m/8185.1477432701@sss.pgh.pa.us
9 years ago
|
|
|
/*
|
|
|
|
|
* NOTE: we do not need to recurse into sublinks, because they should
|
|
|
|
|
* already have been converted to subplans before we see them.
|
|
|
|
|
*/
|
|
|
|
|
Assert(!IsA(node, SubLink));
|
|
|
|
|
Assert(!IsA(node, Query));
|
|
|
|
|
|
|
|
|
|
return expression_tree_mutator(node, adjust_appendrel_attrs_mutator,
|
|
|
|
|
(void *) context);
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/*
|
Teach adjust_appendrel_attrs(_multilevel) to do multiple translations.
Currently, child relations are always base relations, so when we
translate parent relids to child relids, we only need to translate
a singler relid. However, the proposed partition-wise join feature
will create child joins, which will mean we need to translate a set
of parent relids to the corresponding child relids. This is
preliminary refactoring to make that possible.
Ashutosh Bapat. Review and testing of the larger patch set of which
this is a part by Amit Langote, Rajkumar Raghuwanshi, Rafia Sabih,
Thomas Munro, Dilip Kumar, and me. Some adjustments, mostly
cosmetic, by me.
Discussion: http://postgr.es/m/CA+TgmobQK80vtXjAsPZWWXd7c8u13G86gmuLupN+uUJjA+i4nA@mail.gmail.com
8 years ago
|
|
|
* Substitute child relids for parent relids in a Relid set. The array of
|
|
|
|
|
* appinfos specifies the substitutions to be performed.
|
|
|
|
|
*/
|
|
|
|
|
static Relids
|
Teach adjust_appendrel_attrs(_multilevel) to do multiple translations.
Currently, child relations are always base relations, so when we
translate parent relids to child relids, we only need to translate
a singler relid. However, the proposed partition-wise join feature
will create child joins, which will mean we need to translate a set
of parent relids to the corresponding child relids. This is
preliminary refactoring to make that possible.
Ashutosh Bapat. Review and testing of the larger patch set of which
this is a part by Amit Langote, Rajkumar Raghuwanshi, Rafia Sabih,
Thomas Munro, Dilip Kumar, and me. Some adjustments, mostly
cosmetic, by me.
Discussion: http://postgr.es/m/CA+TgmobQK80vtXjAsPZWWXd7c8u13G86gmuLupN+uUJjA+i4nA@mail.gmail.com
8 years ago
|
|
|
adjust_child_relids(Relids relids, int nappinfos, AppendRelInfo **appinfos)
|
|
|
|
|
{
|
Teach adjust_appendrel_attrs(_multilevel) to do multiple translations.
Currently, child relations are always base relations, so when we
translate parent relids to child relids, we only need to translate
a singler relid. However, the proposed partition-wise join feature
will create child joins, which will mean we need to translate a set
of parent relids to the corresponding child relids. This is
preliminary refactoring to make that possible.
Ashutosh Bapat. Review and testing of the larger patch set of which
this is a part by Amit Langote, Rajkumar Raghuwanshi, Rafia Sabih,
Thomas Munro, Dilip Kumar, and me. Some adjustments, mostly
cosmetic, by me.
Discussion: http://postgr.es/m/CA+TgmobQK80vtXjAsPZWWXd7c8u13G86gmuLupN+uUJjA+i4nA@mail.gmail.com
8 years ago
|
|
|
Bitmapset *result = NULL;
|
|
|
|
|
int cnt;
|
|
|
|
|
|
|
|
|
|
for (cnt = 0; cnt < nappinfos; cnt++)
|
|
|
|
|
{
|
Teach adjust_appendrel_attrs(_multilevel) to do multiple translations.
Currently, child relations are always base relations, so when we
translate parent relids to child relids, we only need to translate
a singler relid. However, the proposed partition-wise join feature
will create child joins, which will mean we need to translate a set
of parent relids to the corresponding child relids. This is
preliminary refactoring to make that possible.
Ashutosh Bapat. Review and testing of the larger patch set of which
this is a part by Amit Langote, Rajkumar Raghuwanshi, Rafia Sabih,
Thomas Munro, Dilip Kumar, and me. Some adjustments, mostly
cosmetic, by me.
Discussion: http://postgr.es/m/CA+TgmobQK80vtXjAsPZWWXd7c8u13G86gmuLupN+uUJjA+i4nA@mail.gmail.com
8 years ago
|
|
|
AppendRelInfo *appinfo = appinfos[cnt];
|
|
|
|
|
|
|
|
|
|
/* Remove parent, add child */
|
|
|
|
|
if (bms_is_member(appinfo->parent_relid, relids))
|
|
|
|
|
{
|
|
|
|
|
/* Make a copy if we are changing the set. */
|
|
|
|
|
if (!result)
|
|
|
|
|
result = bms_copy(relids);
|
|
|
|
|
|
|
|
|
|
result = bms_del_member(result, appinfo->parent_relid);
|
|
|
|
|
result = bms_add_member(result, appinfo->child_relid);
|
|
|
|
|
}
|
|
|
|
|
}
|
Teach adjust_appendrel_attrs(_multilevel) to do multiple translations.
Currently, child relations are always base relations, so when we
translate parent relids to child relids, we only need to translate
a singler relid. However, the proposed partition-wise join feature
will create child joins, which will mean we need to translate a set
of parent relids to the corresponding child relids. This is
preliminary refactoring to make that possible.
Ashutosh Bapat. Review and testing of the larger patch set of which
this is a part by Amit Langote, Rajkumar Raghuwanshi, Rafia Sabih,
Thomas Munro, Dilip Kumar, and me. Some adjustments, mostly
cosmetic, by me.
Discussion: http://postgr.es/m/CA+TgmobQK80vtXjAsPZWWXd7c8u13G86gmuLupN+uUJjA+i4nA@mail.gmail.com
8 years ago
|
|
|
|
|
|
|
|
/* If we made any changes, return the modified copy. */
|
|
|
|
|
if (result)
|
|
|
|
|
return result;
|
|
|
|
|
|
|
|
|
|
/* Otherwise, return the original set without modification. */
|
|
|
|
|
return relids;
|
|
|
|
|
}
|
|
|
|
|
|
Basic partition-wise join functionality.
Instead of joining two partitioned tables in their entirety we can, if
it is an equi-join on the partition keys, join the matching partitions
individually. This involves teaching the planner about "other join"
rels, which are related to regular join rels in the same way that
other member rels are related to baserels. This can use significantly
more CPU time and memory than regular join planning, because there may
now be a set of "other" rels not only for every base relation but also
for every join relation. In most practical cases, this probably
shouldn't be a problem, because (1) it's probably unusual to join many
tables each with many partitions using the partition keys for all
joins and (2) if you do that scenario then you probably have a big
enough machine to handle the increased memory cost of planning and (3)
the resulting plan is highly likely to be better, so what you spend in
planning you'll make up on the execution side. All the same, for now,
turn this feature off by default.
Currently, we can only perform joins between two tables whose
partitioning schemes are absolutely identical. It would be nice to
cope with other scenarios, such as extra partitions on one side or the
other with no match on the other side, but that will have to wait for
a future patch.
Ashutosh Bapat, reviewed and tested by Rajkumar Raghuwanshi, Amit
Langote, Rafia Sabih, Thomas Munro, Dilip Kumar, Antonin Houska, Amit
Khandekar, and by me. A few final adjustments by me.
Discussion: http://postgr.es/m/CAFjFpRfQ8GrQvzp3jA2wnLqrHmaXna-urjm_UY9BqXj=EaDTSA@mail.gmail.com
Discussion: http://postgr.es/m/CAFjFpRcitjfrULr5jfuKWRPsGUX0LQ0k8-yG0Qw2+1LBGNpMdw@mail.gmail.com
8 years ago
|
|
|
/*
|
|
|
|
|
* Replace any relid present in top_parent_relids with its child in
|
|
|
|
|
* child_relids. Members of child_relids can be multiple levels below top
|
|
|
|
|
* parent in the partition hierarchy.
|
|
|
|
|
*/
|
|
|
|
|
Relids
|
|
|
|
|
adjust_child_relids_multilevel(PlannerInfo *root, Relids relids,
|
|
|
|
|
Relids child_relids, Relids top_parent_relids)
|
|
|
|
|
{
|
|
|
|
|
AppendRelInfo **appinfos;
|
|
|
|
|
int nappinfos;
|
|
|
|
|
Relids parent_relids = NULL;
|
|
|
|
|
Relids result;
|
|
|
|
|
Relids tmp_result = NULL;
|
|
|
|
|
int cnt;
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* If the given relids set doesn't contain any of the top parent relids,
|
|
|
|
|
* it will remain unchanged.
|
|
|
|
|
*/
|
|
|
|
|
if (!bms_overlap(relids, top_parent_relids))
|
|
|
|
|
return relids;
|
|
|
|
|
|
|
|
|
|
appinfos = find_appinfos_by_relids(root, child_relids, &nappinfos);
|
|
|
|
|
|
|
|
|
|
/* Construct relids set for the immediate parent of the given child. */
|
|
|
|
|
for (cnt = 0; cnt < nappinfos; cnt++)
|
|
|
|
|
{
|
|
|
|
|
AppendRelInfo *appinfo = appinfos[cnt];
|
|
|
|
|
|
|
|
|
|
parent_relids = bms_add_member(parent_relids, appinfo->parent_relid);
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/* Recurse if immediate parent is not the top parent. */
|
|
|
|
|
if (!bms_equal(parent_relids, top_parent_relids))
|
|
|
|
|
{
|
|
|
|
|
tmp_result = adjust_child_relids_multilevel(root, relids,
|
|
|
|
|
parent_relids,
|
|
|
|
|
top_parent_relids);
|
|
|
|
|
relids = tmp_result;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
result = adjust_child_relids(relids, nappinfos, appinfos);
|
|
|
|
|
|
|
|
|
|
/* Free memory consumed by any intermediate result. */
|
|
|
|
|
if (tmp_result)
|
|
|
|
|
bms_free(tmp_result);
|
|
|
|
|
bms_free(parent_relids);
|
|
|
|
|
pfree(appinfos);
|
|
|
|
|
|
|
|
|
|
return result;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Adjust the targetlist entries of an inherited UPDATE operation
|
|
|
|
|
*
|
|
|
|
|
* The expressions have already been fixed, but we have to make sure that
|
|
|
|
|
* the target resnos match the child table (they may not, in the case of
|
|
|
|
|
* a column that was added after-the-fact by ALTER TABLE). In some cases
|
|
|
|
|
* this can force us to re-order the tlist to preserve resno ordering.
|
|
|
|
|
* (We do all this work in special cases so that preptlist.c is fast for
|
|
|
|
|
* the typical case.)
|
|
|
|
|
*
|
|
|
|
|
* The given tlist has already been through expression_tree_mutator;
|
|
|
|
|
* therefore the TargetEntry nodes are fresh copies that it's okay to
|
|
|
|
|
* scribble on.
|
|
|
|
|
*
|
|
|
|
|
* Note that this is not needed for INSERT because INSERT isn't inheritable.
|
|
|
|
|
*/
|
|
|
|
|
static List *
|
|
|
|
|
adjust_inherited_tlist(List *tlist, AppendRelInfo *context)
|
|
|
|
|
{
|
|
|
|
|
bool changed_it = false;
|
|
|
|
|
ListCell *tl;
|
|
|
|
|
List *new_tlist;
|
|
|
|
|
bool more;
|
|
|
|
|
int attrno;
|
|
|
|
|
|
|
|
|
|
/* This should only happen for an inheritance case, not UNION ALL */
|
|
|
|
|
Assert(OidIsValid(context->parent_reloid));
|
|
|
|
|
|
|
|
|
|
/* Scan tlist and update resnos to match attnums of child rel */
|
|
|
|
|
foreach(tl, tlist)
|
|
|
|
|
{
|
|
|
|
|
TargetEntry *tle = (TargetEntry *) lfirst(tl);
|
|
|
|
|
Var *childvar;
|
|
|
|
|
|
|
|
|
|
if (tle->resjunk)
|
|
|
|
|
continue; /* ignore junk items */
|
|
|
|
|
|
|
|
|
|
/* Look up the translation of this column: it must be a Var */
|
|
|
|
|
if (tle->resno <= 0 ||
|
|
|
|
|
tle->resno > list_length(context->translated_vars))
|
|
|
|
|
elog(ERROR, "attribute %d of relation \"%s\" does not exist",
|
|
|
|
|
tle->resno, get_rel_name(context->parent_reloid));
|
|
|
|
|
childvar = (Var *) list_nth(context->translated_vars, tle->resno - 1);
|
|
|
|
|
if (childvar == NULL || !IsA(childvar, Var))
|
|
|
|
|
elog(ERROR, "attribute %d of relation \"%s\" does not exist",
|
|
|
|
|
tle->resno, get_rel_name(context->parent_reloid));
|
|
|
|
|
|
|
|
|
|
if (tle->resno != childvar->varattno)
|
|
|
|
|
{
|
|
|
|
|
tle->resno = childvar->varattno;
|
|
|
|
|
changed_it = true;
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* If we changed anything, re-sort the tlist by resno, and make sure
|
|
|
|
|
* resjunk entries have resnos above the last real resno. The sort
|
|
|
|
|
* algorithm is a bit stupid, but for such a seldom-taken path, small is
|
|
|
|
|
* probably better than fast.
|
|
|
|
|
*/
|
|
|
|
|
if (!changed_it)
|
|
|
|
|
return tlist;
|
|
|
|
|
|
|
|
|
|
new_tlist = NIL;
|
|
|
|
|
more = true;
|
|
|
|
|
for (attrno = 1; more; attrno++)
|
|
|
|
|
{
|
|
|
|
|
more = false;
|
|
|
|
|
foreach(tl, tlist)
|
|
|
|
|
{
|
|
|
|
|
TargetEntry *tle = (TargetEntry *) lfirst(tl);
|
|
|
|
|
|
|
|
|
|
if (tle->resjunk)
|
|
|
|
|
continue; /* ignore junk items */
|
|
|
|
|
|
|
|
|
|
if (tle->resno == attrno)
|
|
|
|
|
new_tlist = lappend(new_tlist, tle);
|
|
|
|
|
else if (tle->resno > attrno)
|
|
|
|
|
more = true;
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
foreach(tl, tlist)
|
|
|
|
|
{
|
|
|
|
|
TargetEntry *tle = (TargetEntry *) lfirst(tl);
|
|
|
|
|
|
|
|
|
|
if (!tle->resjunk)
|
|
|
|
|
continue; /* here, ignore non-junk items */
|
|
|
|
|
|
|
|
|
|
tle->resno = attrno;
|
|
|
|
|
new_tlist = lappend(new_tlist, tle);
|
|
|
|
|
attrno++;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
return new_tlist;
|
|
|
|
|
}
|
Fix some more problems with nested append relations.
As of commit a87c72915 (which later got backpatched as far as 9.1),
we're explicitly supporting the notion that append relations can be
nested; this can occur when UNION ALL constructs are nested, or when
a UNION ALL contains a table with inheritance children.
Bug #11457 from Nelson Page, as well as an earlier report from Elvis
Pranskevichus, showed that there were still nasty bugs associated with such
cases: in particular the EquivalenceClass mechanism could try to generate
"join" clauses connecting an appendrel child to some grandparent appendrel,
which would result in assertion failures or bogus plans.
Upon investigation I concluded that all current callers of
find_childrel_appendrelinfo() need to be fixed to explicitly consider
multiple levels of parent appendrels. The most complex fix was in
processing of "broken" EquivalenceClasses, which are ECs for which we have
been unable to generate all the derived equality clauses we would like to
because of missing cross-type equality operators in the underlying btree
operator family. That code path is more or less entirely untested by
the regression tests to date, because no standard opfamilies have such
holes in them. So I wrote a new regression test script to try to exercise
it a bit, which turned out to be quite a worthwhile activity as it exposed
existing bugs in all supported branches.
The present patch is essentially the same as far back as 9.2, which is
where parameterized paths were introduced. In 9.0 and 9.1, we only need
to back-patch a small fragment of commit 5b7b5518d, which fixes failure to
propagate out the original WHERE clauses when a broken EC contains constant
members. (The regression test case results show that these older branches
are noticeably stupider than 9.2+ in terms of the quality of the plans
generated; but we don't really care about plan quality in such cases,
only that the plan not be outright wrong. A more invasive fix in the
older branches would not be a good idea anyway from a plan-stability
standpoint.)
11 years ago
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* adjust_appendrel_attrs_multilevel
|
|
|
|
|
* Apply Var translations from a toplevel appendrel parent down to a child.
|
|
|
|
|
*
|
Teach adjust_appendrel_attrs(_multilevel) to do multiple translations.
Currently, child relations are always base relations, so when we
translate parent relids to child relids, we only need to translate
a singler relid. However, the proposed partition-wise join feature
will create child joins, which will mean we need to translate a set
of parent relids to the corresponding child relids. This is
preliminary refactoring to make that possible.
Ashutosh Bapat. Review and testing of the larger patch set of which
this is a part by Amit Langote, Rajkumar Raghuwanshi, Rafia Sabih,
Thomas Munro, Dilip Kumar, and me. Some adjustments, mostly
cosmetic, by me.
Discussion: http://postgr.es/m/CA+TgmobQK80vtXjAsPZWWXd7c8u13G86gmuLupN+uUJjA+i4nA@mail.gmail.com
8 years ago
|
|
|
* In some cases we need to translate expressions referencing a parent relation
|
Fix some more problems with nested append relations.
As of commit a87c72915 (which later got backpatched as far as 9.1),
we're explicitly supporting the notion that append relations can be
nested; this can occur when UNION ALL constructs are nested, or when
a UNION ALL contains a table with inheritance children.
Bug #11457 from Nelson Page, as well as an earlier report from Elvis
Pranskevichus, showed that there were still nasty bugs associated with such
cases: in particular the EquivalenceClass mechanism could try to generate
"join" clauses connecting an appendrel child to some grandparent appendrel,
which would result in assertion failures or bogus plans.
Upon investigation I concluded that all current callers of
find_childrel_appendrelinfo() need to be fixed to explicitly consider
multiple levels of parent appendrels. The most complex fix was in
processing of "broken" EquivalenceClasses, which are ECs for which we have
been unable to generate all the derived equality clauses we would like to
because of missing cross-type equality operators in the underlying btree
operator family. That code path is more or less entirely untested by
the regression tests to date, because no standard opfamilies have such
holes in them. So I wrote a new regression test script to try to exercise
it a bit, which turned out to be quite a worthwhile activity as it exposed
existing bugs in all supported branches.
The present patch is essentially the same as far back as 9.2, which is
where parameterized paths were introduced. In 9.0 and 9.1, we only need
to back-patch a small fragment of commit 5b7b5518d, which fixes failure to
propagate out the original WHERE clauses when a broken EC contains constant
members. (The regression test case results show that these older branches
are noticeably stupider than 9.2+ in terms of the quality of the plans
generated; but we don't really care about plan quality in such cases,
only that the plan not be outright wrong. A more invasive fix in the
older branches would not be a good idea anyway from a plan-stability
standpoint.)
11 years ago
|
|
|
* to reference an appendrel child that's multiple levels removed from it.
|
|
|
|
|
*/
|
|
|
|
|
Node *
|
|
|
|
|
adjust_appendrel_attrs_multilevel(PlannerInfo *root, Node *node,
|
Teach adjust_appendrel_attrs(_multilevel) to do multiple translations.
Currently, child relations are always base relations, so when we
translate parent relids to child relids, we only need to translate
a singler relid. However, the proposed partition-wise join feature
will create child joins, which will mean we need to translate a set
of parent relids to the corresponding child relids. This is
preliminary refactoring to make that possible.
Ashutosh Bapat. Review and testing of the larger patch set of which
this is a part by Amit Langote, Rajkumar Raghuwanshi, Rafia Sabih,
Thomas Munro, Dilip Kumar, and me. Some adjustments, mostly
cosmetic, by me.
Discussion: http://postgr.es/m/CA+TgmobQK80vtXjAsPZWWXd7c8u13G86gmuLupN+uUJjA+i4nA@mail.gmail.com
8 years ago
|
|
|
Relids child_relids,
|
|
|
|
|
Relids top_parent_relids)
|
Fix some more problems with nested append relations.
As of commit a87c72915 (which later got backpatched as far as 9.1),
we're explicitly supporting the notion that append relations can be
nested; this can occur when UNION ALL constructs are nested, or when
a UNION ALL contains a table with inheritance children.
Bug #11457 from Nelson Page, as well as an earlier report from Elvis
Pranskevichus, showed that there were still nasty bugs associated with such
cases: in particular the EquivalenceClass mechanism could try to generate
"join" clauses connecting an appendrel child to some grandparent appendrel,
which would result in assertion failures or bogus plans.
Upon investigation I concluded that all current callers of
find_childrel_appendrelinfo() need to be fixed to explicitly consider
multiple levels of parent appendrels. The most complex fix was in
processing of "broken" EquivalenceClasses, which are ECs for which we have
been unable to generate all the derived equality clauses we would like to
because of missing cross-type equality operators in the underlying btree
operator family. That code path is more or less entirely untested by
the regression tests to date, because no standard opfamilies have such
holes in them. So I wrote a new regression test script to try to exercise
it a bit, which turned out to be quite a worthwhile activity as it exposed
existing bugs in all supported branches.
The present patch is essentially the same as far back as 9.2, which is
where parameterized paths were introduced. In 9.0 and 9.1, we only need
to back-patch a small fragment of commit 5b7b5518d, which fixes failure to
propagate out the original WHERE clauses when a broken EC contains constant
members. (The regression test case results show that these older branches
are noticeably stupider than 9.2+ in terms of the quality of the plans
generated; but we don't really care about plan quality in such cases,
only that the plan not be outright wrong. A more invasive fix in the
older branches would not be a good idea anyway from a plan-stability
standpoint.)
11 years ago
|
|
|
{
|
Teach adjust_appendrel_attrs(_multilevel) to do multiple translations.
Currently, child relations are always base relations, so when we
translate parent relids to child relids, we only need to translate
a singler relid. However, the proposed partition-wise join feature
will create child joins, which will mean we need to translate a set
of parent relids to the corresponding child relids. This is
preliminary refactoring to make that possible.
Ashutosh Bapat. Review and testing of the larger patch set of which
this is a part by Amit Langote, Rajkumar Raghuwanshi, Rafia Sabih,
Thomas Munro, Dilip Kumar, and me. Some adjustments, mostly
cosmetic, by me.
Discussion: http://postgr.es/m/CA+TgmobQK80vtXjAsPZWWXd7c8u13G86gmuLupN+uUJjA+i4nA@mail.gmail.com
8 years ago
|
|
|
AppendRelInfo **appinfos;
|
|
|
|
|
Bitmapset *parent_relids = NULL;
|
|
|
|
|
int nappinfos;
|
|
|
|
|
int cnt;
|
|
|
|
|
|
|
|
|
|
Assert(bms_num_members(child_relids) == bms_num_members(top_parent_relids));
|
|
|
|
|
|
|
|
|
|
appinfos = find_appinfos_by_relids(root, child_relids, &nappinfos);
|
|
|
|
|
|
|
|
|
|
/* Construct relids set for the immediate parent of given child. */
|
|
|
|
|
for (cnt = 0; cnt < nappinfos; cnt++)
|
|
|
|
|
{
|
|
|
|
|
AppendRelInfo *appinfo = appinfos[cnt];
|
|
|
|
|
|
|
|
|
|
parent_relids = bms_add_member(parent_relids, appinfo->parent_relid);
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/* Recurse if immediate parent is not the top parent. */
|
|
|
|
|
if (!bms_equal(parent_relids, top_parent_relids))
|
|
|
|
|
node = adjust_appendrel_attrs_multilevel(root, node, parent_relids,
|
|
|
|
|
top_parent_relids);
|
Fix some more problems with nested append relations.
As of commit a87c72915 (which later got backpatched as far as 9.1),
we're explicitly supporting the notion that append relations can be
nested; this can occur when UNION ALL constructs are nested, or when
a UNION ALL contains a table with inheritance children.
Bug #11457 from Nelson Page, as well as an earlier report from Elvis
Pranskevichus, showed that there were still nasty bugs associated with such
cases: in particular the EquivalenceClass mechanism could try to generate
"join" clauses connecting an appendrel child to some grandparent appendrel,
which would result in assertion failures or bogus plans.
Upon investigation I concluded that all current callers of
find_childrel_appendrelinfo() need to be fixed to explicitly consider
multiple levels of parent appendrels. The most complex fix was in
processing of "broken" EquivalenceClasses, which are ECs for which we have
been unable to generate all the derived equality clauses we would like to
because of missing cross-type equality operators in the underlying btree
operator family. That code path is more or less entirely untested by
the regression tests to date, because no standard opfamilies have such
holes in them. So I wrote a new regression test script to try to exercise
it a bit, which turned out to be quite a worthwhile activity as it exposed
existing bugs in all supported branches.
The present patch is essentially the same as far back as 9.2, which is
where parameterized paths were introduced. In 9.0 and 9.1, we only need
to back-patch a small fragment of commit 5b7b5518d, which fixes failure to
propagate out the original WHERE clauses when a broken EC contains constant
members. (The regression test case results show that these older branches
are noticeably stupider than 9.2+ in terms of the quality of the plans
generated; but we don't really care about plan quality in such cases,
only that the plan not be outright wrong. A more invasive fix in the
older branches would not be a good idea anyway from a plan-stability
standpoint.)
11 years ago
|
|
|
|
|
|
|
|
/* Now translate for this child */
|
Teach adjust_appendrel_attrs(_multilevel) to do multiple translations.
Currently, child relations are always base relations, so when we
translate parent relids to child relids, we only need to translate
a singler relid. However, the proposed partition-wise join feature
will create child joins, which will mean we need to translate a set
of parent relids to the corresponding child relids. This is
preliminary refactoring to make that possible.
Ashutosh Bapat. Review and testing of the larger patch set of which
this is a part by Amit Langote, Rajkumar Raghuwanshi, Rafia Sabih,
Thomas Munro, Dilip Kumar, and me. Some adjustments, mostly
cosmetic, by me.
Discussion: http://postgr.es/m/CA+TgmobQK80vtXjAsPZWWXd7c8u13G86gmuLupN+uUJjA+i4nA@mail.gmail.com
8 years ago
|
|
|
node = adjust_appendrel_attrs(root, node, nappinfos, appinfos);
|
|
|
|
|
|
|
|
|
|
pfree(appinfos);
|
|
|
|
|
|
|
|
|
|
return node;
|
|
|
|
|
}
|
|
|
|
|
|
Basic partition-wise join functionality.
Instead of joining two partitioned tables in their entirety we can, if
it is an equi-join on the partition keys, join the matching partitions
individually. This involves teaching the planner about "other join"
rels, which are related to regular join rels in the same way that
other member rels are related to baserels. This can use significantly
more CPU time and memory than regular join planning, because there may
now be a set of "other" rels not only for every base relation but also
for every join relation. In most practical cases, this probably
shouldn't be a problem, because (1) it's probably unusual to join many
tables each with many partitions using the partition keys for all
joins and (2) if you do that scenario then you probably have a big
enough machine to handle the increased memory cost of planning and (3)
the resulting plan is highly likely to be better, so what you spend in
planning you'll make up on the execution side. All the same, for now,
turn this feature off by default.
Currently, we can only perform joins between two tables whose
partitioning schemes are absolutely identical. It would be nice to
cope with other scenarios, such as extra partitions on one side or the
other with no match on the other side, but that will have to wait for
a future patch.
Ashutosh Bapat, reviewed and tested by Rajkumar Raghuwanshi, Amit
Langote, Rafia Sabih, Thomas Munro, Dilip Kumar, Antonin Houska, Amit
Khandekar, and by me. A few final adjustments by me.
Discussion: http://postgr.es/m/CAFjFpRfQ8GrQvzp3jA2wnLqrHmaXna-urjm_UY9BqXj=EaDTSA@mail.gmail.com
Discussion: http://postgr.es/m/CAFjFpRcitjfrULr5jfuKWRPsGUX0LQ0k8-yG0Qw2+1LBGNpMdw@mail.gmail.com
8 years ago
|
|
|
/*
|
|
|
|
|
* Construct the SpecialJoinInfo for a child-join by translating
|
|
|
|
|
* SpecialJoinInfo for the join between parents. left_relids and right_relids
|
|
|
|
|
* are the relids of left and right side of the join respectively.
|
|
|
|
|
*/
|
|
|
|
|
SpecialJoinInfo *
|
|
|
|
|
build_child_join_sjinfo(PlannerInfo *root, SpecialJoinInfo *parent_sjinfo,
|
|
|
|
|
Relids left_relids, Relids right_relids)
|
|
|
|
|
{
|
|
|
|
|
SpecialJoinInfo *sjinfo = makeNode(SpecialJoinInfo);
|
|
|
|
|
AppendRelInfo **left_appinfos;
|
|
|
|
|
int left_nappinfos;
|
|
|
|
|
AppendRelInfo **right_appinfos;
|
|
|
|
|
int right_nappinfos;
|
|
|
|
|
|
|
|
|
|
memcpy(sjinfo, parent_sjinfo, sizeof(SpecialJoinInfo));
|
|
|
|
|
left_appinfos = find_appinfos_by_relids(root, left_relids,
|
|
|
|
|
&left_nappinfos);
|
|
|
|
|
right_appinfos = find_appinfos_by_relids(root, right_relids,
|
|
|
|
|
&right_nappinfos);
|
|
|
|
|
|
|
|
|
|
sjinfo->min_lefthand = adjust_child_relids(sjinfo->min_lefthand,
|
|
|
|
|
left_nappinfos, left_appinfos);
|
|
|
|
|
sjinfo->min_righthand = adjust_child_relids(sjinfo->min_righthand,
|
|
|
|
|
right_nappinfos,
|
|
|
|
|
right_appinfos);
|
|
|
|
|
sjinfo->syn_lefthand = adjust_child_relids(sjinfo->syn_lefthand,
|
|
|
|
|
left_nappinfos, left_appinfos);
|
|
|
|
|
sjinfo->syn_righthand = adjust_child_relids(sjinfo->syn_righthand,
|
|
|
|
|
right_nappinfos,
|
|
|
|
|
right_appinfos);
|
|
|
|
|
sjinfo->semi_rhs_exprs = (List *) adjust_appendrel_attrs(root,
|
|
|
|
|
(Node *) sjinfo->semi_rhs_exprs,
|
|
|
|
|
right_nappinfos,
|
|
|
|
|
right_appinfos);
|
|
|
|
|
|
|
|
|
|
pfree(left_appinfos);
|
|
|
|
|
pfree(right_appinfos);
|
|
|
|
|
|
|
|
|
|
return sjinfo;
|
|
|
|
|
}
|
|
|
|
|
|
Teach adjust_appendrel_attrs(_multilevel) to do multiple translations.
Currently, child relations are always base relations, so when we
translate parent relids to child relids, we only need to translate
a singler relid. However, the proposed partition-wise join feature
will create child joins, which will mean we need to translate a set
of parent relids to the corresponding child relids. This is
preliminary refactoring to make that possible.
Ashutosh Bapat. Review and testing of the larger patch set of which
this is a part by Amit Langote, Rajkumar Raghuwanshi, Rafia Sabih,
Thomas Munro, Dilip Kumar, and me. Some adjustments, mostly
cosmetic, by me.
Discussion: http://postgr.es/m/CA+TgmobQK80vtXjAsPZWWXd7c8u13G86gmuLupN+uUJjA+i4nA@mail.gmail.com
8 years ago
|
|
|
/*
|
|
|
|
|
* find_appinfos_by_relids
|
|
|
|
|
* Find AppendRelInfo structures for all relations specified by relids.
|
|
|
|
|
*
|
|
|
|
|
* The AppendRelInfos are returned in an array, which can be pfree'd by the
|
|
|
|
|
* caller. *nappinfos is set to the number of entries in the array.
|
Teach adjust_appendrel_attrs(_multilevel) to do multiple translations.
Currently, child relations are always base relations, so when we
translate parent relids to child relids, we only need to translate
a singler relid. However, the proposed partition-wise join feature
will create child joins, which will mean we need to translate a set
of parent relids to the corresponding child relids. This is
preliminary refactoring to make that possible.
Ashutosh Bapat. Review and testing of the larger patch set of which
this is a part by Amit Langote, Rajkumar Raghuwanshi, Rafia Sabih,
Thomas Munro, Dilip Kumar, and me. Some adjustments, mostly
cosmetic, by me.
Discussion: http://postgr.es/m/CA+TgmobQK80vtXjAsPZWWXd7c8u13G86gmuLupN+uUJjA+i4nA@mail.gmail.com
8 years ago
|
|
|
*/
|
|
|
|
|
AppendRelInfo **
|
|
|
|
|
find_appinfos_by_relids(PlannerInfo *root, Relids relids, int *nappinfos)
|
|
|
|
|
{
|
|
|
|
|
AppendRelInfo **appinfos;
|
|
|
|
|
int cnt = 0;
|
|
|
|
|
int i;
|
Teach adjust_appendrel_attrs(_multilevel) to do multiple translations.
Currently, child relations are always base relations, so when we
translate parent relids to child relids, we only need to translate
a singler relid. However, the proposed partition-wise join feature
will create child joins, which will mean we need to translate a set
of parent relids to the corresponding child relids. This is
preliminary refactoring to make that possible.
Ashutosh Bapat. Review and testing of the larger patch set of which
this is a part by Amit Langote, Rajkumar Raghuwanshi, Rafia Sabih,
Thomas Munro, Dilip Kumar, and me. Some adjustments, mostly
cosmetic, by me.
Discussion: http://postgr.es/m/CA+TgmobQK80vtXjAsPZWWXd7c8u13G86gmuLupN+uUJjA+i4nA@mail.gmail.com
8 years ago
|
|
|
|
|
|
|
|
*nappinfos = bms_num_members(relids);
|
|
|
|
|
appinfos = (AppendRelInfo **) palloc(sizeof(AppendRelInfo *) * *nappinfos);
|
|
|
|
|
|
|
|
|
|
i = -1;
|
|
|
|
|
while ((i = bms_next_member(relids, i)) >= 0)
|
Teach adjust_appendrel_attrs(_multilevel) to do multiple translations.
Currently, child relations are always base relations, so when we
translate parent relids to child relids, we only need to translate
a singler relid. However, the proposed partition-wise join feature
will create child joins, which will mean we need to translate a set
of parent relids to the corresponding child relids. This is
preliminary refactoring to make that possible.
Ashutosh Bapat. Review and testing of the larger patch set of which
this is a part by Amit Langote, Rajkumar Raghuwanshi, Rafia Sabih,
Thomas Munro, Dilip Kumar, and me. Some adjustments, mostly
cosmetic, by me.
Discussion: http://postgr.es/m/CA+TgmobQK80vtXjAsPZWWXd7c8u13G86gmuLupN+uUJjA+i4nA@mail.gmail.com
8 years ago
|
|
|
{
|
|
|
|
|
AppendRelInfo *appinfo = root->append_rel_array[i];
|
Teach adjust_appendrel_attrs(_multilevel) to do multiple translations.
Currently, child relations are always base relations, so when we
translate parent relids to child relids, we only need to translate
a singler relid. However, the proposed partition-wise join feature
will create child joins, which will mean we need to translate a set
of parent relids to the corresponding child relids. This is
preliminary refactoring to make that possible.
Ashutosh Bapat. Review and testing of the larger patch set of which
this is a part by Amit Langote, Rajkumar Raghuwanshi, Rafia Sabih,
Thomas Munro, Dilip Kumar, and me. Some adjustments, mostly
cosmetic, by me.
Discussion: http://postgr.es/m/CA+TgmobQK80vtXjAsPZWWXd7c8u13G86gmuLupN+uUJjA+i4nA@mail.gmail.com
8 years ago
|
|
|
|
|
|
|
|
if (!appinfo)
|
|
|
|
|
elog(ERROR, "child rel %d not found in append_rel_array", i);
|
Teach adjust_appendrel_attrs(_multilevel) to do multiple translations.
Currently, child relations are always base relations, so when we
translate parent relids to child relids, we only need to translate
a singler relid. However, the proposed partition-wise join feature
will create child joins, which will mean we need to translate a set
of parent relids to the corresponding child relids. This is
preliminary refactoring to make that possible.
Ashutosh Bapat. Review and testing of the larger patch set of which
this is a part by Amit Langote, Rajkumar Raghuwanshi, Rafia Sabih,
Thomas Munro, Dilip Kumar, and me. Some adjustments, mostly
cosmetic, by me.
Discussion: http://postgr.es/m/CA+TgmobQK80vtXjAsPZWWXd7c8u13G86gmuLupN+uUJjA+i4nA@mail.gmail.com
8 years ago
|
|
|
|
|
|
|
|
appinfos[cnt++] = appinfo;
|
Teach adjust_appendrel_attrs(_multilevel) to do multiple translations.
Currently, child relations are always base relations, so when we
translate parent relids to child relids, we only need to translate
a singler relid. However, the proposed partition-wise join feature
will create child joins, which will mean we need to translate a set
of parent relids to the corresponding child relids. This is
preliminary refactoring to make that possible.
Ashutosh Bapat. Review and testing of the larger patch set of which
this is a part by Amit Langote, Rajkumar Raghuwanshi, Rafia Sabih,
Thomas Munro, Dilip Kumar, and me. Some adjustments, mostly
cosmetic, by me.
Discussion: http://postgr.es/m/CA+TgmobQK80vtXjAsPZWWXd7c8u13G86gmuLupN+uUJjA+i4nA@mail.gmail.com
8 years ago
|
|
|
}
|
|
|
|
|
return appinfos;
|
Fix some more problems with nested append relations.
As of commit a87c72915 (which later got backpatched as far as 9.1),
we're explicitly supporting the notion that append relations can be
nested; this can occur when UNION ALL constructs are nested, or when
a UNION ALL contains a table with inheritance children.
Bug #11457 from Nelson Page, as well as an earlier report from Elvis
Pranskevichus, showed that there were still nasty bugs associated with such
cases: in particular the EquivalenceClass mechanism could try to generate
"join" clauses connecting an appendrel child to some grandparent appendrel,
which would result in assertion failures or bogus plans.
Upon investigation I concluded that all current callers of
find_childrel_appendrelinfo() need to be fixed to explicitly consider
multiple levels of parent appendrels. The most complex fix was in
processing of "broken" EquivalenceClasses, which are ECs for which we have
been unable to generate all the derived equality clauses we would like to
because of missing cross-type equality operators in the underlying btree
operator family. That code path is more or less entirely untested by
the regression tests to date, because no standard opfamilies have such
holes in them. So I wrote a new regression test script to try to exercise
it a bit, which turned out to be quite a worthwhile activity as it exposed
existing bugs in all supported branches.
The present patch is essentially the same as far back as 9.2, which is
where parameterized paths were introduced. In 9.0 and 9.1, we only need
to back-patch a small fragment of commit 5b7b5518d, which fixes failure to
propagate out the original WHERE clauses when a broken EC contains constant
members. (The regression test case results show that these older branches
are noticeably stupider than 9.2+ in terms of the quality of the plans
generated; but we don't really care about plan quality in such cases,
only that the plan not be outright wrong. A more invasive fix in the
older branches would not be a good idea anyway from a plan-stability
standpoint.)
11 years ago
|
|
|
}
|
