mirror of https://github.com/postgres/postgres
This commit provides the core code and documentation needed. A contrib module test case will follow shortly. Shigeru Hanada, Jan Urbanski, Heikki Linnakangaspull/1/head
Tom Lane
15 years ago
parent
d5813488a4
commit
bb74240794
@ -0,0 +1,212 @@ |
||||
<!-- doc/src/sgml/fdwhandler.sgml --> |
||||
|
||||
<chapter id="fdwhandler"> |
||||
<title>Writing A Foreign Data Wrapper</title> |
||||
|
||||
<indexterm zone="fdwhandler"> |
||||
<primary>foreign data wrapper</primary> |
||||
<secondary>handler for</secondary> |
||||
</indexterm> |
||||
|
||||
<para> |
||||
All operations on a foreign table are handled through its foreign data |
||||
wrapper, which consists of a set of functions that the planner and |
||||
executor call. The foreign data wrapper is responsible for fetching |
||||
data from the remote data source and returning it to the |
||||
<productname>PostgreSQL</productname> executor. This chapter outlines how |
||||
to write a new foreign data wrapper. |
||||
</para> |
||||
|
||||
<para> |
||||
The FDW author needs to implement a handler function, and optionally |
||||
a validator function. Both functions must be written in a compiled |
||||
language such as C, using the version-1 interface. |
||||
For details on C language calling conventions and dynamic loading, |
||||
see <xref linkend="xfunc-c">. |
||||
</para> |
||||
|
||||
<para> |
||||
The handler function simply returns a struct of function pointers to |
||||
callback functions that will be called by the planner and executor. |
||||
Most of the effort in writing an FDW is in implementing these callback |
||||
functions. |
||||
The handler function must be registered with |
||||
<productname>PostgreSQL</productname> as taking no arguments and returning |
||||
the special pseudo-type <type>fdw_handler</type>. |
||||
The callback functions are plain C functions and are not visible or |
||||
callable at the SQL level. |
||||
</para> |
||||
|
||||
<para> |
||||
The validator function is responsible for validating options given in the |
||||
<command>CREATE FOREIGN DATA WRAPPER</command>, <command>CREATE |
||||
SERVER</command> and <command>CREATE FOREIGN TABLE</command> commands. |
||||
The validator function must be registered as taking two arguments, a text |
||||
array containing the options to be validated, and an OID representing the |
||||
type of object the options are associated with (in the form of the OID |
||||
of the system catalog the object would be stored in). If no validator |
||||
function is supplied, the options are not checked at object creation time. |
||||
</para> |
||||
|
||||
<para> |
||||
The foreign data wrappers included in the standard distribution are good |
||||
references when trying to write your own. Look into the |
||||
<filename>contrib/file_fdw</> subdirectory of the source tree. |
||||
The <xref linkend="sql-createforeigndatawrapper"> reference page also has |
||||
some useful details. |
||||
</para> |
||||
|
||||
<note> |
||||
<para> |
||||
The SQL standard specifies an interface for writing foreign data wrappers. |
||||
However, PostgreSQL does not implement that API, because the effort to |
||||
accommodate it into PostgreSQL would be large, and the standard API hasn't |
||||
gained wide adoption anyway. |
||||
</para> |
||||
</note> |
||||
|
||||
<sect1 id="fdw-routines"> |
||||
<title>Foreign Data Wrapper Callback Routines</title> |
||||
|
||||
<para> |
||||
The FDW handler function returns a palloc'd <structname>FdwRoutine</> |
||||
struct containing pointers to the following callback functions: |
||||
</para> |
||||
|
||||
<para> |
||||
<programlisting> |
||||
FdwPlan * |
||||
PlanForeignScan (Oid foreigntableid, |
||||
PlannerInfo *root, |
||||
RelOptInfo *baserel); |
||||
</programlisting> |
||||
|
||||
Plan a scan on a foreign table. This is called when a query is planned. |
||||
<literal>foreigntableid</> is the <structname>pg_class</> OID of the |
||||
foreign table. <literal>root</> is the planner's global information |
||||
about the query, and <literal>baserel</> is the planner's information |
||||
about this table. |
||||
The function must return a palloc'd struct that contains cost estimates |
||||
plus any FDW-private information that is needed to execute the foreign |
||||
scan at a later time. (Note that the private information must be |
||||
represented in a form that <function>copyObject</> knows how to copy.) |
||||
</para> |
||||
|
||||
<para> |
||||
The information in <literal>root</> and <literal>baserel</> can be used |
||||
to reduce the amount of information that has to be fetched from the |
||||
foreign table (and therefore reduce the cost estimate). |
||||
<literal>baserel->baserestrictinfo</> is particularly interesting, as |
||||
it contains restriction quals (<literal>WHERE</> clauses) that can be |
||||
used to filter the rows to be fetched. (The FDW is not required to |
||||
enforce these quals, as the finished plan will recheck them anyway.) |
||||
<literal>baserel->reltargetlist</> can be used to determine which |
||||
columns need to be fetched. |
||||
</para> |
||||
|
||||
<para> |
||||
In addition to returning cost estimates, the function should update |
||||
<literal>baserel->rows</> to be the expected number of rows returned |
||||
by the scan, after accounting for the filtering done by the restriction |
||||
quals. The initial value of <literal>baserel->rows</> is just a |
||||
constant default estimate, which should be replaced if at all possible. |
||||
The function may also choose to update <literal>baserel->width</> if |
||||
it can compute a better estimate of the average result row width. |
||||
</para> |
||||
|
||||
<para> |
||||
<programlisting> |
||||
void |
||||
ExplainForeignScan (ForeignScanState *node, |
||||
ExplainState *es); |
||||
</programlisting> |
||||
|
||||
Print additional <command>EXPLAIN</> output for a foreign table scan. |
||||
This can just return if there is no need to print anything. |
||||
Otherwise, it should call <function>ExplainPropertyText</> and |
||||
related functions to add fields to the <command>EXPLAIN</> output. |
||||
The flag fields in <literal>es</> can be used to determine what to |
||||
print, and the state of the <structname>ForeignScanState</> node |
||||
can be inspected to provide runtime statistics in the <command>EXPLAIN |
||||
ANALYZE</> case. |
||||
</para> |
||||
|
||||
<para> |
||||
<programlisting> |
||||
void |
||||
BeginForeignScan (ForeignScanState *node, |
||||
int eflags); |
||||
</programlisting> |
||||
|
||||
Begin executing a foreign scan. This is called during executor startup. |
||||
It should perform any initialization needed before the scan can start. |
||||
The <structname>ForeignScanState</> node has already been created, but |
||||
its <structfield>fdw_state</> field is still NULL. Information about |
||||
the table to scan is accessible through the |
||||
<structname>ForeignScanState</> node (in particular, from the underlying |
||||
<structname>ForeignScan</> plan node, which contains a pointer to the |
||||
<structname>FdwPlan</> structure returned by |
||||
<function>PlanForeignScan</>). |
||||
</para> |
||||
|
||||
<para> |
||||
Note that when <literal>(eflags & EXEC_FLAG_EXPLAIN_ONLY)</> is |
||||
true, this function should not perform any externally-visible actions; |
||||
it should only do the minimum required to make the node state valid |
||||
for <function>ExplainForeignScan</> and <function>EndForeignScan</>. |
||||
</para> |
||||
|
||||
<para> |
||||
<programlisting> |
||||
TupleTableSlot * |
||||
IterateForeignScan (ForeignScanState *node); |
||||
</programlisting> |
||||
|
||||
Fetch one row from the foreign source, returning it in a tuple table slot |
||||
(the node's <structfield>ScanTupleSlot</> should be used for this |
||||
purpose). Return NULL if no more rows are available. The tuple table |
||||
slot infrastructure allows either a physical or virtual tuple to be |
||||
returned; in most cases the latter choice is preferable from a |
||||
performance standpoint. Note that this is called in a short-lived memory |
||||
context that will be reset between invocations. Create a memory context |
||||
in <function>BeginForeignScan</> if you need longer-lived storage, or use |
||||
the <structfield>es_query_cxt</> of the node's <structname>EState</>. |
||||
</para> |
||||
|
||||
<para> |
||||
The rows returned must match the column signature of the foreign table |
||||
being scanned. If you choose to optimize away fetching columns that |
||||
are not needed, you should insert nulls in those column positions. |
||||
</para> |
||||
|
||||
<para> |
||||
<programlisting> |
||||
void |
||||
ReScanForeignScan (ForeignScanState *node); |
||||
</programlisting> |
||||
|
||||
Restart the scan from the beginning. Note that any parameters the |
||||
scan depends on may have changed value, so the new scan does not |
||||
necessarily return exactly the same rows. |
||||
</para> |
||||
|
||||
<para> |
||||
<programlisting> |
||||
void |
||||
EndForeignScan (ForeignScanState *node); |
||||
</programlisting> |
||||
|
||||
End the scan and release resources. It is normally not important |
||||
to release palloc'd memory, but for example open files and connections |
||||
to remote servers should be cleaned up. |
||||
</para> |
||||
|
||||
<para> |
||||
The <structname>FdwRoutine</> and <structname>FdwPlan</> struct types |
||||
are declared in <filename>src/include/foreign/fdwapi.h</>, which see |
||||
for additional details. |
||||
</para> |
||||
|
||||
</sect1> |
||||
|
||||
</chapter> |
||||
@ -0,0 +1,209 @@ |
||||
/*-------------------------------------------------------------------------
|
||||
* |
||||
* nodeForeignscan.c |
||||
* Routines to support scans of foreign tables |
||||
* |
||||
* Portions Copyright (c) 1996-2011, PostgreSQL Global Development Group |
||||
* Portions Copyright (c) 1994, Regents of the University of California |
||||
* |
||||
* |
||||
* IDENTIFICATION |
||||
* src/backend/executor/nodeForeignscan.c |
||||
* |
||||
*------------------------------------------------------------------------- |
||||
*/ |
||||
/*
|
||||
* INTERFACE ROUTINES |
||||
* |
||||
* ExecForeignScan scans a foreign table. |
||||
* ExecInitForeignScan creates and initializes state info. |
||||
* ExecReScanForeignScan rescans the foreign relation. |
||||
* ExecEndForeignScan releases any resources allocated. |
||||
*/ |
||||
#include "postgres.h" |
||||
|
||||
#include "executor/executor.h" |
||||
#include "executor/nodeForeignscan.h" |
||||
#include "foreign/fdwapi.h" |
||||
|
||||
static TupleTableSlot *ForeignNext(ForeignScanState *node); |
||||
static bool ForeignRecheck(ForeignScanState *node, TupleTableSlot *slot); |
||||
|
||||
|
||||
/* ----------------------------------------------------------------
|
||||
* ForeignNext |
||||
* |
||||
* This is a workhorse for ExecForeignScan |
||||
* ---------------------------------------------------------------- |
||||
*/ |
||||
static TupleTableSlot * |
||||
ForeignNext(ForeignScanState *node) |
||||
{ |
||||
TupleTableSlot *slot; |
||||
ForeignScan *plan = (ForeignScan *) node->ss.ps.plan; |
||||
ExprContext *econtext = node->ss.ps.ps_ExprContext; |
||||
MemoryContext oldcontext; |
||||
|
||||
/* Call the Iterate function in short-lived context */ |
||||
oldcontext = MemoryContextSwitchTo(econtext->ecxt_per_tuple_memory); |
||||
slot = node->fdwroutine->IterateForeignScan(node); |
||||
MemoryContextSwitchTo(oldcontext); |
||||
|
||||
/*
|
||||
* If any system columns are requested, we have to force the tuple into |
||||
* physical-tuple form to avoid "cannot extract system attribute from |
||||
* virtual tuple" errors later. We also insert a valid value for |
||||
* tableoid, which is the only actually-useful system column. |
||||
*/ |
||||
if (plan->fsSystemCol && !TupIsNull(slot)) |
||||
{ |
||||
HeapTuple tup = ExecMaterializeSlot(slot); |
||||
|
||||
tup->t_tableOid = RelationGetRelid(node->ss.ss_currentRelation); |
||||
} |
||||
|
||||
return slot; |
||||
} |
||||
|
||||
/*
|
||||
* ForeignRecheck -- access method routine to recheck a tuple in EvalPlanQual |
||||
*/ |
||||
static bool |
||||
ForeignRecheck(ForeignScanState *node, TupleTableSlot *slot) |
||||
{ |
||||
/* There are no access-method-specific conditions to recheck. */ |
||||
return true; |
||||
} |
||||
|
||||
/* ----------------------------------------------------------------
|
||||
* ExecForeignScan(node) |
||||
* |
||||
* Fetches the next tuple from the FDW, checks local quals, and |
||||
* returns it. |
||||
* We call the ExecScan() routine and pass it the appropriate |
||||
* access method functions. |
||||
* ---------------------------------------------------------------- |
||||
*/ |
||||
TupleTableSlot * |
||||
ExecForeignScan(ForeignScanState *node) |
||||
{ |
||||
return ExecScan((ScanState *) node, |
||||
(ExecScanAccessMtd) ForeignNext, |
||||
(ExecScanRecheckMtd) ForeignRecheck); |
||||
} |
||||
|
||||
|
||||
/* ----------------------------------------------------------------
|
||||
* ExecInitForeignScan |
||||
* ---------------------------------------------------------------- |
||||
*/ |
||||
ForeignScanState * |
||||
ExecInitForeignScan(ForeignScan *node, EState *estate, int eflags) |
||||
{ |
||||
ForeignScanState *scanstate; |
||||
Relation currentRelation; |
||||
FdwRoutine *fdwroutine; |
||||
|
||||
/* check for unsupported flags */ |
||||
Assert(!(eflags & (EXEC_FLAG_BACKWARD | EXEC_FLAG_MARK))); |
||||
|
||||
/*
|
||||
* create state structure |
||||
*/ |
||||
scanstate = makeNode(ForeignScanState); |
||||
scanstate->ss.ps.plan = (Plan *) node; |
||||
scanstate->ss.ps.state = estate; |
||||
|
||||
/*
|
||||
* Miscellaneous initialization |
||||
* |
||||
* create expression context for node |
||||
*/ |
||||
ExecAssignExprContext(estate, &scanstate->ss.ps); |
||||
|
||||
scanstate->ss.ps.ps_TupFromTlist = false; |
||||
|
||||
/*
|
||||
* initialize child expressions |
||||
*/ |
||||
scanstate->ss.ps.targetlist = (List *) |
||||
ExecInitExpr((Expr *) node->scan.plan.targetlist, |
||||
(PlanState *) scanstate); |
||||
scanstate->ss.ps.qual = (List *) |
||||
ExecInitExpr((Expr *) node->scan.plan.qual, |
||||
(PlanState *) scanstate); |
||||
|
||||
/*
|
||||
* tuple table initialization |
||||
*/ |
||||
ExecInitResultTupleSlot(estate, &scanstate->ss.ps); |
||||
ExecInitScanTupleSlot(estate, &scanstate->ss); |
||||
|
||||
/*
|
||||
* open the base relation and acquire appropriate lock on it. |
||||
*/ |
||||
currentRelation = ExecOpenScanRelation(estate, node->scan.scanrelid); |
||||
scanstate->ss.ss_currentRelation = currentRelation; |
||||
|
||||
/*
|
||||
* get the scan type from the relation descriptor. |
||||
*/ |
||||
ExecAssignScanType(&scanstate->ss, RelationGetDescr(currentRelation)); |
||||
|
||||
/*
|
||||
* Initialize result tuple type and projection info. |
||||
*/ |
||||
ExecAssignResultTypeFromTL(&scanstate->ss.ps); |
||||
ExecAssignScanProjectionInfo(&scanstate->ss); |
||||
|
||||
/*
|
||||
* Acquire function pointers from the FDW's handler, and init fdw_state. |
||||
*/ |
||||
fdwroutine = GetFdwRoutineByRelId(RelationGetRelid(currentRelation)); |
||||
scanstate->fdwroutine = fdwroutine; |
||||
scanstate->fdw_state = NULL; |
||||
|
||||
/*
|
||||
* Tell the FDW to initiate the scan. |
||||
*/ |
||||
fdwroutine->BeginForeignScan(scanstate, eflags); |
||||
|
||||
return scanstate; |
||||
} |
||||
|
||||
/* ----------------------------------------------------------------
|
||||
* ExecEndForeignScan |
||||
* |
||||
* frees any storage allocated through C routines. |
||||
* ---------------------------------------------------------------- |
||||
*/ |
||||
void |
||||
ExecEndForeignScan(ForeignScanState *node) |
||||
{ |
||||
/* Let the FDW shut down */ |
||||
node->fdwroutine->EndForeignScan(node); |
||||
|
||||
/* Free the exprcontext */ |
||||
ExecFreeExprContext(&node->ss.ps); |
||||
|
||||
/* clean out the tuple table */ |
||||
ExecClearTuple(node->ss.ps.ps_ResultTupleSlot); |
||||
ExecClearTuple(node->ss.ss_ScanTupleSlot); |
||||
|
||||
/* close the relation. */ |
||||
ExecCloseScanRelation(node->ss.ss_currentRelation); |
||||
} |
||||
|
||||
/* ----------------------------------------------------------------
|
||||
* ExecReScanForeignScan |
||||
* |
||||
* Rescans the relation. |
||||
* ---------------------------------------------------------------- |
||||
*/ |
||||
void |
||||
ExecReScanForeignScan(ForeignScanState *node) |
||||
{ |
||||
node->fdwroutine->ReScanForeignScan(node); |
||||
|
||||
ExecScanReScan(&node->ss); |
||||
} |
||||
@ -0,0 +1,24 @@ |
||||
/*-------------------------------------------------------------------------
|
||||
* |
||||
* nodeForeignscan.h |
||||
* |
||||
* |
||||
* |
||||
* Portions Copyright (c) 1996-2011, PostgreSQL Global Development Group |
||||
* Portions Copyright (c) 1994, Regents of the University of California |
||||
* |
||||
* src/include/executor/nodeForeignscan.h |
||||
* |
||||
*------------------------------------------------------------------------- |
||||
*/ |
||||
#ifndef NODEFOREIGNSCAN_H |
||||
#define NODEFOREIGNSCAN_H |
||||
|
||||
#include "nodes/execnodes.h" |
||||
|
||||
extern ForeignScanState *ExecInitForeignScan(ForeignScan *node, EState *estate, int eflags); |
||||
extern TupleTableSlot *ExecForeignScan(ForeignScanState *node); |
||||
extern void ExecEndForeignScan(ForeignScanState *node); |
||||
extern void ExecReScanForeignScan(ForeignScanState *node); |
||||
|
||||
#endif /* NODEFOREIGNSCAN_H */ |
||||
@ -0,0 +1,98 @@ |
||||
/*-------------------------------------------------------------------------
|
||||
* |
||||
* fdwapi.h |
||||
* API for foreign-data wrappers |
||||
* |
||||
* Copyright (c) 2010-2011, PostgreSQL Global Development Group |
||||
* |
||||
* src/include/foreign/fdwapi.h |
||||
* |
||||
*------------------------------------------------------------------------- |
||||
*/ |
||||
#ifndef FDWAPI_H |
||||
#define FDWAPI_H |
||||
|
||||
#include "nodes/execnodes.h" |
||||
#include "nodes/relation.h" |
||||
|
||||
/* To avoid including explain.h here, reference ExplainState thus: */ |
||||
struct ExplainState; |
||||
|
||||
|
||||
/*
|
||||
* FdwPlan is the information returned to the planner by PlanForeignScan. |
||||
*/ |
||||
typedef struct FdwPlan |
||||
{ |
||||
NodeTag type; |
||||
|
||||
/*
|
||||
* Cost estimation info. The startup_cost is time before retrieving |
||||
* the first row, so it should include costs of connecting to the remote |
||||
* host, sending over the query, etc. Note that PlanForeignScan also |
||||
* ought to set baserel->rows and baserel->width if it can produce any |
||||
* usable estimates of those values. |
||||
*/ |
||||
Cost startup_cost; /* cost expended before fetching any tuples */ |
||||
Cost total_cost; /* total cost (assuming all tuples fetched) */ |
||||
|
||||
/*
|
||||
* FDW private data, which will be available at execution time. |
||||
* |
||||
* Note that everything in this list must be copiable by copyObject(). |
||||
* One way to store an arbitrary blob of bytes is to represent it as a |
||||
* bytea Const. Usually, though, you'll be better off choosing a |
||||
* representation that can be dumped usefully by nodeToString(). |
||||
*/ |
||||
List *fdw_private; |
||||
} FdwPlan; |
||||
|
||||
|
||||
/*
|
||||
* Callback function signatures --- see fdwhandler.sgml for more info. |
||||
*/ |
||||
|
||||
typedef FdwPlan * (*PlanForeignScan_function) (Oid foreigntableid, |
||||
PlannerInfo *root, |
||||
RelOptInfo *baserel); |
||||
|
||||
typedef void (*ExplainForeignScan_function) (ForeignScanState *node, |
||||
struct ExplainState *es); |
||||
|
||||
typedef void (*BeginForeignScan_function) (ForeignScanState *node, |
||||
int eflags); |
||||
|
||||
typedef TupleTableSlot * (*IterateForeignScan_function) (ForeignScanState *node); |
||||
|
||||
typedef void (*ReScanForeignScan_function) (ForeignScanState *node); |
||||
|
||||
typedef void (*EndForeignScan_function) (ForeignScanState *node); |
||||
|
||||
|
||||
/*
|
||||
* FdwRoutine is the struct returned by a foreign-data wrapper's handler |
||||
* function. It provides pointers to the callback functions needed by the |
||||
* planner and executor. |
||||
* |
||||
* Currently, all functions must be supplied. Later there may be optional |
||||
* additions. It's recommended that the handler initialize the struct with |
||||
* makeNode(FdwRoutine) so that all fields are set to zero. |
||||
*/ |
||||
typedef struct FdwRoutine |
||||
{ |
||||
NodeTag type; |
||||
|
||||
PlanForeignScan_function PlanForeignScan; |
||||
ExplainForeignScan_function ExplainForeignScan; |
||||
BeginForeignScan_function BeginForeignScan; |
||||
IterateForeignScan_function IterateForeignScan; |
||||
ReScanForeignScan_function ReScanForeignScan; |
||||
EndForeignScan_function EndForeignScan; |
||||
} FdwRoutine; |
||||
|
||||
|
||||
/* Functions in foreign/foreign.c */ |
||||
extern FdwRoutine *GetFdwRoutine(Oid fdwhandler); |
||||
extern FdwRoutine *GetFdwRoutineByRelId(Oid relid); |
||||
|
||||
#endif /* FDWAPI_H */ |
||||
Loading…
Reference in new issue