17 Commits (70d6226e4fba26765877fc3c2ec6c468d3ff4084)

Author	SHA1	Message	Date
Peter Eisentraut	70d6226e4f	Use portal pinning in PL/Perl and PL/Python ... PL/pgSQL "pins" internally generated portals so that user code cannot close them by guessing their names. Add this functionality to PL/Perl and PL/Python as well, preventing users from manually closing cursors created by spi_query and plpy.cursor, respectively. (PL/Tcl does not currently offer any cursor functionality.)	8 years ago
Peter Eisentraut	d0aa965c0a	Consistently catch errors from Python _New() functions ... Python Py*_New() functions can fail and return NULL in out-of-memory conditions. The previous code handled that inconsistently or not at all. This change organizes that better. If we are in a function that is called from Python, we just check for failure and return NULL ourselves, which will cause any exception information to be passed up. If we are called from PostgreSQL, we consistently create an "out of memory" error. Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>	8 years ago
Tom Lane	687f096ea9	Make PL/Python handle domain-type conversions correctly. ... Fix PL/Python so that it can handle domains over composite, and so that it enforces domain constraints correctly in other cases that were not always done properly before. Notably, it didn't do arrays of domains right (oversight in commit c12d570fa), and it failed to enforce domain constraints when returning a composite type containing a domain field, and if a transform function is being used for a domain's base type then it failed to enforce domain constraints on the result. Also, in many places it missed checking domain constraints on null values, because the plpy_typeio code simply wasn't called for Py_None. Rather than try to band-aid these problems, I made a significant refactoring of the plpy_typeio logic. The existing design of recursing for array and composite members is extended to also treat domains as containers requiring recursion, and the APIs for the module are cleaned up and simplified. The patch also modifies plpy_typeio to rely on the typcache more than it did before (which was pretty much not at all). This reduces the need for repetitive lookups, and lets us get rid of an ad-hoc scheme for detecting changes in composite types. I added a couple of small features to typcache to help with that. Although some of this is fixing bugs that long predate v11, I don't think we should risk a back-patch: it's a significant amount of code churn, and there've been no complaints from the field about the bugs. Tom Lane, reviewed by Anthony Bykov Discussion: https://postgr.es/m/24449.1509393613@sss.pgh.pa.us	8 years ago
Peter Eisentraut	0008a106d4	Use Py_RETURN_NONE where suitable ... This is more idiomatic style and available as of Python 2.4, which is our minimum.	8 years ago
Tom Lane	382ceffdf7	Phase 3 of pgindent updates. ... Don't move parenthesized lines to the left, even if that means they flow past the right margin. By default, BSD indent lines up statement continuation lines that are within parentheses so that they start just to the right of the preceding left parenthesis. However, traditionally, if that resulted in the continuation line extending to the right of the desired right margin, then indent would push it left just far enough to not overrun the margin, if it could do so without making the continuation line start to the left of the current statement indent. That makes for a weird mix of indentations unless one has been completely rigid about never violating the 80-column limit. This behavior has been pretty universally panned by Postgres developers. Hence, disable it with indent's new -lpl switch, so that parenthesized lines are always lined up with the preceding left paren. This patch is much less interesting than the first round of indent changes, but also bulkier, so I thought it best to separate the effects. Discussion: https://postgr.es/m/E1dAmxK-0006EE-1r@gemulon.postgresql.org Discussion: https://postgr.es/m/30527.1495162840@sss.pgh.pa.us	9 years ago
Peter Eisentraut	70ec3f1f8f	PL/Python: Add cursor and execute methods to plan object ... Instead of plan = plpy.prepare(...) res = plpy.execute(plan, ...) you can now write plan = plpy.prepare(...) res = plan.execute(...) or even res = plpy.prepare(...).execute(...) and similarly for the cursor() method. This is more in object oriented style, and makes the hybrid nature of the existing execute() function less confusing. Reviewed-by: Andrew Dunstan <andrew.dunstan@2ndquadrant.com>	9 years ago
Peter Eisentraut	eaed88ce12	Add function name to PyArg_ParseTuple() ... This causes the supplied function name to appear in any error message, making the error message friendlier and relieving us from having to provide our own in some cases.	9 years ago
Heikki Linnakangas	510e1b8ecf	Give a hint, when [] is incorrectly used for a composite type in array. ... That used to be accepted, so let's try to give a hint to users on why their PL/python functions no longer work. Reviewed by Pavel Stehule. Discussion: <CAH38_tmbqwaUyKs9yagyRra=SMaT45FPBxk1pmTYcM0TyXGG7Q@mail.gmail.com>	9 years ago
Tom Lane	ea268cdc9a	Add macros to make AllocSetContextCreate() calls simpler and safer. ... I found that half a dozen (nearly 5%) of our AllocSetContextCreate calls had typos in the context-sizing parameters. While none of these led to especially significant problems, they did create minor inefficiencies, and it's now clear that expecting people to copy-and-paste those calls accurately is not a great idea. Let's reduce the risk of future errors by introducing single macros that encapsulate the common use-cases. Three such macros are enough to cover all but two special-purpose contexts; those two calls can be left as-is, I think. While this patch doesn't in itself improve matters for third-party extensions, it doesn't break anything for them either, and they can gradually adopt the simplified notation over time. In passing, change TopMemoryContext to use the default allocation parameters. Formerly it could only be extended 8K at a time. That was probably reasonable when this code was written; but nowadays we create many more contexts than we did then, so that it's not unusual to have a couple hundred K in TopMemoryContext, even without considering various dubious code that sticks other things there. There seems no good reason not to let it use growing blocks like most other contexts. Back-patch to 9.6, mostly because that's still close enough to HEAD that it's easy to do so, and keeping the branches in sync can be expected to avoid some future back-patching pain. The bugs fixed by these changes don't seem to be significant enough to justify fixing them further back. Discussion: <21072.1472321324@sss.pgh.pa.us>	10 years ago
Tom Lane	23a27b039d	Widen query numbers-of-tuples-processed counters to uint64. ... This patch widens SPI_processed, EState's es_processed field, PortalData's portalPos field, FuncCallContext's call_cntr and max_calls fields, ExecutorRun's count argument, PortalRunFetch's result, and the max number of rows in a SPITupleTable to uint64, and deals with (I hope) all the ensuing fallout. Some of these values were declared uint32 before, and others "long". I also removed PortalData's posOverflow field, since that logic seems pretty useless given that portalPos is now always 64 bits. The user-visible results are that command tags for SELECT etc will correctly report tuple counts larger than 4G, as will plpgsql's GET GET DIAGNOSTICS ... ROW_COUNT command. Queries processing more tuples than that are still not exactly the norm, but they're becoming more common. Most values associated with FETCH/MOVE distances, such as PortalRun's count argument and the count argument of most SPI functions that have one, remain declared as "long". It's not clear whether it would be worth promoting those to int64; but it would definitely be a large dollop of additional API churn on top of this, and it would only help 32-bit platforms which seem relatively less likely to see any benefit. Andreas Scherbaum, reviewed by Christian Ullrich, additional hacking by me	10 years ago
Tom Lane	8c75ad436f	Fix memory leaks in PL/Python. ... Previously, plpython was in the habit of allocating a lot of stuff in TopMemoryContext, and it was very slipshod about making sure that stuff got cleaned up; in particular, use of TopMemoryContext as fn_mcxt for function calls represents an unfixable leak, since we generally don't know what the called function might have allocated in fn_mcxt. This results in session-lifespan leakage in certain usage scenarios, as for example in a case reported by Ed Behn back in July. To fix, get rid of all the retail allocations in TopMemoryContext. All long-lived allocations are now made in sub-contexts that are associated with specific objects (either pl/python procedures, or Python-visible objects such as cursors and plans). We can clean these up when the associated object is deleted. I went so far as to get rid of PLy_malloc completely. There were a couple of places where it could still have been used safely, but on the whole it was just an invitation to bad coding. Haribabu Kommi, based on a draft patch by Heikki Linnakangas; some further work by me	10 years ago
Tom Lane	3d13623d75	Prevent leakage of SPI tuple tables during subtransaction abort. ... plpgsql often just remembers SPI-result tuple tables in local variables, and has no mechanism for freeing them if an ereport(ERROR) causes an escape out of the execution function whose local variable it is. In the original coding, that wasn't a problem because the tuple table would be cleaned up when the function's SPI context went away during transaction abort. However, once plpgsql grew the ability to trap exceptions, repeated trapping of errors within a function could result in significant intra-function-call memory leakage, as illustrated in bug #8279 from Chad Wagner. We could fix this locally in plpgsql with a bunch of PG_TRY/PG_CATCH coding, but that would be tedious, probably slow, and prone to bugs of omission; moreover it would do nothing for similar risks elsewhere. What seems like a better plan is to make SPI itself responsible for freeing tuple tables at subtransaction abort. This patch attacks the problem that way, keeping a list of live tuple tables within each SPI function context. Currently, such freeing is automatic for tuple tables made within the failed subtransaction. We might later add a SPI call to mark a tuple table as not to be freed this way, allowing callers to opt out; but until someone exhibits a clear use-case for such behavior, it doesn't seem worth bothering. A very useful side-effect of this change is that SPI_freetuptable() can now defend itself against bad calls, such as duplicate free requests; this should make things more robust in many places. (In particular, this reduces the risks involved if a third-party extension contains now-redundant SPI_freetuptable() calls in error cleanup code.) Even though the leakage problem is of long standing, it seems imprudent to back-patch this into stable branches, since it does represent an API semantics change for SPI users. We'll patch this in 9.3, but live with the leakage in older branches.	13 years ago
Bruce Momjian	927d61eeff	Run pgindent on 9.2 source tree in preparation for first 9.3 ... commit-fest.	14 years ago
Tom Lane	ed75380bda	Create a stack of pl/python "execution contexts". ... This replaces the former global variable PLy_curr_procedure, and provides a place to stash per-call-level information. In particular we create a per-call-level scratch memory context. For the moment, the scratch context is just used to avoid leaking memory from datatype output function calls in PLyDict_FromTuple. There probably will be more use-cases in future. Although this is a fix for a pre-existing memory leakage bug, it seems sufficiently invasive to not want to back-patch; it feels better as part of the major rearrangement of plpython code that we've already done as part of 9.2. Jan Urbański	14 years ago
Peter Eisentraut	26e89e7f23	Fix typos	14 years ago
Peter Eisentraut	f9de1e9a96	PL/Python: Add argument names to function declarations ... For easier source reading	14 years ago
Peter Eisentraut	147c248254	Split plpython.c into smaller pieces ... This moves the code around from one huge file into hopefully logical and more manageable modules. For the most part, the code itself was not touched, except: PLy_function_handler and PLy_trigger_handler were renamed to PLy_exec_function and PLy_exec_trigger, because they were not actually handlers in the PL handler sense, and it makes the naming more similar to the way PL/pgSQL is organized. The initialization of the procedure caches was separated into a new function init_procedure_caches to keep the hash tables private to plpy_procedures.c. Jan Urbański and Peter Eisentraut	14 years ago