postgres

History

Tom Lane 9fa6f00b13 Rethink MemoryContext creation to improve performance. This patch makes a number of interrelated changes to reduce the overhead involved in creating/deleting memory contexts. The key ideas are: * Include the AllocSetContext header of an aset.c context in its first malloc request, rather than allocating it separately in TopMemoryContext. This means that we now always create an initial or "keeper" block in an aset, even if it never receives any allocation requests. * Create freelists in which we can save and recycle recently-destroyed asets (this idea is due to Robert Haas). * In the common case where the name of a context is a constant string, just store a pointer to it in the context header, rather than copying the string. The first change eliminates a palloc/pfree cycle per context, and also avoids bloat in TopMemoryContext, at the price that creating a context now involves a malloc/free cycle even if the context never receives any allocations. That would be a loser for some common usage patterns, but recycling short-lived contexts via the freelist eliminates that pain. Avoiding copying constant strings not only saves strlen() and strcpy() overhead, but is an essential part of the freelist optimization because it makes the context header size constant. Currently we make no attempt to use the freelist for contexts with non-constant names. (Perhaps someday we'll need to think harder about that, but in current usage, most contexts with custom names are long-lived anyway.) The freelist management in this initial commit is pretty simplistic, and we might want to refine it later --- but in common workloads that will never matter because the freelists will never get full anyway. To create a context with a non-constant name, one is now required to call AllocSetContextCreateExtended and specify the MEMCONTEXT_COPY_NAME option. AllocSetContextCreate becomes a wrapper macro, and it includes a test that will complain about non-string-literal context name parameters on gcc and similar compilers. An unfortunate side effect of making AllocSetContextCreate a macro is that one is now required to use the size parameter abstraction macros (ALLOCSET_DEFAULT_SIZES and friends) with it; the pre-9.6 habit of writing out individual size parameters no longer works unless you switch to AllocSetContextCreateExtended. Internally to the memory-context-related modules, the context creation APIs are simplified, removing the rather baroque original design whereby a context-type module called mcxt.c which then called back into the context-type module. That saved a bit of code duplication, but not much, and it prevented context-type modules from exercising control over the allocation of context headers. In passing, I converted the test-and-elog validation of aset size parameters into Asserts to save a few more cycles. The original thought was that callers might compute size parameters on the fly, but in practice nobody does that, so it's useless to expend cycles on checking those numbers in production builds. Also, mark the memory context method-pointer structs "const", just for cleanliness. Discussion: https://postgr.es/m/2264.1512870796@sss.pgh.pa.us		8 years ago
..
Makefile	Hash tables backed by DSA shared memory.	8 years ago
README	Fix typos, update README.	11 years ago
binaryheap.c	Initial pgindent run with pg_bsd_indent version 2.0.	9 years ago
bipartite_match.c	Update copyright via script for 2017	9 years ago
dshash.c	Fix uninitialized variable in dshash.c.	8 years ago
hyperloglog.c	Update copyright via script for 2017	9 years ago
ilist.c	Phase 2 of pgindent updates.	9 years ago
knapsack.c	Rethink MemoryContext creation to improve performance.	8 years ago
pairingheap.c	Phase 3 of pgindent updates.	9 years ago
rbtree.c	Remove pre-order and post-order traversal logic for red-black trees.	8 years ago
stringinfo.c	Allow to avoid NUL-byte management for stringinfos and use in format.c.	8 years ago

README

This directory contains a general purpose data structures, for use anywhere
in the backend:

binaryheap.c - a binary heap

hyperloglog.c - a streaming cardinality estimator

pairingheap.c - a pairing heap

rbtree.c - a red-black tree

ilist.c - single and double-linked lists.

stringinfo.c - an extensible string type


Aside from the inherent characteristics of the data structures, there are a
few practical differences between the binary heap and the pairing heap. The
binary heap is fully allocated at creation, and cannot be expanded beyond the
allocated size. The pairing heap on the other hand has no inherent maximum
size, but the caller needs to allocate each element being stored in the heap,
while the binary heap works with plain Datums or pointers.

The linked-lists in ilist.c can be embedded directly into other structs, as
opposed to the List interface in nodes/pg_list.h.