postgres

Commit Graph

Author	SHA1	Message	Date
Robert Haas	bc7fcab5e3	Read from the same worker repeatedly until it returns no tuple. The original coding read tuples from workers in round-robin fashion, but performance testing shows that it works much better to read enough to empty one queue before moving on to the next. I believe the reason for this is that, with the old approach, we could easily wake up a worker repeatedly to write only one new tuple into the shm_mq each time. With this approach, by the time the process gets scheduled, it has a decent chance of being able to fill the entire buffer in one go. Patch by me. Dilip Kumar helped with performance testing.	10 years ago
Robert Haas	3690dc6b03	Fix obsolete comment. It's amazing how fast things become obsolete these days. Amit Langote	10 years ago
Robert Haas	6c878a7553	Avoid server crash when worker registration fails at execution time. The previous coding attempts to destroy the DSM in this case, but child nodes might have stored data there and still be holding onto pointers in this case. So don't do that. Also, free the reader array instead of leaking it. Extracted from two different patch versions both by Amit Kapila.	10 years ago
Robert Haas	166b61a88e	Avoid aggregating worker instrumentation multiple times. Amit Kapila, per design ideas from me.	10 years ago
Tom Lane	b05ae27e9a	Add missing "static" qualifier. Per buildfarm member pademelon.	10 years ago
Robert Haas	bf3d015631	Fix rebasing mistake in nodeGather.c The patches committed as `6e71dd7ce9` and `3a1f8611f2` were developed in parallel but dependent on each other in a way that I failed to notice. This patch to fix the problem was prepared by Amit Kapila.	10 years ago
Robert Haas	6e71dd7ce9	Modify tqueue infrastructure to support transient record types. Commit `4a4e6893aa`, which introduced this mechanism, failed to account for the fact that the RECORD pseudo-type uses transient typmods that are only meaningful within a single backend. Transferring such tuples without modification between two cooperating backends does not work. This commit installs a system for passing the tuple descriptors over the same shm_mq being used to send the tuples themselves. The two sides might not assign the same transient typmod to any given tuple descriptor, so we must also substitute the appropriate receiver-side typmod for the one used by the sender. That adds some CPU overhead, but still seems better than being unable to pass records between cooperating parallel processes. Along the way, move the logic for handling multiple tuple queues from tqueue.c to nodeGather.c; tqueue.c now provides a TupleQueueReader, which reads from a single queue, rather than a TupleQueueFunnel, which potentially reads from multiple queues. This change was suggested previously as a way to make sure that nodeGather.c rather than tqueue.c had policy control over the order in which to read from queues, but it wasn't clear to me until now how good an idea it was. typmod mapping needs to be performed separately for each queue, and it is much simpler if the tqueue.c code handles that and leaves multiplexing multiple queues to higher layers of the stack.	10 years ago
Robert Haas	3a1f8611f2	Update parallel executor support to reuse the same DSM. Commit `b0b0d84b3d` purported to make it possible to relaunch workers using the same parallel context, but it had an unpleasant race condition: we might reinitialize after the workers have sent their last control message but before they have dettached the DSM, leaving to crashes. Repair by introducing a new ParallelContext operation, ReinitializeParallelDSM. Adjust execParallel.c to use this new support, so that we can rescan a Gather node by relaunching workers but without needing to recreate the DSM. Amit Kapila, with some adjustments by me. Extracted from latest parallel sequential scan patch.	10 years ago
Robert Haas	8538a63070	Make Gather node projection-capable. The original Gather code failed to mark a Gather node as not able to do projection, but it couldn't, even though it did call initialize its projection info via ExecAssignProjectionInfo. There doesn't seem to be any good reason for this node not to have projection capability, so clean things up so that it does. Without this, plans using Gather nodes might need to carry extra Result nodes to do projection.	10 years ago
Robert Haas	31ba62ce32	Fix typos in comments. CharSyam	10 years ago
Robert Haas	1a219fa15b	Add header comments to execParallel.c and nodeGather.c. Patch by me, per a note from Simon Riggs. Reviewed by Amit Kapila and Amit Langote.	10 years ago
Robert Haas	bfc78d7196	Rewrite interaction of parallel mode with parallel executor support. In the previous coding, before returning from ExecutorRun, we'd shut down all parallel workers. This was dead wrong if ExecutorRun was called with a non-zero tuple count; it had the effect of truncating the query output. To fix, give ExecutePlan control over whether to enter parallel mode, and have it refuse to do so if the tuple count is non-zero. Rewrite the Gather logic so that it can cope with being called outside parallel mode. Commit `7aea8e4f2d` is largely to blame for this problem, though this patch modifies some subsequently-committed code which relied on the guarantees it purported to make.	10 years ago
Tom Lane	bf686796a0	Add missing "static" specifier. Per buildfarm (pademelon, at least, doesn't like this).	10 years ago
Robert Haas	3bd909b220	Add a Gather executor node. A Gather executor node runs any number of copies of a plan in an equal number of workers and merges all of the results into a single tuple stream. It can also run the plan itself, if the workers are unavailable or haven't started up yet. It is intended to work with the Partial Seq Scan node which will be added in future commits. It could also be used to implement parallel query of a different sort by itself, without help from Partial Seq Scan, if the single_copy mode is used. In that mode, a worker executes the plan, and the parallel leader does not, merely collecting the worker's results. So, a Gather node could be inserted into a plan to split the execution of that plan across two processes. Nested Gather nodes aren't currently supported, but we might want to add support for that in the future. There's nothing in the planner to actually generate Gather nodes yet, so it's not quite time to break out the champagne. But we're getting close. Amit Kapila. Some designs suggestions were provided by me, and I also reviewed the patch. Single-copy mode, documentation, and other minor changes also by me.	10 years ago

14 Commits (dfcd9cb30237f882b7308bdcbfb0318b22b1e224)