26 Commits (728a3e73e96173bb0b0b2afc0da061991f9e3d5d)

Author	SHA1	Message	Date
Peter Eisentraut	728a3e73e9	Switch pg_basebackup commands in Postgres.pm to use --nosync ... On slow machines, this greatly reduces the I/O pressure induced by the tests. From: Michael Paquier <michael.paquier@gmail.com>	9 years ago
Peter Eisentraut	8b845520fb	Add tests for various connection string issues ... Add tests for consistent support of connection strings in frontend programs as well as proper handling of unusual characters in database and user names. These tests were developed for the issues of CVE-2016-5424. To allow testing of names with spaces, change the pg_regress command-line options --create-role and --dbname to split their arguments by comma only, not space or comma as before. Only commas were actually used in existing uses. Noah Misch, Michael Paquier, Peter Eisentraut	9 years ago
Tom Lane	b5bce6c1ec	Final pgindent + perltidy run for 9.6.	9 years ago
Alvaro Herrera	2a0f89cd71	Give recovery tests more time to finish ... These tests are currently only running in buildfarm member hamster, which is purposefully very slow. This suite has failed a couple of times recently because of timeouts, so increase the allowed number of iterations to avoid spurious failures. Author: Michaël Paquier	9 years ago
Tom Lane	30b2731bd2	Fix TAP tests and MSVC scripts for pathnames with spaces. ... Change assorted places in our Perl code that did things like system("prog $path/file"); to do it more like system('prog', "$path/file"); which is safe against spaces and other special characters in the path variable. The latter was already the prevailing style, but a few bits of code hadn't gotten this memo. Back-patch to 9.4 as relevant. Michael Paquier, Kyotaro Horiguchi Discussion: <20160704.160213.111134711.horiguchi.kyotaro@lab.ntt.co.jp>	9 years ago
Noah Misch	3be0a62ffe	Finish pgindent run for 9.6: Perl files.	9 years ago
Tom Lane	08af921906	Fix order of shutdown cleanup operations in PostgresNode.pm. ... Previously, database clusters created by a TAP test were shut down by DESTROY methods attached to the PostgresNode objects representing them. The trouble with that is that if the objects survive into the final global destruction phase (which they do), Perl executes the DESTROY methods in an unspecified order. Thus, the order of shutdown of multiple clusters was indeterminate, which might lead to not-very-reproducible errors getting logged (eg from a slave whose master might or might not get killed first). Worse, the File::Temp objects representing the temporary PGDATA directories might get destroyed before the PostgresNode objects, resulting in attempts to delete PGDATA directories that still have live servers in them. On Windows, this would lead to directory deletion failures; on Unix, it usually had no effects worse than erratic "could not open temporary statistics file "pg_stat/global.tmp": No such file or directory" log messages. While none of this would affect the reported result of the TAP test, which is already determined, it could be very confusing when one is trying to understand from the logs what went wrong with a failed test. To fix, do the postmaster shutdowns in an END block rather than at object destruction time. The END block will execute at a well-defined (and reasonable) time during script termination, and it will stop the postmasters in order of PostgresNode object creation. (Perhaps we should change that to be reverse order of creation, but the main point here is that we now have control which we did not before.) Use "pg_ctl stop", not an asynchronous kill(SIGQUIT), so that we wait for the postmasters to shut down before proceeding with directory deletion. Deletion of temporary directories still happens in an unspecified order during global destruction, but I can see no reason to care about that once the postmasters are stopped.	10 years ago
Tom Lane	40e89e2ab8	Try harder to detect a port conflict in PostgresNode.pm. ... Commit fab84c7787 tried to get away without doing an actual bind(), but buildfarm results show that that doesn't get the job done. So we must really bind to the target port --- and at least on my Linux box, we need a listen() as well, or conflicts won't be detected. We rely on SO_REUSEADDR to prevent problems from starting a postmaster on the socket immediately after we've bound to it in the test code. (There may be platforms where that doesn't work too well. But fortunately, we only really care whether this works on Windows, and there the default behavior should be OK.)	10 years ago
Tom Lane	fab84c7787	Improve PostgresNode.pm's logic for detecting already-in-use ports. ... Buildfarm members bowerbird and jacana have shown intermittent "could not bind IPv4 socket" failures in the BinInstallCheck stage since mid-December, shortly after commits 1caef31d9e and 9821492ee4 changed the logic for selecting which port to use in temporary installations. One plausible explanation is that we are randomly selecting ports that are already in use for some non-Postgres purpose. Although the code tried to defend against already-in-use ports, it used pg_isready to probe the port which is quite unhelpful: if some non-Postgres server responds at the given address, pg_isready will generally say "no response", leading to exactly the wrong conclusion about whether the port is free. Instead, let's use a simple TCP connect() call to see if anything answers without making assumptions about what it is. Note that this means there's no direct check for a conflicting Unix socket, but that should be okay because there should be no other Unix sockets in use in the temporary socket directory created for a test run. This is only a partial solution for the TCP case, since if the port number is in use for an outgoing connection rather than a listening socket, we'll fail to detect that. We could try to bind() to the proposed port as a means of detecting that case, but that would introduce its own failure modes, since the system might consider the address to remain reserved for some period of time after we drop the bound socket. Close study of the errors returned by bowerbird and jacana suggests that what we're seeing there may be conflicts with listening not outgoing sockets, so let's try this and see if it improves matters. It's certainly better than what's there now, in any case. Michael Paquier, adjusted by me to work on non-Windows as well as Windows	10 years ago
Fujii Masao	196b72fb9a	Add regression tests for multiple synchronous standbys. ... Authors: Suraj Kharage, Michael Paquier, Masahiko Sawada, refactored by me Reviewed-By: Kyotaro Horiguchi	10 years ago
Alvaro Herrera	9bd61311bd	PostgresNode: initialize $timed_out if passed ... Corrects an oversight in 2c83f435a3 where the $timed_out reference var isn't initialized; using it would require the caller to initialize it beforehand, which is cumbersome. Author: Craig Ringer	10 years ago
Peter Eisentraut	b555ed8102	Merge wal_level "archive" and "hot_standby" into new name "replica" ... The distinction between "archive" and "hot_standby" existed only because at the time "hot_standby" was added, there was some uncertainty about stability. This is now a long time ago. We would like to move forward with simplifying the replication configuration, but this distinction is in the way, because a primary server cannot tell (without asking a standby or predicting the future) which one of these would be the appropriate level. Pick a new name for the combined setting to make it clearer that it covers all (non-logical) backup and replication uses. The old values are still accepted but are converted internally. Reviewed-by: Michael Paquier <michael.paquier@gmail.com> Reviewed-by: David Steele <david@pgmasters.net>	10 years ago
Alvaro Herrera	28f6df3c36	PostgresNode: add backup_fs_hot and backup_fs_cold ... These simple methods rely on RecursiveCopy to create a filesystem-level backup of a server. They aren't currently used anywhere yet,but will be useful for future tests. Author: Craig Ringer Reviewed-By: Michael Paquier, Salvador Fandino, Álvaro Herrera Commitfest-URL: https://commitfest.postgresql.org/9/569/	10 years ago
Alvaro Herrera	2c83f435a3	Rework PostgresNode's psql method ... This makes the psql() method much more capable: it captures both stdout and stderr; it now returns the psql exit code rather than stdout; a timeout can now be specified, as can ON_ERROR_STOP behavior; it gained a new "on_error_die" (defaulting to off) parameter to raise an exception if there's any problem. Finally, additional parameters to psql can be passed if there's need for further tweaking. For convenience, a new safe_psql() method retains much of the old behavior of psql(), except that it uses on_error_die on, so that problems like syntax errors in SQL commands can be detected more easily. Many existing TAP test files now use safe_psql, which is what is really wanted. A couple of ->psql() calls are now added in the commit_ts tests, which verify that the right thing is happening on certain errors. Some ->command_fails() calls in recovery tests that were verifying that psql failed also became ->psql() calls now. Author: Craig Ringer. Some tweaks by Álvaro Herrera Reviewed-By: Michaël Paquier	10 years ago
Alvaro Herrera	7d9a4301c0	perltidy PostgresNode and SimpleTee ... Also, mention in README that Perl files should be perltidy'ed. This isn't really the best place (since we have Perl files elsewhere in the tree) and this is already in pgindent's README, but this subdir is likely to get hacked a whole lot more than the other Perl files, so it seems okay to spend two lines on this. Author: Craig Ringer	10 years ago
Alvaro Herrera	cc6077d4d5	Prefix temp data dirs with the node name ... This makes it easier to relate the temporary data dirs to each node in a test script. Author: Kyotaro Horiguchi Reviewed-By: Craig Ringer, Alvaro Herrera	10 years ago
Alvaro Herrera	74d58425c7	Apply last revision of recovery patch ... I applied the previous-to-last revision of Michaël's submitted patch instead of the last; these two tweaks pointed out by Craig were left out of the previous commit by accident.	10 years ago
Alvaro Herrera	49148645f7	Add a test framework for recovery ... This long-awaited framework is an expansion of the existing PostgresNode stuff to support additional features for recovery testing; the recovery tests included in this commit are a starting point that cover some of the recovery features we have. More scripts are expected to be added later. Author: Michaël Paquier, a bit of help from Amir Rohan Reviewed by: Amir Rohan, Stas Kelvich, Kyotaro Horiguchi, Victor Wagner, Craig Ringer, Álvaro Herrera Discussion: http://www.postgresql.org/message-id/CAB7nPqTf7V6rswrFa=q_rrWeETUWagP=h8LX8XAov2Jcxw0DRg@mail.gmail.com Discussion: http://www.postgresql.org/message-id/trinity-b4a8035d-59af-4c42-a37e-258f0f28e44a-1443795007012@3capp-mailcom-lxa08	10 years ago
Alvaro Herrera	89ac7004da	Move some code from RewindTest into PostgresNode ... Some code in the RewindTest test suite is more generally useful than just for that suite, so put it where other test suites can reach it. Some postgresql.conf parameters change their default values when a cluster is initialized with 'allows_streaming' than the previous behavior; most notably, autovacuum is no longer turned off. (Also, we no longer call pg_ctl promote with -w, but that flag doesn't actually do anything in promote so there's no behavior change.) Author: Michael Paquier	10 years ago
Alvaro Herrera	e64009303d	Add POD docs to PostgresNode ... Also, the dump_info method got split into another method that returns the stuff as a string instead of just printing it to stdout. Add a new README in src/test/perl too. Author: Craig Ringer Reviewed by: Michaël Paquier	10 years ago
Noah Misch	64d89a93c0	In pg_rewind test suite, triple promote timeout to 90s. ... Thirty seconds was not consistently enough for promotion to complete on buildfarm members sungazer and tern. Experiments suggest 43s would have been enough. Back-patch to 9.5, where pg_rewind was introduced.	10 years ago
Alvaro Herrera	c8642d909f	PostgresNode: Add names to nodes ... This makes the log files easier to follow when investigating a test failure. Author: Michael Paquier Review: Noah Misch	10 years ago
Robert Haas	d5563d7df9	psql: Support multiple -c and -f options, and allow mixing them. ... To support this, we must reconcile some historical anomalies in the behavior of -c. In particular, as a backward-incompatibility, -c no longer implies --no-psqlrc. Pavel Stehule (code) and Catalin Iacob (documentation). Review by Michael Paquier and myself. Proposed behavior per Tom Lane.	10 years ago
Alvaro Herrera	7ac5d9b316	PostgresNode: wrap correctly around port number range end ... Per note from Tom Lane	10 years ago
Alvaro Herrera	9821492ee4	Cleanup some problems in new Perl test code ... Noted by Tom Lane: - PostgresNode had a BEGIN block which created files, contrary to perlmod suggestions to do that only on INIT blocks. - Assign ports randomly rather than starting from 90600. Noted by Noah Misch: - Change use of no-longer-set PGPORT environment variable to $node->port - Don't start a server in pg_controldata test - PostgresNode was reading the PID file incorrectly; test the right thing, and chomp the line we read from the PID file. - Remove an unused $devnull variable - Use 'pg_ctl kill' instead of "kill" directly, for Windos portability. - Make server log names more informative. Author: Michael Paquier	10 years ago
Alvaro Herrera	1caef31d9e	Refactor Perl test code ... The original code was a bit clunky; make it more amenable for further reuse by creating a new Perl package PostgresNode, which is an object-oriented representation of a single server, with some support routines such as init, start, stop, psql. This serves as a better basis on which to build further test code, and enables writing tests that use more than one server without too much complication. This commit modifies a lot of the existing test files, mostly to remove explicit calls to system commands (pg_ctl) replacing them with method calls of a PostgresNode object. The result is quite a bit more straightforward. Also move some initialization code to BEGIN and INIT blocks instead of having it straight in as top-level code. This commit also introduces package RecursiveCopy so that we can copy whole directories without having to depend on packages that may not be present on vanilla Perl 5.8 installations. I also ran perltidy on the modified files, which changes some code sites that are not otherwise touched by this patch. I tried to avoid this, but it ended up being more trouble than it's worth. Authors: Michael Paquier, Álvaro Herrera Review: Noah Misch	10 years ago