In d0366bfb3b I said:
Instead of trying to do this in meson.build, call out to the implementation
meson install uses. This isn't pretty, but it's more reliable than what we had
before.
Unfortunately it was too ugly - to fix a bug, meson 1.0.1 changed the way the
meson internal runpython helper works, resulting in the previous sys.argv[]
indices not working anymore. Just open-code it - it's just a few characters
longer.
Committing this quickly to allow ci/cfbot to work with meson 1.0.1 on windows.
OpenSSL 1.1.1 and newer versions have added support for RSA-PSS
certificates, which requires the use of a specific routine in OpenSSL to
determine which hash function to use when compiling it when using
channel binding in SCRAM-SHA-256. X509_get_signature_nid(), that is the
original routine the channel binding code has relied on, is not able to
determine which hash algorithm to use for such certificates. However,
X509_get_signature_info(), new to OpenSSL 1.1.1, is able to do it. This
commit switches the channel binding logic to rely on
X509_get_signature_info() over X509_get_signature_nid(), which would be
the choice when building with 1.1.1 or newer.
The error could have been triggered on the client or the server, hence
libpq and the backend need to have their related code paths patched.
Note that attempting to load an RSA-PSS certificate with OpenSSL 1.1.0
or older leads to a failure due to an unsupported algorithm.
The discovery of relying on X509_get_signature_info() comes from Jacob,
the tests have been written by Heikki (with few tweaks from me), while I
have bundled the whole together while adding the bits needed for MSVC
and meson.
This issue exists since channel binding exists, so backpatch all the way
down. Some tests are added in 15~, triggered if compiling with OpenSSL
1.1.1 or newer, where the certificate and key files can easily be
generated for RSA-PSS.
Reported-by: Gunnar "Nick" Bluth
Author: Jacob Champion, Heikki Linnakangas
Discussion: https://postgr.es/m/17760-b6c61e752ec07060@postgresql.org
Backpatch-through: 11
We have the long-standing logic to append "postgresql" to some
installation paths if it does not already contain "pgsql" or
"postgres". The existing meson implementation of that only considered
the subdirectory under the prefix, not the prefix itself. Fix that,
so that it now works the same way as the implementation in
Makefile.global.
Reviewed-by: Andres Freund <andres@anarazel.de>
Discussion: https://www.postgresql.org/message-id/a6a6de12-f705-2b33-2fd9-9743277deb08@enterprisedb.com
Testing on CI showed that Cygwin's semctl() can fail with EAGAIN
(possibly due to resource limits in cygserver that could be tuned, not
examined). Switch to so-called POSIX semaphores instead, which don't
seem to fail in that way (possibly due to a more direct implementation
using Windows semaphore primitives instead of talking to cygserver,
based on a cursory glance at the source).
Other known problems still prevent PostgreSQL from running on Cygwin
without random crashes, but this rarer problem was noticed while
testing.
Discussion: https://postgr.es/m/CA%2BhUKG%2BQ6DU4Ov9LrvUyDcF3oHS4KMRVSKmVGaeePq-kOyG9gA%40mail.gmail.com
While on it, newlines are removed from the end of two elog() strings.
The others are simple grammar mistakes. One comment in pg_upgrade
referred incorrectly to sequences since a7e5457.
Author: Justin Pryzby
Discussion: https://postgr.es/m/20221230231257.GI1153@telsasoft.com
Backpatch-through: 11
Clang 16 is still in development, but seawasp reveals that it has
started warning about many of our casts of function pointers (those
introduced by commit 1c27d16e, and some older ones). Disable the new
warning for now, since otherwise buildfarm animal seawasp fails, and we
have no current plans to change our strategy for these callback function
types.
May be back-patched with other Clang/LLVM 16 changes around release
time.
Discussion: https://postgr.es/m/CA%2BhUKGJvX%2BL3aMN84ksT-cGy08VHErRNip3nV-WmTx7f6Pqhyw%40mail.gmail.com
Previously the host operating system and 32/64 bit were not included and the
build machine's cpu was used, which is potentially wrong for cross builds.
Author: Juan José Santamaría Flecha <juanjo.santamaria@gmail.com>
Author: Andres Freund <andres@anarazel.de>
Discussion: https://postgr.es/m/CAC+AXB16gwYhKCdS+t0pk3U7kKtpVj5L-ynmhK3Gbea330At3w@mail.gmail.com
The tests don't have much coverage of segment related code, as we don't create
large enough tables. To make it easier to test these paths, add a new option
specifying the segment size in blocks.
Set the new option to 6 blocks in one of the CI tasks. Smaller numbers
currently fail one of the tests, for understandable reasons.
While at it, fix some segment size related issues in the meson build.
Author: Andres Freund <andres@anarazel.de>
Discussion: https://postgr.es/m/20221107171355.c23fzwanfzq2pmgt@awork3.anarazel.de
To run all tests that support running against existing server:
$ meson test --setup running
To run just the main pg_regress tests against existing server:
$ meson test --setup running regress-running/regress
To ensure the 'running' setup continues to work, test it as part of the
freebsd CI task.
Discussion: https://postgr.es/m/CAH2-Wz=XDQcmLoo7RR_i6FKQdDmcyb9q5gStnfuuQXrOGhB2sQ@mail.gmail.com
Generate a Makefile.global that's complete enough for PGXS to work for some
extensions. It is likely that this compatibility layer will not suffice for
every extension and not all platforms - we can expand it over time.
This allows extensions to use a single buildsystem across all the supported
postgres versions. Once all supported PG versions support meson, we can remove
the compatibility layer.
Reviewed-by: Peter Eisentraut <peter.eisentraut@enterprisedb.com>
Discussion: https://postgr.es/m/20221005200710.luvw5evhwf6clig6@awork3.anarazel.de
There likely are further issues, but as evidenced by the CI task proposed by
Justin in the referenced thread, this suffices to build and run basic tests in
cygwin (some fixes for the test infrastructure are needed, but that's
independent of the meson aspect).
Author: Justin Pryzby <pryzby@telsasoft.com>
Reviewed-by: Andres Freund <andres@anarazel.de>
Discussion: https://postgr.es/m/20221021034040.GT16921@telsasoft.com
Various test suites use the "openssl" program as part of their setup.
There isn't a way to override which openssl program is to be used,
other than by fiddling with the path, perhaps. This has gotten
increasingly problematic because different versions of openssl have
different capabilities and do different things by default.
This patch checks for an openssl binary in configure and meson setup,
with appropriate ways to override it. This is similar to how "lz4"
and "zstd" are handled, for example. The meson build system actually
already did this, but the result was only used in some places. This
is now applied more uniformly.
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Discussion: https://www.postgresql.org/message-id/flat/dc638b75-a16a-007d-9e1c-d16ed6cf0ad2%40enterprisedb.com
I (Andres) missed a few recent changes to configure when merging
e6927270cd "meson: Add initial version of meson based build system". Mirror
the changes in
- ec3c9cc202 "Add definition pg_attribute_aligned() for MSVC"
- b086a47a27 "Bump minimum version of Bison to 2.3"
- 8b878bffa8 "Bump minimum version of Flex to 2.5.35"
As MSVC does not implement 128 bit integers, the oversight of not using
pg_attribute_aligned() should not have current practical consequences. But of
course the code from c.h should still be correctly mirrored.
I (Andres) also hadn't implemented the minimum perl version check. Added that
now.
Reported-by: Junwang Zhao <zhjwpku@gmail.com>
Author: Junwang Zhao <zhjwpku@gmail.com>
Author: Andres Freund <andres@anarazel.de>
Discussion: https://postgr.es/m/CAEG8a3K9c87EwAwmdOgmS0Li1J6P_7r-Uc0-zN6cJtrMr7VvPg@mail.gmail.com
Since cd4e8caaa, we've been able to build the source tree with
-Wshadow=compatible-local without any warnings. Lots of work was done by
Justin Pryzby and I (David) to get all our code to compile warning free
with that flag. In that process, 2 bugs (16d69ec29 and af7d270dd) were
discovered and fixed. Additionally, "git log --grep=shadow" shows that
there is no shortage of other bugs that have been fixed over the years
which were caused by variable shadowing.
In light of the above, it seems very much worthwhile to add at least
-Wshadow=compatible-local to our standard compilation flags. We *may*
want to go further and take this to -Wshadow=local in the future, but
we're not ready for that today, so let's add -Wshadow=compatible-local now
to help make sure we don't introduce further local variable shadowing.
Author: Andres Freund
Discussion: https://postgr.es/m/20221006003920.6xlqaoccxwisza5k@awork3.anarazel.de
This substantially speeds up building for windows, due to the vast amount of
headers included via windows.h. A cross build from linux targetting mingw goes
from
994.11user 136.43system 0:31.58elapsed 3579%CPU
to
422.41user 89.05system 0:14.35elapsed 3562%CPU
The wins on windows are similar-ish (but I don't have a system at hand just
now for actual numbers). Targetting other operating systems the wins are far
smaller (tested linux, macOS, FreeBSD).
For now precompiled headers are disabled by default, it's not clear how well
they work on all platforms. E.g. on FreeBSD gcc doesn't seem to have working
support, but clang does.
When doing a full build precompiled headers are only beneficial for targets
with multiple .c files, as meson builds a separate precompiled header for each
target (so that different compilation options take effect). This commit
therefore only changes target with at least two .c files to use precompiled
headers.
Because this commit adds b_pch=false to the default_options new build
directories will have precompiled headers disabled by default, however
existing build directories will continue use the default value of b_pch, which
is true.
Note that using precompiled headers with ccache requires setting
CCACHE_SLOPPINESS=pch_defines,time_macros to get hits.
Reviewed-by: Peter Eisentraut <peter.eisentraut@enterprisedb.com>
Reviewed-by: Justin Pryzby <pryzby@telsasoft.com>
Discussion: https://postgr.es/m/CA+hUKG+50eOUbN++ocDc0Qnp9Pvmou23DSXu=ZA6fepOcftKqA@mail.gmail.com
Discussion: https://postgr.es/m/c5736f70-bb6d-8d25-e35c-e3d886e4e905@enterprisedb.com
Discussion: https://postgr.es/m/20190826054000.GE7005%40paquier.xyz
The generated resource files aren't exactly the same ones as the old
buildsystems generate. Previously "InternalName" and "OriginalFileName" were
mostly wrong / not set (despite being required), but that was hard to fix in
at least the make build. Additionally, the meson build falls back to a
"auto-generated" description when not set, and doesn't set it in a few cases -
unlikely that anybody looks at these descriptions in detail.
Author: Andres Freund <andres@anarazel.de>
Author: Nazir Bilal Yavuz <byavuz81@gmail.com>
Reviewed-by: Peter Eisentraut <peter.eisentraut@enterprisedb.com>
Improvements:
- we don't need -DFRONTEND for libpq anymore since 1d77afefbd
- the .pc file contents for a static libpq were wrong (referencing
{pgport, common}_shlib)
- incidentally fixes meson not supporting link_whole on AIX yet
- added explanatory comments
Previously I tried to avoid building libpq's sources twice, once for the
static and once for the shared library. We could still do so, but it's not
clear that it's worth the complication.
Reviewed-by: Peter Eisentraut <peter.eisentraut@enterprisedb.com>
autoconf set PREFIX to /usr/local/pgsql, so using the same default for meson
makes sense. The effect on windows is that installation defaults to installing
to C:/usr/local/pgsql rather than meson's default of C:/, which doesn't seem
perfect, but OK enough.
Signed-off-by: Junwang Zhao <zhjwpku@gmail.com>
Author: Junwang Zhao <zhjwpku@gmail.com>
Discussion: https://postgr.es/CAEG8a3LGWE-gG6vuddmH91RORhi8gWs0mMB-hcTmP3_NVgM7dg@mail.gmail.com
Previously some paths (like c:\ or d:/) worked, but plenty others (like
/path/to or //computer/share/path) didn't. As we'd like to change the default
prefix to /usr/local/pgsql, that's a problem.
Instead of trying to do this in meson.build, call out to the implementation
meson install uses. This isn't pretty, but it's more reliable than what we had
before.
Discussion: https://postgr.es/CAEG8a3LGWE-gG6vuddmH91RORhi8gWs0mMB-hcTmP3_NVgM7dg@mail.gmail.com
I hadn't ported -Wl,--disable-auto-import over from the win32 template as I
had focused on msvc for windows. The flag is desirable as it makes it easier
to find problems one would have with msvc, particularly useful during cross
compilation.
This turned out to be a somewhat happy accident, as it allowed me to realize
that readline actually works on windows these days, as long as auto imports to
enable. Therefore enable auto-import again as part of linking to readline.
We perhaps can come up with a better solution for the readline issue, but this
seems good enough for now.
Discussion: http://postgr.es/m/20220928022724.erzuk5v4ai4b53do@awork3.anarazel.de
Not replacing getopt/getopt_long definitely causes issues on mingw. It's not
as clear whether the solaris & openbsd aspect is still needed, but if not, we
should remove it from both autoconf and meson.
Discussion: http://postgr.es/m/20220928022724.erzuk5v4ai4b53do@awork3.anarazel.de
I didn't carry this forward from the win32 template. It's not needed anymore
for the reason stated therein, but it turns out to be required to
e.g. override getopt. Possibly a better solution exists, but that's for later.
Discussion: http://postgr.es/m/20220928022724.erzuk5v4ai4b53do@awork3.anarazel.de
This fixes a build issue on windows, when the prefix is set to a path with
forward slashes. Windows python defaults to a path with a backslash, but mingw
ucrt python defaults to a forward slash. This in turn lead to a wrong PATH set
during tests, causing tests to fail.
Reported-by: Melih Mutlu <m.melihmutlu@gmail.com>
Discussion: http://postgr.es/m/20220928022724.erzuk5v4ai4b53do@awork3.anarazel.de
While I (Andres) had planned to use relative rpaths, it looks like agreeing on
the precise path might take a bit. So set up absolute rpaths for now.
I'm pushing this fairly quickly after posting the patch as several hackers
fought with this issue.
Discussion: https://postgr.es/m/20220927011951.j3h4o7n6bhf7dwau@awork3.anarazel.de
Autoconf is showing its age, fewer and fewer contributors know how to wrangle
it. Recursive make has a lot of hard to resolve dependency issues and slow
incremental rebuilds. Our home-grown MSVC build system is hard to maintain for
developers not using Windows and runs tests serially. While these and other
issues could individually be addressed with incremental improvements, together
they seem best addressed by moving to a more modern build system.
After evaluating different build system choices, we chose to use meson, to a
good degree based on the adoption by other open source projects.
We decided that it's more realistic to commit a relatively early version of
the new build system and mature it in tree.
This commit adds an initial version of a meson based build system. It supports
building postgres on at least AIX, FreeBSD, Linux, macOS, NetBSD, OpenBSD,
Solaris and Windows (however only gcc is supported on aix, solaris). For
Windows/MSVC postgres can now be built with ninja (faster, particularly for
incremental builds) and msbuild (supporting the visual studio GUI, but
building slower).
Several aspects (e.g. Windows rc file generation, PGXS compatibility, LLVM
bitcode generation, documentation adjustments) are done in subsequent commits
requiring further review. Other aspects (e.g. not installing test-only
extensions) are not yet addressed.
When building on Windows with msbuild, builds are slower when using a visual
studio version older than 2019, because those versions do not support
MultiToolTask, required by meson for intra-target parallelism.
The plan is to remove the MSVC specific build system in src/tools/msvc soon
after reaching feature parity. However, we're not planning to remove the
autoconf/make build system in the near future. Likely we're going to keep at
least the parts required for PGXS to keep working around until all supported
versions build with meson.
Some initial help for postgres developers is at
https://wiki.postgresql.org/wiki/Meson
With contributions from Thomas Munro, John Naylor, Stone Tickle and others.
Author: Andres Freund <andres@anarazel.de>
Author: Nazir Bilal Yavuz <byavuz81@gmail.com>
Author: Peter Eisentraut <peter@eisentraut.org>
Reviewed-By: Peter Eisentraut <peter.eisentraut@enterprisedb.com>
Discussion: https://postgr.es/m/20211012083721.hvixq4pnh2pixr3j@alap3.anarazel.de
Andres Freund