24 Commits (master)

Author	SHA1	Message	Date
Tomas Vondra	65c298f61f	Add support for basic NUMA awareness ... Add basic NUMA awareness routines, using a minimal src/port/pg_numa.c portability wrapper and an optional build dependency, enabled by --with-libnuma configure option. For now this is Linux-only, other platforms may be supported later. A built-in SQL function pg_numa_available() allows checking NUMA support, i.e. that the server was built/linked with the NUMA library. The main function introduced is pg_numa_query_pages(), which allows determining the NUMA node for individual memory pages. Internally the function uses move_pages(2) syscall, as it allows batching, and is more efficient than get_mempolicy(2). Author: Jakub Wartak <jakub.wartak@enterprisedb.com> Co-authored-by: Bertrand Drouvot <bertranddrouvot.pg@gmail.com> Reviewed-by: Andres Freund <andres@anarazel.de> Reviewed-by: Álvaro Herrera <alvherre@alvh.no-ip.org> Reviewed-by: Tomas Vondra <tomas@vondra.me> Discussion: https://postgr.es/m/CAKZiRmxh6KWo0aqRqvmcoaX2jUxZYb4kGp3N%3Dq1w%2BDiH-696Xw%40mail.gmail.com	3 months ago
Andres Freund	8eadd5c73c	aio: Add liburing dependency ... Will be used in a subsequent commit, to implement io_method=io_uring. Kept separate for easier review. Reviewed-by: Noah Misch <noah@leadboat.com> Discussion: https://postgr.es/m/uvrtrknj4kdytuboidbhwclo4gxhswwcpgadptsjvjqcluzmah%40brqs62irg4dt	3 months ago
Daniel Gustafsson	b3f0be788a	Add support for OAUTHBEARER SASL mechanism ... This commit implements OAUTHBEARER, RFC 7628, and OAuth 2.0 Device Authorization Grants, RFC 8628. In order to use this there is a new pg_hba auth method called oauth. When speaking to a OAuth- enabled server, it looks a bit like this: $ psql 'host=example.org oauth_issuer=... oauth_client_id=...' Visit https://oauth.example.org/login and enter the code: FPQ2-M4BG Device authorization is currently the only supported flow so the OAuth issuer must support that in order for users to authenticate. Third-party clients may however extend this and provide their own flows. The built-in device authorization flow is currently not supported on Windows. In order for validation to happen server side a new framework for plugging in OAuth validation modules is added. As validation is implementation specific, with no default specified in the standard, PostgreSQL does not ship with one built-in. Each pg_hba entry can specify a specific validator or be left blank for the validator installed as default. This adds a requirement on libcurl for the client side support, which is optional to build, but the server side has no additional build requirements. In order to run the tests, Python is required as this adds a https server written in Python. Tests are gated behind PG_TEST_EXTRA as they open ports. This patch has been a multi-year project with many contributors involved with reviews and in-depth discussions: Michael Paquier, Heikki Linnakangas, Zhihong Yu, Mahendrakar Srinivasarao, Andrey Chudnovsky and Stephen Frost to name a few. While Jacob Champion is the main author there have been some levels of hacking by others. Daniel Gustafsson contributed the validation module and various bits and pieces; Thomas Munro wrote the client side support for kqueue. Author: Jacob Champion <jacob.champion@enterprisedb.com> Co-authored-by: Daniel Gustafsson <daniel@yesql.se> Co-authored-by: Thomas Munro <thomas.munro@gmail.com> Reviewed-by: Daniel Gustafsson <daniel@yesql.se> Reviewed-by: Peter Eisentraut <peter@eisentraut.org> Reviewed-by: Antonin Houska <ah@cybertec.at> Reviewed-by: Kashif Zeeshan <kashi.zeeshan@gmail.com> Discussion: https://postgr.es/m/d1b467a78e0e36ed85a09adf979d04cf124a9d4b.camel@vmware.com	4 months ago
Bruce Momjian	50e6eb731d	Update copyright for 2025 ... Backpatch-through: 13	6 months ago
Heikki Linnakangas	3d1aec225a	Make PG_TEST_EXTRA env var override the "meson setup" option ... "meson test" used to ignore the PG_TEST_EXTRA environment variable, which meant that in order to run additional tests, you had to run "meson setup -DPG_TEST_EXTRA=...". That's somewhat expensive, and not consistent with autoconf builds. Allow PG_TEST_EXTRA environment variable to override the setup-time option at run time, so that you can do "PG_TEST_EXTRA=... meson test". To implement this, the configuration time value is passed as an extra "--pg-test-extra" argument to testwrap instead of adding it to the test environment. If the environment variable is set at the time of running test, testwrap uses the value from the environment variable and ignores the --pg-test-extra option. Now that "meson test" obeys the environment variable, we can remove it from the "meson setup" steps in the CI script. It will now be picked up from the environment variable like with "make check". Author: Nazir Bilal Yavuzk, Ashutosh Bapat Reviewed-by: Ashutosh Bapat with inputs from Tom Lane and Andrew Dunstan	8 months ago
Thomas Munro	06ffce4559	Remove spinlocks and atomics from meson_options.txt. ... Commits e2562667 and 81385261 removed the configure equivalents, but forgot to remove these options from meson_options.txt. Revealed by the fact that build farm animals rorqual and francolin didn't fail, despite being configured to set those options to off. They should now fail with unknown option, until they are adjusted.	11 months ago
Tom Lane	01df147634	Allow selecting the git revision to be packaged by "make dist". ... Commit 619bc23a1 changed "make dist" to invoke "git archive", but hard-wired the call to specify that the HEAD revision should be packaged. Our tarball building process needs to be able to specify which git commit to package (notably, for packaging back branches). While we could make that work with some hackery to operate in detached-HEAD state, it's a lot nicer just to expose git archive's ability to specify what to package. Hence, invent a new make variable PG_GIT_REVISION. This is undocumented, but so is "make dist". Also make corresponding changes in the meson scripts. We have no near-term intention of using that for package building, but it will likely happen eventually, so stay prepared. Discussion: https://postgr.es/m/3552543.1713909947@sss.pgh.pa.us	1 year ago
Michael Paquier	d86d20f0ba	Add backend support for injection points ... Injection points are a new facility that makes possible for developers to run custom code in pre-defined code paths. Its goal is to provide ways to design and run advanced tests, for cases like: - Race conditions, where processes need to do actions in a controlled ordered manner. - Forcing a state, like an ERROR, FATAL or even PANIC for OOM, to force recovery, etc. - Arbitrary sleeps. This implements some basics, and there are plans to extend it more in the future depending on what's required. Hence, this commit adds a set of routines in the backend that allows developers to attach, detach and run injection points: - A code path calling an injection point can be declared with the macro INJECTION_POINT(name). - InjectionPointAttach() and InjectionPointDetach() to respectively attach and detach a callback to/from an injection point. An injection point name is registered in a shmem hash table with a library name and a function name, which will be used to load the callback attached to an injection point when its code path is run. Injection point names are just strings, so as an injection point can be declared and run by out-of-core extensions and modules, with callbacks defined in external libraries. This facility is hidden behind a dedicated switch for ./configure and meson, disabled by default. Note that backends use a local cache to store callbacks already loaded, cleaning up their cache if a callback has found to be removed on a best-effort basis. This could be refined further but any tests but what we have here was fine with the tests I've written while implementing these backend APIs. Author: Michael Paquier, with doc suggestions from Ashutosh Bapat. Reviewed-by: Ashutosh Bapat, Nathan Bossart, Álvaro Herrera, Dilip Kumar, Amul Sul, Nazir Bilal Yavuz Discussion: https://postgr.es/m/ZTiV8tn_MIb_H2rE@paquier.xyz	1 year ago
Bruce Momjian	29275b1d17	Update copyright for 2024 ... Reported-by: Michael Paquier Discussion: https://postgr.es/m/ZZKTDPxBBMt3C0J9@paquier.xyz Backpatch-through: 12	2 years ago
Andres Freund	c04f3de3dc	meson: Change default of 'selinux' feature option to auto ... There is really no reason for selinux to behave differently than other options. Reviewed-by: Peter Eisentraut <peter@eisentraut.org> Discussion: https://postgr.es/m/20231103211601.bgqx3cfq6pz2lmjw@awork3.anarazel.de	2 years ago
Peter Eisentraut	c55d31e59c	meson: Attach colon to keyword argument ... This matches the style found in the rest of the Meson build description. Author: Tristan Partin <tristan@neon.tech> Discussion: https://www.postgresql.org/message-id/flat/CSPIJVUDZFKX.3KHMOAVGF94RV%40c3po	2 years ago
Peter Eisentraut	b2ec7f543f	meson: Use consistent Meson option description formats ... Author: Tristan Partin <tristan@neon.tech> Discussion: https://www.postgresql.org/message-id/flat/CSPIJVUDZFKX.3KHMOAVGF94RV%40c3po	2 years ago
Peter Eisentraut	529449c379	meson: Use consistent casing in Meson option descriptions ... Meson itself uses capital letters for option descriptions, so follow that. Author: Tristan Partin <tristan@neon.tech> Discussion: https://www.postgresql.org/message-id/flat/CSPIJVUDZFKX.3KHMOAVGF94RV%40c3po	2 years ago
Peter Eisentraut	559bdd58eb	meson: Remove triple-quoted strings ... Triple-quoted strings are for multiline strings in Meson. None of the descriptions that got changed were multiline and the entire file uses single-line descriptions. Author: Tristan Partin <tristan@neon.tech> Discussion: https://www.postgresql.org/message-id/flat/CSPIJVUDZFKX.3KHMOAVGF94RV%40c3po	2 years ago
Andres Freund	9d2a464a04	meson: docs: Allow configuring simple/website style ... The meson docs generation hardcoded using the website style so far. Make it configurable via a meson option. Reviewed-by: Justin Pryzby <pryzby@telsasoft.com> Reported-by: Peter Eisentraut <peter.eisentraut@enterprisedb.com> Discussion: https://postgr.es/m/3fc3bb9b-f7f8-d442-35c1-ec82280c564a@enterprisedb.com	2 years ago
Andres Freund	ef07d351a1	meson: add docs, docs_pdf options ... Detect and report if the tools necessary to build documentation are available during configure. This is represented as two new options 'docs' and 'docs_pdf', both defaulting to 'auto'. This should also fix a meson error about the installdocs target, when none of the doc tools are found. Reviewed-by: Justin Pryzby <pryzby@telsasoft.com> Discussion: https://postgr.es/m/20230325201414.sh7c6xlut2fpunnv@awork3.anarazel.de Discussion: https://postgr.es/m/ZB8331v5IhUA/pNu@telsasoft.com	2 years ago
Peter Eisentraut	6a3002715e	meson: Make auto the default of the ssl option ... The 'ssl' option is of type 'combo', but we add a choice 'auto' that simulates the behavior of a feature option. This way, openssl is used automatically by default if present, but we retain the ability to potentially select another ssl library. Author: Nazir Bilal Yavuz <byavuz81@gmail.com> Discussion: https://www.postgresql.org/message-id/flat/ad65ffd1-a9a7-fda1-59c6-f7dc763c3051%40enterprisedb.com	2 years ago
Peter Eisentraut	d2f44cc36e	meson: Add equivalent of configure --disable-rpath option ... Discussion: https://www.postgresql.org/message-id/flat/33e957e6-4b4e-b0ed-1cc1-6335a24543ff%40enterprisedb.com	2 years ago
Bruce Momjian	c8e1ba736b	Update copyright for 2023 ... Backpatch-through: 11	3 years ago
Andrew Dunstan	8284cf5f74	Add copyright notices to meson files ... Discussion: https://postgr.es/m/222b43a5-2fb3-2c1b-9cd0-375d376c8246@dunslane.net	3 years ago
Andres Freund	d3b111e320	Add option to specify segment size in blocks ... The tests don't have much coverage of segment related code, as we don't create large enough tables. To make it easier to test these paths, add a new option specifying the segment size in blocks. Set the new option to 6 blocks in one of the CI tasks. Smaller numbers currently fail one of the tests, for understandable reasons. While at it, fix some segment size related issues in the meson build. Author: Andres Freund <andres@anarazel.de> Discussion: https://postgr.es/m/20221107171355.c23fzwanfzq2pmgt@awork3.anarazel.de	3 years ago
Andres Freund	5bdd0cfb91	meson: Add basic PGXS compatibility ... Generate a Makefile.global that's complete enough for PGXS to work for some extensions. It is likely that this compatibility layer will not suffice for every extension and not all platforms - we can expand it over time. This allows extensions to use a single buildsystem across all the supported postgres versions. Once all supported PG versions support meson, we can remove the compatibility layer. Reviewed-by: Peter Eisentraut <peter.eisentraut@enterprisedb.com> Discussion: https://postgr.es/m/20221005200710.luvw5evhwf6clig6@awork3.anarazel.de	3 years ago
Peter Eisentraut	c8e4030d1b	Make finding openssl program a configure or meson option ... Various test suites use the "openssl" program as part of their setup. There isn't a way to override which openssl program is to be used, other than by fiddling with the path, perhaps. This has gotten increasingly problematic because different versions of openssl have different capabilities and do different things by default. This patch checks for an openssl binary in configure and meson setup, with appropriate ways to override it. This is similar to how "lz4" and "zstd" are handled, for example. The meson build system actually already did this, but the result was only used in some places. This is now applied more uniformly. Reviewed-by: Michael Paquier <michael@paquier.xyz> Discussion: https://www.postgresql.org/message-id/flat/dc638b75-a16a-007d-9e1c-d16ed6cf0ad2%40enterprisedb.com	3 years ago
Andres Freund	e6927270cd	meson: Add initial version of meson based build system ... Autoconf is showing its age, fewer and fewer contributors know how to wrangle it. Recursive make has a lot of hard to resolve dependency issues and slow incremental rebuilds. Our home-grown MSVC build system is hard to maintain for developers not using Windows and runs tests serially. While these and other issues could individually be addressed with incremental improvements, together they seem best addressed by moving to a more modern build system. After evaluating different build system choices, we chose to use meson, to a good degree based on the adoption by other open source projects. We decided that it's more realistic to commit a relatively early version of the new build system and mature it in tree. This commit adds an initial version of a meson based build system. It supports building postgres on at least AIX, FreeBSD, Linux, macOS, NetBSD, OpenBSD, Solaris and Windows (however only gcc is supported on aix, solaris). For Windows/MSVC postgres can now be built with ninja (faster, particularly for incremental builds) and msbuild (supporting the visual studio GUI, but building slower). Several aspects (e.g. Windows rc file generation, PGXS compatibility, LLVM bitcode generation, documentation adjustments) are done in subsequent commits requiring further review. Other aspects (e.g. not installing test-only extensions) are not yet addressed. When building on Windows with msbuild, builds are slower when using a visual studio version older than 2019, because those versions do not support MultiToolTask, required by meson for intra-target parallelism. The plan is to remove the MSVC specific build system in src/tools/msvc soon after reaching feature parity. However, we're not planning to remove the autoconf/make build system in the near future. Likely we're going to keep at least the parts required for PGXS to keep working around until all supported versions build with meson. Some initial help for postgres developers is at https://wiki.postgresql.org/wiki/Meson With contributions from Thomas Munro, John Naylor, Stone Tickle and others. Author: Andres Freund <andres@anarazel.de> Author: Nazir Bilal Yavuz <byavuz81@gmail.com> Author: Peter Eisentraut <peter@eisentraut.org> Reviewed-By: Peter Eisentraut <peter.eisentraut@enterprisedb.com> Discussion: https://postgr.es/m/20211012083721.hvixq4pnh2pixr3j@alap3.anarazel.de	3 years ago