190 Commits (4ea1254f35b27ed35a389e720ab2e614fc1e1d4d)

Author	SHA1	Message	Date
Jacob Champion	b0635bfda0	oauth: Move the builtin flow into a separate module ... The additional packaging footprint of the OAuth Curl dependency, as well as the existence of libcurl in the address space even if OAuth isn't ever used by a client, has raised some concerns. Split off this dependency into a separate loadable module called libpq-oauth. When configured using --with-libcurl, libpq.so searches for this new module via dlopen(). End users may choose not to install the libpq-oauth module, in which case the default flow is disabled. For static applications using libpq.a, the libpq-oauth staticlib is a mandatory link-time dependency for --with-libcurl builds. libpq.pc has been updated accordingly. The default flow relies on some libpq internals. Some of these can be safely duplicated (such as the SIGPIPE handlers), but others need to be shared between libpq and libpq-oauth for thread-safety. To avoid exporting these internals to all libpq clients forever, these dependencies are instead injected from the libpq side via an initialization function. This also lets libpq communicate the offsets of PGconn struct members to libpq-oauth, so that we can function without crashing if the module on the search path came from a different build of Postgres. (A minor-version upgrade could swap the libpq-oauth module out from under a long-running libpq client before it does its first load of the OAuth flow.) This ABI is considered "private". The module has no SONAME or version symlinks, and it's named libpq-oauth-<major>.so to avoid mixing and matching across Postgres versions. (Future improvements may promote this "OAuth flow plugin" to a first-class concept, at which point we would need a public API to replace this anyway.) Additionally, NLS support for error messages in b3f0be788a was incomplete, because the new error macros weren't being scanned by xgettext. Fix that now. Per request from Tom Lane and Bruce Momjian. Based on an initial patch by Daniel Gustafsson, who also contributed docs changes. The "bare" dlopen() concept came from Thomas Munro. Many people reviewed the design and implementation; thank you! Co-authored-by: Daniel Gustafsson <daniel@yesql.se> Reviewed-by: Andres Freund <andres@anarazel.de> Reviewed-by: Christoph Berg <myon@debian.org> Reviewed-by: Daniel Gustafsson <daniel@yesql.se> Reviewed-by: Jelte Fennema-Nio <postgres@jeltef.nl> Reviewed-by: Peter Eisentraut <peter@eisentraut.org> Reviewed-by: Wolfgang Walther <walther@technowledgy.de> Discussion: https://postgr.es/m/641687.1742360249%40sss.pgh.pa.us	8 months ago
Tomas Vondra	65c298f61f	Add support for basic NUMA awareness ... Add basic NUMA awareness routines, using a minimal src/port/pg_numa.c portability wrapper and an optional build dependency, enabled by --with-libnuma configure option. For now this is Linux-only, other platforms may be supported later. A built-in SQL function pg_numa_available() allows checking NUMA support, i.e. that the server was built/linked with the NUMA library. The main function introduced is pg_numa_query_pages(), which allows determining the NUMA node for individual memory pages. Internally the function uses move_pages(2) syscall, as it allows batching, and is more efficient than get_mempolicy(2). Author: Jakub Wartak <jakub.wartak@enterprisedb.com> Co-authored-by: Bertrand Drouvot <bertranddrouvot.pg@gmail.com> Reviewed-by: Andres Freund <andres@anarazel.de> Reviewed-by: Álvaro Herrera <alvherre@alvh.no-ip.org> Reviewed-by: Tomas Vondra <tomas@vondra.me> Discussion: https://postgr.es/m/CAKZiRmxh6KWo0aqRqvmcoaX2jUxZYb4kGp3N%3Dq1w%2BDiH-696Xw%40mail.gmail.com	8 months ago
John Naylor	3c6e8c1238	Compute CRC32C using AVX-512 instructions where available ... The previous implementation of CRC32C on x86 relied on the native CRC32 instruction from the SSE 4.2 extension, which operates on up to 8 bytes at a time. We can get a substantial speedup by using carryless multiplication on SIMD registers, processing 64 bytes per loop iteration. Shorter inputs fall back to ordinary CRC instructions. On Intel Tiger Lake hardware (2020), CRC is now 50% faster for inputs between 64 and 112 bytes, and 3x faster for 256 bytes. The VPCLMULQDQ instruction on 512-bit registers has been available on Intel hardware since 2019 and AMD since 2022. There is an older variant for 128-bit registers, but at least on Zen 2 it performs worse than normal CRC instructions for short inputs. We must now do a runtime check, even for builds that target SSE 4.2. This doesn't matter in practice for WAL (arguably the most critical case), because since commit e2809e3a1 the final computation with the 20-byte WAL header is inlined and unrolled when targeting that extension. Compared with two direct function calls, testing showed equal or slightly faster performance in performing an indirect function call on several dozen bytes followed by inlined instructions on constant input of 20 bytes. The MIT-licensed implementation was generated with the "generate" program from https://github.com/corsix/fast-crc32/ Based on: "Fast CRC Computation for Generic Polynomials Using PCLMULQDQ Instruction" V. Gopal, E. Ozturk, et al., 2009 Co-authored-by: Raghuveer Devulapalli <raghuveer.devulapalli@intel.com> Co-authored-by: Paul Amonson <paul.d.amonson@intel.com> Reviewed-by: Nathan Bossart <nathandbossart@gmail.com> Reviewed-by: Andres Freund <andres@anarazel.de> (earlier version) Reviewed-by: Matthew Sterrett <matthewsterrett2@gmail.com> (earlier version) Tested-by: Raghuveer Devulapalli <raghuveer.devulapalli@intel.com> Tested-by: David Rowley <<dgrowleyml@gmail.com>> (earlier version) Discussion: https://postgr.es/m/BL1PR11MB530401FA7E9B1CA432CF9DC3DC192@BL1PR11MB5304.namprd11.prod.outlook.com Discussion: https://postgr.es/m/PH8PR11MB82869FF741DFA4E9A029FF13FBF72@PH8PR11MB8286.namprd11.prod.outlook.com	9 months ago
Daniel Gustafsson	2da74d8d64	libpq: Add support for dumping SSL key material to file ... This adds a new connection parameter which instructs libpq to write out keymaterial clientside into a file in order to make connection debugging with Wireshark and similar tools possible. The file format used is the standardized NSS format. Author: Abhishek Chanda <abhishek.becs@gmail.com> Co-authored-by: Daniel Gustafsson <daniel@yesql.se> Reviewed-by: Jacob Champion <jacob.champion@enterprisedb.com> Discussion: https://postgr.es/m/CAKiP-K85C8uQbzXKWf5wHQPkuygGUGcufke713iHmYWOe9q2dA@mail.gmail.com	9 months ago
Heikki Linnakangas	09be391126	Add timingsafe_bcmp(), for constant-time memory comparison ... timingsafe_bcmp() should be used instead of memcmp() or a naive for-loop, when comparing passwords or secret tokens, to avoid leaking information about the secret token by timing. This commit just introduces the function but does not change any existing code to use it yet. Co-authored-by: Jelte Fennema-Nio <github-tech@jeltef.nl> Discussion: https://www.postgresql.org/message-id/7b86da3b-9356-4e50-aa1b-56570825e234@iki.fi	9 months ago
Tom Lane	6da2ba1d8a	Fix detection and handling of strchrnul() for macOS 15.4. ... As of 15.4, macOS has strchrnul(), but access to it is blocked behind a check for MACOSX_DEPLOYMENT_TARGET >= 15.4. But our does-it-link configure check finds it, so we try to use it, and fail with the present default deployment target (namely 15.0). This accounts for today's buildfarm failures on indri and sifaka. This is the identical problem that we faced some years ago when Apple introduced preadv and pwritev in the same way. We solved that in commit f014b1b9b by using AC_CHECK_DECLS instead of AC_CHECK_FUNCS to check the functions' availability. So do the same now for strchrnul(). Interestingly, we already had a workaround for "the link check doesn't agree with <string.h>" cases with glibc, which we no longer need since only the header declaration is being checked. Testing this revealed that the meson version of this check has never worked, because it failed to use "-Werror=unguarded-availability-new". (Apparently nobody's tried to build with meson on macOS versions that lack preadv/pwritev as standard.) Adjust that while at it. Also, we had never put support for "-Werror=unguarded-availability-new" into v13, but we need that now. Co-authored-by: Tom Lane <tgl@sss.pgh.pa.us> Co-authored-by: Peter Eisentraut <peter@eisentraut.org> Discussion: https://postgr.es/m/385134.1743523038@sss.pgh.pa.us Backpatch-through: 13	9 months ago
Nathan Bossart	519338ace4	Optimize popcount functions with ARM SVE intrinsics. ... This commit introduces SVE implementations of pg_popcount{32,64}. Unlike the Neon versions, we need an additional configure-time check to determine if the compiler supports SVE intrinsics, and we need a runtime check to determine if the current CPU supports SVE instructions. Our testing showed that the SVE implementations are much faster for larger inputs and are comparable to the status quo for smaller inputs. Author: "Devanga.Susmitha@fujitsu.com" <Devanga.Susmitha@fujitsu.com> Co-authored-by: "Chiranmoy.Bhattacharya@fujitsu.com" <Chiranmoy.Bhattacharya@fujitsu.com> Co-authored-by: "Malladi, Rama" <ramamalladi@hotmail.com> Reviewed-by: John Naylor <johncnaylorls@gmail.com> Reviewed-by: Kirill Reshke <reshkekirill@gmail.com> Discussion: https://postgr.es/m/010101936e4aaa70-b474ab9e-b9ce-474d-a3ba-a3dc223d295c-000000%40us-west-2.amazonses.com Discussion: https://postgr.es/m/OSZPR01MB84990A9A02A3515C6E85A65B8B2A2%40OSZPR01MB8499.jpnprd01.prod.outlook.com	9 months ago
Peter Eisentraut	3c8e463b0d	Revert "Tidy up locale thread safety in ECPG library." ... This reverts commit 8e993bff53. It causes various build failures on the buildfarm, to be investigated. Discussion: https://postgr.es/m/CWZBBRR6YA8D.8EHMDRGLCKCD%40neon.tech	9 months ago
Peter Eisentraut	8e993bff53	Tidy up locale thread safety in ECPG library. ... Remove setlocale() and _configthreadlocal() as fallback strategy on systems that don't have uselocale(), where ECPG tries to control LC_NUMERIC formatting on input and output of floating point numbers. It was probably broken on some systems (NetBSD), and the code was also quite messy and complicated, with obsolete configure tests (Windows). It was also arguably broken, or at least had unstated environmental requirements, if pgtypeslib code was called directly. Instead, introduce PG_C_LOCALE to refer to the "C" locale as a locale_t value. It maps to the special constant LC_C_LOCALE when defined by libc (macOS, NetBSD), or otherwise uses a process-lifetime locale_t that is allocated on first use, just as ECPG previously did itself. The new replacement might be more widely useful. Then change the float parsing and printing code to pass that to _l() functions where appropriate. Unfortunately the portability of those functions is a bit complicated. First, many obvious and useful _l() functions are missing from POSIX, though most standard libraries define some of them anyway. Second, although the thread-safe save/restore technique can be used to replace the missing ones, Windows and NetBSD refused to implement standard uselocale(). They might have a point: "wide scope" uselocale() is hard to combine with other code and error-prone, especially in library code. Luckily they have the _l() functions we want so far anyway. So we have to be prepared for both ways of doing things: 1. In ECPG, use strtod_l() for parsing, and supply a port.h replacement using uselocale() over a limited scope if missing. 2. Inside our own snprintf.c, use three different approaches to format floats. For frontend code, call libc's snprintf_l(), or wrap libc's snprintf() in uselocale() if it's missing. For backend code, snprintf.c can keep assuming that the global locale's LC_NUMERIC is "C" and call libc's snprintf() without change, for now. (It might eventually be possible to call our in-tree Ryū routines to display floats in snprintf.c, given the C-locale-always remit of our in-tree snprintf(), but this patch doesn't risk changing anything that complicated.) Author: Thomas Munro <thomas.munro@gmail.com> Reviewed-by: Peter Eisentraut <peter@eisentraut.org> Reviewed-by: Tristan Partin <tristan@partin.io> Reviewed-by: Heikki Linnakangas <hlinnaka@iki.fi> Discussion: https://postgr.es/m/CWZBBRR6YA8D.8EHMDRGLCKCD%40neon.tech	9 months ago
Peter Eisentraut	b98be8a2a2	Provide thread-safe pg_localeconv_r(). ... This involves four different implementation strategies: 1. For Windows, we now require _configthreadlocale() to be available and work (commit f1da075d9a), and the documentation says that the object returned by localeconv() is in thread-local memory. 2. For glibc, we translate to nl_langinfo_l() calls, because it offers the same information that way as an extension, and that API is thread-safe. 3. For macOS/*BSD, use localeconv_l(), which is thread-safe. 4. For everything else, use uselocale() to set the locale for the thread, and use a big ugly lock to defend against the returned object being concurrently clobbered. In practice this currently means only Solaris. The new call is used in pg_locale.c, replacing calls to setlocale() and localeconv(). Author: Thomas Munro <thomas.munro@gmail.com> Reviewed-by: Heikki Linnakangas <hlinnaka@iki.fi> Reviewed-by: Peter Eisentraut <peter@eisentraut.org> Discussion: https://postgr.es/m/CA%2BhUKGJqVe0%2BPv9dvC9dSums_PXxGo9SWcxYAMBguWJUGbWz-A%40mail.gmail.com	9 months ago
Andres Freund	8eadd5c73c	aio: Add liburing dependency ... Will be used in a subsequent commit, to implement io_method=io_uring. Kept separate for easier review. Reviewed-by: Noah Misch <noah@leadboat.com> Discussion: https://postgr.es/m/uvrtrknj4kdytuboidbhwclo4gxhswwcpgadptsjvjqcluzmah%40brqs62irg4dt	9 months ago
Daniel Gustafsson	b3f0be788a	Add support for OAUTHBEARER SASL mechanism ... This commit implements OAUTHBEARER, RFC 7628, and OAuth 2.0 Device Authorization Grants, RFC 8628. In order to use this there is a new pg_hba auth method called oauth. When speaking to a OAuth- enabled server, it looks a bit like this: $ psql 'host=example.org oauth_issuer=... oauth_client_id=...' Visit https://oauth.example.org/login and enter the code: FPQ2-M4BG Device authorization is currently the only supported flow so the OAuth issuer must support that in order for users to authenticate. Third-party clients may however extend this and provide their own flows. The built-in device authorization flow is currently not supported on Windows. In order for validation to happen server side a new framework for plugging in OAuth validation modules is added. As validation is implementation specific, with no default specified in the standard, PostgreSQL does not ship with one built-in. Each pg_hba entry can specify a specific validator or be left blank for the validator installed as default. This adds a requirement on libcurl for the client side support, which is optional to build, but the server side has no additional build requirements. In order to run the tests, Python is required as this adds a https server written in Python. Tests are gated behind PG_TEST_EXTRA as they open ports. This patch has been a multi-year project with many contributors involved with reviews and in-depth discussions: Michael Paquier, Heikki Linnakangas, Zhihong Yu, Mahendrakar Srinivasarao, Andrey Chudnovsky and Stephen Frost to name a few. While Jacob Champion is the main author there have been some levels of hacking by others. Daniel Gustafsson contributed the validation module and various bits and pieces; Thomas Munro wrote the client side support for kqueue. Author: Jacob Champion <jacob.champion@enterprisedb.com> Co-authored-by: Daniel Gustafsson <daniel@yesql.se> Co-authored-by: Thomas Munro <thomas.munro@gmail.com> Reviewed-by: Daniel Gustafsson <daniel@yesql.se> Reviewed-by: Peter Eisentraut <peter@eisentraut.org> Reviewed-by: Antonin Houska <ah@cybertec.at> Reviewed-by: Kashif Zeeshan <kashi.zeeshan@gmail.com> Discussion: https://postgr.es/m/d1b467a78e0e36ed85a09adf979d04cf124a9d4b.camel@vmware.com	10 months ago
Daniel Gustafsson	44ec095751	Remove support for linking with libeay32 and ssleay32 ... The OpenSSL project stopped using the eay names back in 2016 on platforms other than Microsoft Windows, and version 1.1.0 removed the names from Windows as well. Since we now require OpenSSL 1.1.1 we can remove support for using the eay names from our tree as well. Author: Daniel Gustafsson <daniel@yesql.se> Reviewed-by: Michael Paquier <michael@paquier.xyz> Discussion: https://postgr.es/m/3C445F8E-D43E-4970-9CD9-A54882197714@yesql.se Discussion: https://postgr.es/m/CAHrt6656W9OnFomQTHBGYDcM5CKZ7hcgzFt8L+N0ezBZfcN3zA@mail.gmail.com	10 months ago
Bruce Momjian	50e6eb731d	Update copyright for 2025 ... Backpatch-through: 13	12 months ago
Thomas Munro	962da900ac	Use <stdint.h> and <inttypes.h> for c.h integers. ... Redefine our exact width types with standard C99 types and macros, including int64_t, INT64_MAX, INT64_C(), PRId64 etc. We were already using <stdint.h> types in a few places. One complication is that Windows' <inttypes.h> uses format strings like "%I64d", "%I32", "%I" for PRI*64, PRI*32, PTR*PTR, instead of mapping to other standardized format strings like "%lld" etc as seen on other known systems. Teach our snprintf.c to understand them. This removes a lot of configure clutter, and should also allow 64-bit numbers and other standard types to be used in localized messages without casting. Reviewed-by: Peter Eisentraut <peter@eisentraut.org> Discussion: https://postgr.es/m/ME3P282MB3166F9D1F71F787929C0C7E7B6312%40ME3P282MB3166.AUSP282.PROD.OUTLOOK.COM	1 year ago
Tom Lane	3b08d5224d	Define __EXTENSIONS__ on Solaris, too. ... Apparently, if you define _POSIX_C_SOURCE on Solaris, that's interpreted as "you get ONLY what's defined by POSIX". Results from BF member hake show that that breaks perl.h, and doubtless it'd cause more problems if we got past that. Adopt the suggestion from standards(7) that we also need to define __EXTENSIONS__, in hopes of un-breaking things. Discussion: https://postgr.es/m/1654508.1733162761@sss.pgh.pa.us	1 year ago
Tom Lane	32a7deb2a0	Define _POSIX_C_SOURCE as 200112L on Solaris. ... This is an attempt to suppress some compiler warnings that appeared in the wake of commit 7f798aca1: it seems that by default Solaris/illumos declares shmdt() to take "char *" not "void *". We'd like the system headers to provide modern POSIX APIs, and POSIX 2001 seems to be as modern as is available there. illumos' standards(7) man page suggests that we might also need to define __EXTENSIONS__, but let's see what happens with just this. Discussion: https://postgr.es/m/1654508.1733162761@sss.pgh.pa.us	1 year ago
Thomas Munro	97525bc5c8	Require sizeof(bool) == 1. ... The C standard says that sizeof(bool) is implementation-defined, but we know of no current systems where it is not 1. The last known systems seem to have been Apple macOS/PowerPC 10.5 and Microsoft Visual C++ 4, both long defunct. PostgreSQL has always required sizeof(bool) == 1 for the definition of bool that it used, but previously it would define its own type if the system-provided bool had a different size. That was liable to cause memory layout problems when interacting with system and third-party libraries on (by now hypothetical) computers with wider _Bool, and now C23 has introduced a new problem by making bool a built-in datatype (like C++), so the fallback code doesn't even compile. We could probably work around that, but then we'd be writing new untested code for a computer that doesn't exist. Instead, delete the unreachable and C23-uncompilable fallback code, and let existing static assertions fail if the system-provided bool is too wide. If we ever get a problem report from a real system, then it will be time to figure out what to do about it in a way that also works on modern compilers. Note on C++: Previously we avoided including <stdbool.h> or trying to define a new bool type in headers that might be included by C++ code. These days we might as well just include <stdbool.h> unconditionally: it should be visible to C++11 but do nothing, just as in C23. We already include <stdint.h> without C++ guards in c.h, and that falls under the same C99-compatibility section of the C++11 standard as <stdbool.h>, so let's remove the guards here too. Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us> Discussion: https://postgr.es/m/3198438.1731895163%40sss.pgh.pa.us	1 year ago
Nathan Bossart	4b03a27faf	Use __attribute__((target(...))) for SSE4.2 CRC-32C support. ... Presently, we check for compiler support for the required intrinsics both with and without the -msse4.2 compiler flag, and then depending on the results of those checks, we pick which files to compile with which flags. This is tedious and complicated, and it results in unsustainable coding patterns such as separate files for each portion of code that may need to be built with different compiler flags. This commit makes use of the newly-added support for __attribute__((target(...))) in the SSE4.2 CRC-32C code. This simplifies both the configure-time checks and the build scripts, and it allows us to place the functions that use the intrinsics in files that we otherwise do not want to build with special CPU instructions (although this commit refrains from doing so). This is also preparatory work for a proposed follow-up commit that will further optimize the CRC-32C code with AVX-512 instructions. While at it, this commit modifies meson's checks for SSE4.2 CRC support to be the same as autoconf's. meson was choosing whether to use a runtime check based purely on whether -msse4.2 is required, while autoconf has long checked for the __SSE4_2__ preprocessor symbol to decide. meson's previous approach seems to work just fine, but this change avoids needing to build multiple test programs and to keep track of whether to actually use pg_attribute_target(). Ideally we'd use __attribute__((target(...))) for ARMv8 CRC support, too, but there's little point in doing so because until clang 16, using the ARM intrinsics still requires special compiler flags. Perhaps we can re-evaluate this decision after some time has passed. Author: Raghuveer Devulapalli Discussion: https://postgr.es/m/PH8PR11MB8286BE735A463468415D46B5FB5C2%40PH8PR11MB8286.namprd11.prod.outlook.com	1 year ago
Thomas Munro	f1da075d9a	Remove configure check for _configthreadlocale(). ... All modern Windows systems have _configthreadlocale(). It was first introduced in msvcr80.dll from Visual Studio 2005. Historically, MinGW was stuck on even older msvcrt.dll, but added its own dummy implementation of the function when using msvcrt.dll years ago anyway, effectively rendering the configure test useless. In practice we don't encounter the dummy anymore because modern MinGW uses ucrt. Reviewed-by: Peter Eisentraut <peter@eisentraut.org> Discussion: https://postgr.es/m/CWZBBRR6YA8D.8EHMDRGLCKCD%40neon.tech	1 year ago
Tom Lane	5980f1884f	Update configure probes for CFLAGS needed for ARM CRC instructions. ... On ARM platforms where the baseline CPU target lacks CRC instructions, we need to supply a -march flag to persuade the compiler to compile such instructions. It turns out that our existing choice of "-march=armv8-a+crc" has not worked for some time, because recent gcc will interpret that as selecting software floating point, and then will spit up if the platform requires hard-float ABI, as most do nowadays. The end result was to silently fall back to software CRC, which isn't very desirable since in practice almost all currently produced ARM chips do have hardware CRC. We can fix this by using "-march=armv8-a+crc+simd" to enable the correct ABI choice. (This has no impact on the code actually generated, since neither of the files we compile with this flag does any floating-point stuff, let alone SIMD.) Keep the test for "-march=armv8-a+crc" since that's required for soft-float ABI, but try that second since most platforms we're likely to build on use hard-float. Since this isn't working as-intended on the last several years' worth of gcc releases, back-patch to all supported branches. Discussion: https://postgr.es/m/4496616.iHFcN1HehY@portable-bastien	1 year ago
Thomas Munro	bc5a4dfcf7	Assume that <stdbool.h> conforms to the C standard. ... Previously we checked "for <stdbool.h> that conforms to C99" using autoconf's AC_HEADER_STDBOOL macro. We've required C99 since PostgreSQL 12, so the test was redundant, and under C23 it was broken: autoconf 2.69's implementation doesn't understand C23's new empty header (the macros it's looking for went away, replaced by language keywords). Later autoconf versions fixed that, but let's just remove the anachronistic test. HAVE_STDBOOL_H and HAVE__BOOL will no longer be defined, but they weren't directly tested in core or likely extensions (except in 11, see below). PG_USE_STDBOOL (or USE_STDBOOL in 11 and 12) is still defined when sizeof(bool) is 1, which should be true on all modern systems. Otherwise we define our own bool type and values of size 1, which would fail to compile under C23 as revealed by the broken test. (We'll probably clean that dead code up in master, but here we want a minimal back-patchable change.) This came to our attention when GCC 15 recently started using using C23 by default and failed to compile the replacement code, as reported by Sam James and build farm animal alligator. Back-patch to all supported releases, and then two older versions that also know about <stdbool.h>, per the recently-out-of-support policy[1]. 12 requires C99 so it's much like the supported releases, but 11 only assumes C89 so it now uses AC_CHECK_HEADERS instead of the overly picky AC_HEADER_STDBOOL. (I could find no discussion of which historical systems had <stdbool.h> but failed the conformance test; if they ever existed, they surely aren't relevant to that policy's goals.) [1] https://wiki.postgresql.org/wiki/Committing_checklist#Policies Reported-by: Sam James <sam@gentoo.org> Reviewed-by: Peter Eisentraut <peter@eisentraut.org> (master version) Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us> (approach) Discussion: https://www.postgresql.org/message-id/flat/87o72eo9iu.fsf%40gentoo.org	1 year ago
Thomas Munro	aac831cafa	Use auxv to check for CRC32 instructions on ARM. ... Previously we probed for CRC32 instructions by testing if they caused SIGILL. Some have expressed doubts about that technique, the Linux documentation advises not to use it, and it's not exactly beautiful. Now that more operating systems expose CPU features to userspace via the ELF loader in approximately the same way, let's use that instead. This is expected to work on Linux, FreeBSD and recent OpenBSD. OpenBSD/ARM has not been tested and is not present in our build farm, but the API matches FreeBSD. On macOS, compilers use a more recent baseline ISA so the runtime test mechanism isn't reached. (A similar situation is expected for Windows/ARM when that port lands.) On NetBSD, runtime feature probing is lost for armv8-a builds. It looks potentially doable with sysctl following the example of the cpuctl program; patches are welcome. No back-patch for now, since we don't have any evidence of actual breakage from the previous technique. Suggested-by: Bastien Roucariès <rouca@debian.org> Discussion: https://postgr.es/m/4496616.iHFcN1HehY%40portable-bastien	1 year ago
Thomas Munro	78c09bd9f9	jit: Use -mno-outline-atomics for bitcode on ARM. ... If the executable's .o files were produced by a compiler (probably gcc) not using -moutline-atomics, and the corresponding .bc files were produced by clang using -moutline-atomics (probably by default), then the generated bitcode functions would have the target attribute "+outline-atomics", and could fail at runtime when inlined. If the target ISA at bitcode generation time was armv8-a (the most conservative aarch64 target, no LSE), then LLVM IR atomic instructions would generate calls to functions in libgcc.a or libclang_rt.*.a that switch between LL/SC and faster LSE instructions depending on a runtime AT_HWCAP check. Since the corresponding .o files didn't need those functions, they wouldn't have been included in the executable, and resolution would fail. At least Debian and Ubuntu are known to ship gcc and clang compilers that target armv8-a but differ on the use of outline atomics by default. Fix, by suppressing the outline atomics attribute in bitcode explicitly. Inline LL/SC instructions will be generated for atomic operations in bitcode built for armv8-a. Only configure scripts are adjusted for now, because the meson build system doesn't generate bitcode yet. This doesn't seem to be a new phenomenon, so real cases of functions using atomics that are inlined by JIT must be rare in the wild given how long it took for a bug report to arrive. The reported case could be reduced to: postgres=# set jit_inline_above_cost = 0; SET postgres=# set jit_above_cost = 0; SET postgres=# select pg_last_wal_receive_lsn(); WARNING: failed to resolve name __aarch64_swp4_acq_rel FATAL: fatal llvm error: Program used external function '__aarch64_swp4_acq_rel' which could not be resolved! The change doesn't affect non-ARM systems or later target ISAs. Back-patch to all supported releases. Reported-by: Alexander Kozhemyakin <a.kozhemyakin@postgrespro.ru> Discussion: https://postgr.es/m/18610-37bf303f904fede3%40postgresql.org	1 year ago
Peter Eisentraut	f05b5e6346	configure.ac: Remove useless AC_SUBST ... No longer used since commit 805e431a38.	1 year ago
Nathan Bossart	f78667bd91	Use __attribute__((target(...))) for AVX-512 support. ... Presently, we check for compiler support for the required intrinsics both with and without extra compiler flags (e.g., -mxsave), and then depending on the results of those checks, we pick which files to compile with which flags. This is tedious and complicated, and it results in unsustainable coding patterns such as separate files for each portion of code may need to be built with different compiler flags. This commit introduces support for __attribute__((target(...))) and uses it for the AVX-512 code. This simplifies both the configure-time checks and the build scripts, and it allows us to place the functions that use the intrinsics in files that we otherwise do not want to build with special CPU instructions. We are careful to avoid using __attribute__((target(...))) on compilers that do not understand it, but we still perform the configure-time checks in case the compiler allows using the intrinsics without it (e.g., MSVC). A similar change could likely be made for some of the CRC-32C code, but that is left as a future exercise. Suggested-by: Andres Freund Reviewed-by: Raghuveer Devulapalli, Andres Freund Discussion: https://postgr.es/m/20240731205254.vfpap7uxwmebqeaf%40awork3.anarazel.de	1 year ago
Heikki Linnakangas	99b937a44f	Add PG_TEST_EXTRA configure option to the Make builds ... The Meson builds have PG_TEST_EXTRA as a configure-time variable, which was not available in the Make builds. To ensure both build systems are in sync, PG_TEST_EXTRA is now added as a configure-time variable. It can be set like this: ./configure PG_TEST_EXTRA="kerberos, ssl, ..." Note that to preserve the old behavior, this configure-time variable is overridden by the PG_TEST_EXTRA environment variable when you run the tests. Author: Jacob Champion Reviewed by: Ashutosh Bapat, Nazir Bilal Yavuz	1 year ago
Daniel Gustafsson	6c66b7443c	Raise the minimum supported OpenSSL version to 1.1.1 ... Commit a70e01d430 retired support for OpenSSL 1.0.2 in order to get rid of the need for manual initialization of the library. This left our API usage compatible with 1.1.0 which was defined as the minimum required version. Also mention that 3.4 is the minimum version required when using LibreSSL. An upcoming commit will introduce support for configuring TLSv1.3 cipher suites which require an API call in OpenSSL 1.1.1 and onwards. In order to support this setting this commit will set v1.1.1 as the new minimum required version. The version-specific call for randomness init added in commit c3333dbc0c is removed as it's no longer needed. Author: Daniel Gustafsson <daniel@yesql.se> Discussion: https://postgr.es/m/909A668B-06AD-47D1-B8EB-A164211AAD16@yesql.se Discussion: https://postgr.es/m/tencent_063F89FA72CCF2E48A0DF5338841988E9809@qq.com	1 year ago
Peter Eisentraut	a2d9a9b95a	Remove traces of BeOS. ... Commit 15abc7788e tolerated namespace pollution from BeOS system headers. Commit 44f902122 de-supported BeOS. Since that stuff didn't make it into the Meson build system, synchronize by removing from configure. Author: Thomas Munro <thomas.munro@gmail.com> Reviewed-by: Peter Eisentraut <peter@eisentraut.org> Reviewed-by: Heikki Linnakangas <hlinnaka@iki.fi> Reviewed-by: Japin Li <japinli@hotmail.com> Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us> (the idea, not the patch) Discussion: https://postgr.es/m/ME3P282MB3166F9D1F71F787929C0C7E7B6312%40ME3P282MB3166.AUSP282.PROD.OUTLOOK.COM	1 year ago
Peter Eisentraut	9c2a6c5a5f	Simplify checking for xlocale.h ... Instead of XXX_IN_XLOCALE_H for several features XXX, let's just include <xlocale.h> if HAVE_XLOCALE_H. The reason for the extra complication was apparently that some old glibc systems also had an <xlocale.h>, and you weren't supposed to include it directly, but it's gone now (as far as I can tell it was harmless to do so anyway). Author: Thomas Munro <thomas.munro@gmail.com> Discussion: https://postgr.es/m/CWZBBRR6YA8D.8EHMDRGLCKCD%40neon.tech	1 year ago
Peter Eisentraut	ee4859123e	jit: Use opaque pointers in all supported LLVM versions. ... LLVM's opaque pointer change began in LLVM 14, but remained optional until LLVM 16. When commit 37d5babb added opaque pointer support, we didn't turn it on for LLVM 14 and 15 yet because we didn't want to risk weird bitcode incompatibility problems in released branches of PostgreSQL. (That might have been overly cautious, I don't know.) Now that PostgreSQL 18 has dropped support for LLVM versions < 14, and since it hasn't been released yet and no extensions or bitcode have been built against it in the wild yet, we can be more aggressive. We can rip out the support code and build system clutter that made opaque pointer use optional. Author: Thomas Munro <thomas.munro@gmail.com> Reviewed-by: Peter Eisentraut <peter@eisentraut.org> Discussions: https://postgr.es/m/CA%2BhUKGLhNs5geZaVNj2EJ79Dx9W8fyWUU3HxcpZy55sMGcY%3DiA%40mail.gmail.com	1 year ago
Daniel Gustafsson	a70e01d430	Remove support for OpenSSL older than 1.1.0 ... OpenSSL 1.0.2 has been EOL from the upstream OpenSSL project for some time, and is no longer the default OpenSSL version with any vendor which package PostgreSQL. By retiring support for OpenSSL 1.0.2 we can remove a lot of no longer required complexity for managing state within libcrypto which is now handled by OpenSSL. Reviewed-by: Jacob Champion <jacob.champion@enterprisedb.com> Reviewed-by: Peter Eisentraut <peter@eisentraut.org> Reviewed-by: Michael Paquier <michael@paquier.xyz> Discussion: https://postgr.es/m/ZG3JNursG69dz1lr@paquier.xyz Discussion: https://postgr.es/m/CA+hUKGKh7QrYzu=8yWEUJvXtMVm_CNWH1L_TLWCbZMwbi1XP2Q@mail.gmail.com	1 year ago
Jeff Davis	0fb0f68933	Improve configure error for ICU libraries if pkg-config is absent. ... If pkg-config is not installed, the ICU libraries cannot be found, but the custom configure error message did not mention this. This might lead to confusion about the actual problem. To improve this, remove the explicit error message and rely on PKG_CHECK_MODULES' generic error message. Author: Michael Banck Reported-by: Holger Jakobs Discussion: https://postgr.es/m/ccd579ed-4949-d3de-ab13-9e6456fd2caf%40jakobs.com Discussion: https://postgr.es/m/66b5d05c.050a0220.7c8ce.a951@mx.google.com	1 year ago
Thomas Munro	14c648ff00	All POSIX systems have langinfo.h and CODESET. ... We don't need configure probes for HAVE_LANGINFO_H (it is implied by !WIN32), and we don't need to consider systems that have it but don't define CODESET (that was for OpenBSD in commit 81cca218, but it has now had it for 19 years). Reviewed-by: Heikki Linnakangas <hlinnaka@iki.fi> Discussion: https://postgr.es/m/CA%2BhUKGJqVe0%2BPv9dvC9dSums_PXxGo9SWcxYAMBguWJUGbWz-A%40mail.gmail.com	1 year ago
Peter Eisentraut	6618891256	Add -Wmissing-variable-declarations to the standard compilation flags ... This warning flag detects global variables not declared in header files. This is similar to what -Wmissing-prototypes does for functions. (More correctly, it is similar to what -Wmissing-declarations does for functions, but -Wmissing-prototypes is a superset of that in C.) This flag is new in GCC 14. Clang has supported it for a while. Several recent commits have cleaned up warnings triggered by this, so it should now be clean. Reviewed-by: Andres Freund <andres@anarazel.de> Discussion: https://www.postgresql.org/message-id/flat/e0a62134-83da-4ba4-8cdb-ceb0111c95ce@eisentraut.org	1 year ago
Peter Eisentraut	73275f093f	Make building with LTO work on macOS ... When building with -flto, the backend binary must keep many otherwise unused symbols to make them available to dynamically loaded modules / extensions. This has been done via -Wl,--export-dynamic on many platforms for years. This flag is not supported by the macOS linker, though. Here it's called -Wl,-export_dynamic instead. Thus, make configure pick up on this variant of the flag as well. Meson has the logic upstream as of version 1.5.0. Without this fix, building with -flto fails with errors similar to [1] and [2]. [1]: https://postgr.es/m/1581936537572-0.post%40n3.nabble.com [2]: https://postgr.es/m/21800.1499270547%40sss.pgh.pa.us Author: Wolfgang Walther <walther@technowledgy.de> Discussion: https://www.postgresql.org/message-id/flat/427c7c25-e8e1-4fc5-a1fb-01ceff185e5b@technowledgy.de	1 year ago
Thomas Munro	8138526136	Remove --disable-atomics, require 32 bit atomics. ... Modern versions of all relevant architectures and tool chains have atomics support. Since edadeb07, there is no remaining reason to carry code that simulates atomic flags and uint32 imperfectly with spinlocks. 64 bit atomics are still emulated with spinlocks, if needed, for now. Any modern compiler capable of implementing C11 <stdatomic.h> must have the underlying operations we need, though we don't require C11 yet. We detect certain compilers and architectures, so hypothetical new systems might need adjustments here. Reviewed-by: Heikki Linnakangas <hlinnaka@iki.fi> Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us> (concept, not the patch) Reviewed-by: Andres Freund <andres@anarazel.de> (concept, not the patch) Discussion: https://postgr.es/m/3351991.1697728588%40sss.pgh.pa.us	1 year ago
Thomas Munro	e25626677f	Remove --disable-spinlocks. ... A later change will require atomic support, so it wouldn't make sense for a hypothetical new system not to be able to implement spinlocks. Reviewed-by: Heikki Linnakangas <hlinnaka@iki.fi> Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us> (concept, not the patch) Reviewed-by: Andres Freund <andres@anarazel.de> (concept, not the patch) Discussion: https://postgr.es/m/3351991.1697728588%40sss.pgh.pa.us	1 year ago
Daniel Gustafsson	274bbced85	Disable all TLS session tickets ... OpenSSL supports two types of session tickets for TLSv1.3, stateless and stateful. The option we've used only turns off stateless tickets leaving stateful tickets active. Use the new API introduced in 1.1.1 to disable all types of tickets. Backpatch to all supported versions. Reviewed-by: Heikki Linnakangas <hlinnaka@iki.fi> Reported-by: Andres Freund <andres@anarazel.de> Discussion: https://postgr.es/m/20240617173803.6alnafnxpiqvlh3g@awork3.anarazel.de Backpatch-through: v12	1 year ago
Peter Eisentraut	683be87fbb	Add port/ replacement for strsep() ... from OpenBSD, similar to strlcat, strlcpy There are currently no uses, but some will be added soon. Reviewed-by: Kyotaro Horiguchi <horikyota.ntt@gmail.com> Reviewed-by: David Steele <david@pgmasters.net> Discussion: https://www.postgresql.org/message-id/flat/79692bf9-17d3-41e6-b9c9-fc8c3944222a@eisentraut.org	1 year ago
Michael Paquier	e26810d01d	Stamp HEAD as 18devel. ... Let the hacking begin ...	1 year ago
Joe Conway	23c5a0e7d4	Stamp 17beta2.	2 years ago
Tom Lane	86a2d2a321	Stamp 17beta1.	2 years ago
Andres Freund	6d4bc96eb7	Avoid unnecessary "touch meson.build" in vpath builds ... In e6927270cd I added a 'touch meson.build' to configure.ac, to ensure conflicts between in-tree configure based builds and meson builds are automatically detected. Unfortunately I omitted spaces around the condition restricting this to in-tree builds, leading to touch meson.build to also be executed in vpath builds. While the only consequence of this buglet is an unnecessary empty file in build directories, it seems worth backpatching. Reported-by: Christoph Berg <myon@debian.org> Discussion: https://postgr.es/m/20240417230002.mb2gv3hyetyn67gk@awork3.anarazel.de Backpatch: 16-, where the meson based build was added	2 years ago
Nathan Bossart	792752af4e	Optimize pg_popcount() with AVX-512 instructions. ... Presently, pg_popcount() processes data in 32-bit or 64-bit chunks when possible. Newer hardware that supports AVX-512 instructions can use 512-bit chunks, which provides a nice speedup, especially for larger buffers. This commit introduces the infrastructure required to detect compiler and CPU support for the required AVX-512 intrinsic functions, and it adds a new pg_popcount() implementation that uses these functions. If CPU support for this optimized implementation is detected at runtime, a function pointer is updated so that it is used by subsequent calls to pg_popcount(). Most of the existing in-tree calls to pg_popcount() should benefit from these instructions, and calls with smaller buffers should at least not regress compared to v16. The new infrastructure introduced by this commit can also be used to optimize visibilitymap_count(), but that is left for a follow-up commit. Co-authored-by: Paul Amonson, Ants Aasma Reviewed-by: Matthias van de Meent, Tom Lane, Noah Misch, Akash Shankaran, Alvaro Herrera, Andres Freund, David Rowley Discussion: https://postgr.es/m/BL1PR11MB5304097DF7EA81D04C33F3D1DCA6A%40BL1PR11MB5304.namprd11.prod.outlook.com	2 years ago
Thomas Munro	d93627bcbe	Add --copy-file-range option to pg_upgrade. ... The copy_file_range() system call is available on at least Linux and FreeBSD, and asks the kernel to use efficient ways to copy ranges of a file. Options available to the kernel include sharing block ranges (similar to --clone mode), and pushing down block copies to the storage layer. For automated testing, see PG_TEST_PG_UPGRADE_MODE. (Perhaps in a later commit we could consider setting this mode for one of the CI targets.) Reviewed-by: Peter Eisentraut <peter@eisentraut.org> Discussion: https://postgr.es/m/CA%2BhUKGKe7Hb0-UNih8VD5UNZy5-ojxFb3Pr3xSBBL8qj2M2%3DdQ%40mail.gmail.com	2 years ago
Heikki Linnakangas	1c1eec0f2d	Remove configure --with-CC option ... It's been deprecated since commit cb292206c5 from July 2000. Discussion: https://www.postgresql.org/message-id/a2a0f5d8-2d80-48e5-b8f7-0a0a6018c3b7@iki.fi	2 years ago
Heikki Linnakangas	0b16bb8776	Remove AIX support ... There isn't a lot of user demand for AIX support, we have a bunch of hacks to work around AIX-specific compiler bugs and idiosyncrasies, and no one has stepped up to the plate to properly maintain it. Remove support for AIX to get rid of that maintenance overhead. It's still supported for stable versions. The acute issue that triggered this decision was that after commit 8af2565248, the AIX buildfarm members have been hitting this assertion: TRAP: failed Assert("(uintptr_t) buffer == TYPEALIGN(PG_IO_ALIGN_SIZE, buffer)"), File: "md.c", Line: 472, PID: 2949728 Apperently the "pg_attribute_aligned(a)" attribute doesn't work on AIX for values larger than PG_IO_ALIGN_SIZE, for a static const variable. That could be worked around, but we decided to just drop the AIX support instead. Discussion: https://www.postgresql.org/message-id/20240224172345.32@rfd.leadboat.com Reviewed-by: Andres Freund, Noah Misch, Thomas Munro	2 years ago
Thomas Munro	65f438471b	Fix gai_strerror() thread-safety on Windows. ... Commit 5579388d removed code that supplied a fallback implementation of getaddrinfo(), which was dead code on modern systems. One tiny piece of the removed code was still doing something useful on Windows, though: that OS's own gai_strerror()/gai_strerrorA() function returns a pointer to a static buffer that it overwrites each time, so it's not thread-safe. In rare circumstances, a multi-threaded client program could get an incorrect or corrupted error message. Restore the replacement gai_strerror() function, though now that it's only for Windows we can put it into a win32-specific file and cut it down to the errors that Windows documents. The error messages here are taken from FreeBSD, because Windows' own messages seemed too verbose. Back-patch to 16. Reviewed-by: Kyotaro Horiguchi <horikyota.ntt@gmail.com> Discussion: https://postgr.es/m/CA%2BhUKGKz%2BF9d2PTiXwfYV7qJw%2BWg2jzACgSDgPizUw7UG%3Di58A%40mail.gmail.com	2 years ago
Michael Paquier	d86d20f0ba	Add backend support for injection points ... Injection points are a new facility that makes possible for developers to run custom code in pre-defined code paths. Its goal is to provide ways to design and run advanced tests, for cases like: - Race conditions, where processes need to do actions in a controlled ordered manner. - Forcing a state, like an ERROR, FATAL or even PANIC for OOM, to force recovery, etc. - Arbitrary sleeps. This implements some basics, and there are plans to extend it more in the future depending on what's required. Hence, this commit adds a set of routines in the backend that allows developers to attach, detach and run injection points: - A code path calling an injection point can be declared with the macro INJECTION_POINT(name). - InjectionPointAttach() and InjectionPointDetach() to respectively attach and detach a callback to/from an injection point. An injection point name is registered in a shmem hash table with a library name and a function name, which will be used to load the callback attached to an injection point when its code path is run. Injection point names are just strings, so as an injection point can be declared and run by out-of-core extensions and modules, with callbacks defined in external libraries. This facility is hidden behind a dedicated switch for ./configure and meson, disabled by default. Note that backends use a local cache to store callbacks already loaded, cleaning up their cache if a callback has found to be removed on a best-effort basis. This could be refined further but any tests but what we have here was fine with the tests I've written while implementing these backend APIs. Author: Michael Paquier, with doc suggestions from Ashutosh Bapat. Reviewed-by: Ashutosh Bapat, Nathan Bossart, Álvaro Herrera, Dilip Kumar, Amul Sul, Nazir Bilal Yavuz Discussion: https://postgr.es/m/ZTiV8tn_MIb_H2rE@paquier.xyz	2 years ago