4 Commits (3d9c03429a82c199a77563ae5d57c4c9cefa3d41)

Author	SHA1	Message	Date
Jacob Champion	3d9c03429a	oauth: Ensure unused socket registrations are removed ... If Curl needs to switch the direction of a socket's registration (e.g. from CURL_POLL_IN to CURL_POLL_OUT), it expects the old registration to be discarded. For epoll, this happened via EPOLL_CTL_MOD, but for kqueue, the old registration would remain if it was not explicitly removed by Curl. Explicitly remove the opposite-direction event during registrations. (If that event doesn't exist, we'll just get an ENOENT, which will be ignored by the same code that handles CURL_POLL_REMOVE.) A few assertions are also added to strengthen the relationship between the number of events added, the number of events pulled off the queue, and the lengths of the kevent arrays. Reviewed-by: Thomas Munro <thomas.munro@gmail.com> Backpatch-through: 18 Discussion: https://postgr.es/m/CAOYmi+nDZxJHaWj9_jRSyf8uMToCADAmOfJEggsKW-kY7aUwHA@mail.gmail.com	1 month ago
Jacob Champion	ff5b0824b3	oauth: Remove stale events from the kqueue multiplexer ... If a socket is added to the kqueue, becomes readable/writable, and subsequently becomes non-readable/writable again, the kqueue itself will remain readable until either the socket registration is removed, or the stale event is cleared via a call to kevent(). In many simple cases, Curl itself will remove the socket registration quickly, but in real-world usage, this is not guaranteed to happen. The kqueue can then remain stuck in a permanently readable state until the request ends, which results in pointless wakeups for the client and wasted CPU time. Implement comb_multiplexer() to call kevent() and unstick any stale events that would cause unnecessary callbacks. This is called right after drive_request(), before we return control to the client to wait. Suggested-by: Thomas Munro <thomas.munro@gmail.com> Co-authored-by: Thomas Munro <thomas.munro@gmail.com> Reviewed-by: Thomas Munro <thomas.munro@gmail.com> Backpatch-through: 18 Discussion: https://postgr.es/m/CAOYmi+nDZxJHaWj9_jRSyf8uMToCADAmOfJEggsKW-kY7aUwHA@mail.gmail.com	1 month ago
Jacob Champion	cbc8fd0c9a	oauth: Limit JSON parsing depth in the client ... Check the ctx->nested level as we go, to prevent a server from running the client out of stack space. The limit we choose when communicating with authorization servers can't be overly strict, since those servers will continue to add extensions in their JSON documents which we need to correctly ignore. For the SASL communication, we can be more conservative, since there are no defined extensions (and the peer is probably more Postgres code). Reviewed-by: Aleksander Alekseev <aleksander@timescale.com> Discussion: https://postgr.es/m/CAOYmi%2Bm71aRUEi0oQE9ciBnBS8xVtMn3CifaPu2kmJzUfhOZgA%40mail.gmail.com	4 months ago
Jacob Champion	b0635bfda0	oauth: Move the builtin flow into a separate module ... The additional packaging footprint of the OAuth Curl dependency, as well as the existence of libcurl in the address space even if OAuth isn't ever used by a client, has raised some concerns. Split off this dependency into a separate loadable module called libpq-oauth. When configured using --with-libcurl, libpq.so searches for this new module via dlopen(). End users may choose not to install the libpq-oauth module, in which case the default flow is disabled. For static applications using libpq.a, the libpq-oauth staticlib is a mandatory link-time dependency for --with-libcurl builds. libpq.pc has been updated accordingly. The default flow relies on some libpq internals. Some of these can be safely duplicated (such as the SIGPIPE handlers), but others need to be shared between libpq and libpq-oauth for thread-safety. To avoid exporting these internals to all libpq clients forever, these dependencies are instead injected from the libpq side via an initialization function. This also lets libpq communicate the offsets of PGconn struct members to libpq-oauth, so that we can function without crashing if the module on the search path came from a different build of Postgres. (A minor-version upgrade could swap the libpq-oauth module out from under a long-running libpq client before it does its first load of the OAuth flow.) This ABI is considered "private". The module has no SONAME or version symlinks, and it's named libpq-oauth-<major>.so to avoid mixing and matching across Postgres versions. (Future improvements may promote this "OAuth flow plugin" to a first-class concept, at which point we would need a public API to replace this anyway.) Additionally, NLS support for error messages in b3f0be788a was incomplete, because the new error macros weren't being scanned by xgettext. Fix that now. Per request from Tom Lane and Bruce Momjian. Based on an initial patch by Daniel Gustafsson, who also contributed docs changes. The "bare" dlopen() concept came from Thomas Munro. Many people reviewed the design and implementation; thank you! Co-authored-by: Daniel Gustafsson <daniel@yesql.se> Reviewed-by: Andres Freund <andres@anarazel.de> Reviewed-by: Christoph Berg <myon@debian.org> Reviewed-by: Daniel Gustafsson <daniel@yesql.se> Reviewed-by: Jelte Fennema-Nio <postgres@jeltef.nl> Reviewed-by: Peter Eisentraut <peter@eisentraut.org> Reviewed-by: Wolfgang Walther <walther@technowledgy.de> Discussion: https://postgr.es/m/641687.1742360249%40sss.pgh.pa.us	5 months ago
Michael Paquier	88e947136b	Fix typos and grammar in the code ... The large majority of these have been introduced by recent commits done in the v18 development cycle. Author: Alexander Lakhin <exclusion@gmail.com> Discussion: https://postgr.es/m/9a7763ab-5252-429d-a943-b28941e0e28b@gmail.com	5 months ago
David Rowley	3fae25cbb3	Fixup various new-to-v18 usages of appendPQExpBuffer ... Use appendPQExpBufferStr when there are no parameters and appendPQExpBufferChar when the string length is 1. Author: David Rowley <drowleyml@gmail.com> Discussion: https://postgr.es/m/CAApHDvoARMvPeXTTC0HnpARBHn-WgVstc8XFCyMGOzvgu_1HvQ@mail.gmail.com	5 months ago
Daniel Gustafsson	8ae0a37932	oauth: Fix build on platforms without epoll/kqueue ... register_socket() missed a variable declaration if neither HAVE_SYS_EPOLL_H nor HAVE_SYS_EVENT_H was defined. While we're fixing that, adjust the tests to check pg_config.h for one of the multiplexer implementations, rather than assuming that Windows is the only platform without support. (Christoph reported this on hurd-amd64, an experimental Debian.) Author: Jacob Champion <jacob.champion@enterprisedb.com> Reported-by: Christoph Berg <myon@debian.org> Discussion: https://postgr.es/m/Z-sPFl27Y0ZC-VBl%40msg.df7cb.de	6 months ago
Peter Eisentraut	618c64ffd3	Revert workarounds for -Wmissing-braces false positives on old GCC ... We have collected several instances of a workaround for GCC bug 53119, which caused false-positive compiler warnings. This bug has long been fixed, but was still seen on the buildfarm, most recently on lapwing with gcc (Debian 4.7.2-5). (The GCC bug tracker mentions that a fix was backported to 4.7.4 and 4.8.3.) That compiler no longer runs warning-free since commit 6fdd5d9563, so we don't need to keep these workarounds. And furthermore, the consensus appears to be that we don't want to keep supporting that era of platform anymore at all. This reverts the following commits: d937904cce 506428d091 b449afb582 6392f2a096 bad0763a4d 5e0c761d0a and makes a few similar fixes to newer code. Discussion: https://www.postgresql.org/message-id/flat/e170d61f-01ab-4cf9-ab68-91cd1fac62c5%40eisentraut.org Discussion: https://www.postgresql.org/message-id/flat/CA%2BTgmoYEAm-KKZibAP3hSqbTFTjUd47XtVcf3xSFDpyecXX9uQ%40mail.gmail.com	6 months ago
Thomas Munro	0b53c08677	Fix compiler warning for commit 434dbf69. ... Reported-by: Tom Lane <tgl@sss.pgh.pa.us>	6 months ago
Thomas Munro	434dbf6907	oauth: Fix postcondition for set_timer on macOS ... On macOS, readding an EVFILT_TIMER to a kqueue does not appear to clear out previously queued timer events, so checks for timer expiration do not work correctly during token retrieval. Switching to IPv4-only communication exposes the problem, because libcurl is no longer clearing out other timeouts related to Happy Eyeballs dual-stack handling. Fully remove and re-register the kqueue timer events during each call to set_timer(), to clear out any stale expirations. Author: Jacob Champion <jacob.champion@enterprisedb.com> Discussion: https://postgr.es/m/CAOYmi%2Bn4EDOOUL27_OqYT2-F2rS6S%2B3mK-ppWb2Ec92UEoUbYA%40mail.gmail.com	6 months ago
Thomas Munro	c301a0a74a	Work around OAuth/EVFILT_TIMER quirk on NetBSD. ... NetBSD's EVFILT_TIMER doesn't like zero timeouts, as introduced by commit b3f0be788. Steal the workaround from the same problem on Linux from a few lines up: round zero up to one. Do this only for NetBSD, as the other systems with the kevent() API accept zero and shouldn't have to insert a small bogus wait. Future improvement ideas: * when NetBSD < 10 falls out of support, we could try NODE_ABSTIME for the "fire now" meaning if timeout == 0 * when libcurl tells us to start a 0ms timer and call it back, we could figure out how to handle that more directly without involving the kernel (the current architecture doesn't make that straightforward) Failures with EINVAL errors could be seen on the new optional NetBSD CI task that we're trying to keep green as a candidate for inclusion as default-enabled CI task. The NetBSD build farm animals aren't testing OAuth yet, so no breakage there. Reviewed-by: Jacob Champion <jacob.champion@enterprisedb.com> Discussion: https://postgr.es/m/CA%2BhUKGJ%2BWyJ26QGvO_nkgvbxgw%2B03U4EQ4Hxw%2BQBft6Np%2BXW7w%40mail.gmail.com	7 months ago
Daniel Gustafsson	d1146dc2a7	oauth: Rename macro to avoid collisions on Windows ... Our json parsing defined the macros OPTIONAL and REQUIRED to decorate the structs with for increased readability. This however collides with macros in the <windef.h> header on Windows. ../src/interfaces/libpq/fe-auth-oauth-curl.c:398:9: warning: "OPTIONAL" redefined 398 | #define OPTIONAL false | ^~~~~~~~ In file included from D:/a/_temp/msys64/ucrt64/include/windef.h:9, from D:/a/_temp/msys64/ucrt64/include/windows.h:69, from D:/a/_temp/msys64/ucrt64/include/winsock2.h:23, from ../src/include/port/win32_port.h:60, from ../src/include/port.h:24, from ../src/include/c.h:1331, from ../src/include/postgres_fe.h:28, from ../src/interfaces/libpq/fe-auth-oauth-curl.c:16: include/minwindef.h:65:9: note: this is the location of the previous definition 65 | #define OPTIONAL | ^~~~~~~~ Rename to avoid compilation errors in anticipation of implementing support for Windows. Reported-by: Dave Cramer (on PostgreSQL Hacking Discord)	7 months ago
Daniel Gustafsson	b3f0be788a	Add support for OAUTHBEARER SASL mechanism ... This commit implements OAUTHBEARER, RFC 7628, and OAuth 2.0 Device Authorization Grants, RFC 8628. In order to use this there is a new pg_hba auth method called oauth. When speaking to a OAuth- enabled server, it looks a bit like this: $ psql 'host=example.org oauth_issuer=... oauth_client_id=...' Visit https://oauth.example.org/login and enter the code: FPQ2-M4BG Device authorization is currently the only supported flow so the OAuth issuer must support that in order for users to authenticate. Third-party clients may however extend this and provide their own flows. The built-in device authorization flow is currently not supported on Windows. In order for validation to happen server side a new framework for plugging in OAuth validation modules is added. As validation is implementation specific, with no default specified in the standard, PostgreSQL does not ship with one built-in. Each pg_hba entry can specify a specific validator or be left blank for the validator installed as default. This adds a requirement on libcurl for the client side support, which is optional to build, but the server side has no additional build requirements. In order to run the tests, Python is required as this adds a https server written in Python. Tests are gated behind PG_TEST_EXTRA as they open ports. This patch has been a multi-year project with many contributors involved with reviews and in-depth discussions: Michael Paquier, Heikki Linnakangas, Zhihong Yu, Mahendrakar Srinivasarao, Andrey Chudnovsky and Stephen Frost to name a few. While Jacob Champion is the main author there have been some levels of hacking by others. Daniel Gustafsson contributed the validation module and various bits and pieces; Thomas Munro wrote the client side support for kqueue. Author: Jacob Champion <jacob.champion@enterprisedb.com> Co-authored-by: Daniel Gustafsson <daniel@yesql.se> Co-authored-by: Thomas Munro <thomas.munro@gmail.com> Reviewed-by: Daniel Gustafsson <daniel@yesql.se> Reviewed-by: Peter Eisentraut <peter@eisentraut.org> Reviewed-by: Antonin Houska <ah@cybertec.at> Reviewed-by: Kashif Zeeshan <kashi.zeeshan@gmail.com> Discussion: https://postgr.es/m/d1b467a78e0e36ed85a09adf979d04cf124a9d4b.camel@vmware.com	7 months ago