gh-153400: Resolve newer libc/syscall wrappers at runtime on Linux

[daandemeyer]

os functions like copy_file_range() are gated on a configure-time AC_CHECK_FUNCS probe and compiled out when the build libc lacks the symbol. A redistributable built against an old glibc (the python-build-standalone builds target glibc 2.17) therefore never exposes them, even when run on a newer glibc or a capable kernel.

Add Modules/posixshims.h, which on Linux always exposes Py(), resolves the libc symbol once at load time via dlsym(RTLD_DEFAULT) from a constructor, and falls back to the raw syscall when the running libc lacks the wrapper. Resolving in a constructor keeps dlsym(), which is not async-signal-safe, out of signal handlers and the fork()/exec() window. Off Linux the classic build-time HAVE_* direct calls are kept.

Limit the shims to wrappers newer than the glibc 2.17 baseline, since anything at or below it is always present in the build libc:

copy_file_range (glibc 2.27)
memfd_create (glibc 2.27)
pidfd_open (glibc 2.36)
pidfd_getfd (glibc 2.36)

pidfd_open() and pidfd_getfd() previously issued raw syscalls unconditionally; they now prefer the glibc wrapper when present.

• Issue: Resolve newer Linux os functions at runtime instead of compiling them out against old glibc #153400

[daandemeyer]

@picnixz @vstinner @gpshead Any chance one of you could give this a look? I don't think CODEOWNERS ended up assigning relevant reviewers for this PR. Together with #153398 which I accidentally messed up this PR has already been open for more than a month.

[thesamesam]

For some syscall wrappers, glibc has _FORTIFY_SOURCE checks on the arguments. CPython right now doesn't ever use glibc wrappers (uses syscall instead), it just uses glibc for the syscall numbers, which means the status quo is the worst-of-both-worlds: you get no protection or sanity checks, but you also need a newer glibc that acknowledges the syscall exists. The number could also come from linux-headers.

I think CPython should use the glibc wrappers if they exist at build-time at least.

I'm not super keen on using dlopen for calling the wrapper if they exist, as this PR does, as it bypasses any safety/compatibility guarantees glibc provides from symbol versioning.

[vstinner]

I understand that you want to build Python with glibc 2.17 and run Python on recent Linux kernel with new syscalls (copy_file_range, memfd_create, pidfd_open, pidfd_getfd). Currently, you don't get the os functions since the glibc functions are missing.

I would be fine with adding a syscall() fallbacks implementations for glibc 2.17, but I'm not sure about loading glibc symbols at runtime.

I would prefer to not implement a function if the syscall number is missing. Do you get all required syscall numbers on glibc 2.17?

You wrote:

pidfd_open and pidfd_getfd already sidestep it by calling the raw syscall directly.

So I don't understand. os.pidfd_open() and os.pidfd_getfd() are already available when you build with glibc 2.17 or not?

[vstinner]

Oh, I didn't notice that glibc 2.36 added pidfd_open() and pidfd_getfd() functions. That was 4 years ago! We should use them if they are available. I wrote PR gh-155518 for that.

[daandemeyer]

@vstinner syscall fallback version: #155520