Skip to content

gh-153400: Add syscall fallbacks for copy_file_range/memfd_create #155520

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
vstinner merged 1 commit into from
Aug 13, 2026
+47 −15
Merged

gh-153400: Add syscall fallbacks for copy_file_range/memfd_create #155520

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 10 additions & 2 deletions Doc/library/os.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1011,10 +1011,14 @@ as internal buffering of data.
It will always copy no bytes and return 0 as if the file was empty
because of a known Linux kernel issue.

.. availability:: Linux >= 4.5 with glibc >= 2.27.
.. availability:: Linux >= 4.5.

.. versionadded:: 3.8

.. versionchanged:: 3.16
The function is now also available when Python is built against a libc
that lacks ``copy_file_range()``, such as glibc older than 2.27.


.. function:: device_encoding(fd)

Expand Down Expand Up @@ -4400,10 +4404,14 @@ The following flags are used in :attr:`statvfs_result.f_flag`.
the file descriptor, and as such multiple files can have the same name
without any side effects.

.. availability:: Linux >= 3.17 with glibc >= 2.27.
.. availability:: Linux >= 3.17.

.. versionadded:: 3.8

.. versionchanged:: 3.16
The function is now also available when Python is built against a libc
that lacks ``memfd_create()``, such as glibc older than 2.27.


.. data:: MFD_CLOEXEC
MFD_ALLOW_SEALING
Expand Down
4 changes: 4 additions & 0 deletions Misc/NEWS.d/next/Library/2026-08-10-22-23-29.gh-issue-153400.nucElC.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
On Linux, :func:`os.copy_file_range` and :func:`os.memfd_create` now fall back
to the raw syscall when the libc Python is built against does not provide the
wrapper function, so they stay available on interpreters built against a libc
older than glibc 2.27.
10 changes: 5 additions & 5 deletions Modules/clinic/posixmodule.c.h
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

36 changes: 28 additions & 8 deletions Modules/posixmodule.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -157,8 +157,8 @@
#ifdef HAVE_LINUX_RANDOM_H
# include <linux/random.h> // GRND_RANDOM
#endif
#ifdef HAVE_GETRANDOM_SYSCALL
# include <sys/syscall.h> // syscall()
#ifdef HAVE_SYS_SYSCALL_H
# include <sys/syscall.h> // syscall(), __NR_xxx syscall numbers
#endif

#ifdef HAVE_POSIX_SPAWN
Expand Down Expand Up @@ -13133,7 +13133,12 @@ os_pwritev_impl(PyObject *module, int fd, PyObject *buffers, Py_off_t offset,
}
#endif /* HAVE_PWRITEV */

Comment thread
daandemeyer marked this conversation as resolved.
#ifdef HAVE_COPY_FILE_RANGE
#if defined(HAVE_COPY_FILE_RANGE) || \
(defined(__linux__) && defined(__NR_copy_file_range))
# define _Py_HAVE_COPY_FILE_RANGE
#endif

#ifdef _Py_HAVE_COPY_FILE_RANGE
/*[clinic input]

os.copy_file_range
Expand Down Expand Up @@ -13185,7 +13190,13 @@ os_copy_file_range_impl(PyObject *module, int src, int dst, Py_ssize_t count,

do {
Py_BEGIN_ALLOW_THREADS
#ifdef HAVE_COPY_FILE_RANGE
ret = copy_file_range(src, p_offset_src, dst, p_offset_dst, count, flags);
#else
/* Largefile support makes off_t 64-bit, as the kernel expects. */
ret = syscall(__NR_copy_file_range, src, p_offset_src, dst, p_offset_dst,
count, flags);
#endif
Py_END_ALLOW_THREADS
} while (ret < 0 && errno == EINTR && !(async_err = PyErr_CheckSignals()));

Expand All @@ -13195,7 +13206,7 @@ os_copy_file_range_impl(PyObject *module, int src, int dst, Py_ssize_t count,

return PyLong_FromSsize_t(ret);
}
#endif /* HAVE_COPY_FILE_RANGE*/
#endif /* _Py_HAVE_COPY_FILE_RANGE */

#if (defined(HAVE_SPLICE) && !defined(_AIX))
/*[clinic input]
Expand Down Expand Up @@ -15897,7 +15908,12 @@ os_urandom_impl(PyObject *module, Py_ssize_t size)
return PyBytesWriter_Finish(writer);
}

Comment thread
daandemeyer marked this conversation as resolved.
#ifdef HAVE_MEMFD_CREATE
#if defined(HAVE_MEMFD_CREATE) || \
(defined(__linux__) && defined(__NR_memfd_create) && defined(MFD_CLOEXEC))
# define _Py_HAVE_MEMFD_CREATE
#endif

#ifdef _Py_HAVE_MEMFD_CREATE
/*[clinic input]
os.memfd_create

Expand All @@ -15913,7 +15929,11 @@ os_memfd_create_impl(PyObject *module, PyObject *name, unsigned int flags)
int fd;
const char *bytes = PyBytes_AS_STRING(name);
Py_BEGIN_ALLOW_THREADS
#ifdef HAVE_MEMFD_CREATE
fd = memfd_create(bytes, flags);
#else
fd = syscall(__NR_memfd_create, bytes, flags);
#endif
Py_END_ALLOW_THREADS
if (fd == -1) {
return PyErr_SetFromErrno(PyExc_OSError);
Expand Down Expand Up @@ -18533,7 +18553,7 @@ all_ins(PyObject *m)
if (PyModule_AddIntMacro(m, GRND_RANDOM)) return -1;
if (PyModule_AddIntMacro(m, GRND_NONBLOCK)) return -1;
#endif
#ifdef HAVE_MEMFD_CREATE
#ifdef _Py_HAVE_MEMFD_CREATE
if (PyModule_AddIntMacro(m, MFD_CLOEXEC)) return -1;
if (PyModule_AddIntMacro(m, MFD_ALLOW_SEALING)) return -1;
#ifdef MFD_HUGETLB
Expand Down Expand Up @@ -18581,7 +18601,7 @@ all_ins(PyObject *m)
#ifdef MFD_HUGE_16GB
if (PyModule_AddIntMacro(m, MFD_HUGE_16GB)) return -1;
#endif
#endif /* HAVE_MEMFD_CREATE */
#endif /* _Py_HAVE_MEMFD_CREATE */

#if defined(HAVE_EVENTFD) && defined(EFD_CLOEXEC)
if (PyModule_AddIntMacro(m, EFD_CLOEXEC)) return -1;
Expand Down Expand Up @@ -18837,7 +18857,7 @@ static const struct have_function {
{ "HAVE_LUTIMES", NULL },
#endif

#ifdef HAVE_MEMFD_CREATE
#ifdef _Py_HAVE_MEMFD_CREATE
{ "HAVE_MEMFD_CREATE", NULL },
#endif

Expand Down
Loading