Update libcxx and libcxxabi from LLVM 21.1.8 to 22.1.8 - #27428
Conversation
llvm/llvm-project#167636 forces us to define `_LIBCPP_ASSERTION_SEMANTIC_DEFAULT`. This defines it to follow the hardening mode.
llvm/llvm-project#169405 has this: ```diff --- a/libcxx/src/filesystem/operations.cpp +++ b/libcxx/src/filesystem/operations.cpp @@ -41,17 +41,10 @@ #include <time.h> // since Linux 4.5 and FreeBSD 13, but the Linux libc wrapper is only provided by glibc >= 2.27 and musl -#if defined(__linux__) -# if defined(_LIBCPP_GLIBC_PREREQ) -# if _LIBCPP_GLIBC_PREREQ(2, 27) -# define _LIBCPP_FILESYSTEM_USE_COPY_FILE_RANGE -# endif -# elif _LIBCPP_HAS_MUSL_LIBC -# define _LIBCPP_FILESYSTEM_USE_COPY_FILE_RANGE -# endif -#elif defined(__FreeBSD__) +#if _LIBCPP_GLIBC_PREREQ(2, 27) || _LIBCPP_HAS_MUSL_LIBC || defined(__FreeBSD__) # define _LIBCPP_FILESYSTEM_USE_COPY_FILE_RANGE #endif + #if __has_include(<sys/sendfile.h>) # include <sys/sendfile.h> # define _LIBCPP_FILESYSTEM_USE_SENDFILE ``` Before the PR, because we didn't define `__linux__`, `_LIBCPP_FILESYSTEM_USE_COPY_FILE_RANGE` was not defined. But after it `#if defined(__linux__)` check was gone, and because we defined `_LIBCPP_HAS_MUSL_LIBC`, `_LIBCPP_FILESYSTEM_USE_COPY_FILE_RANGE` was defined as well. After this PR, we started to have `error: undefined symbol: copy_file_range` error on `core*.test_dylink_exceptions_try_catch_6_*` tests. --- This commit fixes the linker error by wiring up `SYS_copy_file_range` and provides a stub implementation returning `-ENOSYS` for both legacy JS syscalls and WASMFS. This allows `copy_file_range.c` from musl to be compiled into the system library. The libcxx implementation falls back to other mechanisms when receiving `ENOSYS`.
llvm/llvm-project#116261 wrapped `__cxa_thread_atexit` with `#if defined(__linux__) || defined(__Fuchsia__)` ```diff --- a/libcxxabi/src/cxa_thread_atexit.cpp +++ b/libcxxabi/src/cxa_thread_atexit.cpp @@ -106,6 +106,7 @@ namespace { #endif // HAVE___CXA_THREAD_ATEXIT_IMPL +#if defined(__linux__) || defined(__Fuchsia__) extern "C" { _LIBCXXABI_FUNC_VIS int __cxa_thread_atexit(Dtor dtor, void* obj, void* dso_sym bol) throw() { @@ -140,6 +141,6 @@ extern "C" { } #endif // HAVE___CXA_THREAD_ATEXIT_IMPL } - } // extern "C" +#endif // defined(__linux__) || defined(__Fuchsia__) } // namespace __cxxabiv1 ``` which caused it to undefined in Emscripten. This adds `defined(__EMSCRIPTEN__)` to make it available again.
This is an automatic change generated by tools/maint/rebaseline_tests.py. The following (14) test expectation files were updated by running the tests with `--rebaseline`: ``` codesize/test_codesize_cxx_ctors1.json: 151784 => 153973 [+2189 bytes / +1.44%] codesize/test_codesize_cxx_ctors2.json: 151187 => 153379 [+2192 bytes / +1.45%] codesize/test_codesize_cxx_except.json: 195674 => 200129 [+4455 bytes / +2.28%] codesize/test_codesize_cxx_except_wasm.json: 166908 => 169501 [+2593 bytes / +1.55%] codesize/test_codesize_cxx_except_wasm_legacy.json: 164792 => 167361 [+2569 bytes / +1.56%] codesize/test_codesize_cxx_lto.json: 120635 => 119586 [-1049 bytes / -0.87%] codesize/test_codesize_cxx_mangle.json: 262056 => 266459 [+4403 bytes / +1.68%] codesize/test_codesize_cxx_noexcept.json: 153784 => 155883 [+2099 bytes / +1.36%] codesize/test_codesize_cxx_wasmfs.json: 179386 => 180981 [+1595 bytes / +0.89%] codesize/test_codesize_files_wasmfs.json: 63585 => 63232 [-353 bytes / -0.56%] codesize/test_codesize_hello_dylink_all.json: 856538 => 856693 [+155 bytes / +0.02%] codesize/test_codesize_minimal_pthreads.json: 25967 => 25967 [+0 bytes / +0.00%] codesize/test_minimal_runtime_code_size_hello_embind.json: 14911 => 15004 [+93 bytes / +0.62%] codesize/test_minimal_runtime_code_size_hello_embind_val.json: 11638 => 11731 [+93 bytes / +0.80%] Average change: +0.87% (-0.87% - +2.28%) ```
| libc_files += files_in_path( | ||
| path='system/lib/libc/musl/src/linux', | ||
| filenames=['getdents.c', 'gettid.c', 'utimes.c', 'statx.c', 'wait4.c', 'wait3.c', 'epoll.c']) | ||
| filenames=['getdents.c', 'gettid.c', 'utimes.c', 'statx.c', 'wait4.c', 'wait3.c', 'epoll.c', 'copy_file_range.c']) |
There was a problem hiding this comment.
Can we not find a way to instead have libc++ not compile in the call to this function? Presumably its not just assuming it exists on all platforms?
There was a problem hiding this comment.
Why is it preferable not to compile this file?
There was a problem hiding this comment.
Not calling a function at all is always better than calling a stub version isn't it? Doesn't libc++ allow for this linux-specific thing to not exist?
There was a problem hiding this comment.
Which one do you prefer?
Solution 1: Add it to emscripten_libs_stub.c
diff --git a/system/lib/libc/emscripten_libc_stubs.c b/system/lib/libc/emscripten_libc_stubs.c
index bd0759bd6..2cccdd9a0 100644
--- a/system/lib/libc/emscripten_libc_stubs.c
+++ b/system/lib/libc/emscripten_libc_stubs.c
@@ -255,3 +255,12 @@ weak int getloadavg(double loadavg[], int nelem) {
}
return limit;
}
+
+// ==========================================================================
+// unistd.h
+// ==========================================================================
+
+weak ssize_t copy_file_range(int fd_in, off_t *off_in, int fd_out, off_t *off_out, size_t len, unsigned flags) {
+ errno = ENOSYS;
+ return -1;
+}Solution 2: Add if !defined(__EMSCRIPTEN__) directly into source code where _LIBCPP_FILESYSTEM_USE_COPY_FILE_RANGE is defined
diff --git a/system/lib/libcxx/src/filesystem/operations.cpp b/system/lib/libcxx/src/filesystem/operations.cpp
index 745db87ce..4c6853fc4 100644
--- a/system/lib/libcxx/src/filesystem/operations.cpp
+++ b/system/lib/libcxx/src/filesystem/operations.cpp
@@ -41,7 +41,7 @@
#include <time.h>
// since Linux 4.5 and FreeBSD 13, but the Linux libc wrapper is only provided by glibc >= 2.27 and musl
-#if _LIBCPP_GLIBC_PREREQ(2, 27) || _LIBCPP_HAS_MUSL_LIBC || defined(__FreeBSD__)
+#if (_LIBCPP_GLIBC_PREREQ(2, 27) || _LIBCPP_HAS_MUSL_LIBC || defined(__FreeBSD__)) && !defined(__EMSCRIPTEN__)
# define _LIBCPP_FILESYSTEM_USE_COPY_FILE_RANGE
#endif
There was a problem hiding this comment.
Yes, not defining _LIBCPP_FILESYSTEM_USE_COPY_FILE_RANGE seems better.
That solution seems good as you have it, but I also wonder if we should consider not defining _LIBCPP_HAS_MUSL_LIBC too? I guess that could have other knock on effects.
There was a problem hiding this comment.
There was a problem hiding this comment.
Yup, probably not worth it.
I was just thinking it might help in cases like this where _LIBCPP_HAS_MUSL_LIBC implies linux-specific stuff (since musl is normally considered linux-only).
In any case I think your proposed solution of of add !__EMSCRIPTEN__ to the _LIBCPP_FILESYSTEM_USE_COPY_FILE_RANGE definition seem good.
This increase seems to be mainly due to the changes of implementation of `std::num_get` in `libcxx/include/__locale_dir/num.h` in llvm/llvm-project#121795. Note that we had a similar size increase due to changes in `std::num_put` in llvm/llvm-project#133572 in LLVM 21 update (emscripten-core#26058). Even though the `std::num_get` PR was uploaded earlier (hence the lower PR number), it was committed later, so `std::num_put` was included in LLVM 21 release and `std::num_get` is in LLVM 22 release.
We update llvm-libc separately in update_llvm_libc.py.
libcxx used to have `module.modulemap`, but since LLVM 21, it was changed to `module.modulemap.in` in llvm/llvm-project#134699. Rather than we copy this manually in each update, this makes `update_libcxx.py` create `module.modulemap` from `module.modulemap.in` automaticaly.
This is an automatic change generated by tools/maint/rebaseline_tests.py. The following (2) test expectation files were updated by running the tests with `--rebaseline`: ``` codesize/test_codesize_cxx_wasmfs.json: 179355 => 180981 [+1626 bytes / +0.91%] codesize/test_codesize_files_wasmfs.json: 63554 => 63232 [-322 bytes / -0.51%] Average change: +0.20% (-0.51% - +0.91%) ```
Since LLVM 18, llvm/llvm-project#77883 started to provide a default assertion handler and a way to override it. We have been using the default handler ever since. Rather than manually copying it in each release, this automates the copying of the file in `update_libcxx.py`. LLVM 22's default assertion handler doesn't have any changes from that of LLVM 21.
This is an automatic change generated by tools/maint/rebaseline_tests.py. The following (1) test expectation files were updated by running the tests with `--rebaseline`: ``` codesize/test_codesize_hello_dylink_all.json: 856524 => 856679 [+155 bytes / +0.02%] Average change: +0.02% (+0.02% - +0.02%) ```
This reverts commit c7d4b0b.
This reverts commit eb90d7d.
llvm/llvm-project#169405 has this: ```diff --- a/libcxx/src/filesystem/operations.cpp +++ b/libcxx/src/filesystem/operations.cpp @@ -41,17 +41,10 @@ #include <time.h> // since Linux 4.5 and FreeBSD 13, but the Linux libc wrapper is only provided by glibc >= 2.27 and musl -#if defined(__linux__) -# if defined(_LIBCPP_GLIBC_PREREQ) -# if _LIBCPP_GLIBC_PREREQ(2, 27) -# define _LIBCPP_FILESYSTEM_USE_COPY_FILE_RANGE -# endif -# elif _LIBCPP_HAS_MUSL_LIBC -# define _LIBCPP_FILESYSTEM_USE_COPY_FILE_RANGE -# endif -#elif defined(__FreeBSD__) +#if _LIBCPP_GLIBC_PREREQ(2, 27) || _LIBCPP_HAS_MUSL_LIBC || defined(__FreeBSD__) # define _LIBCPP_FILESYSTEM_USE_COPY_FILE_RANGE #endif + #if __has_include(<sys/sendfile.h>) # include <sys/sendfile.h> # define _LIBCPP_FILESYSTEM_USE_SENDFILE ``` Before the PR, because we didn't define `__linux__`, `_LIBCPP_FILESYSTEM_USE_COPY_FILE_RANGE` was not defined. But after it `#if defined(__linux__)` check was gone, and because we defined `_LIBCPP_HAS_MUSL_LIBC`, `_LIBCPP_FILESYSTEM_USE_COPY_FILE_RANGE` was defined as well. After this PR, we started to have `error: undefined symbol: copy_file_range` error on `core*.test_dylink_exceptions_try_catch_6_*` tests. --- This commit adds `!defined(__EMSCRIPTEN__)` clause at the end to make `_LIBCPP_FILESYSTEM_USE_COPY_FILE_RANGE` undefined for Emscripten, as was before this update.
This is an automatic change generated by tools/maint/rebaseline_tests.py. The following (1) test expectation files were updated by running the tests with `--rebaseline`: ``` codesize/test_codesize_hello_dylink_all.json: 856679 => 856524 [-155 bytes / -0.02%] Average change: -0.02% (-0.02% - -0.02%) ```
This updates libcxx and libcxxabi from 21.1.8 to LLVM 22.1.8:
https://github.com/llvm/llvm-project/releases/tag/llvmorg-22.1.8
Additional changes:
(More detailed descriptions are in the commit messages)
Don't copy llvm-libc files in
update_libcxx.py: e7af49c, e81407bIn
update_libcxx.py, we don't copy llvm-libc files that libcxx depend on anymore, because we copy those files inupdate_llvm_libc.pytoo. We just assume llvm-libc has been updated to the same version before doing update, and I wrote that in the comments.Define
_LIBCPP_ASSERTION_SEMANTIC_DEFAULTin__config_site: b119c87[libc++][hardening] Allow setting the assertion semantic via CMake. llvm/llvm-project#167636 forces us to define
_LIBCPP_ASSERTION_SEMANTIC_DEFAULT. This defines it to follow the hardening mode.Undefine
_LIBCPP_FILESYSTEM_USE_COPY_FILE_RANGE: e9f7296After [libc++] Always define _LIBCPP_GLIBC_PREREQ llvm/llvm-project#169405,
_LIBCPP_FILESYSTEM_USE_COPY_FILE_RANGEwas defined, which caused Emscripten to usecopy_file_range. This commit adds!defined(__EMSCRIPTEN__)clause at the end to make_LIBCPP_FILESYSTEM_USE_COPY_FILE_RANGEundefined for Emscripten, as was before this update.Make
__cxa_thread_atexitavailable for Emscripten: 638d0e1[libc++] Enable -Wmissing-prototypes llvm/llvm-project#116261 wrapped
__cxa_thread_atexitwith#if defined(__linux__) || defined(__Fuchsia__), which caused it to be undefined in Emscripten. This addsdefined(__EMSCRIPTEN__)to make it available again.Increase size expectations of
test_malloc_size*: c818aa2This increase seems to be mainly due to the changes of implementation of
std::num_getinlibcxx/include/__locale_dir/num.hin [libc++] Optimize num_get integral functions llvm/llvm-project#121795. Note that we had a similar size increase due to changes instd::num_putin Reapply "[libc++] Optimize num_put integral functions" (#131613) llvm/llvm-project#133572 in LLVM 21 update (Update libcxx and libcxxabi from LLVM 20.1.8 to 21.1.8 #26058).Create
module.modulemapfrommodule.modulemap.in: be1f0a7libcxx used to have
module.modulemap, but since LLVM 21, it was changed tomodule.modulemap.inin [libc++] Make __config_site modular llvm/llvm-project#134699. Rather than we copy this manually in each update, this makesupdate_libcxx.pycreatemodule.modulemapfrommodule.modulemap.inautomaticaly.Copy
default_assertion_handler.into__assertion_handler: 547432fSince LLVM 18, [libc++][hardening] Rework how the assertion handler can be overridden. llvm/llvm-project#77883 started to provide a default assertion handler and a way to override it. We have been using the default handler ever since. Rather than manually copying it in each release, this automates the copying of the file in
update_libcxx.py.