Skip to content

mimalloc integration #2708

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Vansh-kap-98 wants to merge 7 commits into
base: main
Choose a base branch
from
Open

mimalloc integration #2708

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
66cfe1d
perf: integrate mimalloc for core benchmarking
Vansh-kap-98 Aug 6, 2026
0af17b8
Fix mimalloc target naming, sanitizer ordering, and restore corrupted…
Vansh-kap-98 Aug 6, 2026
c8a686c
Link mimalloc unconditionally on Linux
Vansh-kap-98 Aug 12, 2026
5976b67
Merge remote-tracking branch 'origin/main' into feature/mimalloc-inte…
Vansh-kap-98 Aug 12, 2026
d80acb0
CI failure fix
Vansh-kap-98 Aug 12, 2026
3a4b251
Guard mimalloc against sanitizers, centralize enablement condition, c…
Vansh-kap-98 Aug 13, 2026
f463ad2
corrupt patch fix
Vansh-kap-98 Aug 13, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion .gitattributes
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,10 @@
/vendor/** linguist-generated=true
vendor/*.mask text eol=lf
*.sh text eol=lf
test/test/*.txt text eol=lf

# These fixtures are parser input, so their bytes, including their line
# endings, must reach the working directory exactly as committed on every
# platform
/test/json/stub_*.json -text
/test/yaml/stubs/** -text
/test/yaml/stubs/** -text
2 changes: 2 additions & 0 deletions .gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,3 +12,5 @@ _deps
Brewfile.lock.json
.DS_Store
.cache
out/
CMakeSettings.json
9 changes: 9 additions & 0 deletions CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,15 @@ elseif(SOURCEMETA_CORE_UNDEFINED_SANITIZER)
sourcemeta_sanitizer(TYPE undefined)
endif()

if(SOURCEMETA_OS_LINUX
AND NOT SOURCEMETA_CORE_ADDRESS_SANITIZER
AND NOT SOURCEMETA_CORE_UNDEFINED_SANITIZER)
set(SOURCEMETA_CORE_MIMALLOC_ENABLED ON)
find_package(Mimalloc REQUIRED)

@cubic-dev-ai cubic-dev-ai Bot Aug 6, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3: The new SOURCEMETA_CORE_ALLOCATOR option is declared at the top-level and immediately drives find_package(Mimalloc REQUIRED), but the only target that ever links the resulting library is the benchmark (and benchmarks default OFF). So a user who picks "mimalloc" expecting the core library stack to use it gets nothing changed, while still paying for a full mimalloc configure/build and risking a hard configure failure from the REQUIRED find when only the non-benchmark build is wanted. Consider gating the find_package (and ideally the option's effect) behind SOURCEMATA_CORE_BENCHMARK, and either documenting that the allocator only applies to the benchmark or actually threading the link into the core libraries so the option does what its description says.

Prompt for AI agents 
Check if this issue is valid — if so, understand the root cause and fix it. At CMakeLists.txt, line 77:

<comment>The new SOURCEMETA_CORE_ALLOCATOR option is declared at the top-level and immediately drives `find_package(Mimalloc REQUIRED)`, but the only target that ever links the resulting library is the benchmark (and benchmarks default OFF). So a user who picks "mimalloc" expecting the core library stack to use it gets nothing changed, while still paying for a full mimalloc configure/build and risking a hard configure failure from the REQUIRED find when only the non-benchmark build is wanted. Consider gating the find_package (and ideally the option's effect) behind SOURCEMATA_CORE_BENCHMARK, and either documenting that the allocator only applies to the benchmark or actually threading the link into the core libraries so the option does what its description says.</comment>

<file context>
@@ -55,19 +55,29 @@ option(SOURCEMETA_CORE_CONTRIB_GOOGLEBENCHMARK "Build the GoogleBenchmark librar
 
-# TODO: Turn this into a re-usable utility CMake function
+if(SOURCEMETA_CORE_ALLOCATOR STREQUAL "mimalloc")
+  find_package(Mimalloc REQUIRED)
+endif()
+
</file context>

else()
set(SOURCEMETA_CORE_MIMALLOC_ENABLED OFF)
endif()

# TODO: Turn this into a re-usable utility CMake function
if(SOURCEMETA_CORE_INSTALL)
include(GNUInstallDirs)
Expand Down
1 change: 1 addition & 0 deletions DEPENDENCIES
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
vendorpull https://github.com/sourcemeta/vendorpull 89f348a97842e05aeab45d338d41fb02031fad62
mimalloc https://github.com/microsoft/mimalloc v3.4.4
jsontestsuite https://github.com/nst/JSONTestSuite d64aefb55228d9584d3e5b2433f720ea8fd00c82
yaml-test-suite https://github.com/yaml/yaml-test-suite data-2022-01-17
cmark-gfm https://github.com/github/cmark-gfm 587a12bb54d95ac37241377e6ddc93ea0e45439b
Expand Down
66 changes: 66 additions & 0 deletions cmake/FindMimalloc.cmake
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
if(NOT Mimalloc_FOUND)
set(MIMALLOC_DIR "${PROJECT_SOURCE_DIR}/vendor/mimalloc")

set(MI_INSTALL_TOPLEVEL ON CACHE BOOL "" FORCE)
set(MI_BUILD_SHARED ${BUILD_SHARED_LIBS} CACHE BOOL "" FORCE)
if(BUILD_SHARED_LIBS)
set(MI_BUILD_STATIC OFF CACHE BOOL "" FORCE)
set(MIMALLOC_TARGET mimalloc)
else()
set(MI_BUILD_STATIC ON CACHE BOOL "" FORCE)
set(MIMALLOC_TARGET mimalloc-static)
endif()
set(MI_BUILD_OBJECT OFF CACHE BOOL "" FORCE)
set(MI_BUILD_TESTS OFF CACHE BOOL "" FORCE)
set(MI_OVERRIDE ON CACHE BOOL "" FORCE)

set(SOURCEMETA_CORE_MIMALLOC_SKIP_INSTALL "${MI_SKIP_INSTALL}")
set(MI_SKIP_INSTALL ON)
add_subdirectory(
"${MIMALLOC_DIR}"
"${CMAKE_CURRENT_BINARY_DIR}/mimalloc" EXCLUDE_FROM_ALL)
set(MI_SKIP_INSTALL "${SOURCEMETA_CORE_MIMALLOC_SKIP_INSTALL}")
unset(SOURCEMETA_CORE_MIMALLOC_SKIP_INSTALL)

if(TARGET ${MIMALLOC_TARGET})
set_target_properties(${MIMALLOC_TARGET}
PROPERTIES COMPILE_WARNING_AS_ERROR OFF)
set(Mimalloc_FOUND ON)

if(SOURCEMETA_CORE_INSTALL)
include(GNUInstallDirs)
if(BUILD_SHARED_LIBS)
install(TARGETS ${MIMALLOC_TARGET}
EXPORT mimalloc
RUNTIME DESTINATION "${CMAKE_INSTALL_BINDIR}"
COMPONENT sourcemeta_core
LIBRARY DESTINATION "${CMAKE_INSTALL_LIBDIR}"
COMPONENT sourcemeta_core
NAMELINK_COMPONENT sourcemeta_core_dev
ARCHIVE DESTINATION "${CMAKE_INSTALL_LIBDIR}"
COMPONENT sourcemeta_core_dev)
else()
install(TARGETS ${MIMALLOC_TARGET}
EXPORT mimalloc
ARCHIVE DESTINATION "${CMAKE_INSTALL_LIBDIR}"
COMPONENT sourcemeta_core_dev)
endif()
install(EXPORT mimalloc
FILE mimalloc.cmake
DESTINATION "${CMAKE_INSTALL_LIBDIR}/cmake/mimalloc"
COMPONENT sourcemeta_core_dev)
install(FILES
"${MIMALLOC_DIR}/include/mimalloc.h"
"${MIMALLOC_DIR}/include/mimalloc-new-delete.h"
"${MIMALLOC_DIR}/include/mimalloc-override.h"
"${MIMALLOC_DIR}/include/mimalloc-stats.h"
DESTINATION "${CMAKE_INSTALL_INCLUDEDIR}"
COMPONENT sourcemeta_core_dev)
install(FILES
"${MIMALLOC_DIR}/cmake/mimalloc-config.cmake"
"${MIMALLOC_DIR}/cmake/mimalloc-config-version.cmake"
DESTINATION "${CMAKE_INSTALL_LIBDIR}/cmake/mimalloc"
COMPONENT sourcemeta_core_dev)
endif()
endif()
endif()
22 changes: 22 additions & 0 deletions cmake/common/targets/library.cmake
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,28 @@ function(sourcemeta_library)

add_library(${ALIAS_NAME} ALIAS ${TARGET_NAME})

if(SOURCEMETA_CORE_MIMALLOC_ENABLED
AND (TARGET mimalloc-static OR TARGET mimalloc))
if(BUILD_SHARED_LIBS)
set(SOURCEMETA_MIMALLOC_LINK_LIBRARY mimalloc)
else()
set(SOURCEMETA_MIMALLOC_LINK_LIBRARY
"$<LINK_LIBRARY:WHOLE_ARCHIVE,mimalloc-static>")
endif()

if(SOURCEMETA_LIBRARY_SOURCES)
if(NOT BUILD_SHARED_LIBS)
add_dependencies(${TARGET_NAME} mimalloc-static)
endif()

target_link_libraries(${TARGET_NAME} PRIVATE
"${SOURCEMETA_MIMALLOC_LINK_LIBRARY}")
else()
target_link_libraries(${TARGET_NAME} INTERFACE
"${SOURCEMETA_MIMALLOC_LINK_LIBRARY}")
endif()
endif()

if(NOT SOURCEMETA_LIBRARY_VARIANT)
set(include_dir "${CMAKE_CURRENT_SOURCE_DIR}/include")
else()
Expand Down
4 changes: 4 additions & 0 deletions config.cmake.in
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,10 @@ endif()

include(CMakeFindDependencyMacro)

if("@SOURCEMETA_CORE_MIMALLOC_ENABLED@" STREQUAL "ON")
find_dependency(mimalloc CONFIG)
endif()

foreach(component ${SOURCEMETA_CORE_COMPONENTS})
if(component STREQUAL "preprocessor")
include("${CMAKE_CURRENT_LIST_DIR}/sourcemeta_core_preprocessor.cmake")
Expand Down
96 changes: 96 additions & 0 deletions patches/mimalloc/0001-Skip-vendored-install-rules-when-embedded.patch
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,96 @@
diff --git a/CMakeLists.txt b/CMakeLists.txt
index 3189dec42426f4f3cbd88f54e72f8c5eb557073a..f76757cd3ae68624c2dbf1c6ad882bcae6c7bc15 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -705,14 +705,18 @@ if(MI_BUILD_SHARED)
$<BUILD_INTERFACE:${CMAKE_CURRENT_SOURCE_DIR}/include>
$<INSTALL_INTERFACE:${mi_install_incdir}>
)
- install(TARGETS mimalloc EXPORT mimalloc ARCHIVE DESTINATION ${CMAKE_INSTALL_LIBDIR} RUNTIME DESTINATION ${CMAKE_INSTALL_BINDIR} LIBRARY DESTINATION ${CMAKE_INSTALL_LIBDIR})
- install(EXPORT mimalloc DESTINATION ${mi_install_cmakedir})
+ if(NOT MI_SKIP_INSTALL)
Comment thread
cubic-dev-ai[bot] marked this conversation as resolved.
+ install(TARGETS mimalloc EXPORT mimalloc ARCHIVE DESTINATION ${CMAKE_INSTALL_LIBDIR} RUNTIME DESTINATION ${CMAKE_INSTALL_BINDIR} LIBRARY DESTINATION ${CMAKE_INSTALL_LIBDIR})
+ install(EXPORT mimalloc DESTINATION ${mi_install_cmakedir})
+ endif()

if(WIN32 AND NOT MINGW)
# On windows, the import library name for the dll would clash with the static mimalloc.lib library
# so we postfix the dll import library with `.dll.lib` (and also the .pdb debug file)
set_property(TARGET mimalloc PROPERTY ARCHIVE_OUTPUT_NAME "${mi_libname}.dll" )
- install(FILES "$<TARGET_FILE_DIR:mimalloc>/${mi_libname}.dll.lib" DESTINATION ${CMAKE_INSTALL_LIBDIR}/mimalloc-${mi_version})
+ if(NOT MI_SKIP_INSTALL)
+ install(FILES "$<TARGET_FILE_DIR:mimalloc>/${mi_libname}.dll.lib" DESTINATION ${CMAKE_INSTALL_LIBDIR}/mimalloc-${mi_version})
+ endif()
set_property(TARGET mimalloc PROPERTY PDB_NAME "${mi_libname}.dll")
# don't try to install the pdb since it may not be generated depending on the configuration
# install(FILES "$<TARGET_FILE_DIR:mimalloc>/${mi_libname}.dll.pdb" DESTINATION ${CMAKE_INSTALL_LIBDIR})
@@ -740,7 +744,9 @@ if(MI_BUILD_SHARED)
add_custom_command(TARGET mimalloc POST_BUILD
COMMAND "${CMAKE_COMMAND}" -E copy "${CMAKE_CURRENT_SOURCE_DIR}/bin/mimalloc-redirect${MIMALLOC_REDIRECT_SUFFIX}.dll" $<TARGET_FILE_DIR:mimalloc>
COMMENT "Copy mimalloc-redirect${MIMALLOC_REDIRECT_SUFFIX}.dll to output directory")
- install(FILES "$<TARGET_FILE_DIR:mimalloc>/mimalloc-redirect${MIMALLOC_REDIRECT_SUFFIX}.dll" DESTINATION ${CMAKE_INSTALL_BINDIR})
+ if(NOT MI_SKIP_INSTALL)
+ install(FILES "$<TARGET_FILE_DIR:mimalloc>/mimalloc-redirect${MIMALLOC_REDIRECT_SUFFIX}.dll" DESTINATION ${CMAKE_INSTALL_BINDIR})
+ endif()
endif()
endif()

@@ -757,17 +763,21 @@ if (MI_BUILD_STATIC)
$<BUILD_INTERFACE:${CMAKE_CURRENT_SOURCE_DIR}/include>
$<INSTALL_INTERFACE:${mi_install_incdir}>
)
- install(TARGETS mimalloc-static EXPORT mimalloc DESTINATION ${mi_install_objdir} LIBRARY)
- install(EXPORT mimalloc DESTINATION ${mi_install_cmakedir})
+ if(NOT MI_SKIP_INSTALL)
+ install(TARGETS mimalloc-static EXPORT mimalloc DESTINATION ${mi_install_objdir} LIBRARY)
+ install(EXPORT mimalloc DESTINATION ${mi_install_cmakedir})
+ endif()
endif()

-# install include files
-install(FILES include/mimalloc.h DESTINATION ${mi_install_incdir})
-install(FILES include/mimalloc-override.h DESTINATION ${mi_install_incdir})
-install(FILES include/mimalloc-new-delete.h DESTINATION ${mi_install_incdir})
-install(FILES include/mimalloc-stats.h DESTINATION ${mi_install_incdir})
-install(FILES cmake/mimalloc-config.cmake DESTINATION ${mi_install_cmakedir})
-install(FILES cmake/mimalloc-config-version.cmake DESTINATION ${mi_install_cmakedir})
+if(NOT MI_SKIP_INSTALL)
+ # install include files
Comment thread
cubic-dev-ai[bot] marked this conversation as resolved.
+ install(FILES include/mimalloc.h DESTINATION ${mi_install_incdir})
+ install(FILES include/mimalloc-override.h DESTINATION ${mi_install_incdir})
+ install(FILES include/mimalloc-new-delete.h DESTINATION ${mi_install_incdir})
+ install(FILES include/mimalloc-stats.h DESTINATION ${mi_install_incdir})
+ install(FILES cmake/mimalloc-config.cmake DESTINATION ${mi_install_cmakedir})
+ install(FILES cmake/mimalloc-config-version.cmake DESTINATION ${mi_install_cmakedir})
+endif()


# single object file for more predictable static overriding
@@ -798,9 +808,11 @@ if (MI_BUILD_OBJECT)

# the FILES expression can also be: $<TARGET_OBJECTS:mimalloc-obj>
# but that fails cmake versions less than 3.10 so we leave it as is for now
- install(FILES ${mimalloc-obj-static}
- DESTINATION ${mi_install_objdir}
- RENAME ${mi_libname}${CMAKE_C_OUTPUT_EXTENSION} )
+ if(NOT MI_SKIP_INSTALL)
+ install(FILES ${mimalloc-obj-static}
+ DESTINATION ${mi_install_objdir}
+ RENAME ${mi_libname}${CMAKE_C_OUTPUT_EXTENSION} )
+ endif()
endif()


@@ -819,8 +831,10 @@ join_paths(mi_pc_includedir "\${prefix}" "${CMAKE_INSTALL_INCLUDEDIR}")
join_paths(mi_pc_libdir "\${prefix}" "${CMAKE_INSTALL_LIBDIR}")

configure_file(mimalloc.pc.in mimalloc.pc @ONLY)
-install(FILES "${CMAKE_CURRENT_BINARY_DIR}/mimalloc.pc"
- DESTINATION "${CMAKE_INSTALL_LIBDIR}/pkgconfig/")
+if(NOT MI_SKIP_INSTALL)
+ install(FILES "${CMAKE_CURRENT_BINARY_DIR}/mimalloc.pc"

@cubic-dev-ai cubic-dev-ai Bot Aug 13, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: The final hunk of this patch declares @@ -819,8 +831,10 @@, but its content is only 5 old lines and 7 new lines, so strict git apply rejects the entire patch as corrupt (error: corrupt patch at EOF). The count should be -819,5 +831,7. If the vendorpull flow uses git apply, pulling mimalloc breaks; if it uses lenient GNU patch, it is silently tolerated. Fix the hunk header counts to make the patch a well-formed unified diff regardless of the apply tool.

Prompt for AI agents 
Check if this issue is valid — if so, understand the root cause and fix it. At patches/mimalloc/0001-Skip-vendored-install-rules-when-embedded.patch, line 91:

<comment>The final hunk of this patch declares `@@ -819,8 +831,10 @@`, but its content is only 5 old lines and 7 new lines, so strict `git apply` rejects the entire patch as corrupt (`error: corrupt patch` at EOF). The count should be `-819,5 +831,7`. If the vendorpull flow uses `git apply`, pulling mimalloc breaks; if it uses lenient GNU `patch`, it is silently tolerated. Fix the hunk header counts to make the patch a well-formed unified diff regardless of the apply tool.</comment>

<file context>
@@ -46,3 +66,28 @@ index 3189dec42..565852ecd 100644
+-install(FILES "${CMAKE_CURRENT_BINARY_DIR}/mimalloc.pc"
+-        DESTINATION "${CMAKE_INSTALL_LIBDIR}/pkgconfig/")
++if(NOT MI_SKIP_INSTALL)
++  install(FILES "${CMAKE_CURRENT_BINARY_DIR}/mimalloc.pc"
++          DESTINATION "${CMAKE_INSTALL_LIBDIR}/pkgconfig/")
++endif()
</file context>

+ DESTINATION "${CMAKE_INSTALL_LIBDIR}/pkgconfig/")
+endif()



10 changes: 10 additions & 0 deletions vendor/mimalloc.mask
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading
Loading