Skip to content

coverage: repair the Windows coverage wheels - #2508

Open
rluo8 wants to merge 2 commits into
NVIDIA:mainfrom
rluo8:ci/repair-windows-coverage-wheels
Open

coverage: repair the Windows coverage wheels#2508
rluo8 wants to merge 2 commits into
NVIDIA:mainfrom
rluo8:ci/repair-windows-coverage-wheels

Conversation

@rluo8

@rluo8 rluo8 commented Aug 5, 2026

Copy link
Copy Markdown
Contributor

Description

The Windows coverage of CI: Coverage has not collected cuda.core data: the step dies in seconds with exit 139 and no pytest output. Two separate causes, both in this workflow's wheel build.

1. The crash

Cause. A plain pip wheel does not read [tool.cibuildwheel], so the delvewheel repair every other Windows build runs never happened here, and the .pyd files load whatever MSVCP140.dll the test machine has — on this runner 14.00.24215.1, from 2015. _resource_handles.pyd is built by MSVC 14.44 and imports _Mtx_lock/_Mtx_unlock but never _Mtx_init_in_situ, because std::mutex has had a constexpr constructor since VS 2022 17.10. The old runtime still dereferences that uninitialised handle on the first lock, which _stream.pyx takes during import cuda.core:

EXCEPTION_ACCESS_VIOLATION reading 0x0 at msvcp140.dll!mtx_do_lock+0x80
  <- _resource_handles.pyd  (get_legacy_stream -> stream_registry.lookup)
  <- _stream.pyd            (_stream.pyx: LEGACY_DEFAULT_STREAM = ...)

It is the only extension module in either package that locks a mutex, which is why cuda.bindings and cuda.pathfinder have always passed on the same machine and why the PyPI wheels are unaffected.

The fix is to repair these wheels too, with the same flags cibuildwheel uses: 14.44 is vendored into cuda_core.libs and the import tables are rewritten to match, so the process no longer depends on what the test machine carries.

2. The build failure

Cause. PIP_PRE, needed so pip sees the .devN cuda-bindings wheel built one step earlier, also admits PyPI's pre-releases: 13.4.0b1 (published 2026-07-29) outranks it, and its cydriver.pxd comes from CTK 13.4 headers where CUmemLocation has a localized field, which the 13.3.0 mini-CTK used here does not. Cython generates a struct converter over every field, so the build fails with error C2039 — every night since, most recently run 30964133960.

The fix is to constrain cuda-bindings to the version just built, which keeps what PIP_PRE is here for — compiling cuda.core against the same bindings this job then installs and measures — and drops what it lets in.

Windows coverage has not collected a test since 2026-03-17.  The job builds
its wheels with a plain `pip wheel`, which never reads [tool.cibuildwheel],
so the delvewheel repair every other Windows build performs never ran here.
Those wheels import a bare "MSVCP140.dll" and resolve it against whatever the
test machine has in System32, which on the coverage runner is 14.00.24215.1,
built in 2015.

_resource_handles.pyd is compiled by MSVC 14.44 and imports exactly _Mtx_lock
and _Mtx_unlock from that DLL -- never _Mtx_init_in_situ, because std::mutex
has had a constexpr constructor since VS 2022 17.10.  The 2015 runtime still
expects that initialisation and dereferences a null handle on the first lock,
which _stream.pyx takes while cuda.core is still importing.  It is the only
extension module in either package that locks a mutex, which is why
cuda.bindings and cuda.pathfinder have always passed on the same machine.

Repairing the wheels vendors msvcp140 14.44 into cuda_core.libs and rewrites
the import tables to match, so the process no longer depends on what the test
machine carries.  Verified on the coverage runner: 18 failed, 2929 passed,
918 skipped in 346s, against three to seven seconds of dying beforehand, and
the first Windows coverage data since March.

The same commit pins cuda-bindings to the wheel built one step earlier.
PIP_PRE is set so pip will consider that wheel at all -- it carries a .devN
version -- but it also admits PyPI's pre-releases, and cuda-bindings 13.4.0b1,
published 2026-07-29, outranks the local build.  Its cydriver.pxd comes from
CTK 13.4 headers where CUmemLocation has a `localized` field, while cuda.core
compiles against the 13.3.0 mini-CTK where it does not, so the build has been
failing on `error C2039` ever since.

Signed-off-by: Rui Luo <ruluo@nvidia.com>
@copy-pr-bot

copy-pr-bot Bot commented Aug 5, 2026

Copy link
Copy Markdown
Contributor

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

@rluo8
rluo8 marked this pull request as ready for review August 5, 2026 06:09
@github-actions github-actions Bot added the CI/CD CI/CD infrastructure label Aug 5, 2026
@rluo8
rluo8 requested a review from mdboom August 5, 2026 06:09
@rluo8 rluo8 self-assigned this Aug 5, 2026
@github-actions

github-actions Bot commented Aug 5, 2026

Copy link
Copy Markdown

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CI/CD CI/CD infrastructure

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant