Skip to content

fix(ci): repair cross image builds#6617

Open
fmassot wants to merge 7 commits into
mainfrom
codex/fix-cross-image-publish
Open

fix(ci): repair cross image builds#6617
fmassot wants to merge 7 commits into
mainfrom
codex/fix-cross-image-publish

Conversation

@fmassot

@fmassot fmassot commented Jul 19, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • install GCC 10.5 in the GNU cross images to avoid the compiler bug detected by aws-lc
  • install target curl development headers required by librdkafka 2.12.1 even when OIDC support is disabled
  • install target Zlib and expose the ARM64 static library and headers inside the cross CMake sysroot
  • replace the failing zlib.net download with a checksummed archive from zlib's official GitHub repository
  • expose the release Lambda ZIP at its path inside the cross container
  • make cross-image publication stop immediately when an image build or push fails

Validation

  • built both updated GNU cross images locally
  • ran aws-lc's memcmp compiler probe for x86_64 and aarch64 through the production image runners
  • compiled curl-header probes with the x86_64 and aarch64 target compilers
  • ran CMake FindZLIB through the production ARM64 toolchain and resolved static target Zlib from /usr/aarch64-linux-gnu
  • configured and compiled librdkafka 2.12.1 to 100% with the patched ARM64 image
  • downloaded, checksummed, and statically built zlib in the aarch64-musl base image
  • published the first corrected cross-image set successfully
  • RC2 confirmed both Linux builds pass the libclang and GCC probes and embed the Lambda ZIP from /project/lambda-artifact
  • RC3 confirmed x86_64 builds librdkafka and isolated the remaining ARM64 failure to CMake's target Zlib lookup
  • cargo +nightly fmt --all -- --check
  • license-header and log-format checks
  • git diff --check and make -n cross-images

@fmassot
fmassot requested a review from a team as a code owner July 19, 2026 12:58

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: aac5f29e23

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +35 to +37
ENV CC_aarch64_unknown_linux_gnu=aarch64-linux-gnu-gcc-10 \
CXX_aarch64_unknown_linux_gnu=aarch64-linux-gnu-g++-10 \
CARGO_TARGET_AARCH64_UNKNOWN_LINUX_GNU_LINKER=aarch64-linux-gnu-gcc-10

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Point CMake at the GCC 10 cross compiler

For the ARM64 GNU release build, these variables only switch Cargo/cc-rs and the final linker; CMake-based crates still inherit cross-rs's toolchain file, which sets CMAKE_C_COMPILER from CROSS_TOOLCHAIN_PREFIX as ${prefix}gcc, and this image leaves that at the base default. Since .github/actions/cross-build-binary/action.yml builds release-feature-vendored-set and rdkafka-sys is enabled with cmake-build, librdkafka is still compiled with the old /usr/bin/aarch64-linux-gnu-gcc instead of aarch64-linux-gnu-gcc-10, so the GCC 9.4 compiler bug this image is trying to avoid remains for the ARM64 Linux artifact unless the CMake toolchain prefix/suffix or compiler is updated too.

Useful? React with 👍 / 👎.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 6cd39519cd

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +22 to +24
ENV CC_x86_64_unknown_linux_gnu=gcc-10 \
CXX_x86_64_unknown_linux_gnu=g++-10 \
CARGO_TARGET_X86_64_UNKNOWN_LINUX_GNU_LINKER=gcc-10

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Route x86 CMake builds through GCC 10

In the Linux release path checked in .github/actions/cross-build-binary/action.yml, release-feature-vendored-set builds aws-lc/rdkafka via CMake; the x86_64 cross-rs base image supplies CMAKE_TOOLCHAIN_FILE_x86_64_unknown_linux_gnu=/opt/toolchain.cmake, and that toolchain chooses ${CROSS_TOOLCHAIN_PREFIX}gcc (x86_64-linux-gnu-gcc) independently of these CC_*/Cargo linker overrides. For the x86_64 GNU artifact, CMake-built deps therefore still use the image's default GCC 9.4 and can hit the aws-lc memcmp compiler rejection unless the CMake compiler or cross toolchain prefix/suffix is switched to gcc-10 too.

Useful? React with 👍 / 👎.

Comment on lines 17 to 18
unzip && \
rm -rf /var/lib/apt/lists/*

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Preserve CMake dependency prefixes for x86 builds

When the x86_64 Linux release builds release-feature-vendored-set, libz-sys/static builds zlib and rdkafka-sys registers it as a CMake dependency, but cmake-rs passes that prefix via the CMAKE_PREFIX_PATH environment variable while the cross-rs toolchain consumes the CMake variable in CMAKE_FIND_ROOT_PATH. This file never materializes the env value for the x86 image (and no longer installs zlib1g-dev), so find_package(ZLIB) in librdkafka's CMake build is limited to the cross sysroot and misses the vendored zlib prefix, breaking the x86_64 GNU artifact unless the same prefix preservation is applied here.

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants