Extend v1 EROFS to OCI and booting#297
Draft
cgwalters wants to merge 10 commits into
Draft
Conversation
Add set_write_concurrency() to Repository for overriding the default parallelism. Add read_filesystem_with_semaphore() as a public entry point that accepts an explicit Semaphore, and refactor the internal read_filesystem_impl() to centralize semaphore selection. Prep for wiring up --threads in mkcomposefs. Assisted-by: OpenCode (Claude Sonnet 4.6) Signed-off-by: Colin Walters <walters@verbum.org>
The patch recipe referenced crates/cfsctl which was never a valid path; the crate has always been named composefs-ctl. Also relax the clean-tree check to allow untracked files (only committed changes need to match the pinned revision). Assisted-by: OpenCode (Claude Sonnet 4.6) Signed-off-by: Colin Walters <walters@verbum.org>
import_oci_layout() was opening the layout directory before calling ensure_writable(), so pulling into a read-only repo produced a misleading ENOENT error instead of a clear 'not writable' message. Move the write check to the top of the function, matching the existing skopeo pull path. Fixes privileged_pull_readonly_repo integration test. Signed-off-by: Colin Walters <walters@verbum.org>
For compatibility with the C composefs, we need to support writing directly to a flat XX/DIGEST path, without a leading `objects/`. Assisted-by: OpenCode (Claude Sonnet 4.6) Signed-off-by: Colin Walters <walters@verbum.org>
The script hardcoded /usr/share/edk2/ovmf/OVMF_CODE.fd which is only present on Fedora. Probe a list of common paths (Ubuntu's ovmf package uses /usr/share/ovmf/OVMF.fd, Arch uses /usr/share/edk2/x64/OVMF.4m.fd) so the script works across distros without manual adjustment. Also add -machine q35, required on newer QEMU builds (e.g. RHEL10/CentOS Stream 10) where the default pc-i440fx machine type doesn't pair well with OVMF for EFI boot. Assisted-by: OpenCode (claude-sonnet-4-6@default) Signed-off-by: Colin Walters <walters@verbum.org>
The combined OVMF.qemuvars.fd with -bios hangs indefinitely on RHEL10/ CentOS Stream 10 QEMU (qemu-kvm 9.x). Use the split OVMF_CODE.fd + OVMF_VARS.fd files with -drive if=pflash and -machine q35 instead, which works correctly. Fall back to -bios with the combined image on distros that only ship the combined file (Ubuntu, Arch). Updated both testthing.py (which drives the example integration tests) and the fix-verity helper script (which runs the in-VM verity fixup pass). A temporary copy of OVMF_VARS.fd is made so UEFI can write to it without modifying the original system file. Assisted-by: OpenCode (claude-sonnet-4-6@default) Signed-off-by: Colin Walters <walters@verbum.org>
composefs-setup-root validates that the repo's meta.json has fs-verity enabled before trusting the repo. The dracut hook was only enabling verity on the content objects, so setup-root would see the repo as insecure and refuse to proceed. Switch the working directory to /sysroot/composefs (instead of the objects subdirectory) so we can enable verity on meta.json in addition to all the content objects. Also quote the loop variable and use the full relative path for clarity. Assisted-by: OpenCode (claude-sonnet-4-6@default) Signed-off-by: Colin Walters <walters@verbum.org>
The 30s default is tight on slower hardware (e.g. CentOS Stream 10 with OVMF pflash init overhead) — the VM boots successfully but just barely misses the window. 60s gives enough headroom while still being short enough to catch genuinely broken VMs. CI on Ubuntu with KVM acceleration boots well under 30s so the extra budget costs nothing. Assisted-by: OpenCode (claude-sonnet-4-6@default) Signed-off-by: Colin Walters <walters@verbum.org>
…info CLI Add support for generating V1 EROFS images compatible with the C composefs tools (mkcomposefs/composefs-info 1.0.8+). V1 uses compact inodes, BFS layout, and a simpler on-disk structure. Adds --erofs-version flag to cfsctl, new mkcomposefs and composefs-info compatibility subcommands, and RepositoryConfig for cleaner repo initialization. Note: this commit does not compile with --features oci (the default) until the following commit migrates OCI crate callers. Assisted-by: OpenCode (Claude Sonnet 4.5) Signed-off-by: Colin Walters <walters@verbum.org>
…ne support Migrate OCI crate callers to the new RepositoryConfig API and add dual-format (V1+V2) EROFS image generation during OCI pull. Add the composefs.digest= karg for V1 EROFS images and update boot integration to generate the appropriate karg based on repository format version. Assisted-by: OpenCode (Claude Sonnet 4.5) Signed-off-by: Colin Walters <walters@verbum.org>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Builds on #225
In a nutshell: I want to do a cutover to generating V1 EROFS (i.e. bit-for-bit compatible with the original https://github.com/composefs/composefs ). The rationale here is basically we want to be able to replace that project entirely in a compatible way and we need to support RHEL9 and our current EROFS ("v2") isn't compatible because we were implicitly relying on https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=9ed50b8231e37b1ae863f5dec8153b98d9f389b4
Handling EROFS versions
In an OCI image splitstream, we can now generate and reference either or both versions. For new repositories, we default to v1. Apart from UKIs, we don't care about the binary checksum of the EROFS, so it's totally fine to just change what we do there. One can use composefs to fetch OCI images and mount them as before, and the fact the EROFS is different is largely invisible.
Sealed UKIs
However, we do care for sealed UKIs that have a checksum of the EROFS. A
composefs=karg signals that V2 EROFS is used. A newcomposefs.digest=karg signals that V1 EROFS is used.Conveniently, I'm also using this transition to get rid of the annoying kernel warning because presence of a
.signals it's for userspace, as is the case here.