local test improvments by zhijianli88 · Pull Request #4037 · flatcar/scripts

zhijianli88 · 2026-05-19T08:01:23Z

Fix and improve the run_local_tests.sh and related scripts

The local test runner ./run_local_tests.sh invokes the mantle container with the scripts checkout mounted as /work. When using a git worktree for the scripts directory, git inside the container may see a .git/worktrees path that does not resolve correctly in that environment and fails with: ./run_local_tests.sh ... fatal: not a git repository: ../flatcar/scripts/.git/worktrees/... ... Commit 87e13eb ("ci-automation: Allow git to work on directory owned by other user") added "git config --global --add safe.directory /work" to the test docker invocation to fix an issue where get_git_channel() failed inside the mantle container, which caused AWS publishing to skip the Alpha channel. However, this git config was never necessary for the test path. In _test_run_impl(), both get_git_version() and get_git_channel() are called on the host side before the docker run, and their outputs are stored as files for the container to consume. Since commit 1d6f38a ("ci-automation: Reduce boilerplate in vendor tests"), all vendor testing scripts read git version and channel from these precomputed files via vendor_test.sh rather than running git commands. No vendor script (qemu_uefi, qemu, qemu_update, aws, azure, gce, etc.) executes git inside the mantle container. The git config in the test docker invocation was therefore redundant from the start. Remove it to avoid failures in worktree-based local test setups, while CI behavior remains unchanged since the non-worktree case is unaffected. Signed-off-by: Li Zhijian <lizhijian@fujitsu.com>

run_local_tests.sh wraps ci-automation/test.sh for local qemu_uefi and qemu_update testing, but it did not document its arguments or defaults. Add a usage header and a --help option that describe the ARCH, PARALLEL and TEST parameters so that developers can discover how to invoke the script and what behavior to expect without having to read through the implementation. Signed-off-by: Li Zhijian <lizhijian@fujitsu.com>

zhijianli88 · 2026-07-09T03:55:19Z

Rebase against the latest main branch.
Kindly ping. if you want to split to small patches, feel free to let me know.

Copilot

Pull request overview

This PR improves the local/CI testing scripts around Mantle/kola to make local runs more ergonomic (help/list options) and to adjust test execution behavior (cleanup + result validation).

Changes:

Add --help and --list-tests support to run_local_tests.sh.
Clean up prior test artifacts and add an “empty results” guard in ci-automation/test.sh.
Remove a zstd presence warning from ci-automation/ci-config.env.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 4 comments.

File	Description
run_local_tests.sh	Adds a usage helper and a `--list-tests` mode for local runs.
ci-automation/test.sh	Adjusts test execution container runs and adds local result cleanup/validation.
ci-automation/ci-config.env	Removes a startup warning about missing `zstd`.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

zhijianli88 · 2026-07-09T05:11:58Z

+    --list-tests)
+      # List all tests that would be run by default and exit.
+      mantle_container="$(cat "sdk_container/.repo/manifests/mantle-container")"
+      docker run --rm "${mantle_container}" kola list --platform qemu
+      exit 0
+      ;;


I prefer to update the comment instead of filtering out the output.

When working with the mantle image it is useful to inspect the set of available kola tests before running any local qemu_uefi or qemu_update runs, but run_local_tests.sh did not provide a way to query them. Add a --list-tests helper option that runs kola list --platform qemu in the configured mantle container and prints the result without starting any tests, allowing developers to quickly discover all qemu/qemu_update tests. Signed-off-by: Li Zhijian <lizhijian@fujitsu.com>

Copilot

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

Stop the retry flow when the vendor test runner does not produce a TAP file. This avoids handing an empty result to test_update_reruns.sh and makes the underlying failure visible earlier in the log. After this patch, when it failed to download the qemu image: $ ./run_local_tests.sh amd64 2 docker.base ================================= Using Mantle docker image 'ghcr.io/flatcar/mantle:git-ca80a2eaee4cc195ae6e17f9202c1d72e729d460' ================================= Running qemu_uefi tests git-ca80a2eaee4cc195ae6e17f9202c1d72e729d460: Pulling from flatcar/mantle Digest: sha256:cd37c1ba21b8051eb2632e171d4f1f9bc7e91708883575343265d3d58940d5df Status: Image is up to date for ghcr.io/flatcar/mantle:git-ca80a2eaee4cc195ae6e17f9202c1d72e729d460 zstd could not be found. unpacking container image may fail. ++++ Running qemu_uefi.sh inside __TESTS__/qemu_uefi ++++ ++++ qemu_uefi.sh: downloading /work/__build__/images/images/amd64-usr/latest/flatcar_production_image.bin for 4593.2.0+4-gbc1fd0010d (amd64) ++++ curl: (22) The requested URL returned error: 404 image_escaped: qemu_uefi.sh return 22, __TESTS__/qemu_uefi/results-run-1.tap ERROR: something went wrong, result is empty: __TESTS__/qemu_uefi/results-run-1.tap ########### All re-runs exhausted (5). Giving up. ########### Signed-off-by: Li Zhijian <lizhijian@fujitsu.com>

This reverts the --pull always addition from commit 60a45ef ("ci-automation/test: Always pull mantle image when running tests"). MANTLE_REF points to an immutable tag (ghcr.io/flatcar/mantle:git-<sha>) rather than a mutable tag like :latest, so --pull always only adds an unnecessary registry lookup without any functional effect. Removing it avoids pointless network round-trips and ensures reruns use the same local image for consistent results. Signed-off-by: Li Zhijian <lizhijian@fujitsu.com>

When running `./run_local_tests.sh`, a warning is printed if `zstd` is not found in the host environment: $ ./run_local_tests.sh arm64 2 cl.cloudinit.script ... zstd could not be found. unpacking container image may fail. ... This check is performed because `ci-config.env` is sourced by the script. However, the `run_local_tests.sh` workflow itself does not depend on `zstd`. The `zstd` command is required for building Flatcar[1], but that process runs inside the Flatcar SDK container, which is guaranteed to have `zstd` installed. The check on the host running the local tests is therefore redundant and produces confusing noise for developers. This patch removes the check to eliminate the spurious warning. [1] flatcar/mantle#853 (comment) Signed-off-by: Li Zhijian <lizhijian@fujitsu.com>

Copilot

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

zhijianli88 requested a review from a team as a code owner May 19, 2026 08:01

zhijianli88 changed the title ~~local test improments~~ local test improvments May 19, 2026

zhijianli88 force-pushed the run-local-test branch from 505fa4f to f1708fd Compare May 22, 2026 01:08

github-actions Bot mentioned this pull request May 22, 2026

Monthly contributions report 2026-04-22 - 2026-05-21 flatcar/Flatcar#2136

Closed

zhijianli88 force-pushed the run-local-test branch from f1708fd to 534d92c Compare May 22, 2026 01:27

zhijianli88 force-pushed the run-local-test branch from 534d92c to 2ccc978 Compare June 5, 2026 03:27

zhijianli88 added 2 commits July 9, 2026 12:53

Copilot AI review requested due to automatic review settings July 9, 2026 03:53

zhijianli88 force-pushed the run-local-test branch from 2ccc978 to 95fb667 Compare July 9, 2026 03:53

Copilot started reviewing on behalf of zhijianli88 July 9, 2026 03:53 View session

Copilot AI reviewed Jul 9, 2026

View reviewed changes

Copilot AI review requested due to automatic review settings July 9, 2026 05:20

zhijianli88 force-pushed the run-local-test branch from 95fb667 to 58f65a0 Compare July 9, 2026 05:20

Copilot started reviewing on behalf of zhijianli88 July 9, 2026 05:20 View session

Copilot AI reviewed Jul 9, 2026

View reviewed changes

Comment thread ci-automation/test.sh Outdated

zhijianli88 added 3 commits July 9, 2026 14:31

Copilot AI review requested due to automatic review settings July 9, 2026 05:31

zhijianli88 force-pushed the run-local-test branch from 58f65a0 to 8064def Compare July 9, 2026 05:31

Copilot started reviewing on behalf of zhijianli88 July 9, 2026 05:32 View session

Copilot AI reviewed Jul 9, 2026

View reviewed changes

Comment thread ci-automation/test.sh

Comment thread ci-automation/test.sh

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

local test improvments#4037

local test improvments#4037
zhijianli88 wants to merge 6 commits into
flatcar:mainfrom
zhijianli88:run-local-test

zhijianli88 commented May 19, 2026

Uh oh!

zhijianli88 commented Jul 9, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

zhijianli88 Jul 9, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

zhijianli88 commented May 19, 2026

Uh oh!

zhijianli88 commented Jul 9, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

zhijianli88 Jul 9, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants