Skip to content

ci: workflows/build-yocto: move the sdk build to the second stage #2254

Open
quaresmajose wants to merge 4 commits into
qualcomm-linux:masterfrom
quaresmajose:sdk
Open

ci: workflows/build-yocto: move the sdk build to the second stage #2254
quaresmajose wants to merge 4 commits into
qualcomm-linux:masterfrom
quaresmajose:sdk

Conversation

@quaresmajose
Copy link
Copy Markdown
Contributor

@quaresmajose quaresmajose commented May 20, 2026

The SDK is taking too long, reaching 3 times the build time of the target image.

The SDK main pain point It's really in the last image:do_populate_sdk.
It currently takes approximately 30 minutes minutes to complete.

Perhaps a solution for the SDK would be to move it from the first (warmup) stage
to the second stage of the build. Thus, the first stage would finish earlier and
the second would begin earlier, with the SDK build starting directly to the image
deployment.
However, the gains weren't significant because the 30 minutes will always be
necessary; we were just doing part of that in parallel with the other builds
of the second stage.

buildstats.log example for the best case (when everything comes from sstate-cache):

0:00:01    gcc-cross-canadian-aarch64:do_package_write_rpm_setscene
0:00:02    nativesdk-clang:do_package_write_rpm_setscene
0:00:02    nativesdk-llvm:do_package_write_rpm_setscene
0:00:02    nativesdk-spirv-tools:do_package_write_rpm_setscene
0:00:04    nativesdk-glibc-locale:do_package_write_rpm_setscene
0:00:05    nativesdk-glibc-locale:do_packagedata_setscene
0:00:08    nativesdk-qemu:do_package_write_rpm_setscene
0:34:07    qcom-multimedia-image:do_populate_sdk
0:35:56    qcom-multimedia-proprietary-image:do_populate_sdk

@quaresmajose quaresmajose force-pushed the sdk branch 3 times, most recently from 079c47d to ddfb16d Compare May 20, 2026 21:37
@quaresmajose quaresmajose changed the title ci: move the sdk build to the second stage ci: workflows/build-yocto: move the sdk build to the second stage May 21, 2026
@quaresmajose quaresmajose force-pushed the sdk branch 2 times, most recently from 88bc8d9 to 773f826 Compare May 21, 2026 14:46
@github-actions
Copy link
Copy Markdown

Test run workflow

Test jobs for commit 773f826

qcom-distro
Pass: 8 | Fail: 1 | Total: 9
nodistro
Pass: 9 | Fail: 0 | Total: 9
qcom-distro_linux-qcom-6.18
Pass: 228 | Fail: 2 | Total: 245

@test-reporting-app
Copy link
Copy Markdown

test-reporting-app Bot commented May 21, 2026

Test Results

   61 files  ±0    303 suites  ±0   2h 47m 28s ⏱️ - 8m 24s
  105 tests ±0    100 ✅ +2   0 💤 ±0  5 ❌  - 2 
2 860 runs  ±0  2 828 ✅ +5  27 💤  - 3  5 ❌  - 2 

For more details on these failures, see this check.

Results for commit 4181053. ± Comparison against base commit 0c252ae.

♻️ This comment has been updated with latest results.

@quaresmajose quaresmajose force-pushed the sdk branch 2 times, most recently from e8bc9e7 to 128ac26 Compare May 22, 2026 09:40
@github-actions
Copy link
Copy Markdown

Test run workflow

Test jobs for commit 128ac26

qcom-distro
Pass: 253 | Fail: 5 | Total: 285
nodistro
Pass: 9 | Fail: 0 | Total: 9
qcom-distro_linux-qcom-6.18
Pass: 7 | Fail: 0 | Total: 7

Comment thread .github/workflows/compile.yml Outdated
@github-actions
Copy link
Copy Markdown

Test run workflow

Test jobs for commit 4f65ba5

qcom-distro
Pass: 271 | Fail: 1 | Total: 291
nodistro
Pass: 9 | Fail: 0 | Total: 9
qcom-distro_linux-qcom-6.18
Pass: 216 | Fail: 4 | Total: 249

@quaresmajose quaresmajose force-pushed the sdk branch 2 times, most recently from 1c6ae5c to b9b5642 Compare May 22, 2026 19:06
@github-actions
Copy link
Copy Markdown

Test run workflow

Test jobs for commit b9b5642

qcom-distro
Pass: 269 | Fail: 4 | Total: 296
nodistro
Pass: 9 | Fail: 0 | Total: 9
qcom-distro_linux-qcom-6.18
Pass: 222 | Fail: 1 | Total: 249

@quaresmajose quaresmajose self-assigned this May 26, 2026
@github-actions
Copy link
Copy Markdown

Test run workflow

Test jobs for commit b9b5642

qcom-distro_linux-qcom-6.18
Pass: 222 | Fail: 1 | Total: 249
qcom-distro
Pass: 270 | Fail: 3 | Total: 296
nodistro
Pass: 9 | Fail: 0 | Total: 9

There are scenarios where the deploy_dir may not exist.
This happens, for example, when we skip target builds.

Signed-off-by: Jose Quaresma <jose.quaresma@oss.qualcomm.com>
The SDK is taking too long, reaching 3 times the build time of the target image.

The SDK main pain point It's really in the last image:do_populate_sdk.
It currently takes approximately 30 minutes minutes to complete.

Perhaps a solution for the SDK would be to move it from the first (warmup) stage
to the second stage of the build. Thus, the first stage would finish earlier and
the second would begin earlier, with the SDK build starting directly to the image
deployment.
However, the gains weren't significant because the 30 minutes will always be
necessary; we were just doing part of that in parallel with the other builds
of the second stage.

Signed-off-by: Jose Quaresma <jose.quaresma@oss.qualcomm.com>
This will allow us to define the name outside the block,
so it can be customized depending on how the block will be used.

This also allows you to shorten the name displayed in the UI when
the block is skipped and not executed:

-| ${{matrix.machine}}/${{matrix.distro.name}}${{matrix.kernel.dirname}}
+| inputs.name

Signed-off-by: Jose Quaresma <jose.quaresma@oss.qualcomm.com>
Creating a specific step for buildstats facilitates the analysis
of data and allows direct access to this data in the UI.

Signed-off-by: Jose Quaresma <jose.quaresma@oss.qualcomm.com>
@lumag
Copy link
Copy Markdown
Contributor

lumag commented May 28, 2026

@quaresmajose @mwasilew did we test it at next?

@mwasilew
Copy link
Copy Markdown
Contributor

I pushed it to next just now: https://github.com/qualcomm-linux/meta-qcom/actions/runs/26576552463

@github-actions
Copy link
Copy Markdown

Test run workflow

Test jobs for commit 4181053

qcom-distro_linux-qcom-6.18
Pass: 221 | Fail: 1 | Total: 249
qcom-distro
Pass: 8 | Fail: 1 | Total: 9
nodistro
Pass: 9 | Fail: 0 | Total: 9

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants