Skip to content

Batch libraries test Helix work items by compatibility metadata#129632

Draft
mmitche wants to merge 2 commits into
dotnet:mainfrom
mmitche:dev/helix-library-batching-structured
Draft

Batch libraries test Helix work items by compatibility metadata#129632
mmitche wants to merge 2 commits into
dotnet:mainfrom
mmitche:dev/helix-library-batching-structured

Conversation

@mmitche

@mmitche mmitche commented Jun 19, 2026

Copy link
Copy Markdown
Member

Summary

Reduces the number of libraries-test Helix work items by batching multiple test assemblies into a single work item, cutting per-work-item overhead (queueing, machine provisioning, payload/runtime download) while keeping total test execution time roughly flat.

Today each libraries test assembly is sent to Helix as its own work item, so the fixed per-work-item overhead is paid thousands of times. This change groups compatible desktop libraries work items together and runs them through a structured batch runner.

How it works

  • GroupWorkItems MSBuild task (src/tasks/HelixTestTasks/GroupWorkItems.cs): greedy bin-packs assemblies into batches by file size. Items above a size threshold get their own solo batch. Assemblies are first partitioned by a compatibility-key tuple so only co-runnable assemblies share a work item:
    RunnerType;PlatformFamily;RuntimeFlavor;TargetOS;TargetArchitecture;Scenario;TimeoutClass;PayloadShape
  • DesktopBatchRunner.sh / .cmd (eng/testing/): inside a single work item, iterates the batched suites, gives each its own upload root so results don't collide, runs each suite's existing RunTests script, and aggregates per-suite exit codes/durations.
  • sendtohelixhelp.proj: composes the desktop batch command from structured runner parts and emits batched work items. Non-batch-compatible (e.g. stress Scenario) items are emitted directly so they keep their original runner and timeout instead of flowing through ComputeBatchTimeout.
  • Unit tests added in HelixTestTasks.Tests/GroupWorkItemsTests.cs.

Batching can be disabled via BatchLibraryHelixWorkItems=false.

Validation

A manual pipeline run of this branch showed:

  • ~90% fewer libraries Helix work items vs a comparable main run, with total test-execution time roughly flat.
  • Work-item efficiency (test-execution time / wall-clock time) improved substantially.
  • No new failures observed inside the batched libraries work items.

Opening as draft to get full CI coverage and review on the approach.

Note

This pull request description was drafted with the assistance of GitHub Copilot (AI-generated).

mmitche and others added 2 commits June 5, 2026 14:19
Compose desktop batch commands from structured runner parts and group library Helix work items by compatibility metadata.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Emit non-batch-compatible desktop library work items directly so stress scenarios keep the original runner and timeout instead of flowing through ComputeBatchTimeout.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@dotnet-policy-service

Copy link
Copy Markdown
Contributor

Tagging subscribers to this area: @akoeplinger, @matouskozak, @simonrozsival
See info in area-owners.md if you want to be subscribed.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR changes the libraries Helix submission flow to batch multiple compatible test payload ZIPs into fewer Helix work items, and adds a desktop batch runner that executes each suite inside a single work item while keeping per-suite results separated.

Changes:

  • Extend the GroupWorkItems MSBuild task to partition candidates by a compatibility-metadata tuple (and optionally force “solo” batches).
  • Update sendtohelixhelp.proj to emit either classic per-assembly work items or new batched work items (with per-batch payload creation and timeouts).
  • Add DesktopBatchRunner scripts and a new HelixTestTasks.Tests project with unit tests for the grouping logic.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
src/tasks/HelixTestTasks/GroupWorkItems.cs Adds compatibility partitioning + “solo” batching logic and assigns globally-unique batch IDs across partitions.
src/tasks/HelixTestTasks.Tests/HelixTestTasks.Tests.csproj Introduces a new xUnit test project for HelixTestTasks.
src/tasks/HelixTestTasks.Tests/GroupWorkItemsTests.cs Adds unit tests validating partitioning and solo-batch behaviors.
src/libraries/sendtohelixhelp.proj Adds batching opt-in default + emits batched work items, payload staging, and batch-runner command composition.
src/libraries/pretest.proj Ensures HelixTestTasks is built for desktop library Helix batching scenarios (not just browser).
eng/testing/DesktopBatchRunner.sh Adds Unix batch runner that unpacks suite ZIPs, sets per-suite upload roots, runs each suite, and summarizes outcomes.
eng/testing/DesktopBatchRunner.cmd Adds Windows batch runner with similar behavior to the Unix runner.

Comment on lines +63 to +66
string[] compatibilityKeys = CompatibilityMetadataKeys
.Split(s_compatibilityMetadataKeySeparators, StringSplitOptions.RemoveEmptyEntries | StringSplitOptions.TrimEntries);
bool hasCompatibilityKeys = compatibilityKeys.Length > 0;
string batchCompatibleMetadataKey = BatchCompatibleMetadataKey.Trim();
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants