Deadline-aware cancellation (prototype) by nohwnd · Pull Request #10018 · microsoft/testfx

nohwnd · 2026-07-16T15:29:17Z

CI hard-cancels a job at a fixed wall-clock time. When it fires the runner kills the test process, and we lose the TRX/HTML/AzDO reports and get no dump for a hanging test. This is a prototype that tells MTP when that deadline is, so it can react a little early while it still owns its shutdown.

What it does

The deadline arrives as an environment variable, an absolute instant:

TESTINGPLATFORM_DEADLINE: ISO 8601, parsed to UTC.
TESTINGPLATFORM_DEADLINE_STOP_MARGIN: lead time for the graceful stop. Default 60s.
TESTINGPLATFORM_DEADLINE_DUMP_MARGIN: lead time for the hang dump. Default 30s.

Two reactions, armed off the same instant:

At deadline - stopMargin an in-process extension asks the framework to gracefully stop scheduling new tests (IGracefulStopTestExecutionCapability). In-flight tests finish, the session ends normally, and every reporter gets to finalize.
At deadline - dumpMargin the out-of-process HangDump controller dumps the process tree and kills the host. This is the fallback for a wedged host that never reaches the graceful stop. It reuses the controller that HangDump already runs, so it costs nothing extra when --hangdump is on.

stopMargin > dumpMargin on purpose: try the clean stop first, dump only if that did not happen in time.

It is opt-in. No deadline set, both timers stay unarmed, no behavior change. The graceful stop also degrades to a no-op if the framework does not expose the capability.

Where the deadline comes from

Out of scope here, that is the "solve timing later" part. MTP has no hardcoded timeout, it only reacts to whatever instant the environment gives it. The CI side is a few lines of YAML that compute job start + timeout and export it. Rough sketches:

GitHub Actions (job has timeout-minutes: 60):

- name: Compute deadline
  shell: pwsh
  run: echo "TESTINGPLATFORM_DEADLINE=$((Get-Date).ToUniversalTime().AddMinutes(55).ToString('o'))" >> $env:GITHUB_ENV

Azure DevOps (job has timeoutInMinutes: 60):

- pwsh: echo "##vso[task.setvariable variable=TESTINGPLATFORM_DEADLINE]$((Get-Date).ToUniversalTime().AddMinutes(55).ToString('o'))"
  displayName: Compute deadline

Both approximate "now" as the job start, which is close enough. The nicer version is Arcade computing the real deadline once and exporting it for everyone, so nobody has to copy YAML around. Left for a follow-up.

Verified

build.cmd -pack passes 0/0. New acceptance tests:

AbortAtDeadlineTests (5): past deadline stops immediately, future deadline stops when the timer fires, the stop margin is subtracted from the deadline, no deadline stays silent, missing capability is a no-op.
HangDumpTests.HangDump_AbsoluteDeadline_CreateDump (3 tfms): 30 minute inactivity timeout so only the deadline path can fire, and it produces a dump. Full HangDumpTests still 30/30.

This is a prototype, so LMK what you think about the shape before I polish it. Open questions:

Names for the env vars and the margins.
Whether the dump-at-deadline message should say "deadline" instead of reusing the inactivity-timeout text.

CI systems (AzDO, GitHub Actions) hard-cancel a job at a fixed wall-clock time. When that happens the runner kills the test process, so we lose the TRX/HTML/AzDO reports and get no dump for a hanging test. This teaches MTP about that deadline through an environment variable and lets it react a bit early, while it still controls its own shutdown: - TESTINGPLATFORM_DEADLINE is an absolute instant (ISO 8601, parsed to UTC). - At deadline minus stop margin (default 60s) an in-process extension asks the framework to gracefully stop scheduling new tests, so the session ends normally and every reporter finalizes. - At deadline minus dump margin (default 30s) the out-of-process HangDump controller takes a dump of the process tree and kills the host, for the case where the host is wedged and never reaches the graceful stop. The deadline comes from the environment, so there is no hardcoded timeout in MTP. The margins are MTP side policy and are env overridable. Wiring the real deadline from the CI timeout is a small bit of YAML, left for a follow-up. It is opt-in: with no deadline set both timers stay unarmed and there is no behavior change. The graceful stop also degrades to a no-op when the framework does not expose IGracefulStopTestExecutionCapability. Verified: build.cmd -pack passes 0/0, and the new acceptance tests pass (AbortAtDeadlineTests 5/5, HangDumpTests 30/30 including the deadline dump). Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot

Pull request overview

Adds a prototype “deadline-aware cancellation” mechanism to Microsoft.Testing.Platform (MTP) so CI can provide an absolute wall-clock deadline and MTP can proactively (a) request a graceful stop before the runner hard-kills the process, and (b) trigger HangDump as a fallback before the deadline.

Changes:

Introduces DeadlineHelper + new env vars (TESTINGPLATFORM_DEADLINE, *_STOP_MARGIN, *_DUMP_MARGIN) for parsing an absolute UTC deadline and margins.
Registers a new in-proc AbortAtDeadlineExtension that schedules a timer to request IGracefulStopTestExecutionCapability.
Extends HangDump to arm an additional one-shot timer for the absolute deadline and adds acceptance coverage for both graceful stop and deadline-driven dump.

Reviewed changes

Copilot reviewed 14 out of 14 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/HangDumpTests.cs	Adds acceptance coverage ensuring HangDump can be triggered via absolute deadline env var.
test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/AbortAtDeadlineTests.cs	New acceptance suite validating the graceful-stop behavior around deadlines/margins and missing capability.
src/Platform/Microsoft.Testing.Platform/InternalAPI/InternalAPI.Unshipped.txt	Tracks newly added internal APIs/constants for PublicAPIAnalyzers.
src/Platform/Microsoft.Testing.Platform/Hosts/TestHostBuilder.Framework.cs	Registers `AbortAtDeadlineExtension` into the message bus when enabled.
src/Platform/Microsoft.Testing.Platform/Helpers/EnvironmentVariableConstants.cs	Adds constants for the new deadline-related environment variables.
src/Platform/Microsoft.Testing.Platform/Helpers/DeadlineHelper.cs	New helper to read/parse deadline + margins from environment.
src/Platform/Microsoft.Testing.Platform/Extensions/AbortAtDeadlineExtension.cs	New extension that arms a timer to request graceful stop before the deadline.
src/Platform/Microsoft.Testing.Extensions.TrxReport/InternalAPI/InternalAPI.Unshipped.txt	Tracks newly added env-var constants due to shared-source inclusion.
src/Platform/Microsoft.Testing.Extensions.Retry/InternalAPI/InternalAPI.Unshipped.txt	Tracks newly added env-var constants due to shared-source inclusion.
src/Platform/Microsoft.Testing.Extensions.MSBuild/InternalAPI.Unshipped.txt	Tracks newly added env-var constants due to shared-source inclusion.
src/Platform/Microsoft.Testing.Extensions.HotReload/InternalAPI/InternalAPI.Unshipped.txt	Tracks newly added env-var constants due to shared-source inclusion.
src/Platform/Microsoft.Testing.Extensions.HangDump/Microsoft.Testing.Extensions.HangDump.csproj	Links `DeadlineHelper.cs` into HangDump extension for shared deadline parsing.
src/Platform/Microsoft.Testing.Extensions.HangDump/InternalAPI/InternalAPI.Unshipped.txt	Tracks `DeadlineHelper` + env-var constants for this assembly.
src/Platform/Microsoft.Testing.Extensions.HangDump/HangDumpProcessLifetimeHandler.cs	Arms a new deadline-driven dump timer and prevents double dump via `_dumpTaken`.

nohwnd · 2026-07-16T16:04:57Z

@@ -191,6 +203,23 @@
            null,
            _activityTimerValue!.Value,
            TimeSpan.FromMilliseconds(-1));


Fixed in 1ddfd5a. Both timer callbacks now go through a TriggerDumpOnce trampoline that owns the Interlocked one-shot gate and assigns _activityIndicatorTask only for the winning caller. The loser returns without touching it, so it can no longer overwrite the real dump task, and Dispose keeps awaiting the right one.

nohwnd · 2026-07-16T16:04:48Z

+            _deadlineTimer = new Timer(
+                _ => _activityIndicatorTask = TakeDumpOfTreeAsync(cancellationToken),
+                null,
+                dueTime,
+                TimeSpan.FromMilliseconds(-1));


Fixed in 1ddfd5a, same TriggerDumpOnce trampoline. The deadline timer no longer assigns _activityIndicatorTask directly; only the winner of the one-shot gate does.

nohwnd · 2026-07-16T16:04:52Z

+        // The inactivity timer and the deadline timer can both fire; only dump once.
+        if (Interlocked.Exchange(ref _dumpTaken, 1) != 0)
+        {
+            return;
+        }


Fixed in 1ddfd5a. The deadline path now carries its own reason and its own output message (new HangDumpDeadlineReached resource, xlf regenerated) instead of reusing the inactivity "timeout expired" text.

nohwnd · 2026-07-16T16:04:47Z

+// Copyright (c) Microsoft Corporation. All rights reserved.
+// Licensed under the MIT license. See LICENSE file in the project root for full license information.


Fixed in 1ddfd5a. Added the UTF-8 BOM.

nohwnd · 2026-07-16T16:04:53Z

+// Copyright (c) Microsoft Corporation. All rights reserved.
+// Licensed under the MIT license. See LICENSE file in the project root for full license information.


Fixed in 1ddfd5a. Added the UTF-8 BOM.

nohwnd · 2026-07-16T16:04:50Z

+// Copyright (c) Microsoft Corporation. All rights reserved.
+// Licensed under the MIT license. See LICENSE file in the project root for full license information.


Fixed in 1ddfd5a. Added the UTF-8 BOM.

…-cancellation # Conflicts: # src/Platform/Microsoft.Testing.Extensions.TrxReport/InternalAPI/InternalAPI.Unshipped.txt

Fixes from the review of #10018: - HangDump: the deadline timer and the inactivity timer both wrote _activityIndicatorTask, so the losing one could overwrite the winner's real dump task with a completed no-op, and Dispose would stop waiting for the dump mid-flight. Move the one-shot guard into a TriggerDumpOnce trampoline so only the winning timer assigns _activityIndicatorTask. - HangDump: the deadline path logged "Hang dump timeout expired", which is misleading because no inactivity timeout expired. Give the deadline case its own reason and its own output message (new HangDumpDeadlineReached resource + regenerated xlf). - AbortAtDeadlineExtension: compute a local non-null stopAt instead of dereferencing _stopAt.Value, so there is no nullable deref. - Add the UTF-8 BOM to the three new source files to satisfy the charset=utf-8-bom editorconfig rule. Verified: build 0/0, AbortAtDeadline 5/5, full HangDump suite 30/30. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot

Pull request overview

Copilot reviewed 28 out of 28 changed files in this pull request and generated 4 comments.

nohwnd · 2026-07-20T11:45:42Z

+            _deadlineTimer = new Timer(
+                _ => TriggerDumpOnce(cancellationToken, triggeredByDeadline: true),
+                null,
+                dueTime,
+                TimeSpan.FromMilliseconds(-1));


Good catch. I moved the deadline timer so it is armed right after we have the test host process info, before the pipe handshake. Now if the host wedges during startup and never connects back over the pipe, the deadline dump and kill are still armed, which is the case the deadline is meant for. The in-progress-test list needs the consumer pipe, so I made that part best-effort and skip it when the pipe never connected.

🤖

nohwnd · 2026-07-20T11:45:41Z

+        if (Interlocked.Exchange(ref _dumpTaken, 1) != 0)
+        {
+            return;
+        }
+
+        _activityIndicatorTask = TakeDumpOfTreeAsync(cancellationToken, triggeredByDeadline);


Right. The Interlocked.Exchange claimed the gate but did not publish _activityIndicatorTask atomically, so disposal could read it as null and tear the pipes down while a dump the timer just started was still running. I now take the dump-once gate and publish the task under one lock, and Dispose/DisposeAsync take the same lock, claim the gate so no new dump can start, and capture the in-flight task to wait on outside the lock.

🤖

nohwnd · 2026-07-20T11:45:44Z

+
+        if (DeadlineHelper.TryGetDeadline(environment, out DateTimeOffset deadline) && capability is not null)
+        {
+            DateTimeOffset stopAt = deadline - DeadlineHelper.GetStopMargin(environment);


Agreed. deadline - margin can underflow DateTimeOffset for a very old (but valid) deadline or a large margin. I added DeadlineHelper.SubtractSaturating, which clamps at DateTimeOffset.MinValue, so underflow just means the stop instant is already in the past and we act immediately.

🤖

nohwnd · 2026-07-20T11:45:39Z

+        // has a chance to complete before the CI runner hard-kills the process.
+        if (DeadlineHelper.TryGetDeadline(_environment, out DateTimeOffset deadline))
+        {
+            _deadlineDumpAt = deadline - DeadlineHelper.GetDumpMargin(_environment);


Same underflow class here, so this path now uses DeadlineHelper.SubtractSaturating too.

🤖

…-cancellation # Conflicts: # src/Platform/Microsoft.Testing.Platform/InternalAPI/InternalAPI.Unshipped.txt

More fixes from the review of #10018: - HangDump: the absolute deadline timer was armed only after the pipe handshake in OnTestHostProcessStartedAsync. A test host that wedges during startup never connects back over the pipe, so those waits blocked past the deadline and the deadline dump/kill was never armed, which is exactly the case the deadline is for. Arm the deadline timer right after we have the test host process info (before the handshake). The dump path only needs the PID; the in-progress-test list needs the consumer pipe, so I make it best-effort and skip it when the pipe never connected. - HangDump: the winning timer published _activityIndicatorTask without any ordering against disposal. Disposal could read the field as null, release, and tear the pipes down while a dump the timer just started was still running. Guard the "take the dump once" gate and the task publish under one lock, and have Dispose/DisposeAsync take that lock, claim the gate so no new dump can start, and capture the in-flight task to wait on outside the lock. The lock is a System.Threading.Lock on net9.0 and an object below it, so it still compiles on netstandard2.0. - AbortAtDeadline and HangDump: deadline - margin on DateTimeOffset throws for a very old (but valid) deadline or a large margin. Add a shared DeadlineHelper.SubtractSaturating that clamps at DateTimeOffset.MinValue, so underflow means "already in the past" -> act immediately. Verified: build 0/0, AbortAtDeadline 5/5, full HangDump suite 30/30. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot

Pull request overview

Copilot reviewed 28 out of 28 changed files in this pull request and generated 7 comments.

nohwnd · 2026-07-22T06:42:24Z

+        }
+    }
+
+    public Type[] DataTypesConsumed { get; } = [typeof(TestNodeUpdateMessage)];


Good catch. DataTypesConsumed is now an empty array, so the bus keeps the reference for the run without routing every test result into a no-op ConsumeAsync.

🤖

nohwnd · 2026-07-22T06:42:26Z

+            return;
+        }
+
+        _ = Task.Run(HandleDeadlineAsync);


Switched to calling HandleDeadlineAsync directly instead of Task.Run, so single-threaded runtimes (browser/WASI) don't queue work that never runs. The method is async and yields at the first await, so it doesn't block the timer thread.

🤖

nohwnd · 2026-07-22T06:42:27Z

+                dueTime = TimeSpan.Zero;
+            }
+
+            _timer = new Timer(static state => ((AbortAtDeadlineExtension)state!).OnDeadlineReached(), this, dueTime, Timeout.InfiniteTimeSpan);


Added a clamp to the Timer maximum (~49.7 days) so the ctor can't throw for a far-future deadline. The run is disposed long before that, so the timer never fires early in practice.

🤖

nohwnd · 2026-07-22T06:42:29Z

+            _deadlineTimer = new Timer(
+                _ => TriggerDumpOnce(cancellationToken, triggeredByDeadline: true),
+                null,
+                dueTime,
+                TimeSpan.FromMilliseconds(-1));


Same clamp added here.

🤖

nohwnd · 2026-07-22T06:42:30Z

+        // The consumer pipe is only present once the test host connected back over it. When the host
+        // wedged during startup and the deadline dump path fired, there is no pipe to ask for the
+        // in-progress tests, so we skip that best-effort list and still take the dump and kill the tree.
+        if (_namedPipeClient is not null)


Right, a non-null client isn't necessarily connected: it's created when the host sends its pipe name but connected later. I wrapped the in-progress query in try/catch so a deadline dump firing in that window can't block taking the dump and killing the tree.

🤖

nohwnd · 2026-07-22T06:42:32Z

    <value>Hang dump file</value>
  </data>
+  <data name="HangDumpDeadlineReached" xml:space="preserve">
+    <value>CI deadline reached: taking a hang dump before the run is hard-cancelled.</value>


Renamed the resource to HangDumpDeadlineApproaching and reworded the text and log reason. The dump fires at deadline minus the dump margin, so "approaching" is accurate. Regenerated the xlf files.

🤖

nohwnd · 2026-07-22T06:42:33Z

+            await _logger.LogInformationAsync($"Deadline approaching (stop scheduled at {_stopAt:o}). Requesting graceful stop of test execution.").ConfigureAwait(false);
+
+            CancellationToken cancellationToken = _cancellationTokenSource.CancellationToken;
+            await _outputDevice.DisplayAsync(
+                this,
+                new FormattedTextOutputDeviceData("Deadline approaching: gracefully stopping the test run so reports can be finalized before the CI hard-cancel."),
+                cancellationToken).ConfigureAwait(false);
+
+            await capability.StopTestExecutionAsync(cancellationToken).ConfigureAwait(false);


Moved the graceful stop into its own try/catch, separate from the best-effort diagnostics, so a logging or output-device failure can't skip the stop. The diagnostics failure log is itself swallowed so it can't re-throw and skip the stop either.

🤖

The automated re-review flagged seven spots after the last push. All are in the two deadline timers and the hang dump resource text. AbortAtDeadlineExtension: - DataTypesConsumed is now empty. The extension only implements IDataConsumer so the message bus keeps a live reference to it (which keeps its timer alive). Returning [TestNodeUpdateMessage] made the bus route every test result to a no-op ConsumeAsync, which is O(test-count) for nothing. - The timer callback calls HandleDeadlineAsync directly instead of Task.Run. On single-threaded runtimes (browser/WASI) Task.Run can queue work that never runs; the method is already async and yields at the first await. - Arming the timer clamps a far-future due time to the Timer maximum (~49.7 days) instead of throwing. The run is disposed long before that, so the timer never fires early in practice. - The graceful stop now runs in its own try/catch, separate from the best-effort diagnostics. A logging or output-device failure can no longer skip the stop, and the diagnostics failure log is itself swallowed so it cannot re-throw and skip the stop either. HangDumpProcessLifetimeHandler: - Same far-future clamp on the deadline dump timer. - The in-progress-test query before dumping is wrapped in try/catch. A non-null pipe client is not necessarily connected (it is created when the host sends its pipe name but connected later), so a deadline dump firing in that window could hit an unconnected pipe. Any failure is logged and swallowed so it cannot block taking the dump and killing the tree. - Renamed the resource HangDumpDeadlineReached to HangDumpDeadlineApproaching and reworded the text and log reason to "approaching". The dump fires at deadline minus the dump margin, so the deadline has not been reached yet. Regenerated the xlf files. Local Debug pack is 0/0. AbortAtDeadline acceptance tests 5/5 and the full HangDump acceptance suite 30/30. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot

Pull request overview

Copilot reviewed 28 out of 28 changed files in this pull request and generated 3 comments.

Comments suppressed due to low confidence (1)

src/Platform/Microsoft.Testing.Extensions.HangDump/HangDumpProcessLifetimeHandler.cs:459

This request is not actually best-effort when the host is wedged. NamedPipeClient.RequestReplyAsync waits for a response until the supplied run token is canceled (NamedPipeClient.cs:103-105), so a connected host whose control-pipe callback no longer runs can block here indefinitely and prevent both dump creation and process-tree termination. Skip this query for deadline-triggered dumps or bound it to a short deadline-specific budget.

                GetInProgressTestsResponse tests = await _namedPipeClient.RequestReplyAsync<GetInProgressTestsRequest, GetInProgressTestsResponse>(new GetInProgressTestsRequest(), cancellationToken).ConfigureAwait(false);

+        }
+    }
+
+    public void Dispose() => _timer?.Dispose();


+            await _outputDevice.DisplayAsync(
+                this,
+                new FormattedTextOutputDeviceData("Deadline approaching: gracefully stopping the test run so reports can be finalized before the CI hard-cancel."),
+                _cancellationTokenSource.CancellationToken).ConfigureAwait(false);


+    // Prototype defaults. stopMargin > dumpMargin so the graceful stop is attempted first and the
+    // hang dump is the fallback for a host that did not stop in time.
+    private static readonly TimeSpan DefaultStopMargin = TimeSpan.FromSeconds(60);
+    private static readonly TimeSpan DefaultDumpMargin = TimeSpan.FromSeconds(30);


+            catch
+            {
+                // Ignore: the graceful stop below is the only thing that must happen.
+            }


Evangelink

Review — deadline-aware cancellation

I ran this through several review passes (expert MTP reviewer + a bug-focused diff reviewer) and then validated every finding against the code on 14d6b88. Skipping everything the earlier Copilot round already covered and you already fixed.

The shape is good and the concurrency work after the last round (the _dumpLock gate + TriggerDumpOnce trampoline, SubtractSaturating, the timer clamp) reads correctly to me. What follows is what survived validation, roughly in priority order. The first three are the ones I'd want settled before this stops being a prototype.

Things I checked and found clean, so you don't have to re-litigate them: the empty DataTypesConsumed is legal (AsynchronousMessageBus.InitAsync just creates no processor, and the consumer is still disposed via CommonTestHost line 401's messageBus.DataConsumerServices loop); the InternalAPI.Unshipped.txt entries are exact across all five projects that link-compile EnvironmentVariableConstants.cs; the .xlf files are genuinely generated (state="new", source==target, alphabetical); [Embedded] on DeadlineHelper matches every neighbour in that folder; and no new test hard-codes a \(\d+ms\) duration pattern.

Evangelink · 2026-07-28T12:39:22Z

+
+        try
+        {
+            await capability.StopTestExecutionAsync(_cancellationTokenSource.CancellationToken).ConfigureAwait(false);


Major — a run truncated by the deadline still exits with code 0.

TestApplicationResult.GetProcessExitCode() derives a non-success code from exactly four signals:

// TestApplicationResult.cs:142-145 exitCode = exitCode == ExitCode.Success && _policiesService.IsMaxFailedTestsTriggered ? ExitCode.TestExecutionStoppedForMaxFailedTests : exitCode; exitCode = exitCode == ExitCode.Success && _testAdapterTestSessionFailure ? ExitCode.TestAdapterTestSessionFailure : exitCode; exitCode = exitCode == ExitCode.Success && _failedTestsCount > 0 ? ExitCode.AtLeastOneTestFailed : exitCode; exitCode = exitCode == ExitCode.Success && _policiesService.IsAbortTriggered ? ExitCode.TestSessionAborted : exitCode;

AbortForMaxFailedTestsExtension covers itself by calling _policiesService.ExecuteMaxFailedTestsCallbacksAsync right after StopTestExecutionAsync. This extension never touches IStopPoliciesService, so a 3000-test suite that gets cut off after 400 tests returns 0.

That inverts the value proposition: today the CI hard-cancel at least fails the job. After this change the job goes green on a run that never finished, and the only trace is one console line. AbortAtDeadlineTests currently enshrines it — all five methods assert ExitCode.Success.

Suggest a dedicated stop-policy flag plus a distinct ExitCode (mirroring TestExecutionStoppedForMaxFailedTests), mapped in GetProcessExitCode. If green-on-truncation is a deliberate prototype choice, it needs to be explicit in the XML doc, because nothing in the code hints at it.

Evangelink · 2026-07-28T12:39:22Z

            dataConsumersBuilder.Add(abortForMaxFailedTestsExtension);
        }

+        var abortAtDeadlineExtension = new AbortAtDeadlineExtension(


Major — in server mode the deadline is sticky and silently zeroes out every subsequent request.

BuildTestFrameworkAsync is invoked per JSON-RPC request, not once per process — see ServerTestHost.RequestExecution.cs:111. So every testing/runTests and testing/discoverTests constructs a fresh AbortAtDeadlineExtension and arms a fresh Timer, all reading the same absolute instant the server process inherited at startup.

Concrete failure: a VS / VS Code test-explorer session (or dotnet test in server mode) launched from a shell that happens to have TESTINGPLATFORM_DEADLINE exported. Once wall-clock passes the deadline, every later request computes dueTime = 0, fires immediately, and requests a graceful stop before anything is scheduled. The user gets "0 tests, success" forever with no way to recover short of restarting the IDE.

Disposal is fine (line 159 of RequestExecution.cs disposes the per-request provider, which walks DataConsumerServices), so this is behavioural rather than a leak.

Also worth noting the same path runs for discovery / --list-tests, where "gracefully stop test execution" doesn't really mean anything.

Suggest gating registration on a single-shot console run, and/or treating a deadline that is already in the past at construction time as "not applicable" (logged) rather than "fire now".

Evangelink · 2026-07-28T12:39:22Z

+                dueTime = MaxTimerDueTime;
+            }
+
+            _deadlineTimer = new Timer(


Major — arming the timer here means the deadline dump can be taken and then thrown away.

Moving the arming above the handshake is right for the reason in the comment, but nothing downstream was adjusted for the case where the timer actually wins. Sequence:

Test host wedges during startup, never connects; _waitConnectionTask never completes.

_deadlineTimer fires. TakeDumpOfTreeAsync writes the dumps into _dumpFiles, then its finally kills the process tree.

Killing the host does not complete WaitConnectionAsync, so line 236 keeps waiting the full TimeoutHelper.DefaultHangTimeSpanTimeout and then throws TimeoutException. (TimeoutAfterAsync there takes no cancellation token, so nothing can shorten it.) If the connection landed but the consumer pipe name didn't, line 239 / the ApplicationStateGuard at 240 throw instead.

TestHostControllersTestHost.cs:319 awaits OnTestHostProcessStartedAsync with no try/catch, so that exception unwinds straight past the OnTestHostProcessExitedAsync loop.

Net result in exactly the scenario this feature exists for: the dump files are on disk but never published as FileArtifacts (publication lives only in OnTestHostProcessExitedAsync, lines 299-302), and the process burns another 5+ minutes past the deadline — so CI hard-cancels it anyway.

Two options, not mutually exclusive: publish _dumpFiles from the dump path itself, and/or wrap 235-242 with catch (Exception ex) when (Volatile.Read(ref _dumpTaken) != 0) → log and return, so the controller proceeds to WaitForExitAsync / OnTestHostProcessExitedAsync normally.

Evangelink · 2026-07-28T12:39:22Z

+    // Prototype defaults. stopMargin > dumpMargin so the graceful stop is attempted first and the
+    // hang dump is the fallback for a host that did not stop in time.
+    private static readonly TimeSpan DefaultStopMargin = TimeSpan.FromSeconds(60);
+    private static readonly TimeSpan DefaultDumpMargin = TimeSpan.FromSeconds(30);


Major — stopMargin > dumpMargin is the invariant the whole design rests on, and it's only a comment about the defaults.

GetStopMargin and GetDumpMargin are read independently, in two different processes, with no cross-check and no warning anywhere.

Set TESTINGPLATFORM_DEADLINE_DUMP_MARGIN=120 (wanting headroom to write a big dump) and leave the stop margin at its 60s default, and the ordering inverts: at deadline-120s the HangDump controller dumps the whole process tree and kills a perfectly healthy run, 60 seconds before the graceful stop would have fired. All reports are lost. TESTINGPLATFORM_DEADLINE_STOP_MARGIN=10 does the same thing from the other side. Neither produces any diagnostic.

Cheapest fix that keeps this honest: resolve both margins in one helper and emit a warning to the output device when dumpMargin >= stopMargin. At minimum, log both resolved values in both processes so it's diagnosable after the fact.

Evangelink · 2026-07-28T12:39:22Z

+            return false;
+        }
+
+        if (!DateTimeOffset.TryParse(raw, CultureInfo.InvariantCulture, DateTimeStyles.AssumeUniversal | DateTimeStyles.AdjustToUniversal, out DateTimeOffset parsed))


Major — every misconfiguration is a silent no-op, and an offset-less timestamp is silently reinterpreted as UTC.

Three separate silent failures land here:

Offset-less input. AssumeUniversal means 2026-03-01T14:50:00 (no offset, no Z) is taken as UTC. A UTC+2 pipeline doing [DateTime]::Now.AddMinutes(50).ToString("s") therefore hands you an instant that is already two hours in the past. dueTime clamps to zero, the extension requests a graceful stop during startup, the suite runs ~nothing, and (per the exit-code comment) reports success. Every acceptance test uses DateTimeOffset.UtcNow.ToString("o"), which always carries +00:00, so none of them can catch this.

Typo / malformed date. TESTINGPLATFORM_DEADLIINE, or a value TryParse rejects, disables the entire safety net with zero output.

Unparsable margin. GetMargin silently reverts to the default, so TESTINGPLATFORM_DEADLINE_DUMP_MARGIN=2min (unsupported syntax) quietly becomes 30s.

And a fourth, one level up at line 56: when the framework doesn't expose IGracefulStopTestExecutionCapability — which is every VSTestBridge-based framework — the extension disables itself with no message at all. Compare --maximum-failed-tests, which produces an explicit validation error when the capability is absent.

Suggest: require an explicit offset/Z (or at least log the resolved instant), warn on an unparsable deadline or margin, and log once when a deadline is configured but the framework can't honour it. Right now "nothing happened" and "it's not supported here" are indistinguishable from CI output.

Evangelink · 2026-07-28T12:39:22Z

+        catch (Exception ex)
+        {
+            // Best-effort: never let the timer callback crash the process during teardown.
+            await _logger.LogErrorAsync("Failed to request graceful stop at deadline.", ex).ConfigureAwait(false);


Minor — this is the one logging call in the method that isn't double-guarded, and it can FailFast the process.

Lines 139-146 carefully wrap the recovery log in a nested try/catch, with a comment explaining exactly why. The identical hazard here is bare. StopTestExecutionAsync is a public [TPEXP] extensibility point (a third-party framework can throw anything, and ObjectDisposedException is plausible if the timer fires during teardown); if LogErrorAsync then also throws — the FileLoggerProvider is disposed at CommonTestHost.cs:93, before process end — the exception escapes HandleDeadlineAsync.

Because line 114 discards the task, nobody observes it. With PlatformExitProcessOnUnhandledException / TESTINGPLATFORM_EXIT_PROCESS_ON_UNHANDLED_EXCEPTION=1 set, UnhandledExceptionHandler.OnTaskSchedulerUnobservedTaskException (line 45) calls LogErrorAndExit(..., forceClose: true) → _environment.FailFast(...). So the finalizer thread hard-crashes the process and every result is lost — the exact opposite of what this feature is for.

Two-line fix: wrap this in the same nested try/catch as above. Separately, it'd be worth storing the task in a field so Dispose() can drain it with a bounded wait — Timer.Dispose() doesn't wait for an in-flight callback, so today HandleDeadlineAsync can still be touching _logger / _outputDevice / the capability after their providers are gone.

Evangelink · 2026-07-28T12:39:22Z

+            // Claim the gate so no timer callback can start a new dump once we begin tearing down the
+            // pipes, and capture any dump already in flight so we wait for it below.
+            _dumpTaken = 1;
+            activityIndicatorTask = _activityIndicatorTask;


Minor — neither Dispose() nor DisposeAsync() disposes _activityTimer / _deadlineTimer.

Both dispose paths claim the _dumpTaken gate (so a late callback is a no-op — good) and dispose the pipes, but the two Timer objects are only disposed on the happy path in OnTestHostProcessExitedAsync (lines 276-292). That method is skipped in at least two reachable cases: its own cancellationToken.ThrowIfCancellationRequested() at line 274 on Ctrl+C, and the exception path I described on the arming block above.

Timer.Dispose() is idempotent, so this is a two-line addition here and in DisposeAsync without touching the existing disposal:

_deadlineTimer?.Dispose(); _activityTimer?.Dispose();

Evangelink · 2026-07-28T12:39:22Z

+            }
+
+            _dumpTaken = 1;
+            _activityIndicatorTask = TakeDumpOfTreeAsync(cancellationToken, triggeredByDeadline);


Minor — the dump is invoked under the lock, not just scheduled.

TakeDumpOfTreeAsync(...) runs synchronously up to its first incomplete await. _logger.LogInformationAsync and _outputDisplay.DisplayAsync both return completed tasks in ordinary configurations (no file logger, buffered output device), in which case execution continues under _dumpLock into GetProcessById / GetProcessTreeAsync — real work, on the timer's thread-pool thread.

Meanwhile Dispose() (560) and DisposeAsync() (596) both block on that same lock, on whatever thread the platform tears down on. Not a deadlock, but an unbounded stall on a path that already has its own timeout semantics.

Keep the decision under the lock and start the task outside it — set _dumpTaken inside, then assign _activityIndicatorTask after releasing.

Evangelink · 2026-07-28T12:39:22Z

+    /// the stop/dump instant. Saturating means "this instant is already in the past", which for both
+    /// callers translates to "act immediately".
+    /// </summary>
+    public static DateTimeOffset SubtractSaturating(DateTimeOffset instant, TimeSpan margin)


Minor — this class has no unit tests, and it's where all the subtle logic lives.

TryGetDeadline (parsing + timezone semantics), GetMargin (fallback behaviour) and SubtractSaturating (underflow clamping) are pure, static, and trivially testable — and test/UnitTests/Microsoft.Testing.Platform.UnitTests/Helpers/ already has the exact precedent in TimeSpanParserTests.cs.

Today they're only exercised indirectly by timing-sensitive acceptance tests that always feed a well-formed DateTimeOffset.UtcNow.ToString("o"), so both the offset-less footgun and the saturation clamp are invisible to CI.

A DeadlineHelperTests.cs with [DataRow] coverage for Z-suffixed / +02:00 / offset-less / empty / garbage deadlines, margins "30" "30s" "1m" "" "abc", and SubtractSaturating(DateTimeOffset.MinValue + 1s, 1h) == MinValue would pin all of it down for a few minutes' work.

(Small aside while you're in here: the parsed >= TimeSpan.Zero guard in GetMargin is dead — TimeSpanParser's pattern is \d+, so it can never produce a negative. Harmless, but it reads as protection that isn't there.)

Evangelink · 2026-07-28T12:39:22Z

+                // Stop margin 0 means the graceful stop is scheduled for the deadline itself, a few
+                // seconds out. The framework blocks until the stop is requested, so this proves the
+                // timer fires on schedule (not only when the deadline is already past).
+                ["TESTINGPLATFORM_DEADLINE"] = DateTimeOffset.UtcNow.AddSeconds(6).ToString("o"),


Minor — this test can't fail for the reason it exists.

UtcNow.AddSeconds(6) is evaluated in the test process, but the timer is armed in the child after process launch + runtime startup + the MTP builder runs. On a loaded CI agent that easily exceeds 6s, at which point dueTime clamps to zero and the run takes the "already past" path — which is what WhenDeadlineIsInThePast_GracefullyStopsImmediately already covers. The assertions still pass, so the test silently stops testing "the timer fires on schedule".

Same applies to WhenStopMarginIsSubtracted (60s out, 60s margin → stop instant ≈ now).

If you want this to actually pin the timer down, have the test asset report when the stop arrived (e.g. print elapsed-since-start from StopTestExecutionAsync) and assert it's non-trivially greater than zero.

Also: await GracefulStop.Instance.TCS.Task in the asset has no timeout, so a regression in the extension turns these into hangs rather than failures — the acceptance harness timeout is what would eventually catch it. A WaitAsync/Task.WhenAny with a generous cap would fail fast with a readable message instead.

Copilot AI review requested due to automatic review settings July 16, 2026 15:29

Copilot started reviewing on behalf of nohwnd July 16, 2026 15:30 View session

Copilot AI reviewed Jul 16, 2026

View reviewed changes

github-code-quality Bot found potential problems Jul 16, 2026

View reviewed changes

Comment thread src/Platform/Microsoft.Testing.Platform/Extensions/AbortAtDeadlineExtension.cs Fixed

nohwnd and others added 2 commits July 16, 2026 17:51

Merge remote-tracking branch 'origin/main' into nohwnd-deadline-aware…

30fe9a1

…-cancellation # Conflicts: # src/Platform/Microsoft.Testing.Extensions.TrxReport/InternalAPI/InternalAPI.Unshipped.txt

Copilot AI review requested due to automatic review settings July 16, 2026 16:03

Copilot started reviewing on behalf of nohwnd July 16, 2026 16:04 View session

Copilot AI reviewed Jul 16, 2026

View reviewed changes

nohwnd and others added 2 commits July 20, 2026 13:22

Merge remote-tracking branch 'origin/main' into nohwnd-deadline-aware…

1cbf4df

…-cancellation # Conflicts: # src/Platform/Microsoft.Testing.Platform/InternalAPI/InternalAPI.Unshipped.txt

Copilot AI review requested due to automatic review settings July 20, 2026 11:45

Copilot started reviewing on behalf of nohwnd July 20, 2026 11:45 View session

Copilot AI reviewed Jul 20, 2026

View reviewed changes

Copilot AI review requested due to automatic review settings July 22, 2026 06:41

Copilot started reviewing on behalf of nohwnd July 22, 2026 06:42 View session

Copilot AI reviewed Jul 22, 2026

View reviewed changes

github-code-quality Bot found potential problems Jul 22, 2026

View reviewed changes

Comment thread src/Platform/Microsoft.Testing.Platform/Extensions/AbortAtDeadlineExtension.cs

Comment on lines +143 to +146

catch

{

// Ignore: the graceful stop below is the only thing that must happen.

}

Evangelink reviewed Jul 28, 2026

View reviewed changes

		// Copyright (c) Microsoft Corporation. All rights reserved.
		// Licensed under the MIT license. See LICENSE file in the project root for full license information.

Uh oh!

Conversation

nohwnd commented Jul 16, 2026

What it does

Where the deadline comes from

Verified

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Evangelink left a comment

Choose a reason for hiding this comment

Review — deadline-aware cancellation

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment