Skip to content

fix(manifest): correct published digests and make drift detectable - #1062

Merged
abrichr merged 2 commits into
mainfrom
fix/platform-manifest-release-drift
Jul 27, 2026
Merged

fix(manifest): correct published digests and make drift detectable#1062
abrichr merged 2 commits into
mainfrom
fix/platform-manifest-release-drift

Conversation

@abrichr

@abrichr abrichr commented Jul 27, 2026

Copy link
Copy Markdown
Member

The wrong claim

platform-manifest.json is served publicly from raw.githubusercontent.com/OpenAdaptAI/OpenAdapt/main/platform-manifest.json. It is OpenAdapt's only cryptographic claim about the current release, and it pinned openadapt-flow 1.23.0 with that release's wheel and sdist sha256 (b5d10dfc…, b16e181a…). openadapt-flow 1.24.0 published at 2026-07-27T03:58Z. Anyone verifying today's published artifact against the manifest failed.

Regenerated from the live PyPI state. All eight artifact digests were re-verified by downloading each file and hashing it, not by trusting a metadata field:

component artifact sha256 (downloaded)
launcher openadapt-1.9.0-py3-none-any.whl 5acc4273c2db403e…
launcher openadapt-1.9.0.tar.gz a181627615e3ff74…
flow openadapt_flow-1.24.0-py3-none-any.whl 170fdac154794292…
flow openadapt_flow-1.24.0.tar.gz 2d4702e5ccbdfed0…
capture openadapt_capture-1.2.0-py3-none-any.whl da3b2ef52b3eb24c…
capture openadapt_capture-1.2.0.tar.gz 5eb5f7df329e58a1…
desktop openadapt_desktop-0.14.0-py3-none-any.whl 7ee5a1d2e00f5cbd…
desktop openadapt_desktop-0.14.0.tar.gz 44a50bd2c126524a…

Why the drift detector did not fire in time — the more important bug

1. The staleness originates in another repository. Flow released; nothing was committed here; only the schedule could notice. It was weekly (17 6 * * 1), so the public copy could advertise superseded digests for up to seven days. Now daily, plus a component-released repository_dispatch receiver so a component release can trigger the check immediately. The job is stdlib-only — no install, no cache — so this costs seconds a day.

2. The manifest has no automatic remediation path for non-launcher releases. reconcile-platform-manifest lives in this repo's release workflow and only runs when the launcher releases.

3. The check went benignly red after every single release, so a real failure was indistinguishable from noise. Two normal conditions were classified as errors:

  • run 30226357202 (push of the 1.9.0 version commit) — manifest has '1.8.0' but pyproject.toml has '1.9.0'. Guaranteed at every release: semantic-release pushes the bump ~90s before the reconcile job regenerates the manifest.
  • run 30226399528 (push of the reconcile commit) — manifest has '1.9.0' but PyPI's latest openadapt is '1.8.0'. Guaranteed at every release: PyPI's info.version lags an upload on some CDN edges.

main was therefore already red from the previous night's 1.9.0 release when the genuine flow drift turned run 30257087341 red. A guard that cries wolf at every release stops being read.

Both conditions now warn. Every filename, URL, and digest comparison stays fatal. An unreachable PyPI warns rather than fails (an index outage is not evidence of drift) — except at the release gate, which now passes --require-network.

4. A red scheduled run on a repository nobody has open is not a signal. A failed scheduled or dispatched run now files (or comments on) a platform-manifest.json has drifted issue, so drift gets an owner.

Proof the guard fails when it should

tests/test_platform_manifest_drift.py (12 tests) exercises the pure comparison functions against a simulated future release, a tampered digest, a tampered URL, and a version PyPI never published — and pins that the two transient release-time conditions do not fail.

End-to-end against live PyPI:

$ python scripts/validate_platform_manifest.py                        # this PR
OK: platform manifest validated (including published artifacts); 4 warning(s).   exit=0

$ python scripts/validate_platform_manifest.py --manifest <origin/main copy>
ERROR: flow version drift: manifest has '1.23.0' but PyPI's latest
openadapt-flow is '1.24.0'. The manifest advertises a superseded release
and its digests.                                                                 exit=1

$ python scripts/validate_platform_manifest.py --manifest <digest tampered>
ERROR: flow artifact openadapt_flow-1.24.0-py3-none-any.whl sha256 drift …
A consumer verifying the published artifact against this manifest would fail.    exit=1

Follow-up (out of scope here)

The sending half of the dispatch is a one-step gh api repos/OpenAdaptAI/OpenAdapt/dispatches -f event_type=component-released in openadapt-flow / -capture / -desktop's release workflow. Until that lands, the daily sweep bounds exposure at 24h instead of 7 days.

🤖 Generated with Claude Code

https://claude.ai/code/session_01NyCHrzA1psrKMFfroYbzaM

abrichr and others added 2 commits July 27, 2026 10:14
platform-manifest.json is served publicly from raw.githubusercontent and is
OpenAdapt's only cryptographic claim about the current release. It pinned
openadapt-flow 1.23.0 with that release's wheel and sdist sha256, so anyone
verifying today's published artifact against it failed.

Regenerate from the live PyPI state (flow 1.24.0). All eight artifact digests
were re-verified by downloading each file and hashing it, not by trusting a
metadata field.

The more important bug is that the drift detector missed the drift for hours
and could have missed it for a week:

* The staleness originates in ANOTHER repository, so no commit lands here and
  only the schedule can notice. It was weekly. It is now daily, plus a
  `component-released` repository_dispatch receiver so a component release can
  trigger the check immediately.
* The check failed benignly after every single release, which is how a real
  failure became invisible: the semantic-release version commit leaves
  pyproject.toml ahead of the not-yet-reconciled manifest, and PyPI's
  info.version lags an upload by minutes. Main was already red from the 1.9.0
  release when the genuine flow drift arrived. Both conditions now warn.
  Every filename, URL, and digest comparison stays fatal, and an unreachable
  PyPI warns rather than fails (an index outage is not evidence of drift)
  except at the release gate, which passes --require-network.
* A red scheduled run on a repository nobody has open is not a signal, so a
  failed scheduled or dispatched run now files or comments on a drift issue.

tests/test_platform_manifest_drift.py proves the guard FAILS when it should:
against a simulated future release, a tampered digest, a tampered URL, and a
version PyPI never published -- and that it does not fail on the two transient
release-time conditions above.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NyCHrzA1psrKMFfroYbzaM
The daily and repository_dispatch runs exist to compare the committed manifest
against PyPI, and that comparison needs nothing but the standard library. Gate
the characterization suite to pull requests and pushes, where the guard's own
source can change, so the remote drift probe stays fast and installs nothing.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NyCHrzA1psrKMFfroYbzaM
@abrichr
abrichr merged commit 5e6ef4e into main Jul 27, 2026
14 checks passed
@abrichr
abrichr deleted the fix/platform-manifest-release-drift branch July 27, 2026 14:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant