Implement registry version check script

.github/workflows/package-set-version-check.yml

@@ -0,0 +1,55 @@
+name: Package set version check
+
+on:
+  workflow_dispatch:
+
+permissions:
+  contents: read
+  issues: write
+
+concurrency:
+  group: package-set-version-check
+  cancel-in-progress: false
+
+jobs:
+  version-check:
+    runs-on: ubuntu-latest
+    # Planning performs at most 100 incremental compile probes against the
+    # package set, each of which downloads and compiles the proposed set.
+    timeout-minutes: 120
+
+    steps:
+      - name: Checkout code
+        uses: actions/checkout@v6
+
+      - name: Install Nix
+        uses: DeterminateSystems/determinate-nix-action@v3
+
+      - name: Setup Nix cache
+        uses: DeterminateSystems/magic-nix-cache-action@v14
+        with:
+          use-flakehub: false
+
+      - name: Generate compile-verified version check report
+        run: nix run .#package-set-version-checker -- --output version-check.md
+
+      - name: Upload report
+        if: ${{ !cancelled() }}
+        uses: actions/upload-artifact@v4
+        with:
+          name: package-set-version-check
+          path: version-check.md
+          if-no-files-found: warn
+          retention-days: 14
+
+      - name: Open report issue
+        # The checker exits successfully without writing a report when there
+        # is no package set to check, so only open an issue when the report
+        # file exists.
+        if: ${{ hashFiles('version-check.md') != '' }}
+        env:
+          GH_TOKEN: ${{ github.token }}
+        run: |
+          gh issue create \
+            --title "Package set version check ($(date --utc +%Y-%m-%d))" \
+            --body-file version-check.md

SPEC.md

@@ -784,7 +784,7 @@ A package set update is an object with two keys: `compiler`, an optional field t
 ```jsonc
 { // Sets the package set compiler version to 0.15.2
   "compiler": "0.15.2",
-  "packages" {
+  "packages": {
     // Updates the `aff` package to v8.0.0
     "aff": "8.0.0",
     // Removes the `argonaut` package from the package sets altogether
@@ -812,13 +812,15 @@ The suggested new package set can only be released if:
 1. All packages in the set depend only on other packages in the set
 2. All packages in the set can be compiled together
 
+Package set self-containment concerns dependency names, not the version ranges in package manifests. Every dependency name must occur in the package set, but its manifest range need not include the exact version selected by the set. Compiling the exact package versions in the set with the set's exact compiler version is the authoritative validity check.
+
 To verify the package set, we take the following steps:
 
-1. We apply the suggested changes to the package set and verify that the package set is still self-contained. If not, the update is rejected.
+1. We apply all suggested changes to the package set as one exact batch and verify that the package set is still self-contained. If not, the update is rejected.
 2. We install the previous package set and compile it. This ensures we are beginning from a known good state.
 3. We install the new package set (uninstalling any packages that are removed) and compile the package set again. If compilation fails then the update is rejected.
 
-When we have verified the package set is self-contained and all packages compile together then we can release the package set.
+When we have verified the package set is self-contained and all packages compile together then we can release the package set. A submitted package set update is atomic: either its entire exact payload is released or it fails without releasing a subset.
 
 ##### Releasing the Package Set
 
@@ -874,27 +876,25 @@ If documentation publishing fails (for example, due to a transient network error
 
 #### 6.3 Publish to Package Sets
 
-The registry attempts to produce a new package set automatically every day, so long as packages have been uploaded that could be added or updated. No packages are ever dropped from a package set automatically; the only time packages are dropped from the package sets are during manual releases.
+The registry attempts to produce a new package set automatically every day when eligible package versions exist. The latest package set and its compiler version are the starting point for every automatic plan.
 
-Every day, the registry executes the following steps:
+Newer registry versions of packages already in the package set remain candidates until they are included or superseded by the package set itself. They do not cease to be candidates merely because an automatic run could not include them or because time has passed since publication. An automatic plan may coordinate upgrades to several packages and may select an intermediate newer version instead of only the latest release when that produces a valid set.
 
-First, we read the contents of the latest package set release and gather all package versions that have been uploaded to the registry since that release. These package versions are the "batch" of packages that we are considering for automatic inclusion to the next package set.
+Packages that have never appeared in a package set are proposed as additions while they are recent uploads. The candidate set is recursively expanded with any dependency names absent from the current package set so the planner can consider a self-contained plan; these dependencies are included whenever they were published.
 
-Second, we filter out any packages where, based on their metadata and manifest files alone, we know they can't be added to the package set. This happens for one of three reasons:
+Every automatic plan must satisfy these constraints:
 
-1. They have a dependency that is neither in the package sets nor in the batch that is up for consideration
-2. They have had multiple releases since the last package set, in which case we only take the highest version
-3. They already have a higher version published in a previous package set
+1. It does not remove a package.
+2. It does not downgrade a package.
+3. It does not change the compiler version.
+4. Every dependency name in the resulting set occurs in that set. Manifest version ranges are ignored for this self-containment check.
+5. The exact resulting package set compiles as a whole with the unchanged, exact compiler version.
 
-Third, we attempt to add the rest of the batch of package versions to the package set. Processing the batch follows these steps:
+The whole-set compiler result is authoritative: manifest version ranges do not prevent an exact selection from being a valid package set, and manifest compatibility alone does not prove that a selection is valid. Each selected plan is submitted and released as one exact atomic package set update according to [Section 5.5 (Update the Package Set)](#55-update-the-package-set).
 
-1. We install the previous package set and compile it.
-2. We attempt to upgrade all package versions from the batch at once in the package set. Once the new versions are installed, we compile the package set. If it succeeds, then we're done.
-3. If we couldn't compile the whole batch, then we order the batch first by their dependencies and then by their upload time. Packages with no dependencies on other packages in the batch go first, and ties are broken by upload time: older uploads go first. Then, we attempt to add packages to the package set one-by-one.
-4. If a package fails to compile with the rest of the package set, then it is filtered from the batch.
-5. Once there are no more packages to consider in the batch, the new package set is ready.
+If an otherwise desirable upgrade requires removing packages, it is not an automatic plan. Registry Trustees handle such cases using the manual intervention process in [Section 9.5 (Package Sets)](#manual-intervention-in-the-package-sets-via-the-package-sets-api).
 
-Fourth, we release the new package set (if we could produce one). Automatic package sets follow the versioning policy described in [Section 5.5 (Update the Package Set)](#releasing-the-package-set).
+The automation reports residual upgrades it could not apply, including compiler evidence. It may report an exact removal-containing payload as compile-verified only after compiling that full payload against its original package set baseline. Such payloads are advisory and are never submitted automatically; a Registry Trustee must decide whether to apply them. Infrastructure failures and bounded searches must be reported distinctly from package incompatibility.
 
 ## 7. Non-JavaScript Backends
 

app-e2e/src/Test/E2E/Scripts.purs

@@ -20,7 +20,10 @@ import Registry.App.Effect.Cache as Cache
 import Registry.App.Effect.Env as Env
 import Registry.App.Effect.GitHub as GitHub
 import Registry.App.Effect.Log as Log
+import Registry.App.Effect.PackageSets as PackageSets
 import Registry.App.Effect.Registry as Registry
+import Registry.App.Effect.Storage as Storage
+import Registry.Foreign.FSExtra as FS.Extra
 import Registry.Foreign.Octokit as Octokit
 import Registry.Location (Location(..))
 import Registry.Operation (AuthenticatedPackageOperation(..))
@@ -140,19 +143,20 @@ spec = do
       jobs <- Client.getJobs
       let packageSetJobs = Array.filter isPackageSetJob jobs
       case Array.head packageSetJobs of
-        Just (PackageSetJob { payload }) ->
-          case payload of
-            Operation.PackageSetUpdate { packages } ->
-              case Map.lookup typeEqualityName packages of
-                Just (Just _) -> pure unit
-                _ -> Assert.fail "Expected type-equality in package set update"
+        Just (PackageSetJob { payload }) -> do
+          let Operation.PackageSetUpdate { packages } = payload
+          case Map.lookup typeEqualityName packages of
+            Just (Just _) -> pure unit
+            _ -> Assert.fail "Expected type-equality in package set update"
         Just _ -> Assert.fail "Expected PackageSetJob but got different job type"
         Nothing -> Assert.fail "Expected package set job to be enqueued"
 
 -- | Common environment for running read-only registry scripts in E2E tests
 type RegistryScriptSetup =
-  { resourceEnv :: Env.ResourceEnv
+  { cache :: FilePath
+  , resourceEnv :: Env.ResourceEnv
   , registryEnv :: Registry.RegistryEnv
+  , workdir :: FilePath
   }
 
 type GitHubScriptSetup =
@@ -171,18 +175,21 @@ setupRegistryScript = do
   resourceEnv <- liftEffect Env.lookupResourceEnv
   registryCacheRef <- liftAff Cache.newCacheRef
   debouncer <- liftAff Registry.newDebouncer
+  let workdir = Path.concat [ stateDir, "scratch" ]
+  let cache = Path.concat [ workdir, ".cache" ]
+  liftAff $ FS.Extra.ensureDirectory cache
   let
     registryEnv :: Registry.RegistryEnv
     registryEnv =
       { jobId: Nothing
       , pull: Git.Autostash
       , write: Registry.ReadOnly
       , repos: Registry.defaultRepos
-      , workdir: Path.concat [ stateDir, "scratch" ]
+      , workdir
       , debouncer
       , cacheRef: registryCacheRef
       }
-  pure { resourceEnv, registryEnv }
+  pure { cache, resourceEnv, registryEnv, workdir }
 
 setupGitHubScript :: E2E GitHubScriptSetup
 setupGitHubScript = do
@@ -229,12 +236,15 @@ runPackageTransferrerScript = do
 -- | Run the PackageSetUpdater script in Submit mode
 runPackageSetUpdaterScript :: E2E Unit
 runPackageSetUpdaterScript = do
-  { resourceEnv, registryEnv } <- setupRegistryScript
+  { cache, resourceEnv, registryEnv, workdir } <- setupRegistryScript
   result <- liftAff
     $ PackageSetUpdater.runPackageSetUpdater PackageSetUpdater.Submit resourceEnv.registryApiUrl
+    # PackageSets.interpret (PackageSets.handle { workdir })
     # Except.runExcept
     # Registry.interpretRead (Registry.handleRead registryEnv)
+    # Storage.interpret (Storage.handleReadOnly cache)
     # Log.interpret (Log.handleTerminal Quiet)
+    # Env.runResourceEnv resourceEnv
     # Run.runBaseAff'
   case result of
     Left err -> liftAff $ Aff.throwError $ Aff.error $ "PackageSetUpdater failed: " <> err

app/src/App/API.purs

@@ -153,7 +153,7 @@ packageSetUpdate :: forall r. PackageSetJobData -> Run (PackageSetUpdateEffects
 packageSetUpdate details = do
   let Operation.PackageSetUpdate payload = details.payload
 
-  Log.debug $ "Package set update job starting with payload:\n" <> stringifyJson Operation.packageSetUpdateCodec payload
+  Log.debug $ "Package set update job starting with payload:\n" <> stringifyJson Operation.packageSetOperationCodec details.payload
 
   latestPackageSet <- Registry.readLatestPackageSet >>= case _ of
     Nothing -> do
@@ -225,18 +225,13 @@ packageSetUpdate details = do
 
   let changeSet = candidates.accepted <#> maybe Remove Update
   Log.notice "Attempting to build package set update."
-  PackageSets.upgradeSequential latestPackageSet (fromMaybe prevCompiler payload.compiler) changeSet >>= case _ of
-    Nothing ->
-      Except.throw "No packages could be added to the package set. All packages failed to compile."
-    Just { failed, succeeded, result: packageSet } -> do
-      unless (Map.isEmpty failed) do
-        let
-          formatFailed = String.joinWith "\n" $ Array.catMaybes $ flip map (Map.toUnfoldable failed) \(Tuple name change) ->
-            case change of
-              PackageSets.Update version -> Just $ "  - " <> formatPackageVersion name version
-              PackageSets.Remove -> Nothing
-        Log.warn $ "Some packages could not be added to the set:\n" <> formatFailed
-      let commitMessage = PackageSets.commitMessage latestPackageSet succeeded (un PackageSet packageSet).version
+  -- A submitted payload is exact. Sequential probing is useful for planning,
+  -- but must not turn this operation into a partial release.
+  PackageSets.upgradeAtomic latestPackageSet (fromMaybe prevCompiler payload.compiler) changeSet >>= case _ of
+    Left error ->
+      Except.throw $ "Package set update failed to compile atomically. No changes were published.\n\n" <> error
+    Right packageSet -> do
+      let commitMessage = PackageSets.commitMessage latestPackageSet changeSet (un PackageSet packageSet).version
       Registry.writePackageSet packageSet commitMessage
       Log.notice "Built and released a new package set! Now mirroring to the package-sets repo..."
       Registry.mirrorPackageSet packageSet

app/src/App/GitHubIssue.purs

@@ -41,7 +41,7 @@ import Registry.Foreign.JsonRepair as JsonRepair
 import Registry.Foreign.Octokit (GitHubToken, IssueNumber(..), Octokit)
 import Registry.Foreign.Octokit as Octokit
 import Registry.Internal.Format as Internal.Format
-import Registry.Operation (AuthenticatedData, AuthenticatedPackageOperation(..), PackageOperation(..), PackageSetOperation(..))
+import Registry.Operation (AuthenticatedData, AuthenticatedPackageOperation(..), PackageOperation(..), PackageSetOperation)
 import Registry.Operation as Operation
 import Run (AFF, EFFECT, Run)
 import Run as Run
@@ -86,9 +86,9 @@ runGitHubIssue env = do
   run do
     -- Determine endpoint and prepare the JSON payload
     { endpoint, jsonBody } <- case env.operation of
-      Left packageSetOp@(PackageSetUpdate payload) -> do
+      Left packageSetOp -> do
         -- Sign with pacchettibotti if submitter is a trustee
-        request <- signPackageSetIfTrustee packageSetOp payload
+        request <- signPackageSetIfTrustee packageSetOp
         pure
           { endpoint: "/v1/package-sets"
           , jsonBody: JSON.print $ CJ.encode Operation.packageSetUpdateRequestCodec request
@@ -366,7 +366,7 @@ readOperation eventPath = do
       let keys = CJ.Object.keys object
       let hasKeys = all (flip Array.elem keys)
       if hasKeys [ "packages" ] then
-        map (Left <<< PackageSetUpdate) (CJ.decode Operation.packageSetUpdateCodec json)
+        map Left (CJ.decode Operation.packageSetOperationCodec json)
       else if hasKeys [ "name", "ref", "version" ] then
         map (Right <<< Publish) (CJ.decode Operation.publishCodec json)
       else if hasKeys [ "payload", "signature" ] then
@@ -473,10 +473,9 @@ signPacchettiBottiIfTrustee auth = do
 signPackageSetIfTrustee
   :: forall r
    . PackageSetOperation
-  -> Operation.PackageSetUpdateData
   -> Run (GITHUB + PACCHETTIBOTTI_ENV + GITHUB_EVENT_ENV + LOG + EXCEPT String + r) Operation.PackageSetUpdateRequest
-signPackageSetIfTrustee packageSetOp payload = do
-  let rawPayload = JSON.print $ CJ.encode Operation.packageSetUpdateCodec payload
+signPackageSetIfTrustee packageSetOp = do
+  let rawPayload = JSON.print $ CJ.encode Operation.packageSetOperationCodec packageSetOp
   GitHub.listTeamMembers API.packagingTeam >>= case _ of
     Left githubError -> do
       Log.warn $ Array.fold