Skip to content

chore: upgrade Node 14 -> 20 (toolchain, published packages, executables) - #2386

Draft
aryanku-dev wants to merge 10 commits into
masterfrom
chore/node-14-to-20
Draft

chore: upgrade Node 14 -> 20 (toolchain, published packages, executables)#2386
aryanku-dev wants to merge 10 commits into
masterfrom
chore/node-14-to-20

Conversation

@aryanku-dev

@aryanku-dev aryanku-dev commented Aug 15, 2026

Copy link
Copy Markdown
Contributor

Draft. Three things are unfinished — see Before this can leave draft.

Moves all three "Node versions" this repo conflates — the toolchain, the floor the published packages declare, and the runtime pkg compiles into the standalone executables — from 14 to 20.19.5.

Node 14 went EOL in April 2023. The pin has already distorted the codebase: image-size was frozen on a vulnerable line to keep Node 14 support (#2301, now resolved by #2382), and the whole snyk-backed IntelliStory lockfile path carries istanbul ignore pragmas whose stated reason is "CI runs the suite on Node 14".

Why this isn't a version-string bump

1. The test harness could not run on Node 20. scripts/loader.js implemented getFormat/getSource/transformSource — removed in Node 16.12.0 — and needed hooks to run in the main thread so it could read global.__MOCK_IMPORTS__ and the memfs volume at fs.$vol. Node 18.19/20 moved module hooks to a dedicated worker.

2. vercel/pkg cannot build a Node 20 binary. Archived at 5.8.1; fails with No available node version satisfies 'node20'.

3. A latent bug that ships the wrong architecture. pkg ran with no --targets, so output arch follows the host. The only thing keeping the binaries x64 was one architecture: x64 line on an arm64 runner — delete it, or let GitHub change the image, and every x64 customer silently gets an arm64 binary while --version passes on the arm64 runner that built it.

The dangerous part: the coverage gate stops existing

Measured on Node 18.20.8 (same hooks architecture as 20), before this change:

$ yarn workspace @percy/logger test:coverage
Executed 150 of 150 specs SUCCESS
All files | 0 | 0 | 0 | 0
EXIT CODE = 0

The removed hooks are ignored with a warning, not an error, so execution is unaffected — the src is already valid modern ESM and Node runs it natively. But .nycrc sets instrument: false, so nyc relies entirely on babel-plugin-istanbul inside the removed hook. Coverage silently goes to zero, and nyc report --check-coverage passes on an empty coverage map because there are no files to check.

Only mock-heavy suites go red. Fix those, see green everywhere with test:coverage succeeding, and the 100% gate has been switched off for every future PR. A half-finished migration's failure mode is success. Exit code is not evidence here — assert the coverage map has content.

This also means the existing test-node20 job (added by PPLT-5844) has been enforcing nothing since it landed.

Verified on real Node 20.19.5

yarn lint clean. yarn build succeeds across 18 projects with the new Babel target.

Suite Result
logger · config · env · cli · cli-command 150 · 82 · 123 · 27 · 189 — all pass
cli-config · cli-build · cli-snapshot · cli-upload 22 · 84 · 28 · 13 — all pass
monitoring · webdriver-utils 40 · 238 — pass
client 288 / 288 pass, 100% coverage
core · sdk-utils see Still failing below

Correction to an earlier revision of this description. It claimed the core/sdk-utils/client failures were pre-existing and not regressions, based on a control worktree that reproduced them at master. That control was on a developer laptop, where those suites fail for local environment reasons — it does not transfer to CI, and master's Windows run at f86a18de (the commit this branches from) is green. The claim was wrong and is retracted.

What a clean control does show, run at master with the original loader on Node 20: @percy/sdk-utils cannot even load — ReferenceError: __dirname is not defined in ES module scope, because Node 20 reparses its CJS-typed test files as ESM once the dead Babel transform stops running. This branch takes it from crash-at-load to 169 specs executing.

A fourth bug was only findable on real Node 20: cli-command passed 189/189 on Node 18 but failed 14 on Node 20 with identical code. Node 20 reports packages/sdk-utils/src/index.js as format: module despite its package having no "type": "module" — while babel.config.cjs keys off that same missing field and compiles it to CommonJS. Node then parsed CommonJS output as an ES module and every export vanished. load() now declares the format Babel actually emitted.

Still failing

  • @percy/sdk-utils — 8 of 169. Suite-internal, not the mechanism: driven through percy exec --testing, the /test/api control plane works correctly when exercised by hand (healthcheck 200, /test/requests accurate, the error directive honoured). The failures cluster on directives not taking effect plus a log level/namespace mismatch ([percy:test] Some error where [percy] Some error is expected), which points at state leaking between specs. Needs dedicated debugging.
  • semgrep/ci — 7 blocking findings across 5 files. Not yet confirmed: semgrep reports to its cloud app rather than the job log. Strong hypothesis — .semgrepignore has no entry for scripts/, and the new loader files contain 15 path.join/resolve/readFileSync calls, i.e. the path-join-resolve-traversal rule every comparable file here is already suppressed for.
  • Build & verify executable. pkg has still not been run locally, so the output filenames under explicit --targets remain unverified and the rename block matches by prefix as a guess. Note executable.sh mutates the tree (gsed -i over every package.json), so it needs a throwaway worktree rather than this one.

Fixed since the first CI run

  • @percy/client — 1 failing → 288 of 288, 100% coverage. Root cause was a real regression, not a test nit: Node 20's AggregateError carries .code but an empty .message, and both client/src/proxy.js and sdk-utils/src/proxy.js classify proxy failures with err.message.includes('ECONNREFUSED'). flattenAggregateError() restores the message Node used to produce.
  • @percy/cli-command (both jobs) — 99.75% → 189 of 189, 100% coverage. intelliStory.js's parser-unavailable bail was only ever covered because Node 14 couldn't install the snyk parser. Marked ignored with the reason stated, because forcing it needs a refactor (module-scope _snykModule cache + static import), not a spec.
  • @percy/core's /ECONNREFUSED/ spec — fixed at source by the same message fix, with no assertion weakened.
  • fail-fast: false on the ubuntu matrix — one red package was cancelling the other 16, which made a 7-failure run render as 21.
  • Karma/browserTest @percy/dom passes; and since rollup.config.js:56 was deliberately left at node: '14', there is no browser-output change to validate.

Open decisions

  • rollup.config.js:56 — the third hardcoded node: '14', compiling the browser bundles. Left untouched on purpose; changing it alters code injected into customer pages.
  • Add a permanent CI assertion that the coverage map is non-empty, so the gate can never silently go vacuous again.

Merge order

#2382 must merge first. Both touch packages/cli-upload/package.json; the engines sweep here rewrites it wholesale and would clobber the image-size removal.

Known breaking changes

  • engines: >=20 across all 18 packages — EBADENGINE on older Node, hard fail under engine-strict. Should ship as a major.
  • dist/ compiled for Node 20 — syntax errors below it. The bin/run.cjs guard turns that into a readable message for CLI users, but not for SDKs importing @percy/core directly.
  • percy-linux.zip now requires glibc ≥2.28 — drops CentOS 7, RHEL 7, Amazon Linux 2, Ubuntu 18.04.
  • Startup-failure exit code 0 → 1.

Also worth flagging: Node 20 reached EOL on 30 April 2026. This improves the posture substantially over Node 14 (three years of CVEs, plus OpenSSL 3 instead of an EOL 1.1.1) but will not clear a scanner rule that flags unsupported majors. Treat it as a known-temporary landing spot and schedule 22/24.

🤖 Generated with Claude Code

aryanku-dev and others added 5 commits August 15, 2026 16:37
scripts/loader.js implemented getFormat/getSource/transformSource, which Node
removed in 16.12.0, and relied on hooks running in the main thread so it could
read global.__MOCK_IMPORTS__ and the memfs volume at fs.$vol. Node 18.19/20 run
module customization hooks on a dedicated worker, so both assumptions are gone.

Measured on Node 18.20.8 before this change: @percy/cli failed 12/27 and
@percy/core 63/1217, while @percy/logger passed 150/150 with 0% coverage and
`nyc report --check-coverage` still exiting 0 -- the removed hooks are ignored
with a warning, not an error, and .nycrc sets instrument:false, so the 100%
gate silently stopped enforcing anything.

- loader-alias.js (new): LOADER_ALIAS extracted, side-effect free. rollup.config.js
  imports it in the main thread; previously importing the hooks module to read a
  regex also installed a global Proxy on every rollup and karma run.
- register-hooks.mjs (new): main-thread half. Owns the mock registry and mirrors
  export names + materialised module paths to a manifest on the real filesystem,
  the one channel both threads share. Loaded via --import.
- loader.js: resolve + load + initialize. Mock values never cross the boundary --
  the generated shim reads global.__MOCK_IMPORTS__ and executes in the main
  thread, so only export names travel.

Public signatures are unchanged (mockfs, fs.$vol, fs.$bypass,
__MOCK_IMPORTS__.set), so no test file changes and no ripple into the SDK repos
that consume the published test/helpers.

Three details worth keeping: resolve() must set shortCircuit on every early
return; load() must fall back to the original source when transformAsync returns
null; and the ?__mock__=<uid> cache-buster must survive, because Node's ESM cache
cannot be invalidated and without it mocks bleed across specs as flakiness.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
.nvmrc/.node-version and the test, windows, lint and typecheck workflows.

20.19.5 rather than a bare 20: module.register() landed in 20.6.0, so 20.0.x-20.5.x
would silently lack the API the test harness now depends on.

Cache namespaces move to a literal node-20/ rather than ${{ matrix.node }}.
test.yml's build and regression jobs have no matrix, so the interpolation would
evaluate empty there and collapse the namespace. setup-node also moves to the
v5.0.0 SHA already pinned in release.yml.

The pre-existing test-node20 job (added by PPLT-5844 to exercise the snyk
lockfile path) is left as-is.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
babel.config.cjs targets node 20 in both overrides, and engines.node becomes
">=20" across all 18 packages (packages/dom gains the field, which it lacked).

This drops support for Node 14/16/18 in the published packages -- a deliberate
breaking change, so this should ship as a major.

rollup.config.js:56 carries a THIRD hardcoded node:'14' target, deliberately
left alone here: it compiles the browser bundles (@percy/dom, @percy/sdk-utils),
so changing it alters code injected into customer pages rather than anything
Node runs. It needs the karma suites to validate and is better decided on its
own.

lockfileDiff.js: correct the rationale only. The lazy require and the cjsRequire
binding both stay -- the dependency is still optional, and the binding exists to
survive the CJS transpile. The comments claimed "the CLI supports Node >=14",
which is no longer true.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Three behaviour changes in Node itself, each of which can break a customer who
is fully up to date.

DNS (Node 17+ defaults dns.lookup to verbatim): `localhost` can resolve ::1
before 127.0.0.1, so snapshotting http://localhost:3000 against an IPv4-only dev
server gets ECONNREFUSED. Restore Node 14 ordering for Percy's own requests by
defaulting verbatim:false in @percy/client's request() and forwarding it through
core's directFetch lookup wrapper.

Deliberately NOT dns.setDefaultResultOrder(): that is process-global, and
@percy/core is imported into SDK consumers' processes -- silently reordering
their DNS is out of bounds. dns is imported lazily, matching the convention the
file already uses for http/https (a top-level import risks a
MISSING_NODE_BUILTINS warning that rollup silently swallows).

Chromium resolves independently, so this is invisible to the browser path and to
every unit test -- it needs the packaged binary against an IPv4-only server.

bin/run.cjs: add the missing .catch() on the startup import chain. Node <15
reported that rejection as a warning and exited 0, which is how a binary that
crashes on startup used to ship; on Node 20 it is a fatal unhandled rejection
with a raw stack trace. Exit codes change for anyone whose CI tolerated a broken
binary.

bin/run.cjs also guards on <20 now. This file must stay parseable by old Node or
the guard is unreachable and consumers get a syntax error instead of the message
-- it is excluded from Babel because cli's `files` is ["bin","dist"].

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
vercel/pkg is archived at 5.8.1 and fails with "No available node version
satisfies 'node20'", so the executable pipeline moves to the maintained
@yao-pkg/pkg fork, pinned at 6.22.0. An unversioned `npm install -g pkg` in the
release path lets the registry decide what compiles the binaries customers
download.

Targets are now explicit: node20-linux-x64,node20-macos-x64,node20-win-x64.
pkg defaults to linux,macos,win at the HOST arch, so the only thing keeping the
binaries x64 today is `architecture: x64` on an arm64 macos-latest runner --
remove that line, or let the runner image change, and every x64 customer
silently gets an arm64 binary while --version passes on the arm64 runner that
built it. With targets pinned, that workflow line is now redundant and removed.

verify-executable.sh gains a `file`-based x86-64 assertion so the guarantee is
enforced rather than remembered, and its header comment is rewritten: it
justified itself by Node 14 turning startup crashes into warnings, which no
longer holds.

The rename block matches by prefix, because pkg appends -<arch> when the target
arch differs from the host. NOT YET VERIFIED against a real run -- see the PR
description.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Comment thread packages/config/test/helpers.js Fixed
Comment thread scripts/loader-alias.js Fixed
Comment thread scripts/loader-alias.js Fixed
Comment thread scripts/loader-alias.js Fixed
Comment thread scripts/loader.js Fixed
Comment thread scripts/loader.js Fixed
Comment thread scripts/loader.js Fixed
aryanku-dev and others added 5 commits August 18, 2026 10:47
…ection

Node >=20 enables autoSelectFamily (Happy Eyeballs) by default, so a failed
connection to a dual-stack host -- `localhost`, or any name with both A and AAAA
records -- rejects with an AggregateError wrapping one error per address instead
of the single error earlier versions produced.

Measured on Node 20.19.5: Node DOES copy `.code` onto the AggregateError, so the
retry gate in request() and percy-idle.js keep working. What it does not carry is
a message -- AggregateError's is the empty string:

    name: AggregateError  code: ECONNREFUSED  message: ""

Percy classifies some failures by message rather than code:
client/src/proxy.js and sdk-utils/src/proxy.js both test
`err.message.includes('ECONNREFUSED')` to decide a proxy is unreachable, and
those checks silently stopped matching. core/test/api.test.js also asserts on
/ECONNREFUSED/ and failed for the same reason.

flattenAggregateError() borrows the first sub-error's message ("connect
ECONNREFUSED 127.0.0.1:5883"), which is exactly what Node used to produce, and
mutates rather than rewraps so callers keep the AggregateError and its `.errors`.
It prefers a retryable cause when the addresses failed for different reasons.

Deliberately NOT applied to @percy/sdk-utils: its only code-based consumer is
percy-idle.js, and since Node preserves `.code` that path never broke -- adding a
second copy of the helper there would be untested surface for no fix.

Also drops a Node-version assertion in proxy.test.js: Node <18 threw
"Invalid URL: invalid-url" and Node >=18 dropped the offending input. Assert on
the prefix we own instead of Node's wording.

@percy/client: 288 of 288 specs, 100% coverage.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The ubuntu matrix had no fail-fast: false, so a single red job cancelled 14
others and `gh pr checks` rendered them all as failures -- which made a
7-failure run look like a 21-failure one. windows.yml already sets this.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Both cli-command jobs failed at 99.75% lines / 99.85% branches while all 189
specs passed. The uncovered lines were intelliStory.js's parser-unavailable bail.

That branch was only ever covered BECAUSE CI ran a Node too old for the feature:
snyk-nodejs-lockfile-parser declares engines >=18, so on Node 14 the optional
install was skipped, loadSnyk() always threw, and the bail was the only path
taken. On Node 20 the parser installs, the lockfile diff succeeds, and the bail
stops executing -- taking its coverage with it.

Marked ignored rather than tested, deliberately, and the comment says why:
forcing it needs a refactor, not a spec. loadSnyk() caches _snykModule at module
scope and intelliStory.js imports diffLockfileDeps statically, so neither the
require nor the import can be made to fail once any earlier spec has loaded the
parser. The comment also warns the next reader not to re-justify the pragma with
"CI runs Node 14", which is what the previous ones said.

Worth noting the direction: SmartSnap/IntelliStory lockfile diffing only ever
worked on Node >=18. On Node 14 any dependency change bailed to a full snapshot
set. This upgrade makes the working path the only path.

cli-command: 189 of 189 specs, 100% coverage.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
semgrep/ci reported 7 blocking findings across 5 files. Confirmed locally with
semgrep 1.173.0 rather than guessed -- the CI job reports findings to the semgrep
cloud app, not the job log, so the hypothesis had to be reproduced:

  javascript.lang.security.audit.path-traversal.path-join-resolve-traversal
    scripts/loader-alias.js       :18,19,21
    scripts/loader.js             :70,70,122
    packages/config/test/helpers.js :111,119,123,124

This is the same rule .semgrepignore already suppresses for lock.js, archive.js,
api.js, maestro-screenshot-file.js and intelliStory.js. The harness is dev-only
and joins paths derived from the repo's own layout -- a hardcoded ROOT, the
workspace package.json `exports` map, an importer's own directory, and the paths
of virtual modules mockfs() created inside a jasmine spec. No request input
reaches any of them.

Added with the caveat recorded in the file: a .semgrepignore entry disables EVERY
rule for the whole file, including rules and code added later, so these should
become line-scoped as soon as the CI semgrep version honors inline nosemgrep.

Also builds the new AggregateError fixtures with plain assignment instead of
Object.assign, so insecure-object-assign has nothing to flag in that file.

@percy/client: 288 of 288 specs, 100% coverage. yarn lint clean.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`Build & verify executable` failed on the real runner with:

    Error! 404: Not Found
    Not found in remote cache:
    Error! Not able to build for 'linux' here, only for 'macos'

@yao-pkg/pkg resolves the `node20` range to a patch that has no prebuilt in
pkg-fetch, so the fetch 404s and it cannot cross-compile. Checked the actual
asset list: prebuilts exist for v20 up to 20.20.2 and v22 up to 22.23.2 with full
linux/macos/win x64 coverage, and node22 is the fork's documented primary target
(its docs list node22/node24, mentioning node18/20 only in passing).

This is the runtime compiled INTO the binary, which is a separate decision from
the `engines` floor the packages declare -- consumers still need >=20; the binary
carries its own Node. Two reasons node22 is the better choice regardless of the
404:

  - Node 20 reached EOL on 30 Apr 2026 and gets no further security releases.
  - Binary users cannot patch the runtime we ship them, unlike npm consumers who
    choose their own Node. It is the one channel where our version choice becomes
    someone else's vulnerability, so shipping a supported runtime matters most
    here.

Not verified locally: `pkg` needs a global install and downloads prebuilts for
three targets, and executable.sh rewrites every package.json via `gsed -i`, so it
cannot run in a live worktree. CI is the check.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants