Skip to content

Apps: rspack build prototype#74079

Open
breville wants to merge 10 commits into
stagingfrom
apps-rspack-prototype
Open

Apps: rspack build prototype#74079
breville wants to merge 10 commits into
stagingfrom
apps-rspack-prototype

Conversation

@breville

@breville breville commented Jul 24, 2026

Copy link
Copy Markdown
Member

[written by Claude]

This PR adds an optional, parallel rspack build for apps/ — nothing about the existing build changes. yarn start, yarn build, yarn test, CI, and production builds run exactly as before; the webpack path stays the default and the source of stability. The new system only runs when invoked by its own commands, so folks can try it on real work and fall back instantly, and we can discuss adoption with zero switching pressure. The touch points on shared files are two exported constants in webpack.config.js, added (unused-by-default) scripts in package.json, and one small applab source fix described below.

Rspack is a Rust implementation of webpack: it accepts the webpack 5 config surface and plugin/loader API, so our config translates nearly verbatim rather than being rewritten. It embeds swc ("Speedy Web Compiler", a Rust JS/TS compiler) as a built-in transpiler that can stand in for babel-loader and ts-loader.

The motivation: frontend devs wait ~2.5 minutes for yarn start and 12–47 seconds per rebuild. The rspack dev server starts in ~10 seconds and rebuilds in 6–20 seconds (6–7.5 with the disk copy turned off, below).

rspack.config.js translates createWebpackConfig(): same entries, aliases, loaders (including our EJS loader), externals, named runtime chunk, and splitChunks cache groups. Each deliberate divergence is marked RSPACK-DIFF with rationale. Output parity with the webpack build is 3633/3634 named manifest assets — the one gap is fonts referenced by root-relative url('/fonts/...'), which dashboard serves anyway.

Measured on an 8-core/30GB dev box, full 240-entry dev build, typecheck skipped (both systems run it as a separate process):

webpack rspack + swc
dev-server startup 155s ~10s
rebuild, leaf file edit 12.2s 6.0s
rebuild, util/color.js edit 47.2s 20.3s (7.5s with RSPACK_NO_WRITE=1)
build from scratch: wall / total CPU 130s / 210 CPU-s 10s / 37 CPU-s
dev-server steady-state memory 5.9GB 4.5GB

swc keeps no persistent cache, so its 10-second build IS the from-scratch build. The total-CPU column is the deeper story: rspack+swc does ~6x less compute per build, not just more parallel compute. Rebuilds are bounded by emit, not compilation — a shared-file edit recompiles one module but rewrites every entry bundle it invalidates, which is why rebuilds can exceed a from-scratch startup and why RSPACK_NO_WRITE=1 (skip the dev-server's disk copy; pages load from server memory) cuts the shared-edit rebuild to 7.5s. The disk copy stays on by default because direct-to-Rails access and anything else reading build/package still needs files.

HMR with React Fast Refresh works as under webpack (HOT=1), verified by hot-applying an edit into a live page with no reload and state preserved — so the rebuild rows above are the full editor-save-to-patched-page latency. As with webpack, Fast Refresh hot-swaps function components in place, while edits to class components (most legacy lab UI) fall back to a full re-render through the same hot-update pipe.

The engine swap alone is not the story: rspack running our unchanged babel/ts-loader set only improves the from-scratch build from 130s to 88s. The speedup comes from swc replacing babel-loader and ts-loader. Making that safe required reproducing four babel behaviors the codebase depends on, all in config — no source codemods:

  • lib/add-module-exports-shim-loader.js restores add-module-exports unwrapping and babel's rule of stamping __esModule only on modules with export declarations (unit-tested);
  • isModule: 'unknown' mirrors sourceType: unambiguous, so legacy CJS files stay sloppy scripts instead of gaining "use strict";
  • jsc.assumptions gives loose classes (enumerable prototype methods, which CustomMarshalingInterpreter's for..in marshaling needs) without loose spread, which babel never applies and which passes Sets through [].concat unexpanded.

Two toggles select how much swc takes over: RSPACK_SWC=ts replaces only ts-loader (conservative; the yarn start:rspack default) and RSPACK_SWC=all also replaces babel-loader (the ~10s path). All browser verification was done in RSPACK_SWC=all mode, signed in: App Lab, Game Lab, Dance Party, Sprite Lab, Web Lab 2, Music Lab including playback, and Python Lab including executing code through the pyodide web worker. Applab unit tests pass; a stock-webpack build of the applab entry on this branch succeeds.

The shakedown also surfaced dead code: src/applab/ai/dropletConfig.js (2021, intended to gate the getPrediction block behind level.aiEnabled) has never had an effect — it reads a const before its declaration, which babel silently compiles to [undefined] while swc's spec-compliant helper throws, and two further layers documented in #74080 made the gate a no-op regardless. This PR carries the minimal reorder so the file survives spec-compliant spread; #74080 deletes the module outright and supersedes the reorder once merged.

Example, from apps/:

yarn start:rspack        # dev server on the usual port, proxying to dashboard
yarn build:rspack        # one-shot dev build

Not in scope: production/minified builds, karma, grunt integration, and the circular-dependency allowlist (the rspack plugin reports cycles as warnings rather than failing). webpack.timing.config.js and webpack.rung1.config.js are the measurement harnesses behind the numbers, kept so results are reproducible; two findings from them worth keeping even if rspack is rejected: webpack's own filesystem cache is not viable at our scale (multi-GB on disk, OOM on restore), and experiments.lazyCompilation imports-only cuts stock webpack startup 155s→95s. Separately: fresh webpack builds currently fail the circular-dependency checker on staging — circular_dependencies.json is missing the direct AssetManager ⇄ AudioRecorder cycle.

breville and others added 9 commits July 23, 2026 16:10
rspack.config.js translates createWebpackConfig() to rspack 2.1.5:
same entries, aliases, loaders (including the EJS loader), externals,
named webpack-runtime chunk, and splitChunks cacheGroups.  Webpack-only
plugins map to rspack builtins or forks; divergences are marked
RSPACK-DIFF in the config.  RSPACK_SWC=1 swaps babel/ts-loader for the
builtin swc loader (fast path; output semantics unverified).

webpack.timing.config.js and webpack.rung1.config.js are measurement
harnesses: the former invokes the stock config directly via webpack
CLI, the latter layers filesystem-cache / swc-loader / lazyCompilation
levers behind env flags.

Full dev build, 240 entries, typecheck skipped, this machine:
webpack 127s cold / 58s warm; rspack 86s / 25s; rspack+swc 9s.
Dev-server startup 155s -> 17s (rspack) -> 9s (rspack+swc); HMR
rebuild roughly halves.  webpack filesystem cache is not viable at
this scale: multi-GB on disk, OOM on restore.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Three fixes from loading Dance Party and Music Lab against the rspack
dev server:

- Split RSPACK_SWC into 'ts' and 'all'.  Full-swc breaks at runtime:
  swc has no add-module-exports, and ~128 legacy files mix `import`
  with `module.exports =` (project.js was the first to crash).
  RSPACK_SWC=ts replaces only ts-loader, whose transpileOnly output
  swc matches; js/jsx stays on babel.

- Disable lazyCompilation.  @rspack/cli defaults it on for web
  targets under `serve`; its /_rspack/lazy trigger endpoint loses to
  our catch-all proxy to Rails, so React.lazy chunks 404 and Suspense
  boundaries crash (lab2 ProgressContainer).

- preserveImportMeta in the swc module config.  swc's commonjs
  transform otherwise lowers import.meta.url to a Node-only
  pathToFileURL(__filename), which crashes music lab's patternAi
  worker and silently drops the pyodide/input-service worker chunks.
  With it, `new Worker(new URL(...))` detection works and the named
  worker bundles emit again.

With RSPACK_SWC=ts, Dance Party and Music Lab load with no console
errors beyond the React deprecation noise the webpack build also
shows.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Music lab's play button threw 'Sequencer.newSequence is not a
function'.  CustomMarshalingInterpreter marshals scope objects into
the student-code sandbox with for..in, so it only sees enumerable
properties.  ts-loader's es5 emit assigns methods directly to the
prototype (enumerable); swc's strict emit defines them via
_create_class/defineProperty (non-enumerable), which hides every
method of every class instance handed to the interpreter.

jsc.loose makes swc assign to the prototype like ts-loader does, and
matches TS's useDefineForClassFields:false default at target es5 for
fields too.  Verified: a music program compiles and plays with no
console errors.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Instrumenting dev-server startup exposed that the dominant cost was
not compilation: serving over a populated build/package/js pays ~18s
of writeToDisk reading and comparing the previous 2.8GB of output
before writing (29s startup) versus 11s over an empty dir.  The dev
server serves from memory, so start:rspack now removes the js output
dir first.  start:rspack also defaults to RSPACK_SWC=ts, the
browser-validated hybrid, and build:rspack:swc no longer selects the
known-broken full-swc mode.

lazyCompilation comes back on for dynamic imports only, with
/_rspack/** excluded from the catch-all proxy so the trigger endpoint
reaches the dev server; React.lazy chunks now compile on demand
(verified in music lab).  Deferring entries as well (RSPACK_LAZY_
ENTRIES=1) measures 0.55s startup and ~6.6s to an interactive first
page, but is left off: the entry stub + hot-patch flow races the
Rails inline scripts that synchronously expect entry globals.

For reference, the same imports-only lever on stock webpack cuts
dev-server startup 155s -> 95s (webpack.rung1.config.js).

Net honest numbers, hybrid mode on an 8-core/30GB box: 11s dev-server
startup cold (14x webpack's 155s), HMR unchanged at ~6.7s leaf /
~20.5s shared.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Re-checking the manifest diff after the swc fixes: the ten static/json
assets emit correctly (the earlier gap was an artifact of the broken
full-swc build), worker bundles emit under their webpack names, and
the only remaining difference is three woff2 subsets referenced by
root-relative url('/fonts/...'), which the css-loader url filter skips
on purpose because dashboard serves those paths.  Update the stale
header comment accordingly.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Three babel behaviors reproduced, unlocking the ~9s full build with
no source changes:

- lib/add-module-exports-shim-loader.js post-processes swc output,
  appending the same `module.exports = exports.default` footer that
  babel-plugin-add-module-exports emits for modules whose only export
  is default.  require() sites across src/ depend on it (project.js's
  require('./clientApi').create was the first crash).

- isModule 'unknown' mirrors babel's sourceType 'unambiguous', so
  files without import/export stay sloppy scripts instead of gaining
  "use strict" (the frozen-write class of runtime break).

- jsc.assumptions replaces the blanket `loose: true` in BOTH swc
  rules.  babel only applies loose to transform-classes; swc's global
  loose also degrades spread to [].concat, which passes a Set through
  whole instead of expanding it — Blockly's flyout got an array
  containing a Set and crashed on isDisposed.  The assumptions keep
  class methods as enumerable prototype assignments (which
  CustomMarshalingInterpreter's for..in marshaling requires) while
  spread stays spec-compliant.

Full-swc numbers: 8.9s dev-server startup (clean output dir), HMR
~6.0s leaf / ~20.3s shared — rebuilds are emit-bound, same as every
other mode.  Verified in-browser: Dance Party, Sprite Lab, and Music
Lab including playback, no errors beyond the React deprecation noise
webpack also shows.  start:rspack keeps the ts hybrid default until
'all' has had more hands-on time.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
App Lab crashed at module init under RSPACK_SWC=all.  Two distinct
causes, both babel leniencies the swc path did not yet reproduce:

- src/applab/ai/dropletConfig.js spreads aiBlocks BEFORE its const
  declaration.  babel compiles that spread to [].concat(aiBlocks),
  which silently yields [undefined] on the hoisted var — meaning the
  getPrediction block has never actually registered — while swc's
  spec-compliant helper throws.  Reorder the declarations; the latent
  bug is now fixed under both compilers.

- babel stamps __esModule only on modules with export DECLARATIONS.
  Legacy files mixing `import` with bare `exports.foo =` assignments
  (block_utils.js and friends) carry no marker under babel, so a
  consumer's `import blockUtils from` interop wraps the whole exports
  object as default.  swc stamps the marker on every module-classified
  file, turning those default imports into undefined
  (levels.js: blockUtils.createToolbox).  The shim loader footer now
  rebuilds module.exports without the marker when a module has it but
  declared no ESM exports (detected as: no getter-style export
  properties; the marker itself is non-configurable and cannot be
  deleted in place).

Also adds RSPACK_NO_LAZY=1 to disable lazy dynamic imports when
isolating compilation-order questions; it was needed to rule lazy
compilation out while debugging this.

Verified signed-in: App Lab, Game Lab, Dance, Sprite Lab, Web Lab 2
load clean and Music Lab plays, all under RSPACK_SWC=all.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Covers the three footer behaviors against swc-style CommonJS output:
sole-default unwrapping, __esModule stripping for import-only modules
with manual exports assignments, and leaving genuine named ESM
exports untouched.  Plain CommonJS sources pass through byte-for-byte.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Comments now state the invariant why and stop: discovery narratives,
point-in-time measurements, and validation status move to the PR
description where they belong.  Also corrects a stale claim — the
lazyCompilation comment attributed a 3x startup win to deferred
imports; the win was traced to writeToDisk re-reading a populated
output dir, which the clean-start note already covers.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@breville breville changed the title Apps rspack prototype Apps: rspack build prototype Jul 24, 2026
Incremental rebuild time is dominated by rewriting every entry bundle
an edit invalidates, not by recompiling: a shared-file edit rebuilds
in 7.5s without the disk copy versus 20.3s with it, while a leaf edit
is 6.0s either way — the remaining floor is linking and hot-update
generation across 240 entries.  Pages served through the dev server
load JS from memory, so the copy is skippable; it stays on by default
because direct :3000 access and anything else reading build/package
still needs files on disk.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@breville breville added the AI generated This PR has been substantially generated using AI. label Jul 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

AI generated This PR has been substantially generated using AI.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant