WasmGC compiler: pyex support shims, robustness, and opt-in auto-feed - #2
Open
ivarvong wants to merge 3 commits into
Open
WasmGC compiler: pyex support shims, robustness, and opt-in auto-feed#2ivarvong wants to merge 3 commits into
ivarvong wants to merge 3 commits into
Conversation
Feeding + shims so real pyex (a Python-3 interpreter) runs on WasmGC without the sandbox trapping on dynamic dispatch it can't statically see: - @stdlib: Inspect.Opts + protocol impls, Macro, Code.Identifier, Path, :filename, String.Tokenizer (so Kernel.inspect / atom classification / pathlib resolve instead of hitting missing-external traps). - runtime shims: Code.ensure_loaded?/1 and :erlang.function_exported?/3 -> true (closed world; else Pyex.Stdlib.fetch/1 rejects every import); Task.async/ yield/shutdown run synchronously (re.safe_regex ReDoS guard, gated on $ftab); :elixir_config.identifier_tokenizer/0 -> String.Tokenizer. Net effect: bad indent / missing file / chained-assign now return clean tracebacks, not traps. Compiler robustness: literal_funs_in/make_fun_refs/atoms_in/term_atoms cons-cell recurse so IMPROPER-list literals (iodata/charlists in Erlang stdlib beams) don't crash the reachability/atom walks. Output-neutral for pyex; conformance 219/219. mix wasm.build: --auto-feed (transitive static closure via BEAM import chunks, with a @no_feed exclude of natively-shimmed modules) and --dce/--no-dce, both opt-in. Measured finding: DCE prunes nothing for pyex (10257/10257 kept; --no-dce is byte-identical) because apply/3 dispatch trips keep-everything, and auto-feeding the whole closure is a pessimization (11.7 -> 60 MB). Size is governed by the curated feeding list, not DCE. Also: cli/ (elw/pyex runners), demo runner tweaks, CI matrix + toolchain pin. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RJjxNdAP4EVmwnibzPXHQe
…he 1200x630 cut) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KuxYKoh8pXEna5ohZUgJsn
The site + wasm API + differential harness need pyex AND beam2wasm; pyex_web composes both via git deps. This repo goes back to being a generic Elixir→WasmGC compiler. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KuxYKoh8pXEna5ohZUgJsn
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Makes the real pyex Python-3 interpreter run on WasmGC without the sandbox trapping on dynamic dispatch the compiler can't see statically.
@stdlibaddsInspect.Opts+ protocol impls,Macro,Code.Identifier,Path,:filename,String.Tokenizer. Runtime shims:Code.ensure_loaded?/1&:erlang.function_exported?/3→true(closed world; else every guestimportis rejected),Task.async/yield/shutdownsynchronous (gated on$ftab),:elixir_config.identifier_tokenizer/0→String.Tokenizer. Net: bad indent / missing file / chained-assign return clean tracebacks, not traps.literal_funs_in/make_fun_refs/atoms_in/term_atomscons-cell recurse so improper-list literals (iodata/charlists in Erlang stdlib beams) don't crash the reachability/atom walks. Output-neutral for pyex.mix wasm.buildflags (opt-in):--auto-feed(transitive static closure via BEAM import chunks + a@no_feedexclude) and--no-dce.Measured finding (the "why DCE?" question)
DCE prunes nothing for pyex:
DCE: kept 10257 of 10257, and--no-dceis byte-identical —apply/3dispatch trips keep-everything. Auto-feeding the whole closure is a pessimization (11.7 → 60 MB, more stubs, more externals). Size is governed by the curated feeding list, not DCE — so the targeted approach stays, with the differential harness (pyex PR) as the net.Tests
Conformance 219/219 bit-exact vs the VM. Live demo unchanged (byte-identical wasm).
🤖 Generated with Claude Code
https://claude.ai/code/session_01RJjxNdAP4EVmwnibzPXHQe