Make export discoverable, and its output readable - #227
Open
subwire wants to merge 1 commit into
Open
Conversation
`export` was added in #222 to replace the load -> list_functions -> decompile -> decompile -> ... loop that everyone otherwise scripts by hand. Across 60 solve sessions on a 40-binary benchmark it was invoked **zero times**. Not because agents rejected it. SKILL.md -- 589 lines, the doc agents actually read -- never mentions `export`. It says "Always start with `list_functions` and `list_strings`", which is precisely the loop export exists to collapse, so agents did that instead. The four sessions that found the verb at all found it by running `decompiler export --help` inside a batch of `--help` calls at command 60-85 of ~130, long after they had done the work by hand. One of them then hand-wrote eight per-function .c files -- a worse copy of export's own output -- having spent 112 CLI calls to get there. A 117-function binary exports in 31 seconds. Two changes: **SKILL.md** now leads the "first moves" section with export, shows the export-then-grep pattern, and says when *not* to use it (very large binaries, or when you already know the one function you want). The interrogative path is still documented, just no longer the only thing documented. **`--min-size`** makes the output worth reading rather than only grepping. Import stubs dominate the function count of any dynamically linked binary: on the crackme above, 93 of 117 functions were PLT thunks and they were 45% of the exported pseudocode. `--min-size 16` cuts that binary to 24 functions and 108K (from 117 and 480K) while keeping every function a human analyst annotated by hand. Opt-in; default 0 exports everything exactly as before. The manifest reports `skipped_below_min_size` so the trim is never silent. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #222.
exportworks — it is just invisible, and its output is noisier than it needs to be.It was never used
Across 60 solve sessions on a 40-binary benchmark,
exportwas invoked zero times.Not because agents tried it and rejected it.
SKILL.md— 589 lines, the doc agents actually read — never mentionsexport. What it does say is:…which is exactly the loop
exportexists to collapse. So that is what agents did.The four sessions that encountered the verb at all found it by running
decompiler export --helpinside a batch of--helpcalls at command 60–85 of ~130 — help-surface spelunking two-thirds of the way in, after the analysis was already done by hand. None of them ever ran it.One of those sessions then hand-wrote eight per-function
.cfiles — a worse copy of export's own output — having spent 112 CLI calls to get there. That binary exports in 31 seconds.The verb-frequency table for the arm makes the shape of the loss plain:
listbatchloaddecompilelist_functionsexportIt emits mostly stubs
Import stubs dominate the function count of any dynamically linked binary. On the crackme above:
exportNearly half the bytes are
free,werase,curs_set,__cxa_finalize.--filtercannot help here — it matches on name, and on a stripped binary the names areFUN_*. Size is the discriminator that actually separates the two populations, and the backend already knows it.Changes
SKILL.mdleads the "first moves" section with export and the export-then-grep pattern, and says when not to reach for it (very large binaries; or when you already know the one function you want). The interrogative path is unchanged and still documented — it is just no longer the only thing documented.--min-sizetrims the stubs. On the same binary,--min-size 16gives 24 functions / 108K instead of 117 / 480K, while keeping every function a human analyst annotated by hand. Opt-in: default0exports everything exactly as before. The manifest gainsskipped_below_min_sizeso the trim is never silent.Testing
New test asserts the default is unchanged (
skipped_below_min_size == 0), thatexported + skipped == total, that every kept function clears the threshold, and thatmainsurvives.Note: I could not execute it locally — the
fauxwarefixture isn't in my container — so it runs here for the first time in CI. The behaviour it asserts I did verify directly against a real corpus binary (the 117→24 / 480K→108K figures above).🤖 Generated with Claude Code