Skip to content

Make server startup and teardown failures diagnosable - #223

Open
subwire wants to merge 1 commit into
mainfrom
server-startup-reliability
Open

Make server startup and teardown failures diagnosable#223
subwire wants to merge 1 commit into
mainfrom
server-startup-reliability

Conversation

@subwire

@subwire subwire commented Jul 26, 2026

Copy link
Copy Markdown
Collaborator

Context

We ran DecLib as the sole decompiler interface for a fleet of autonomous CTF agents at D3CTF 2026 and replayed every session afterwards. 54 of the 62 DecLib errors we logged were these three. None told the user what had actually happened, and each one ended with an agent abandoning the CLI for a hand-rolled IDAPython script.


1. Failed to open database <binary> (14×)

Reproduced, and the cause is not what the message suggests. It is what you get when a second server opens a project database another server already holds:

$ decompiler load /tmp/orig --backend ida          # fine
$ decompiler load /tmp/orig --backend ida --force
Decompiler server 7f5bf7f798 exited with status 1 before registering.
  ERROR - Failed to start server: Failed to open database /tmp/orig

--force is documented as "run a second copy", but the default project dir is derived from binary+backend alone — so the second server always collided with the first and died. Forced copies now get their own project dir:

after: two live servers
  project_dir  .../orig-5a3e46a4
  project_dir  .../orig-5a3e46a4-722556f738

The bare message also reads like file corruption, so any server log carrying it now gets an explanation naming the real cause and the three ways out (--id, --replace, --project-dir).

2. Timed out waiting Ns for server to start (19×)

Two problems.

The advice read "Check backend dependencies (e.g. GHIDRA_INSTALL_DIR)" regardless of backend — actively misleading when the backend is IDA. It is now backend-aware.

And the wait was silent, so a legitimately slow load was indistinguishable from a hang. Agents retried at 180s, then 240s, then 300s — ~12 minutes of solve budget per abandonment. The wait now reports every 10s, surfacing the backend's own last log line:

  [10s] INFO | declib.api.decompiler_server | Using headless interface utilizing ida
  [20s] still starting (5s left)...
Timed out waiting 25s for server 89ec2f75dd to start. Check the IDA install and
that its licence/EULA is accepted (`decompiler backend status ida`).

(That run is a real 137 MB libLLVM.so — genuinely slow rather than stuck, which is now visible.)

3. No running decompiler server matches {...} (21×)

The registry prunes dead records during the lookup, so a server that died left no trace and the error read as "you never started one". list_servers / find_servers can now report what they reaped, and the error names the corpse:

Decompiler server 078c974380 is no longer running (binary: /tmp/orig).
It died or was stopped; its analysis is gone.
Reload it with:
  decompiler load /tmp/orig --backend ida

Deliberately not doing silent auto-revive. Re-analysis can take minutes, and quietly spending them — or quietly discarding unsaved annotations — is a worse surprise than an accurate error naming the exact command to run. Happy to add it behind an explicit flag if you'd prefer.


Tests

Backend-aware hints for every backend (asserting GHIDRA_INSTALL_DIR appears for ghidra and only ghidra); the lock explanation fires on lock failures and not on unrelated ones; the registry reports pruned records with enough detail to name the binary.

🤖 Generated with Claude Code

Three failures accounted for 54 of the 62 DecLib errors we logged across
a CTF where DecLib was the sole decompiler interface. None of them told
the user what had actually happened, and each one ended with an agent
abandoning the CLI for a hand-rolled script.

**"Failed to open database <binary>" (14x).** Reproduced: it is what you
get when a second server opens a project another server already holds.
`--force` is documented as "run a second copy", but the default project
dir is derived from binary+backend alone, so the second server always
collided with the first and died. Forced copies now get their own project
dir, and `--force` works:

    before: Failed to open database /tmp/orig
    after:  two live servers, project_dir orig-5a3e46a4
                                      and orig-5a3e46a4-722556f738

The bare message also reads like file corruption, so any log carrying it
now gets an explanation naming the real cause and the three ways out
(`--id`, `--replace`, `--project-dir`).

**"Timed out waiting Ns for server to start" (19x).** Two problems. The
advice said "Check backend dependencies (e.g. GHIDRA_INSTALL_DIR)"
whatever the backend was -- actively misleading on IDA. It is now
backend-aware. And the wait was silent, so a legitimately slow load was
indistinguishable from a hang; agents retried at 180s, 240s, then 300s,
burning ~12 minutes before giving up. The wait now reports every 10s,
surfacing the backend's own last log line:

    [10s] INFO | declib.api.decompiler_server | Using headless interface utilizing ida
    [20s] still starting (5s left)...
    Timed out waiting 25s ... Check the IDA install and that its licence/EULA
    is accepted (`decompiler backend status ida`).

**"No running decompiler server matches {...}" (21x).** The registry
prunes dead records during lookup, so a server that died left no trace
and the error read as "you never started one". list_servers/find_servers
can now report what they reaped, and the error names the corpse:

    Decompiler server 078c974380 is no longer running (binary: /tmp/orig).
    It died or was stopped; its analysis is gone.
    Reload it with:
      decompiler load /tmp/orig --backend ida

Deliberately not doing silent auto-revive: re-analysis can take minutes,
and quietly spending them (or quietly losing unsaved annotations) is a
worse surprise than an accurate error naming the command to run.

Tests: backend-aware hints for every backend, the lock explanation fires
only on lock failures, and the registry reports pruned records.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant