Skip to content

test(agent): vitest suite + CI for the agent runner; fix relay error bug#4784

Draft
mmabrouk wants to merge 2 commits into
feat/agent-runner-enginesfrom
chore/agent-runner-test-setup
Draft

test(agent): vitest suite + CI for the agent runner; fix relay error bug#4784
mmabrouk wants to merge 2 commits into
feat/agent-runner-enginesfrom
chore/agent-runner-test-setup

Conversation

@mmabrouk

Copy link
Copy Markdown
Member

Stacks on feat/agent-runner-engines.

What

The agent runner (services/agent) had 8 hand-run tsx/node:assert test scripts, no pnpm test, no CI, and no typecheck gate. This adds a real test setup and wires it into CI.

  • vitest suite: the 8 scripts become tests/unit/*.test.ts (describe/it, node:assert kept), plus vitest.config.ts (node env, junit, v8 coverage) and scripts (test, test:watch, test:coverage, typecheck).
  • CI: new run-services-node-unit-tests job in 12-check-unit-tests.yml (Node 24, frozen install, typecheck, test:unit, publish junit).
  • Typecheck gate: tsconfig include broadened to cover tests/ + vitest.config.ts.
  • Testability seam: server.ts exports createAgentServer(run) and cli.ts exports runCli(raw, stream, io) with an injectable engine, so the HTTP and CLI paths are tested with a fake engine (server.test.ts, cli.test.ts). src/entry.ts isEntrypoint keeps the entrypoints inert when imported.
  • Runner identity: src/version.ts + a richer GET /health ({status, runner, protocol, engines, harnesses}); version 0.0.0 -> 0.1.0.
  • Docs: services/agent/AGENTS.md + the planning workspace under docs/design/agent-workflows/typescript-structure/.

Bug fix

tools/dispatch.ts relayToolCall referenced an undefined callRef in its error/timeout messages — on a Daytona relay failure it would throw ReferenceError: callRef is not defined and mask the real error. Now uses toolName, with a focused relay regression test. The new typecheck gate catches this class of bug.

Result

pnpm test = 10 files / 47 tests pass; tsc --noEmit clean.

Deferred (dependency on a sibling branch)

  • skills.test.ts needs engines/skills.ts (on feat/agenta-on-rivet).
  • wire-contract.test.ts needs the shared Python golden fixtures (on the SDK/service branch).

Both land when those reach this branch.

https://claude.ai/code/session_01GLA8RywSLTGiJvBrDnqZa2

Convert the hand-run tsx/node:assert scripts under services/agent/test/ into a vitest suite
in tests/unit/ (describe/it, node:assert kept). Add vitest + typescript devDeps, scripts
(test/typecheck/coverage), a node-env vitest.config (junit + v8 coverage), and broaden the
tsconfig include so typecheck covers tests + config. Add a run-services-node-unit-tests job to
12-check-unit-tests.yml, and a services/agent/AGENTS.md.

Testability seam: server.ts exports createAgentServer(run); cli.ts exports
runCli(raw, stream, io) with an injectable engine + output sink, so the HTTP and CLI paths are
tested with a fake engine (server.test.ts, cli.test.ts). src/entry.ts isEntrypoint keeps the
entrypoints inert on import. Add src/version.ts + a richer GET /health; version 0.0.0 -> 0.1.0.

Fix a real bug in tools/dispatch.ts: relayToolCall referenced an undefined callRef in its
error/timeout messages (would throw ReferenceError and mask the real failure); use toolName,
plus a focused relay regression test.

Result: 10 test files / 47 tests pass, tsc --noEmit clean. Two tests are deferred because their
deps live on sibling branches: skills.test.ts (needs engines/skills.ts from feat/agenta-on-rivet)
and wire-contract.test.ts (needs the shared Python golden fixtures). They land when those reach
this branch.

Claude-Session: https://claude.ai/code/session_01GLA8RywSLTGiJvBrDnqZa2
@vercel

vercel Bot commented Jun 20, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
agenta-documentation Ready Ready Preview, Comment Jun 20, 2026 11:16pm

Request Review

@coderabbitai

coderabbitai Bot commented Jun 20, 2026

Copy link
Copy Markdown

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 15018f8b-3a1a-4b01-bf34-a5e661e7321d

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch chore/agent-runner-test-setup

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (3)
docs/design/agent-workflows/typescript-structure/research.md (2)

11-35: 💤 Low value

Add language specifier to the fenced code block (lines 11–35).

The file-tree block would benefit from a language tag for better syntax highlighting and markdown linting compliance.

📝 Proposed fix
-```
+```bash
 services/agent/                 standalone pnpm package "agenta-agent-pi-wrapper"

79-82: 💤 Low value

Add language specifier to the fenced code block (lines 79–82).

The structure listing would benefit from a language tag.

📝 Proposed fix
-```
+```text
 code-tool.test.ts   continuation.test.ts   mcp-servers.test.ts   responder.test.ts
services/agent/src/cli.ts (1)

29-34: 💤 Low value

CLI dispatcher logic differs from server.ts.

The runAgent dispatcher here defaults unknown/unset backends to runPi, but server.ts (lines 53-55) falls back to runRivet when request.harness or request.sandbox is set. This means a request with {harness: "claude"} and no explicit backend runs on Rivet via the HTTP path but Pi via CLI.

If this is intentional for CLI simplicity, consider adding a brief comment. If not, align the fallback logic.


ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 3ada8826-4a3b-4dd7-9ed5-525aea827aee

📥 Commits

Reviewing files that changed from the base of the PR and between 394f007 and 7ff3f42.

⛔ Files ignored due to path filters (1)
  • services/agent/pnpm-lock.yaml is excluded by !**/pnpm-lock.yaml
📒 Files selected for processing (35)
  • .github/workflows/12-check-unit-tests.yml
  • .gitignore
  • docs/design/agent-workflows/typescript-structure/README.md
  • docs/design/agent-workflows/typescript-structure/context.md
  • docs/design/agent-workflows/typescript-structure/plan.md
  • docs/design/agent-workflows/typescript-structure/research.md
  • docs/design/agent-workflows/typescript-structure/status.md
  • services/agent/AGENTS.md
  • services/agent/CLAUDE.md
  • services/agent/package.json
  • services/agent/src/cli.ts
  • services/agent/src/entry.ts
  • services/agent/src/server.ts
  • services/agent/src/tools/dispatch.ts
  • services/agent/src/version.ts
  • services/agent/test/code-tool.test.ts
  • services/agent/test/continuation.test.ts
  • services/agent/test/extension-tools.test.ts
  • services/agent/test/mcp-servers.test.ts
  • services/agent/test/responder.test.ts
  • services/agent/test/stream-events.test.ts
  • services/agent/test/tool-bridge.test.ts
  • services/agent/test/tool-dispatch.test.ts
  • services/agent/tests/unit/cli.test.ts
  • services/agent/tests/unit/code-tool.test.ts
  • services/agent/tests/unit/continuation.test.ts
  • services/agent/tests/unit/extension-tools.test.ts
  • services/agent/tests/unit/mcp-servers.test.ts
  • services/agent/tests/unit/responder.test.ts
  • services/agent/tests/unit/server.test.ts
  • services/agent/tests/unit/stream-events.test.ts
  • services/agent/tests/unit/tool-bridge.test.ts
  • services/agent/tests/unit/tool-dispatch.test.ts
  • services/agent/tsconfig.json
  • services/agent/vitest.config.ts
💤 Files with no reviewable changes (8)
  • services/agent/test/extension-tools.test.ts
  • services/agent/test/responder.test.ts
  • services/agent/test/code-tool.test.ts
  • services/agent/test/tool-dispatch.test.ts
  • services/agent/test/tool-bridge.test.ts
  • services/agent/test/stream-events.test.ts
  • services/agent/test/continuation.test.ts
  • services/agent/test/mcp-servers.test.ts

Comment thread .github/workflows/12-check-unit-tests.yml
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant