Skip to content

refactor: reposition Caplets around whole-stack agent capabilities - #248

Open
ian-pascoe wants to merge 40 commits into
mainfrom
worktree/rapid-river-8e06
Open

refactor: reposition Caplets around whole-stack agent capabilities#248
ian-pascoe wants to merge 40 commits into
mainfrom
worktree/rapid-river-8e06

Conversation

@ian-pascoe

@ian-pascoe ian-pascoe commented Jul 27, 2026

Copy link
Copy Markdown
Contributor

Summary

  • reposition Caplets as the capability layer that lets coding agents work across the user’s intentionally exposed whole stack
  • rebuild the landing journey around backend breadth, capability composition, portability, sharing, and issue-to-production work
  • align docs, README, catalog, blog, metadata, strategy, domain language, and CTA observability with the new positioning
  • preserve trust boundaries, explicit authorization, catalog safety warnings, and benchmarked efficiency as technical proof

Verification

  • pnpm verify
  • focused landing tests, typechecks, and landing/docs/catalog builds
  • desktop and mobile browser smoke tests
  • accessibility audit: 0 violations
  • two-axis standards and specification review, with findings addressed

Closes #244

Summary by CodeRabbit

  • New Features / Updates
    • Added connection breadth and end-to-end workflow sections.
    • Updated activation with copyable setup steps and Caplet Activation guidance.
    • Reframed Caplets as a whole-stack capability layer for coding agents.
    • Improved Code Mode discovery, reusable call templates, session handling, and concise results.
  • SEO & Sharing
    • Updated catalog and Caplet metadata, including unavailable-page messaging.
    • Clarified that shared Caplets never transfer credentials or authorization.
  • Accessibility & Tracking
    • Added accessibility landmarks and more precise CTA tracking.
  • Tests
    • Expanded coverage for Code Mode behavior and extension-free benchmarking.

@ian-pascoe ian-pascoe added the no changeset No package changeset required label Jul 27, 2026
@coderabbitai

coderabbitai Bot commented Jul 27, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

Caplets is repositioned as a capability layer for coding agents. Public messaging, catalog and landing pages, activation, analytics, Code Mode execution, tests, and benchmark tooling are updated.

Changes

Whole Stack positioning

Layer / File(s) Summary
Language, strategy, and catalog positioning
CONTEXT.md, PRODUCT.md, README.md, STRATEGY.md, apps/docs/..., apps/catalog/..., apps/landing/src/content/blog/...
Defines capability-layer terminology and updates product, documentation, strategy, blog, and catalog messaging.
Landing narrative and page structure
apps/landing/src/components/landing/..., apps/landing/src/data/landing.ts, apps/landing/src/pages/index.astro, apps/landing/src/styles/...
Adds breadth and whole-job sections and revises portability, proof, navigation, accessibility, and page composition.
Landing activation, tabs, and CTA measurement
apps/landing/src/components/landing/Activation.astro, apps/landing/src/components/starwind/tabs/..., apps/landing/src/scripts/observability.ts, apps/landing/test/..., apps/landing/vitest.config.ts
Adds setup modes and tab interactions, updates CTA categorization, and validates rendered activation behavior.
Code Mode discovery and call contracts
packages/core/src/code-mode/...
Adds bounded discovery, compact summaries, template-based calls, concise declarations, and updated runtime types.
Code Mode envelope projection and native integration
packages/core/src/native/..., packages/core/src/code-mode/runner.ts
Projects successful execution results, preserves error metadata, and updates local, composite, remote, and agent-guidance paths.
Behavior validation and benchmark tooling
packages/core/test/..., packages/benchmarks/..., autoresearch.sh, .changeset/...
Updates Code Mode tests, adds token-efficiency benchmarks, isolates semantic judging from extensions, and records a patch release.

Estimated code review effort: 4 (Complex) | ~60 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Agent
  participant CapletHandle
  participant CodeModeAPI
  participant NativeService
  Agent->>CapletHandle: Discover bounded tool summaries
  Agent->>CapletHandle: Call tool with template and overrides
  CapletHandle->>CodeModeAPI: Resolve name and merged arguments
  CodeModeAPI->>NativeService: Execute Code Mode run
  NativeService-->>Agent: Return projected success or error envelope
Loading

Possibly related PRs

Poem

A rabbit hops through stacks so wide,
Caplets gather paths side by side.
Copy, connect, and share with care,
No secrets scattered in the air.
“Whole stack!” we cheer, then leap.

🚥 Pre-merge checks | ✅ 2 | ❌ 3

❌ Failed checks (3 warnings)

Check name Status Explanation Resolution
Linked Issues check ⚠️ Warning The PR meets the positioning goals but changes Code Mode runtime APIs, conflicting with the explicit no-runtime-API-change requirement in [#244]. Remove or isolate the Code Mode API and declaration changes, then verify that the PR preserves existing runtime and security contracts.
Out of Scope Changes check ⚠️ Warning Changes in packages/core/src/code-mode and runtime-api.d.ts add template-based calls and exported behavior beyond the positioning scope in [#244]. Move the Code Mode API changes to a separate issue or revert them from this PR; keep only changes required for positioning and approved verification.
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (2 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly summarizes the PR's main change: repositioning Caplets around whole-stack capabilities for coding agents.
✨ Finishing Touches 💡 1
📝 Generate docstrings 💡
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch worktree/rapid-river-8e06

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@greptile-apps

greptile-apps Bot commented Jul 27, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

Repositions Caplets as a controlled, reusable whole-stack capability layer for coding agents.

  • Rebuilds the landing-page journey, navigation, activation flow, capability examples, and benchmark presentation.
  • Aligns catalog, documentation, README, blog, strategy, terminology, metadata, and CTA analytics with the new positioning.
  • Adds catalog social metadata and expands landing observability tests for explicitly categorized CTAs.

Confidence Score: 4/5

The PR appears safe to merge, with a non-blocking catalog metadata issue that should be corrected to keep shared detail URLs canonical.

The landing interactions and analytics contract remain coherent, while catalog detail pages can emit request-specific Open Graph URLs on preview hosts or requests containing query parameters.

Files Needing Attention: apps/catalog/src/pages/caplets/[entryKey].astro

Important Files Changed

Filename Overview
apps/landing/src/pages/index.astro Reorders the landing journey and adds the breadth and issue-to-production sections without an identified functional issue.
apps/landing/src/components/landing/Activation.astro Moves setup and copy controls into the activation section while preserving working tab and copy initialization.
apps/landing/src/scripts/observability.ts Captures explicitly categorized internal CTAs; the emitted categorical values satisfy the shared event contract.
apps/catalog/src/pages/caplets/[entryKey].astro Adds social metadata, but derives og:url from the request and can publish noncanonical preview or query-bearing URLs.
apps/landing/src/data/landing.ts Updates capability, portability, benchmark, and activation content used throughout the redesigned landing page.

Fix All in Codex

Reviews (1): Last reviewed commit: "fix(marketing): align whole-stack public..." | Re-trigger Greptile

Comment thread apps/catalog/src/pages/caplets/[entryKey].astro Outdated

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🧹 Nitpick comments (1)
apps/landing/test/activation-links.test.ts (1)

8-16: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Test rendered behavior instead of component-source literals.

These checks couple to raw Astro markup and data-* strings. Cover the rendered CTA destination/category and setup-tab behavior through the landing test surface instead.

As per coding guidelines, “avoid tests that merely restate implementation literals or superficial metadata/copy/export details.”

Also applies to: 18-20

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@apps/landing/test/activation-links.test.ts` around lines 8 - 16, Replace the
source-text assertions in the activation-links test with landing-surface
behavior tests that render or exercise Hero.astro and verify the secondary
shared-Caplets CTA resolves to the public catalog and exposes the secondary
category. Also cover the setup-tab interaction and its resulting rendered
behavior through the landing test surface, without reading component source or
asserting raw implementation literals.

Source: Coding guidelines

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@apps/docs/src/content/docs/index.mdx`:
- Line 15: Update the Code Mode description to say that each configured Caplet
becomes a typed caplets.<id> handle, replacing the inaccurate reference to each
configured backend while preserving the rest of the explanation.

In `@STRATEGY.md`:
- Around line 22-23: Update the “Caplet Activation” definition in STRATEGY.md to
require both the first successful Caplet backend operation and a subsequent
connection of a capability from the user’s own stack. Alternatively, split first
execution and activation into distinct events, ensuring retention and activation
cohorts use the milestone that includes the follow-up connection.

---

Nitpick comments:
In `@apps/landing/test/activation-links.test.ts`:
- Around line 8-16: Replace the source-text assertions in the activation-links
test with landing-surface behavior tests that render or exercise Hero.astro and
verify the secondary shared-Caplets CTA resolves to the public catalog and
exposes the secondary category. Also cover the setup-tab interaction and its
resulting rendered behavior through the landing test surface, without reading
component source or asserting raw implementation literals.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 6718a552-0756-4e7d-bd5b-3784aeb08aca

📥 Commits

Reviewing files that changed from the base of the PR and between 76f55bb and 7ded03f.

📒 Files selected for processing (29)
  • CONTEXT.md
  • PRODUCT.md
  • README.md
  • STRATEGY.md
  • apps/catalog/src/pages/caplets/[entryKey].astro
  • apps/catalog/src/pages/index.astro
  • apps/catalog/src/styles/catalog.css
  • apps/docs/src/content/docs/index.mdx
  • apps/landing/src/components/landing/Activation.astro
  • apps/landing/src/components/landing/AgentTrace.astro
  • apps/landing/src/components/landing/BlogCta.astro
  • apps/landing/src/components/landing/Breadth.astro
  • apps/landing/src/components/landing/CommandBlock.astro
  • apps/landing/src/components/landing/Footer.astro
  • apps/landing/src/components/landing/Header.astro
  • apps/landing/src/components/landing/Hero.astro
  • apps/landing/src/components/landing/Proof.astro
  • apps/landing/src/components/landing/RemoteStory.astro
  • apps/landing/src/components/landing/WholeJob.astro
  • apps/landing/src/components/landing/WhyCaplets.astro
  • apps/landing/src/content/blog/why-giant-mcp-tool-walls-dont-scale.md
  • apps/landing/src/data/landing.ts
  • apps/landing/src/layouts/LandingLayout.astro
  • apps/landing/src/pages/blog/index.astro
  • apps/landing/src/pages/index.astro
  • apps/landing/src/scripts/observability.ts
  • apps/landing/test/activation-links.test.ts
  • apps/landing/test/blog-navigation.test.ts
  • apps/landing/test/observability.test.ts
💤 Files with no reviewable changes (1)
  • apps/landing/test/blog-navigation.test.ts

Comment thread apps/docs/src/content/docs/index.mdx Outdated
Comment thread STRATEGY.md Outdated
@github-actions

github-actions Bot commented Jul 27, 2026

Copy link
Copy Markdown
Contributor

@ian-pascoe
ian-pascoe force-pushed the worktree/rapid-river-8e06 branch from 7ded03f to aac5cde Compare July 28, 2026 10:59

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@apps/landing/src/components/landing/Header.astro`:
- Line 23: Update the desktop Workflow link in Header.astro and the mobile Issue
to production link in Header.astro to use the same scroll-offset behavior as the
other anchored section links targeting `#work`, ensuring the destination heading
remains visible below the sticky header.
- Line 36: Update the site-header__actions wrapper in Header.astro so it does
not use the “Project links” aria-label when it contains only the mobile “Open
navigation menu” control; keep the wrapper layout-only or move the label to the
container that exclusively contains the GitHub/npm links.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 05eb1dd5-4627-44de-8a6d-508fa37587ff

📥 Commits

Reviewing files that changed from the base of the PR and between 7ded03f and aac5cde.

📒 Files selected for processing (29)
  • CONTEXT.md
  • PRODUCT.md
  • README.md
  • STRATEGY.md
  • apps/catalog/src/pages/caplets/[entryKey].astro
  • apps/catalog/src/pages/index.astro
  • apps/catalog/src/styles/catalog.css
  • apps/docs/src/content/docs/index.mdx
  • apps/landing/src/components/landing/Activation.astro
  • apps/landing/src/components/landing/AgentTrace.astro
  • apps/landing/src/components/landing/BlogCta.astro
  • apps/landing/src/components/landing/Breadth.astro
  • apps/landing/src/components/landing/CommandBlock.astro
  • apps/landing/src/components/landing/Footer.astro
  • apps/landing/src/components/landing/Header.astro
  • apps/landing/src/components/landing/Hero.astro
  • apps/landing/src/components/landing/Proof.astro
  • apps/landing/src/components/landing/RemoteStory.astro
  • apps/landing/src/components/landing/WholeJob.astro
  • apps/landing/src/components/landing/WhyCaplets.astro
  • apps/landing/src/content/blog/why-giant-mcp-tool-walls-dont-scale.md
  • apps/landing/src/data/landing.ts
  • apps/landing/src/layouts/LandingLayout.astro
  • apps/landing/src/pages/blog/index.astro
  • apps/landing/src/pages/index.astro
  • apps/landing/src/scripts/observability.ts
  • apps/landing/test/activation-links.test.ts
  • apps/landing/test/blog-navigation.test.ts
  • apps/landing/test/observability.test.ts
💤 Files with no reviewable changes (1)
  • apps/landing/test/blog-navigation.test.ts
🚧 Files skipped from review as they are similar to previous changes (25)
  • apps/landing/src/pages/blog/index.astro
  • apps/landing/src/components/landing/Footer.astro
  • apps/landing/src/pages/index.astro
  • CONTEXT.md
  • apps/landing/src/components/landing/CommandBlock.astro
  • apps/landing/src/layouts/LandingLayout.astro
  • apps/landing/src/components/landing/Breadth.astro
  • apps/landing/src/components/landing/AgentTrace.astro
  • apps/landing/src/components/landing/Proof.astro
  • apps/catalog/src/pages/index.astro
  • apps/landing/src/components/landing/WholeJob.astro
  • apps/landing/src/components/landing/Activation.astro
  • apps/landing/test/activation-links.test.ts
  • README.md
  • apps/catalog/src/styles/catalog.css
  • apps/landing/src/components/landing/RemoteStory.astro
  • apps/landing/src/data/landing.ts
  • apps/catalog/src/pages/caplets/[entryKey].astro
  • apps/landing/src/components/landing/WhyCaplets.astro
  • apps/landing/src/components/landing/BlogCta.astro
  • PRODUCT.md
  • apps/docs/src/content/docs/index.mdx
  • apps/landing/src/scripts/observability.ts
  • STRATEGY.md
  • apps/landing/src/components/landing/Hero.astro

Comment thread apps/landing/src/components/landing/Header.astro
Comment thread apps/landing/src/components/landing/Header.astro Outdated

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@apps/landing/src/components/landing/RemoteStory.astro`:
- Line 15: The RemoteStory “Create and share a Caplet” CTA is categorized as
“docs” instead of the intended secondary CTA. Add an explicit
data-cta-category="secondary" attribute to that link in
apps/landing/src/components/landing/RemoteStory.astro; the related CTA in
apps/landing/src/components/landing/Proof.astro requires no direct change.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 2b62414b-4f74-4c18-aff6-fd90ee9c9fee

📥 Commits

Reviewing files that changed from the base of the PR and between aac5cde and 863ebc6.

📒 Files selected for processing (12)
  • apps/landing/src/components/landing/Activation.astro
  • apps/landing/src/components/landing/AgentTrace.astro
  • apps/landing/src/components/landing/CommandBlock.astro
  • apps/landing/src/components/landing/Proof.astro
  • apps/landing/src/components/landing/RemoteStory.astro
  • apps/landing/src/components/landing/WhyCaplets.astro
  • apps/landing/src/data/landing.ts
  • apps/landing/src/pages/index.astro
  • apps/landing/src/scripts/agent-trace.ts
  • apps/landing/src/styles/global.css
  • packages/benchmarks/lib/pi-eval/semantic-judge.ts
  • packages/benchmarks/test/benchmark.test.ts
💤 Files with no reviewable changes (4)
  • apps/landing/src/scripts/agent-trace.ts
  • apps/landing/src/components/landing/AgentTrace.astro
  • apps/landing/src/components/landing/CommandBlock.astro
  • apps/landing/src/pages/index.astro

Comment thread apps/landing/src/components/landing/RemoteStory.astro Outdated
Benchmark entrypoint: bash autoresearch.sh
Goal: without overfitting gpt-5.6 or the pi harness/integration, improve the token efficiency while maintaining the success rate
…guidance while preserving discovery, schema, fallback, synthesis, session, recovery, and compact-output semantics.

Result: {"status":"keep","token_burden":10849,"success_rate":1,"surface_tokens":2342,"workflow_tokens":1481,"workload_calls":4}
…guidance without removing tested discovery, schema, triage, output-shaping, or session semantics.

Result: {"status":"keep","token_burden":10633,"success_rate":1,"surface_tokens":2288,"workflow_tokens":1481,"workload_calls":4}
…ng tool description; retained generated Caplet IDs/descriptions plus every handle operation, result envelope, pagination, and debug reference.

Result: {"status":"keep","token_burden":5305,"success_rate":1,"surface_tokens":956,"workflow_tokens":1481,"workload_calls":4}
…e system prompt with one compact purpose/session/recovery line; dedicated tool guidance remains complete.

Result: {"status":"keep","token_burden":5017,"success_rate":1,"surface_tokens":884,"workflow_tokens":1481,"workload_calls":4}
…sion/recovery invariants while preserving all tested session and recovery phrases.

Result: {"status":"keep","token_burden":4909,"success_rate":1,"surface_tokens":857,"workflow_tokens":1481,"workload_calls":4}
…ile retaining all operations, discovery/schema rules, fallback/error handling, result envelopes, evidence synthesis, and debug access.

Result: {"status":"keep","token_burden":4733,"success_rate":1,"surface_tokens":813,"workflow_tokens":1481,"workload_calls":4}
…on-failure wording after focused verification exposed the contract; all workload calls still pass.

Result: {"status":"keep","token_burden":4749,"success_rate":1,"surface_tokens":817,"workflow_tokens":1481,"workload_calls":4}
…ols guidance marker, removing generic discovery instructions already present at the Code Mode level while retaining each Caplet name and domain description.

Result: {"status":"keep","token_burden":4449,"success_rate":1,"surface_tokens":742,"workflow_tokens":1481,"workload_calls":4}
… no longer receives irrelevant snake_case direct-tool instructions, while direct-tool exposures retain their full discovery/schema/triage guidance.

Result: {"status":"keep","token_burden":4049,"success_rate":1,"surface_tokens":642,"workflow_tokens":1481,"workload_calls":4}
Benchmark entrypoint: bash autoresearch.sh
Goal: Without overfitting the executor model or the Pi harness/integration, reduce total Code Mode input plus output token burden while maintaining 100% success on deterministic workload and the mcp-real-world-large Pi eval suite.
… signatures, removed duplicated REPL guidance, and shortened the session schema description; deterministic success remains 100%.

Result: {"status":"keep","token_burden":3077,"success_rate":1,"surface_tokens":500,"workflow_tokens":1077,"workload_calls":4}
… while preserving metadata on failures and readiness errors; deterministic workload and focused contracts pass.

Result: {"status":"keep","token_burden":2925,"success_rate":1,"surface_tokens":500,"workflow_tokens":925,"workload_calls":4}
…pt guidance while shortening duplicate system-level session text; all focused native and Code Mode contracts pass.

Result: {"status":"keep","token_burden":2913,"success_rate":1,"surface_tokens":497,"workflow_tokens":925,"workload_calls":4}
… fence scaffolding while retaining discovery, exact signatures, error handling, persistence, and recovery semantics.

Result: {"status":"keep","token_burden":2737,"success_rate":1,"surface_tokens":453,"workflow_tokens":925,"workload_calls":4}
…mpressed duplicated prompt guidance while retaining session creation, reuse, and recovery behavior.

Result: {"status":"keep","token_burden":2661,"success_rate":1,"surface_tokens":434,"workflow_tokens":925,"workload_calls":4}
…t, and omitted the redundant sole-tool heading while retaining exact behavior and mixed-tool guidance.

Result: {"status":"keep","token_burden":2589,"success_rate":1,"surface_tokens":416,"workflow_tokens":925,"workload_calls":4}
…protocol and tightened workflow prose without removing discovery, schema, fallback, or evidence guidance.

Result: {"status":"keep","token_burden":2529,"success_rate":1,"surface_tokens":401,"workflow_tokens":925,"workload_calls":4}
…tails remain available. Deterministic success stayed 100%, and the architecture live replay passed in 25 rounds at 632,480 total tokens.

Result: {"status":"keep","token_burden":2529,"success_rate":1,"surface_tokens":401,"workflow_tokens":925,"workload_calls":4}
…, session reuse, and recovery safety semantics. Deterministic burden fell 32 tokens to 2,497; architecture replay passed at 467,242 total tokens across 22 calls, the best observed replay.

Result: {"status":"keep","token_burden":2497,"success_rate":1,"surface_tokens":393,"workflow_tokens":925,"workload_calls":4}
Benchmark entrypoint: bash autoresearch.sh
Goal: Reduce caplets-code-mode total input plus output tokens on the full mcp-real-world-large suite while preserving 100% semantic success and remaining strictly more efficient than executor-mcp and vanilla-mcp, without model-, harness-, or Pi-integration overfitting.
Benchmark entrypoint: bash autoresearch.sh
Goal: Reduce caplets-code-mode total input plus output tokens on the full mcp-real-world-large suite while preserving 100% semantic success and remaining strictly more efficient than executor-mcp and vanilla-mcp, without model-, harness-, or Pi-integration overfitting.
Benchmark entrypoint: bash autoresearch.sh
Goal: Reduce caplets-code-mode total input plus output tokens on the full mcp-real-world-large suite while preserving 100% semantic success and remaining strictly more efficient than executor-mcp and vanilla-mcp, without model-, harness-, or Pi-integration overfitting.
…mode pass: all modes completed 5/5, Caplets used 696,705 average tokens versus Executor 1,131,991 and Vanilla 883,006. Caplets still averaged 25 provider requests, so the template hints preserved correctness but did not yet solve turn serialization.

Result: {"status":"keep","total_tokens":696705,"success_rate":1,"executor_total_tokens":1131991,"vanilla_total_tokens":883006,"executor_ratio":0.6154686742209081,"vanilla_ratio":0.7890150236804733,"average_calls":24}
… all three modes passed 5/5; Caplets used 555,132 average tokens versus Vanilla 569,598 and Executor 1,229,127. Caplets improved 20.3% from the 696,705 baseline and reduced average requests from 25 to 21.

Result: {"status":"keep","total_tokens":555132,"success_rate":1,"executor_total_tokens":1229127,"vanilla_total_tokens":569598,"executor_ratio":0.45164738875641003,"vanilla_ratio":0.9746031411627147,"average_calls":20}
…diagnostics/logs, and session/recovery metadata produced a new best: all modes passed 5/5; Caplets used 466,726 tokens versus Vanilla 900,810 and Executor 979,600. Caplets improved 15.9% over the prior best and 33.0% over baseline, with requests down to 17.

Result: {"status":"keep","total_tokens":466726,"success_rate":1,"executor_total_tokens":979600,"vanilla_total_tokens":900810,"executor_ratio":0.47644548795426706,"vanilla_ratio":0.5181181381201363,"average_calls":16}

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@autoresearch.sh`:
- Around line 4-7: Use a fresh run-scoped directory for output_dir instead of
the shared fixed path, creating it with mktemp -d beneath the existing base
location and including it in cleanup alongside wrapper_dir. Ensure the report
collection and selection logic at the reports assignment and latest-report
selection only sees files generated by the current run.

In `@packages/core/src/code-mode/api.ts`:
- Around line 377-392: Update compactDefaultSearchPage so matching items
identified by summaryMatchesQuery(item, query) are placed before non-matching
items before applying the limit and slice. Preserve the existing limit
selection, input-limit bypass, truncation flag, and page behavior while ensuring
partial-match results retain matching items within the truncated output.

In `@packages/core/src/code-mode/runner.ts`:
- Around line 367-385: Add focused local and remote tests for
projectCodeModeToolEnvelope covering omission of empty diagnostics and logs,
retention of visible logs, preservation of non-empty diagnostics/logs, and
unchanged failure envelopes including error and session metadata. Keep existing
runCodeMode coverage separate and assert the projected response shape directly.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 9cda6d65-d2b9-448c-9238-f236e76ad8c8

📥 Commits

Reviewing files that changed from the base of the PR and between 863ebc6 and ec7e46a.

⛔ Files ignored due to path filters (1)
  • packages/core/src/code-mode/runtime-api.generated.ts is excluded by !**/*.generated.*
📒 Files selected for processing (17)
  • .changeset/tidy-code-mode-token-burden.md
  • autoresearch.sh
  • packages/benchmarks/run-token-efficiency.ts
  • packages/core/src/code-mode/api.ts
  • packages/core/src/code-mode/declarations.ts
  • packages/core/src/code-mode/runner.ts
  • packages/core/src/code-mode/runtime-api.d.ts
  • packages/core/src/code-mode/tool.ts
  • packages/core/src/native/remote.ts
  • packages/core/src/native/service.ts
  • packages/core/src/native/tools.ts
  • packages/core/test/code-mode-api.test.ts
  • packages/core/test/code-mode-mcp.test.ts
  • packages/core/test/code-mode-runner.test.ts
  • packages/core/test/native-remote.test.ts
  • packages/core/test/native.test.ts
  • packages/core/test/serve-session.test.ts

Comment thread autoresearch.sh Outdated
Comment thread packages/core/src/code-mode/api.ts
Comment thread packages/core/src/code-mode/runner.ts
@ian-pascoe ian-pascoe changed the title Reposition Caplets around whole-stack agent capabilities refactor: reposition Caplets around whole-stack agent capabilities Aug 1, 2026

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 4

🧹 Nitpick comments (1)
apps/landing/test/activation-links.test.ts (1)

72-72: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Drop the negative copy assertion.

Line 72 asserts that the agent panel does not contain a specific longer sentence. This restates a copy literal rather than a behavior. The assertion fails on any unrelated rewording of the panel text, and it does not protect the tab-switching contract that the rest of the test covers. The positive assertion on Line 71 already proves the agent panel rendered.

As per coding guidelines: "avoid tests that merely restate implementation literals or superficial metadata/copy/export details".

♻️ Proposed change
     expect(agentPanel.textContent).toContain("Read bootstrap skill");
-    expect(agentPanel.textContent).not.toContain("Read and follow this Caplets bootstrap skill");
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@apps/landing/test/activation-links.test.ts` at line 72, Remove the negative
copy assertion from the activation-links test, specifically the expect on
agentPanel.textContent. Keep the positive rendering assertion and existing
tab-switching behavior checks unchanged.

Source: Coding guidelines

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@apps/catalog/src/pages/caplets/`[entryKey].astro:
- Line 40: Add a real canonical link element alongside the existing og:url
metadata, using canonicalUrl as its href. Keep the current og:url tag unchanged
and ensure the page includes link rel="canonical" for search-engine
canonicalization.

In `@apps/landing/src/components/starwind/tabs/tabs-client.ts`:
- Around line 65-77: Update setupSyncListener and the tab group lifecycle to
retain the registered document listener and remove it on astro:before-swap when
the group leaves the document. Ensure cleanup prevents stale handlers from
retaining detached tabs while preserving synchronization for connected tab
groups.
- Around line 47-50: Harden storage access in TabsHandler: update initializeTab
to use a safe localStorage read that returns null on failure, preserving
fallback to tabs.dataset.defaultValue; in
apps/landing/src/components/starwind/tabs/tabs-client.ts lines 47-50, replace
the direct read, and in lines 102-109 wrap the localStorage.setItem write in
dispatchSyncEvent, dispatching the sync event before the write so storage errors
do not interrupt tab switching.

In `@packages/core/src/code-mode/api.ts`:
- Around line 395-397: Update compactDefaultSearchPage so nextCursor is not
forwarded when the prioritizedItems result is compacted and sliced to limit;
clear or recompute it for the truncated output while preserving the existing
cursor when items remain unchanged. Ensure pagination cannot skip prioritized
items after following the returned cursor.

---

Nitpick comments:
In `@apps/landing/test/activation-links.test.ts`:
- Line 72: Remove the negative copy assertion from the activation-links test,
specifically the expect on agentPanel.textContent. Keep the positive rendering
assertion and existing tab-switching behavior checks unchanged.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 277fd5b2-e179-4d3a-959a-9c0a443a2b75

📥 Commits

Reviewing files that changed from the base of the PR and between ec7e46a and a8f1207.

📒 Files selected for processing (19)
  • CONTEXT.md
  • STRATEGY.md
  • apps/catalog/src/pages/caplets/[entryKey].astro
  • apps/docs/src/content/docs/index.mdx
  • apps/docs/src/content/docs/reference/code-mode-api.mdx
  • apps/landing/src/components/landing/Activation.astro
  • apps/landing/src/components/landing/Header.astro
  • apps/landing/src/components/landing/RemoteStory.astro
  • apps/landing/src/components/landing/WholeJob.astro
  • apps/landing/src/components/starwind/tabs/Tabs.astro
  • apps/landing/src/components/starwind/tabs/tabs-client.ts
  • apps/landing/test/activation-links.test.ts
  • apps/landing/vitest.config.ts
  • autoresearch.sh
  • packages/core/src/code-mode/api.ts
  • packages/core/src/native/service.ts
  • packages/core/test/code-mode-api.test.ts
  • packages/core/test/native-remote.test.ts
  • packages/core/test/native.test.ts
🚧 Files skipped from review as they are similar to previous changes (9)
  • apps/landing/src/components/landing/WholeJob.astro
  • apps/landing/src/components/landing/Header.astro
  • packages/core/test/code-mode-api.test.ts
  • packages/core/test/native-remote.test.ts
  • apps/docs/src/content/docs/index.mdx
  • apps/landing/src/components/landing/Activation.astro
  • autoresearch.sh
  • STRATEGY.md
  • packages/core/src/native/service.ts

Comment thread apps/catalog/src/pages/caplets/[entryKey].astro
Comment thread apps/landing/src/components/starwind/tabs/tabs-client.ts
Comment thread apps/landing/src/components/starwind/tabs/tabs-client.ts
Comment thread packages/core/src/code-mode/api.ts
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

no changeset No package changeset required

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Reposition Caplets around Whole Stack agent capabilities

1 participant