Skip to content

feat(soul): agent logic upgrade - #149

Merged
elkaix merged 20 commits into
mainfrom
agent-logic-upgrade
Jun 15, 2026
Merged

feat(soul): agent logic upgrade#149
elkaix merged 20 commits into
mainfrom
agent-logic-upgrade

Conversation

@elkaix

@elkaix elkaix commented Jun 15, 2026

Copy link
Copy Markdown
Member

Summary

  • Port blackbox agent-loop observability into the soul runtime: new Wire events (TodoListUpdated, SubagentToolFallback, AgentListDelta, ToolUseSkipped, ContextOverflowRecovered), opt-in token-budget nudge, and context-overflow recovery signaling.
  • Add agent-facing session tooling (ToolSearch, EnterWorktree, ExitWorktree) and opt-in stop-time memory extraction via memory.harvest_on_stop.
  • Integrate the blackbox port branch into main, resolve duplicate merge paths, and refresh tests/snapshots for the updated wire and fail-fast semantics.

Test plan

  • make check-pythinker-code
  • make test-pythinker-code
  • make test (or targeted e2e wire snapshot tests if full suite is too slow)
  • Smoke-test shell/web session: todo updates, subagent fallback, tool dedup skip, and context-overflow recovery events appear on Wire
  • Confirm CodeRabbit review completes before merge

Made with Cursor

Summary by CodeRabbit

  • New Features

    • Added tool discovery search functionality to find available tools by name or description.
    • Added Git worktree management to create and switch between isolated working directories within a session.
    • Added agent list display showing available agent types and their capabilities.
    • Added optional stop-time memory harvesting to stage important decisions and next steps without writing permanent memory files.
    • Added budget spend ceiling warnings to alert before session cost limits are reached.
  • Improvements

    • Enhanced agent briefing guidance with clearer prompt hygiene best practices for spawning new agents.
    • Improved tool result handling with intelligent output budgeting for better context management.
  • Observability

    • Added detailed telemetry events for tool skipping, context overflow recovery, and agent availability changes.

@coderabbitai

coderabbitai Bot commented Jun 15, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Caution

Review failed

Pull request was closed or merged during review

📝 Walkthrough

Walkthrough

This PR adds three new agent tools (ToolSearch, EnterWorktree, ExitWorktree), five new wire event types, a budget continuation nudge, opt-in stop-time memory harvesting, microcompact tool-result body capping, ToolUseSkipped/SubagentToolFallback/ContextOverflowRecovered telemetry emissions, and an AgentListInjectionProvider dynamic injection. Supporting config fields, snapshot updates, and comprehensive tests are included.

Changes

Agent loop observability, new tools, and context management

Layer / File(s) Summary
Wire event types and config schema additions
src/pythinker_code/wire/types.py, src/pythinker_code/config.py, tests/core/test_wire_types.py, tests/core/test_config.py
Five new frozen Pydantic wire event models (TodoListUpdated, SubagentToolFallback, AgentListDelta, ToolUseSkipped, ContextOverflowRecovered) added to the Event union and __all__. Three new config fields added: budget_nudge_ratio, prune_tool_result_max_chars, harvest_on_stop. Round-trip and immutability tests included.
ToolSearch tool
src/pythinker_code/tools/tool_search/__init__.py, src/pythinker_code/tools/tool_search/tool_search.md, src/pythinker_code/agents/default/agent.yaml, tests/tools/test_tool_search.py
Term-scoring search over the visible toolset, capped at max_results, registered in the default agent. Tests cover name/description match, no-match, and hidden-tool exclusion.
EnterWorktree/ExitWorktree tools and set_work_dir_override
src/pythinker_code/tools/worktree/__init__.py, src/pythinker_code/tools/worktree/enter_worktree.md, src/pythinker_code/tools/worktree/exit_worktree.md, src/pythinker_code/soul/toolset.py, tests/tools/test_worktree_tools.py
Root-only approval-gated tools for Git worktree creation and teardown. set_work_dir_override propagates the new cwd to runtime and tools exposing _work_dir. Process-local _ACTIVE_WORKTREES tracks lifecycle. Tests cover success, failure without state change, rejected approval, root-only, and permission profile blocking.
AgentListInjectionProvider
src/pythinker_code/soul/dynamic_injections/agent_list.py, src/pythinker_code/soul/pythinkersoul.py, tests/core/test_agent_list_injection.py, tests/core/test_pythinkersoul_steer.py
Formats builtin agent types into a sorted bullet list, fingerprints to skip redundant injections, emits AgentListDelta over the wire, suppresses for subagents, and resets on compaction or rearm. Registered in PythinkerSoul._injection_providers.
Budget continuation nudge
src/pythinker_code/prompts/__init__.py, src/pythinker_code/soul/pythinkersoul.py, tests/core/test_pythinkersoul_turn_balance.py
BUDGET_CONTINUATION_NUDGE template and _crossed_budget_nudge_threshold/_budget_nudge_message helpers. After each turn, appends a system-reminder when cost crosses budget_nudge_ratio threshold and stop reason is neither budget_exhausted nor auto_continue.
Stop-time memory harvest
src/pythinker_code/soul/pythinkersoul.py, tests/core/test_memory_phase_bcd.py
_harvest_on_stop(history_start_index) extracts notes from recent context history after a stop, appends to scratchpad, and rearms project_memory injection. Gated by memory.harvest_on_stop. Tests cover disabled no-op, opt-in dedup/filtering, append failures, and non-appended note behavior.
Microcompact: cap stale tool result bodies
src/pythinker_code/soul/compaction.py, src/pythinker_code/soul/pythinkersoul.py, tests/core/test_microcompact.py
CAP_PLACEHOLDER and cap_stale_tool_result_bodies() truncate oversized stale tool message bodies with a budget-aware suffix, preserving tool IDs. prune_context() runs capping before full pruning, combining freed char counts.
ToolUseSkipped telemetry
src/pythinker_code/soul/toolset.py, tests/core/test_toolset.py, tests/core/test_toolset_concurrency.py
_emit_tool_use_skipped/_emit_tool_use_skipped_if_opted_in added with TOOL_USE_SKIPPED_REASONS. Emitted for same-step dedup, cross-step dedup, policy/permission block, and PreToolUse hook block, all gated by emits_tool_use_skipped. Concurrency gate docs clarified; unflagged plugin tools default to exclusive execution.
SubagentToolFallback and ContextOverflowRecovered emission
src/pythinker_code/tools/agent/__init__.py, src/pythinker_code/soul/pythinkersoul.py, tests/tools/test_agent_tool.py, tests/core/test_overflow_recovery.py, tests/core/test_pythinkersoul_retry_recovery.py
AgentTool emits SubagentToolFallback before every ToolError return (policy denied, MCP unavailable, timeout, unavailable type, exception). Overflow recovery emits ContextOverflowRecovered(outcome=recovered/failed) and only continues the loop on success (one-shot per turn).
TodoListUpdated wire event
src/pythinker_code/tools/todo/__init__.py, tests/tools/test_todo.py
_emit_todo_list_updated() sends TodoListUpdated after every write and read operation on SetTodoList, computing a complete flag from item statuses.
Prompt hygiene, snapshots, and miscellaneous fixes
src/pythinker_code/tools/agent/description.md, tests/core/test_agent_spec.py, tests/core/test_default_agent.py, tests/tools/test_tool_descriptions.py, src/pythinker_code/llm.py, examples/feedback-worker/src/index.ts, packages/pythinker-review/src/..., .github/workflows/ci-pythinker-host.yml, .cursor/rules/no-co-author-trailers.mdc, CHANGELOG.md, tests_e2e/..., tests/utils/test_pyinstaller_utils.py
Prompt Hygiene section added to agent description. Agent spec and default-agent snapshots updated for three new tools. Alibaba endpoint detection switched to urlparse hostname checks. parseMailbox reimplemented with linear parsing. Version prefix regex stops stripping =. CI workflow gets contents: read permissions. Cursor co-author rule added. All e2e snapshots updated for the new AgentListDelta event.

Sequence Diagram(s)

sequenceDiagram
  participant PythinkerSoul
  participant AgentListInjectionProvider
  participant BudgetNudge
  participant MemoryHarvest
  participant Wire

  PythinkerSoul->>AgentListInjectionProvider: get_injections(history, soul)
  AgentListInjectionProvider->>Wire: send AgentListDelta (if fingerprint changed)
  AgentListInjectionProvider-->>PythinkerSoul: list[DynamicInjection]

  PythinkerSoul->>BudgetNudge: _crossed_budget_nudge_threshold(before, after, ceiling, ratio)
  BudgetNudge-->>PythinkerSoul: True / False
  PythinkerSoul->>PythinkerSoul: append _budget_nudge_message to context

  PythinkerSoul->>MemoryHarvest: _harvest_on_stop(history_start_index)
  MemoryHarvest->>MemoryHarvest: append_scratch_note for each harvested note
  MemoryHarvest->>AgentListInjectionProvider: rearm("project_memory")
Loading

Estimated code review effort

🎯 5 (Critical) | ⏱️ ~120 minutes

Possibly related PRs

  • Pythoughts-labs/pythinker-code#80: Modifies _is_dashscope_endpoint in src/pythinker_code/llm.py, the same function this PR refactors to use urlparse hostname checks.
  • Pythoughts-labs/pythinker-code#89: Extends src/pythinker_code/soul/pythinkersoul.py with dynamic injection providers and adds new wire event models to src/pythinker_code/wire/types.py, the same surfaces this PR modifies.
  • Pythoughts-labs/pythinker-code#75: Adds fields to StatusUpdate wire payload and updates e2e serialization snapshots, overlapping with this PR's wire type and e2e snapshot changes.

Suggested labels

enhancement

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch agent-logic-upgrade

Comment thread .github/workflows/ci-pythinker-host.yml Fixed
Comment thread examples/feedback-worker/src/index.ts Fixed
Comment thread examples/feedback-worker/src/index.ts Fixed
Comment thread src/pythinker_code/llm.py Fixed
Comment thread src/pythinker_code/llm.py Fixed
Comment thread src/pythinker_code/web/api/sessions.py Fixed
Comment thread tests/auth/test_alibaba_auth.py Fixed
Comment thread tests/test_version_lockstep.py Fixed
Comment thread tests/tools/test_web_allowlist_tools.py Fixed
Comment thread tests/core/test_otel_span_tree.py Fixed
Comment thread tests/core/test_otel_span_tree.py Fixed
Comment thread tests/core/test_toolset_concurrency.py Fixed
elkaix and others added 15 commits June 15, 2026 10:33
Add wire events for todo updates, tool skips, subagent fallback, agent-list
deltas, and overflow recovery. Wire opt-in ToolUseSkipped telemetry,
AgentList injection, budget nudge, microcompact pruning, ToolSearch
discovery, session worktree enter/exit, and opt-in harvest-on-stop memory
capture. Update changelog, architecture docs, and tests.
Add opt-in emits_tool_use_skipped hook for cross-step duplicate visibility,
pin exclusive default for unflagged plugin tools, and document the gate contract.
Drop duplicate budget-nudge, ToolUseSkipped emit paths, and conflicting
tests introduced when merging feat/blackbox-agent-loop-port into main.
@elkaix
elkaix force-pushed the agent-logic-upgrade branch from 1d76d0b to c17ac9c Compare June 15, 2026 14:36
Comment thread tests/core/test_toolset.py Fixed
Comment thread tests/core/test_toolset.py Fixed
Comment thread tests/core/test_toolset.py Fixed
Comment thread tests/core/test_toolset.py Fixed
Comment thread tests/core/test_toolset.py Fixed
Comment thread tests/core/test_toolset.py Fixed
Comment thread tests/core/test_toolset.py Fixed
Comment thread tests/core/test_toolset.py Fixed
Comment thread tests/core/test_toolset.py Fixed
Comment thread tests/core/test_memory_phase_bcd.py Fixed
elkaix added 2 commits June 15, 2026 10:58
…upgrade

4 files were unformatted (ruff would-reformat). 5 e2e snapshot tests were
stale after new slash commands (/recap, /accept-edits, /goal, /learn,
/best-practices) and skill entries were added to the wire initialize handshake.
@codecov

codecov Bot commented Jun 15, 2026

Copy link
Copy Markdown

- ci-pythinker-host.yml: add workflow-level `permissions: contents: read`
- llm.py: use urlparse hostname check instead of substring match for dashscope
  and workspace endpoint detection (prevents false positives from path segments)
- dependencies.py: remove duplicate '=' from _VERSION_PREFIX_RE character class
- test_memory_phase_bcd.py: consolidate to single import style for pythinkersoul module
- test_toolset.py: replace standalone awaits with gather/assignment forms
- test_toolset_concurrency.py: remove always-true redundant assertion
- examples/feedback-worker/index.ts: replace .+? with [^<]*? to prevent ReDoS
Comment thread examples/feedback-worker/src/index.ts Fixed
CodeQL flagged the angle-bracket regex in the feedback-worker example as a
polynomial regular expression on uncontrolled header input. Replace it with
an indexOf/slice parse that is linear in input length while preserving the
"Name <addr>" parsing behavior.
@elkaix
elkaix merged commit 63faa85 into main Jun 15, 2026
46 of 47 checks passed
@elkaix
elkaix deleted the agent-logic-upgrade branch June 15, 2026 15:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants