diff --git a/.gitignore b/.gitignore index ff129039..3baff9b8 100644 --- a/.gitignore +++ b/.gitignore @@ -19,6 +19,7 @@ uv.toml .idea/* .superpowers/ docs/superpowers/ +docs/archive/ # Build dependencies src/pythinker_code/deps/bin diff --git a/CHANGELOG.md b/CHANGELOG.md index 7f0be0c7..a54e30ed 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -15,6 +15,8 @@ GitHub Releases page; `0.8.0` is the new starting line. ## Unreleased +- **Windows in-app update no longer shows a spurious "could not close the program" error.** The native installer now waits for the launching `pythinker.exe` to fully exit (its PID is passed via `/PID`) before its Restart Manager scan runs, so the scan no longer races the launcher's teardown into a false "close the program and retry" dialog. The update already succeeded in that case; now it completes cleanly without the alarming prompt. +- **Simplified the Windows pip/uv/pipx update path.** Now that every shipped Windows install updates through the native installer, the Windows-only detached-spawn upgrade helper is removed; the remaining pip/uv/pipx path (a Windows source checkout, or any macOS/Linux install) runs the upgrade inline like POSIX, surfacing real command output and errors instead of a fire-and-forget process. - **Toggle auto-update from the CLI.** Running `/update` now opens a menu — *Check for updates now* (the default, so a bare `/update` + Enter still checks immediately) or *Auto-update on startup* with its current state — so the toggle is discoverable without knowing a subcommand. `/update auto on|off` still sets it directly, and `/update auto` with no value opens an interactive On/Off picker (cursor defaulted to the current setting). The same toggle appears in the interactive `/settings` panel, and `pythinker info` reports the auto-update status. All surfaces show the *effective* state — an external override (`PYTHINKER_CLI_NO_AUTO_UPDATE` or a source checkout) is surfaced as the reason, renders the `/settings` row read-only, and makes `/update auto` report the read-only state rather than popping a no-op picker, so the toggle is never a silent no-op. ## 0.43.0 (2026-06-13) diff --git a/docs/superpowers/specs/2026-05-05-openai-codex-auth-design.md b/docs/superpowers/specs/2026-05-05-openai-codex-auth-design.md deleted file mode 100644 index cfd7e822..00000000 --- a/docs/superpowers/specs/2026-05-05-openai-codex-auth-design.md +++ /dev/null @@ -1,206 +0,0 @@ -# OpenAI Codex-Compatible Auth Design - -## Goal - -Replace user-facing Pythinker account authentication with OpenAI authentication so Pythinker uses OpenAI/Codex as the default account setup path. - -Pythinker must support three OpenAI setup flows: - -- ChatGPT browser login for Codex-capable ChatGPT accounts. -- ChatGPT headless login using Codex-compatible device-code authentication. -- OpenAI API key setup using the standard OpenAI API. - -The old Pythinker OAuth flow must not remain visible through `pythinker login`, `/login`, or `/setup`. - -## Context - -Pythinker currently has two user-facing auth/setup paths: - -- `pythinker login` and `/login` run Pythinker OAuth in `src/pythinker_code/auth/oauth.py` against `auth.pythinker.com`. -- `/setup` configures API-key platforms through `src/pythinker_code/ui/shell/setup.py` and the platform picker in `src/pythinker_code/auth/platforms.py`. - -Those flows should be replaced at the user-facing layer. OpenAI/Codex login becomes the normal path, and OpenAI API key setup becomes the fallback path for users who do not use ChatGPT managed auth. - -OpenAI API documentation uses API-key authentication for direct API calls. OpenAI Codex CLI also supports ChatGPT managed auth for Codex usage with ChatGPT plans through browser and device-code flows. Pythinker should follow Codex-compatible behavior rather than inventing a separate OpenAI OAuth protocol. - -## User-Facing Behavior - -`pythinker login` becomes the default OpenAI/Codex login entrypoint. With no flags, it starts browser-based ChatGPT/Codex login. - -CLI examples: - -```sh -pythinker login -pythinker login --browser -pythinker login --headless -pythinker login --api-key -``` - -Shell slash examples: - -```text -/login -/login browser -/login headless -/login api-key -/setup -``` - -The commands mean: - -- `pythinker login`, `pythinker login --browser`, `/login`, and `/login browser`: Start Codex-compatible ChatGPT browser login, open the auth URL, receive the localhost callback, store tokens, discover models, and configure OpenAI as default. -- `pythinker login --headless` and `/login headless`: Start Codex-compatible device-code login, print verification URL and user code, poll until authorized, store tokens, discover models, and configure OpenAI as default. -- `pythinker login --api-key`, `/login api-key`, and `/setup`: Prompt for an OpenAI API key, validate it with the OpenAI models endpoint, store it in Pythinker config, discover models, and configure OpenAI as default. - -`/setup` should no longer open a generic platform picker. It should route to OpenAI API-key setup because OpenAI is the default provider. - -## Removed User-Facing Behavior - -The implementation must remove Pythinker account OAuth from normal user-facing login/setup: - -- `pythinker login` must not call `login_pythinker_code`. -- `/login` must not offer or select the `pythinker-code` platform. -- `/setup` must not present the old platform picker. -- Login/help text must not tell users to authenticate with a Pythinker account. - -Existing Pythinker OAuth internals may be deleted during implementation if no tests or runtime code need them. If deletion is too large for the first implementation pass, the code may remain temporarily unused, but no visible command should invoke it. - -## Architecture - -Add a focused OpenAI auth module, for example `src/pythinker_code/auth/openai.py`, to keep OpenAI/Codex-specific logic separate from the old Pythinker OAuth implementation. - -The module should expose event-driven functions compatible with the current `OAuthEvent` rendering style: - -- `login_openai_browser(config) -> AsyncIterator[OAuthEvent]` -- `login_openai_headless(config) -> AsyncIterator[OAuthEvent]` -- `login_openai_api_key(config, api_key: str | None = None) -> AsyncIterator[OAuthEvent]` -- `logout_openai(config) -> AsyncIterator[OAuthEvent]` - -The CLI and shell UI should render these events with the existing JSON-line and terminal status behavior. This preserves the current operational shape while replacing the underlying account provider. - -## OpenAI/Codex Auth Source Of Truth - -The implementation plan must begin by checking the current OpenAI Codex source/docs for auth details. Current Codex app-server docs describe these login modes: - -- `account/login/start` with `type: "apiKey"` for API-key login. -- `account/login/start` with `type: "chatgpt"` for browser ChatGPT managed login. -- `account/login/start` with `type: "chatgptDeviceCode"` for device-code login. - -The implementation should use the current Codex-compatible endpoints, request payloads, token response shape, and refresh behavior found during that research step. Tests must mock those HTTP interactions and must not hit real OpenAI endpoints. - -## Credential Storage - -Use Pythinker’s existing credential storage directory under `get_share_dir()/credentials`. - -Credential keys: - -- `oauth/openai-chatgpt` for ChatGPT managed auth tokens. -- OpenAI API keys remain in `config.toml` as `LLMProvider.api_key`, matching existing API-key provider behavior. - -The ChatGPT token file should preserve enough Codex-compatible fields to support refresh and account metadata. Access tokens, refresh tokens, and API keys must not be written into logs or displayed in terminal output. - -## Provider Configuration - -After successful OpenAI auth, Pythinker should configure OpenAI as the default provider and model. - -For API-key login: - -- Provider key: `managed:openai`. -- Provider type: `openai_responses`. -- Base URL: `https://api.openai.com/v1`. -- API key: user-provided key. -- Model source: list `https://api.openai.com/v1/models` with `Authorization: Bearer `. - -For ChatGPT managed auth: - -- Provider key: `managed:openai-chatgpt`. -- Provider type: a new or extended OpenAI/Codex-compatible provider if ChatGPT managed tokens cannot be used with the existing `openai_responses` provider. -- Store `oauth=OAuthRef(storage="file", key="oauth/openai-chatgpt")` on the provider. -- Store no raw ChatGPT access token in `config.toml`. -- Configure the default model to a Codex/GPT model exposed by the authenticated account. - -The implementation must verify whether the existing `OpenAIResponses` provider can use ChatGPT managed tokens directly. If it cannot, add a distinct provider type rather than overloading `openai_responses` incorrectly. - -## Model Selection - -Model discovery determines which models are configured. Pythinker should write one `LLMModel` entry per discovered model, then set `default_model` using this preference order: - -1. First Codex/GPT coding model exposed by the account. -2. First GPT-5-class model. -3. First available GPT model. -4. First model returned by the discovery endpoint. - -For API-key login, if the models endpoint returns no usable models, the API key must not be saved as the default provider. For ChatGPT managed login, keep valid tokens but do not overwrite `default_model` if model discovery fails. - -## Data Flow - -Browser login: - -1. User runs `pythinker login`, `pythinker login --browser`, `/login`, or `/login browser`. -2. Pythinker starts the Codex-compatible browser auth request. -3. Pythinker opens the returned auth URL in the default browser. -4. The local callback receives the auth result. -5. Pythinker exchanges auth data for tokens. -6. Pythinker stores tokens, refresh metadata, and account metadata. -7. Pythinker discovers available models and writes default provider/model config. - -Headless login: - -1. User runs `pythinker login --headless` or `/login headless`. -2. Pythinker starts the Codex-compatible device-code auth request. -3. Pythinker prints the verification URL and user code. -4. Pythinker polls until success, timeout, or denial. -5. Pythinker stores tokens and configures provider/model on success. - -API-key setup: - -1. User runs `pythinker login --api-key`, `/login api-key`, or `/setup`. -2. Pythinker prompts for a key if none was provided. -3. Pythinker validates the key by listing models. -4. Pythinker saves the managed OpenAI provider and discovered models. -5. Pythinker sets the selected OpenAI model as the default. - -## Logout Behavior - -`pythinker logout` and `/logout` should log out of the active OpenAI-managed provider: - -- For ChatGPT managed auth, delete `oauth/openai-chatgpt` credentials and remove the managed ChatGPT provider/models from config. -- For OpenAI API-key auth, remove the managed OpenAI provider/models from config. -- If the default model pointed at the removed provider, clear it or switch to another configured model. - -Logout should not call the old Pythinker OAuth logout path. - -## Error Handling - -- If browser opening fails, print the URL and continue waiting for the local callback. -- If device-code auth expires, show a clear message and allow the user to retry by rerunning login. -- If device-code auth is disabled for the ChatGPT workspace, print the provider error and recommend browser login or API-key setup. -- If model discovery fails after ChatGPT auth, keep tokens but do not overwrite the current default model. -- If API-key validation returns 401, do not save the key. -- If a ChatGPT token refresh fails with unauthorized, clear the active token cache and tell the user to run `pythinker login` again. -- If ChatGPT managed auth is unavailable in the current environment, `pythinker login --api-key` and `/setup` must remain usable. - -## Testing - -Add tests for: - -- CLI parsing for default browser login, `--browser`, `--headless`, and `--api-key`. -- `/login`, `/login browser`, `/login headless`, `/login api-key`, and `/setup` routing. -- `pythinker login` no longer calls `login_pythinker_code`. -- `/login` and `/setup` no longer open the old platform picker. -- Event rendering for browser, headless, and API-key flows. -- API-key validation success and failure. -- Config writes for managed OpenAI provider/model. -- Token save/load/delete for `oauth/openai-chatgpt`. -- Refresh failure behavior. -- Logout removes OpenAI managed auth and does not call Pythinker OAuth logout. - -Use mocked HTTP responses for auth, token refresh, and model discovery. Do not hit real OpenAI endpoints in tests. - -## Non-Goals - -- Do not scrape ChatGPT web sessions or browser cookies. -- Do not keep Pythinker account OAuth visible in user-facing login/setup. -- Do not store OpenAI API keys outside existing config mechanisms. -- Do not guarantee all ChatGPT subscription tiers expose the same models; model discovery determines what is configured. -- Do not require the external `codex` binary at runtime. diff --git a/docs/superpowers/specs/2026-05-05-opencode-go-auth-design.md b/docs/superpowers/specs/2026-05-05-opencode-go-auth-design.md deleted file mode 100644 index 6ed50c85..00000000 --- a/docs/superpowers/specs/2026-05-05-opencode-go-auth-design.md +++ /dev/null @@ -1,197 +0,0 @@ -# OpenCode Go Auth Design - -## Goal - -Add OpenCode Go as a first-class setup path in Pythinker Code so users with an OpenCode Go subscription can configure the provider from the CLI and use all current OpenCode Go plan models. - -This change is limited to OpenCode Go model access. It does not add Tavily MCP setup, Context7 MCP setup, or generic OpenCode CLI integration. - -## Context - -Pythinker currently has managed provider setup for OpenAI API keys and OpenAI ChatGPT Codex OAuth. That setup writes provider and model entries into `config.toml`, uses managed provider keys, refreshes managed model lists at startup when possible, and exposes login through both `pythinker login` and `/login`. - -OpenCode Go is documented as an API-key-based model provider. Users sign in to OpenCode Zen, subscribe to Go, copy an API key, and connect it in OpenCode. OpenCode Go model IDs use the `opencode-go/` form in OpenCode config. The current API base path is `https://opencode.ai/zen/go/v1`. - -The OpenCode Go model set is mixed: - -- Most models use an OpenAI-compatible `chat/completions` API. -- MiniMax M2.5 and MiniMax M2.7 use an Anthropic-compatible `messages` API. - -Because Pythinker already has provider implementations for OpenAI-compatible and Anthropic-compatible APIs, the smallest complete design is to configure two managed providers that share the same OpenCode Go API key. - -## User-facing behavior - -Add a dedicated OpenCode Go setup mode without changing the default OpenAI login behavior. - -CLI examples: - -```sh -pythinker login --opencode-go -``` - -Shell examples: - -```text -/login opencode-go -``` - -The command prompts for the OpenCode Go API key, validates or accepts it, writes managed providers and model aliases, and reloads the shell after successful setup. - -The setup must also support environment-provided keys. If a key is not provided interactively, use the first available value from: - -1. `OPENCODE_GO_API_KEY` -2. `OPENCODE_API_KEY` -3. `OPENCODE_ZEN_API_KEY` - -The dedicated OpenCode Go mode must not replace `pythinker login`, `pythinker login --browser`, `pythinker login --headless`, `pythinker login --api-key`, `/login`, `/login browser`, `/login headless`, or `/login api-key`. - -## Provider configuration - -Add a provider family ID for model aliases and telemetry: - -```text -opencode-go -``` - -Configure two managed provider entries. These provider keys are intentionally distinct because the APIs are not wire-compatible: - -- `managed:opencode-go-openai` for OpenAI-compatible models. -- `managed:opencode-go-anthropic` for Anthropic-compatible models. - -Both providers use the same API key and the same base URL: - -```text -https://opencode.ai/zen/go/v1 -``` - -The OpenAI-compatible provider uses provider type `openai_legacy`, because OpenCode Go exposes `chat/completions` for these models. The Anthropic-compatible provider uses provider type `anthropic`, because MiniMax models use `messages`. - -Model aliases must follow OpenCode's documented form and use the provider family ID, not the internal provider key: - -```text -opencode-go/ -``` - -The `LLMModel.model` field must contain the raw model ID expected by the API, for example `kimi-k2.6`. - -The implementation must not rely on `managed_provider_key("opencode-go")` for this provider family, because one alias family maps to two provider implementations. Add small OpenCode Go-specific helpers for provider keys, model alias construction, and removal instead of stretching the generic managed-platform helpers. - -## Models - -Configure the current official OpenCode Go model list: - -| Alias | API model ID | Provider type | Display name | -| --- | --- | --- | --- | -| `opencode-go/glm-5` | `glm-5` | `openai_legacy` | GLM-5 | -| `opencode-go/glm-5.1` | `glm-5.1` | `openai_legacy` | GLM-5.1 | -| `opencode-go/kimi-k2.5` | `kimi-k2.5` | `openai_legacy` | Kimi K2.5 | -| `opencode-go/kimi-k2.6` | `kimi-k2.6` | `openai_legacy` | Kimi K2.6 | -| `opencode-go/deepseek-v4-pro` | `deepseek-v4-pro` | `openai_legacy` | DeepSeek V4 Pro | -| `opencode-go/deepseek-v4-flash` | `deepseek-v4-flash` | `openai_legacy` | DeepSeek V4 Flash | -| `opencode-go/mimo-v2-pro` | `mimo-v2-pro` | `openai_legacy` | MiMo-V2-Pro | -| `opencode-go/mimo-v2-omni` | `mimo-v2-omni` | `openai_legacy` | MiMo-V2-Omni | -| `opencode-go/mimo-v2.5-pro` | `mimo-v2.5-pro` | `openai_legacy` | MiMo-V2.5-Pro | -| `opencode-go/mimo-v2.5` | `mimo-v2.5` | `openai_legacy` | MiMo-V2.5 | -| `opencode-go/qwen3.5-plus` | `qwen3.5-plus` | `openai_legacy` | Qwen3.5 Plus | -| `opencode-go/qwen3.6-plus` | `qwen3.6-plus` | `openai_legacy` | Qwen3.6 Plus | -| `opencode-go/minimax-m2.5` | `minimax-m2.5` | `anthropic` | MiniMax M2.5 | -| `opencode-go/minimax-m2.7` | `minimax-m2.7` | `anthropic` | MiniMax M2.7 | - -Use conservative context sizes from official or third-party metadata when available. If metadata is unavailable, use the documented public values surfaced by current model-router docs: - -- `mimo-v2.5` and `mimo-v2.5-pro`: 1,000,000 tokens. -- `qwen3.5-plus` and `qwen3.6-plus`: 262,000 tokens. -- `minimax-m2.5` and `minimax-m2.7`: 205,000 tokens. -- Other OpenCode Go models: 262,000 tokens. If model discovery returns a numeric context length for an official model, prefer that discovered value. - -Set the default model to `opencode-go/kimi-k2.6` after successful setup. If that model is not present because the discovered list changes, use the first configured OpenAI-compatible model. - -## Model discovery - -OpenCode docs list `https://opencode.ai/zen/go/v1/models` as the model metadata endpoint, but current third-party reports indicate the endpoint may return `404` for some valid Go API keys. The implementation must treat model discovery as best effort. - -The setup flow must: - -1. Try to list models from `https://opencode.ai/zen/go/v1/models` with `Authorization: Bearer `. -2. If the endpoint returns a usable list, map returned IDs into the provider split above. -3. If listing fails with `404`, non-JSON HTML, timeout, or a non-auth server error, fall back to the static official model list. -4. If listing fails with `401` or `403`, do not save the key. - -This keeps the CLI usable even when the model metadata endpoint is unavailable while still rejecting clearly invalid credentials. - -## Validation - -API key validation must avoid generation requests. - -Preferred validation order: - -1. `GET /models` against the Go base URL. -2. If that is unavailable for non-auth reasons, accept the key and configure the static model list with a success message that model listing was unavailable. -3. If the response is `401` or `403`, show a clear error and do not save the key. - -Do not log or display the API key. JSON event output must redact secrets. - -## Data flow - -CLI setup: - -1. User runs `pythinker login --opencode-go`. -2. Pythinker reads an environment key or prompts for one. -3. Pythinker attempts model discovery and validates auth failures. -4. Pythinker writes both managed providers and all known OpenCode Go model aliases. -5. Pythinker sets `default_model` to `opencode-go/kimi-k2.6` or a safe fallback. -6. Pythinker reports success with the default model. - -Shell setup: - -1. User runs `/login opencode-go`. -2. Pythinker prompts for the API key when no environment key exists. -3. Pythinker renders setup events through the existing OAuth event renderer. -4. On success, Pythinker tracks a `login` event with provider `opencode-go`, clears the console, and reloads the shell. - -## Logout behavior - -Existing OpenAI logout behavior must not remove OpenCode Go providers unless the user is explicitly logging out of OpenCode Go. - -Add both user-facing logout options: - -- `pythinker logout --opencode-go` -- `/logout opencode-go` - -Logout must remove both managed OpenCode Go providers and all models whose provider is one of those provider keys. If `default_model` points to a removed model, set it to the first remaining configured model or clear it. - -## Error handling - -- If the user enters an empty API key and no environment key exists, print `OpenCode Go API key is required.` -- If model discovery returns `401` or `403`, print `Invalid OpenCode Go API key; the key was not saved.` -- If model discovery is unavailable for non-auth reasons, configure the static official model list and print a warning-level event before the success event. -- If a user selects a MiniMax model and the Anthropic-compatible request fails due to provider incompatibility, surface the provider error and leave the configured model intact. -- If OpenCode changes model IDs, startup refresh must preserve existing config until a successful replacement list is available. Startup refresh may be implemented as an OpenCode Go-specific path; it does not need to reuse `refresh_managed_models` if that would blur the two-provider split. - -## Testing - -Add tests for: - -- CLI parsing for `pythinker login --opencode-go`. -- Shell routing for `/login opencode-go`. -- Environment key precedence: `OPENCODE_GO_API_KEY`, then `OPENCODE_API_KEY`, then `OPENCODE_ZEN_API_KEY`. -- Config writes for both OpenCode Go managed providers. -- All current official model aliases are created. -- MiniMax models are assigned to the Anthropic-compatible provider. -- Other OpenCode Go models are assigned to the OpenAI-compatible provider. -- Default model selection prefers `opencode-go/kimi-k2.6`. -- `401` and `403` model discovery failures do not save the key. -- Non-auth model discovery failures fall back to the static official model list. -- API keys are not included in event messages or JSON output. -- OpenCode Go logout removes only OpenCode Go providers/models and leaves OpenAI providers intact. - -All tests must mock network calls. No tests should call real OpenCode endpoints. - -## Non-goals - -- Do not add Tavily MCP setup. -- Do not add Context7 MCP setup. -- Do not change the default `pythinker login` behavior. -- Do not require the external `opencode` binary. -- Do not integrate with OpenCode's local credential file. -- Do not add a custom provider abstraction unless existing provider types cannot support the documented endpoints. diff --git a/docs/superpowers/specs/2026-05-06-anthropic-direct-auth-design.md b/docs/superpowers/specs/2026-05-06-anthropic-direct-auth-design.md deleted file mode 100644 index 8744a6d9..00000000 --- a/docs/superpowers/specs/2026-05-06-anthropic-direct-auth-design.md +++ /dev/null @@ -1,184 +0,0 @@ -# Anthropic Direct Auth Design - -## Goal - -Add Anthropic (direct API) as a managed provider in Pythinker Code so users with an Anthropic Console API key can configure the provider from the CLI and use the current Claude frontier model lineup (Opus 4.7, Sonnet 4.6, Haiku 4.5). - -This change is limited to the direct Anthropic API (`https://api.anthropic.com`). It does not configure any third-party gateway (Bedrock, Vertex, Foundry, Portkey, etc.) and does not add any non-Anthropic provider. - -## Context - -Pythinker already has managed setup for OpenAI, OpenCode Go, MiniMax, and (in the prior plan) DeepSeek. Anthropic follows the same pattern: a single API key authorizes the Messages API, and the existing Pythinker `anthropic` provider type covers the wire format. Provider construction with the `anthropic` type at a configured base URL was already proven during the OpenCode Go work. - -The key difference from prior providers: Anthropic uses `x-api-key` + `anthropic-version` headers natively, not `Authorization: Bearer`. The `anthropic` provider wrapper handles this internally. - -The new module is named `auth/anthropic_direct.py` (not `auth/anthropic.py`) to disambiguate from the `anthropic` wire-format string used by other managed providers' `LLMProvider.type` field. - -## User-facing behavior - -CLI: - -```sh -pythinker login --anthropic -pythinker logout --anthropic -``` - -Shell: - -```text -/login anthropic -/logout anthropic -``` - -The command prompts for the Anthropic API key (hidden input), validates it via best-effort model discovery, writes the managed provider and model aliases, and reloads the shell after successful setup. - -The setup must also support an environment-provided key. If a key is not provided interactively, use: - -1. `ANTHROPIC_API_KEY` (only documented Anthropic env var). - -The dedicated Anthropic mode must not change any existing login or logout flow. - -## Provider configuration - -Provider family ID: - -```text -anthropic -``` - -Configure ONE managed provider entry: - -- `managed:anthropic` — type `anthropic`, base URL `https://api.anthropic.com`. - -(Note: Anthropic's official SDK appends `/v1/messages` internally; the base URL must NOT include `/v1`. Verify the pythinker_core `anthropic` provider wrapper follows the same convention before writing tests.) - -Model aliases follow the project convention: - -```text -anthropic/ -``` - -The `LLMModel.model` field stores the EXACT API model ID (e.g. `claude-opus-4-7`); the alias uses a lowercase short form (`anthropic/claude-opus-4-7` — verbatim, since the API model ID is already in the alias-friendly format). - -## Models - -Configure exactly three current frontier text models: - -| Alias | API model ID | Provider type | Display name | Max context | -| --- | --- | --- | --- | --- | -| `anthropic/claude-opus-4-7` | `claude-opus-4-7` | `anthropic` | Claude Opus 4.7 | 1_000_000 | -| `anthropic/claude-sonnet-4-6` | `claude-sonnet-4-6` | `anthropic` | Claude Sonnet 4.6 | 200_000 | -| `anthropic/claude-haiku-4-5` | `claude-haiku-4-5-20251001` | `anthropic` | Claude Haiku 4.5 | 200_000 | - -Notes: - -- Opus 4.7 has a 1M-token context window per Anthropic's transparency hub (released 2026-04-16). -- Sonnet 4.6 and Haiku 4.5 use the standard 200K context window. -- The Haiku 4.5 model ID intentionally includes the `-20251001` date suffix because Anthropic ships dated Haiku snapshots; Opus and Sonnet have stable aliases without dates. -- Older models (Opus 4.6, Opus 4.1, Sonnet 4.5, Sonnet 4, Haiku 4) are excluded — frontier-only catalog. - -Set the default model to `anthropic/claude-opus-4-7` after successful setup. If `claude-opus-4-7` is not present in the discovered list, fall back to the first configured Anthropic model (preserving the static catalog order above). - -If model discovery returns numeric `context_length` values for any of the three official models, prefer that discovered value over the static defaults. - -## Model discovery - -Anthropic exposes `https://api.anthropic.com/v1/models` (per platform.claude.com docs). The setup flow must: - -1. Try to list models from `https://api.anthropic.com/v1/models` with `x-api-key: ` and `anthropic-version: 2023-06-01` headers. -2. If the endpoint returns a usable list, map known IDs (the three current frontier IDs above) into the static catalog, optionally overriding `max_context_size` and `display_name`. -3. If listing fails with `404`, non-JSON, timeout, or a non-auth server error, fall back to the static catalog and emit an `info` event before success. -4. If listing fails with `401` or `403`, do not save the key. Emit an `error` event with `"Invalid Anthropic API key; the key was not saved."`. - -Discovery is best-effort. Tests must mock all network calls. - -Note: the response format follows OpenAI's `{"object": "list", "data": [...]}` shape; the parser shares the same structure as DeepSeek's and MiniMax's parsers. - -## Validation - -API-key validation must avoid generation requests. - -Preferred validation order: - -1. `GET https://api.anthropic.com/v1/models` with `x-api-key` and `anthropic-version` headers. -2. If unavailable for non-auth reasons, accept the key and configure the static model list with an info-level event. -3. If the response is `401` or `403`, show a clear error and do not save the key. - -Do not log or display the API key. JSON event output must redact secrets. - -## Data flow - -CLI setup: - -1. User runs `pythinker login --anthropic`. -2. Pythinker reads `ANTHROPIC_API_KEY` from the environment or prompts for one (hidden input). -3. Pythinker attempts model discovery and validates auth failures. -4. Pythinker writes the managed provider entry and the three Anthropic model aliases. -5. Pythinker sets `default_model` to `anthropic/claude-opus-4-7` or the first configured Anthropic model. -6. Pythinker reports success with the default model. - -Shell setup: - -1. User runs `/login anthropic`. -2. Pythinker prompts for the API key when no environment key exists. -3. Pythinker renders setup events through the existing OAuth event renderer. -4. On success, Pythinker tracks a `login` event with provider `anthropic`, clears the console, and reloads the shell. - -## Logout behavior - -Existing OpenAI, OpenCode Go, MiniMax, and DeepSeek logout behavior must not remove Anthropic providers. Add both user-facing logout options: - -- `pythinker logout --anthropic` -- `/logout anthropic` - -Logout removes the `managed:anthropic` provider entry and every model whose `provider` field equals that key. If `default_model` points to a removed Anthropic model, set `default_model` to the first remaining configured model or clear it (`""`). - -## Error handling - -- If the user enters an empty API key and no environment key exists, print `Anthropic API key is required.` -- If model discovery returns `401` or `403`, print `Invalid Anthropic API key; the key was not saved.` -- If model discovery is unavailable for non-auth reasons (timeout, 404, 5xx, malformed JSON), configure the static model list and print an info-level event before the success event. - -## Testing - -Add tests for: - -- Model catalog completeness (three aliases, exact API IDs, all assigned to `managed:anthropic`). -- Env resolution for `ANTHROPIC_API_KEY`. -- Config writes for the single Anthropic managed provider. -- All current model aliases are created. -- Default model selection prefers `anthropic/claude-opus-4-7`. -- `401` and `403` model discovery failures do not save the key. -- Non-auth model discovery failures fall back to the static official model list with an info event before success. -- Discovery sends both `x-api-key` and `anthropic-version` headers (verified via aiohttp request mocking). -- API keys are not included in event messages or JSON output. -- Anthropic logout removes only Anthropic providers/models. -- CLI parsing for `pythinker login --anthropic` and `pythinker logout --anthropic`. -- Shell routing for `/login anthropic` and `/logout anthropic`. - -All tests must mock network calls. - -Provider construction (`create_llm` for the `anthropic` provider type at a custom base URL) was verified during the OpenCode Go work. That coverage is sufficient. - -## Provider chooser update - -Append `Anthropic` as the next sequential option in the `/login` shell chooser. After this change (assuming DeepSeek already landed) the chooser exposes 7 options: - -``` -1. OpenAI ChatGPT (browser) -2. OpenAI ChatGPT (device code) -3. OpenAI API key -4. OpenCode Go -5. MiniMax -6. DeepSeek -7. Anthropic -``` - -## Non-goals - -- Do not configure third-party Anthropic gateways (Bedrock, Vertex, Foundry, Portkey, LaoZhang, etc.). -- Do not add Anthropic OAuth login (the Anthropic Console requires API keys for programmatic access). -- Do not add Anthropic non-text models or beta endpoints (Files, Skills, Agents, Sessions are out of scope). -- Do not add legacy Claude models (3.x, Sonnet 4.5, Opus 4.6, Opus 4.1). -- Do not change any existing login or logout behavior. -- Do not require the `anthropic` SDK; the project's existing `anthropic` provider wrapper is sufficient. diff --git a/docs/superpowers/specs/2026-05-06-deepseek-auth-design.md b/docs/superpowers/specs/2026-05-06-deepseek-auth-design.md deleted file mode 100644 index 9b9d08bb..00000000 --- a/docs/superpowers/specs/2026-05-06-deepseek-auth-design.md +++ /dev/null @@ -1,175 +0,0 @@ -# DeepSeek Auth Design - -## Goal - -Add DeepSeek as a managed provider in Pythinker Code so users with a DeepSeek API key can configure the provider from the CLI and use the current DeepSeek V4 model family (V4-Pro and V4-Flash). - -This change is limited to DeepSeek's OpenAI-compatible chat-completions API for the V4 model family. It does not configure DeepSeek's Anthropic-compatible endpoint, does not include legacy `deepseek-chat` / `deepseek-reasoner` aliases (deprecated 2026-07-24), and does not configure any non-DeepSeek provider. - -## Context - -Pythinker already has managed setup for OpenAI, OpenCode Go, and MiniMax. DeepSeek follows the same pattern: a single API key authorizes a chat-completions endpoint, and the existing Pythinker `openai_legacy` provider type covers the wire format. - -DeepSeek exposes both an OpenAI-compatible endpoint at `https://api.deepseek.com/v1` and an Anthropic-compatible endpoint at `https://api.deepseek.com/anthropic`. We configure only the OpenAI-compatible endpoint because: - -- Every DeepSeek V4 model is exposed on the OpenAI-compatible path, including thinking mode (via `reasoning_effort`). -- A single managed provider keeps the model alias namespace single-routed and simpler. -- Users who want native Anthropic streaming with `thinking` blocks can edit `config.toml` directly. - -## User-facing behavior - -Add a dedicated DeepSeek setup mode without changing any existing login flow. - -CLI: - -```sh -pythinker login --deepseek -pythinker logout --deepseek -``` - -Shell: - -```text -/login deepseek -/logout deepseek -``` - -The command prompts for the DeepSeek API key (hidden input), validates it via best-effort model discovery, writes the managed provider and model aliases, and reloads the shell after successful setup. - -The setup must also support an environment-provided key. If a key is not provided interactively, use: - -1. `DEEPSEEK_API_KEY` (only documented DeepSeek env var). - -The dedicated DeepSeek mode must not change `pythinker login`, `pythinker login --browser`, `pythinker login --headless`, `pythinker login --api-key`, `pythinker login --opencode-go`, `pythinker login --minimax`, `/login`, or any existing `/login ` route. - -## Provider configuration - -Provider family ID: - -```text -deepseek -``` - -Configure ONE managed provider entry: - -- `managed:deepseek` — type `openai_legacy`, base URL `https://api.deepseek.com/v1`. - -Model aliases follow the project convention: - -```text -deepseek/ -``` - -The `LLMModel.model` field stores the EXACT API model ID (e.g. `deepseek-v4-pro`); the alias uses a lowercase short form (`deepseek/v4-pro`). - -## Models - -Configure exactly two current text models: - -| Alias | API model ID | Provider type | Display name | Max context | -| --- | --- | --- | --- | --- | -| `deepseek/v4-pro` | `deepseek-v4-pro` | `openai_legacy` | DeepSeek V4 Pro | 128_000 | -| `deepseek/v4-flash` | `deepseek-v4-flash` | `openai_legacy` | DeepSeek V4 Flash | 128_000 | - -Legacy `deepseek-chat`, `deepseek-reasoner`, and any `-thinking`-suffixed variants are intentionally excluded (the `reasoning_effort` request parameter handles thinking on the canonical V4 model IDs). - -Set the default model to `deepseek/v4-pro` after successful setup. If `v4-pro` is not present in the discovered list, fall back to the first configured DeepSeek model (preserving the static catalog order above). - -If model discovery returns numeric `context_length` values for either of the two official models, prefer that discovered value over the static 128,000 default. - -## Model discovery - -DeepSeek exposes `https://api.deepseek.com/v1/models` (OpenAI-compatible listing). The setup flow must: - -1. Try to list models from `https://api.deepseek.com/v1/models` with `Authorization: Bearer `. -2. If the endpoint returns a usable list, map known IDs (`deepseek-v4-pro`, `deepseek-v4-flash`) into the static catalog above, optionally overriding `max_context_size` and `display_name`. -3. If listing fails with `404`, non-JSON, timeout, or a non-auth server error, fall back to the static catalog and emit an `info` event before success. -4. If listing fails with `401` or `403`, do not save the key. Emit an `error` event with `"Invalid DeepSeek API key; the key was not saved."`. - -Discovery is best-effort. Tests must mock all network calls. - -## Validation - -API-key validation must avoid generation requests. - -Preferred validation order: - -1. `GET https://api.deepseek.com/v1/models` with `Authorization: Bearer `. -2. If unavailable for non-auth reasons, accept the key and configure the static model list with an info-level event. -3. If the response is `401` or `403`, show a clear error and do not save the key. - -Do not log or display the API key. JSON event output must redact secrets. - -## Data flow - -CLI setup: - -1. User runs `pythinker login --deepseek`. -2. Pythinker reads `DEEPSEEK_API_KEY` from the environment or prompts for one (hidden input). -3. Pythinker attempts model discovery and validates auth failures. -4. Pythinker writes the managed provider entry and the two DeepSeek model aliases. -5. Pythinker sets `default_model` to `deepseek/v4-pro` or the first configured DeepSeek model. -6. Pythinker reports success with the default model. - -Shell setup: - -1. User runs `/login deepseek`. -2. Pythinker prompts for the API key when no environment key exists. -3. Pythinker renders setup events through the existing OAuth event renderer. -4. On success, Pythinker tracks a `login` event with provider `deepseek`, clears the console, and reloads the shell. - -## Logout behavior - -Existing OpenAI, OpenCode Go, and MiniMax logout behavior must not remove DeepSeek providers. Add both user-facing logout options: - -- `pythinker logout --deepseek` -- `/logout deepseek` - -Logout removes the `managed:deepseek` provider entry and every model whose `provider` field equals that key. If `default_model` points to a removed DeepSeek model, set `default_model` to the first remaining configured model or clear it (`""`). - -## Error handling - -- If the user enters an empty API key and no environment key exists, print `DeepSeek API key is required.` -- If model discovery returns `401` or `403`, print `Invalid DeepSeek API key; the key was not saved.` -- If model discovery is unavailable for non-auth reasons (timeout, 404, 5xx, malformed JSON), configure the static model list and print an info-level event before the success event. - -## Testing - -Add tests for: - -- Model catalog completeness (two aliases, exact API IDs, all assigned to `managed:deepseek`). -- Env resolution for `DEEPSEEK_API_KEY`. -- Config writes for the single DeepSeek managed provider. -- All current model aliases are created. -- Default model selection prefers `deepseek/v4-pro`. -- `401` and `403` model discovery failures do not save the key. -- Non-auth model discovery failures fall back to the static official model list with an info event before success. -- API keys are not included in event messages or JSON output. -- DeepSeek logout removes only DeepSeek providers/models and leaves OpenAI, OpenCode Go, and MiniMax providers intact. -- CLI parsing for `pythinker login --deepseek` and `pythinker logout --deepseek`. -- Shell routing for `/login deepseek` and `/logout deepseek`. - -All tests must mock network calls. No tests should call real DeepSeek endpoints. - -Provider construction (`create_llm` for the `openai_legacy` provider type at a non-default base URL) was verified during the OpenCode Go work in `tests/core/test_openai_provider.py::test_create_llm_supports_opencode_go_openai_provider`. That coverage is sufficient; no additional provider-construction test is required for DeepSeek. - -## Provider chooser update - -Append `DeepSeek` as option 6 in the `/login` shell chooser. After this change the chooser exposes 6 options: - -``` -1. OpenAI ChatGPT (browser) -2. OpenAI ChatGPT (device code) -3. OpenAI API key -4. OpenCode Go -5. MiniMax -6. DeepSeek -``` - -## Non-goals - -- Do not configure the DeepSeek Anthropic-compatible endpoint (`https://api.deepseek.com/anthropic`). -- Do not add legacy `deepseek-chat` or `deepseek-reasoner` aliases. -- Do not configure DeepSeek non-text models (none are documented at this time, but explicitly out of scope if added later). -- Do not change any existing login or logout behavior. -- Do not require an external CLI binary. diff --git a/docs/superpowers/specs/2026-05-06-minimax-auth-design.md b/docs/superpowers/specs/2026-05-06-minimax-auth-design.md deleted file mode 100644 index 4b9c62b7..00000000 --- a/docs/superpowers/specs/2026-05-06-minimax-auth-design.md +++ /dev/null @@ -1,191 +0,0 @@ -# MiniMax Auth Design - -## Goal - -Add MiniMax as a first-class managed provider in Pythinker Code so users with a MiniMax API key (Open Platform pay-as-you-go OR Token Plan subscription) can configure the provider from the CLI and use the current MiniMax M2.5 / M2.7 text models. - -This change is limited to MiniMax text-model access. It does not add MiniMax speech, image, video, or music models. It does not add any non-MiniMax provider. - -## Context - -Pythinker currently has managed provider setup for OpenAI (API key + ChatGPT OAuth) and OpenCode Go (API key with two-provider split). This task adds an analogous managed setup for MiniMax. - -MiniMax exposes both an OpenAI-compatible chat-completions API and an Anthropic-compatible messages API at distinct base URLs that share a single API key. MiniMax's official documentation recommends the Anthropic-compatible endpoint for full feature support (thinking blocks, tool use, etc.). Because the Anthropic-compatible endpoint is the recommended surface and supports every text model we plan to expose, this design configures **only the Anthropic-compatible managed provider** by default. Users who prefer the OpenAI-compatible surface can edit `config.toml` directly; that is a deliberate simplification, not a limitation. - -MiniMax has two key types that share the same wire format (`Authorization: Bearer `): - -- **Open Platform / pay-as-you-go keys** — billed per token. Prefix typically `sk-`. -- **Token Plan / Coding Plan keys** — subscription, prompt quotas per 5-hour window for text models. Prefix `sk-cp-`. - -The two key types are NOT interchangeable: the server enforces the binding. The client wire format is identical, so Pythinker treats both transparently and only emits an informational event when a Token Plan key is detected. - -## User-facing behavior - -Add a dedicated MiniMax setup mode without changing any existing login flow. - -CLI: - -```sh -pythinker login --minimax -pythinker logout --minimax -``` - -Shell: - -```text -/login minimax -/logout minimax -``` - -The command prompts for the MiniMax API key (hidden input), validates it via best-effort model discovery, writes a managed provider and model aliases, and reloads the shell after successful setup. - -The setup must also support an environment-provided key. If a key is not provided interactively, use: - -1. `MINIMAX_API_KEY` (only documented MiniMax env var) - -No other fallback variable names are supported (MiniMax docs do not document any). - -The dedicated MiniMax mode must not change `pythinker login`, `pythinker login --browser`, `pythinker login --headless`, `pythinker login --api-key`, `pythinker login --opencode-go`, `/login`, `/login browser`, `/login headless`, `/login api-key`, or `/login opencode-go`. - -## Provider configuration - -Add a provider family ID for model aliases and telemetry: - -```text -minimax -``` - -Configure ONE managed provider entry. The Anthropic-compatible endpoint is MiniMax's recommended surface and supports every model we expose: - -- `managed:minimax-anthropic` — type `anthropic`, base URL `https://api.minimax.io/anthropic`. - -Provider key naming preserves the Pythinker convention (`managed:-`). Even though only one provider is configured, the suffix `-anthropic` keeps room for an optional `-openai` provider in the future without renaming the existing key. - -Model aliases follow the project's existing form: - -```text -minimax/ -``` - -The `LLMModel.model` field stores the EXACT API model ID (CamelCase, e.g. `MiniMax-M2.7`), while the alias uses a lowercase short form (`minimax/m2.7`). - -The implementation must add small MiniMax-specific helpers for provider keys, model alias construction, and removal — do not stretch the OpenCode Go helpers, even though the patterns are similar. - -## Models - -Configure exactly four current text models (M2.5 and M2.7, standard and high-speed): - -| Alias | API model ID | Provider type | Display name | Max context | -| --- | --- | --- | --- | --- | -| `minimax/m2.7` | `MiniMax-M2.7` | `anthropic` | MiniMax M2.7 | 192_000 | -| `minimax/m2.7-highspeed` | `MiniMax-M2.7-highspeed` | `anthropic` | MiniMax M2.7 High-Speed | 192_000 | -| `minimax/m2.5` | `MiniMax-M2.5` | `anthropic` | MiniMax M2.5 | 192_000 | -| `minimax/m2.5-highspeed` | `MiniMax-M2.5-highspeed` | `anthropic` | MiniMax M2.5 High-Speed | 192_000 | - -Legacy `MiniMax-M2.1`, `MiniMax-M2`, and `M2-her` are intentionally excluded. - -Set the default model to `minimax/m2.7` after successful setup. If `m2.7` is not present in the discovered list, fall back to the first configured MiniMax model (preserving the static catalog order above). - -If model discovery returns numeric `context_length` values for any of the four official models, prefer that discovered value over the static 192,000 default. - -## Token Plan awareness - -After successful setup, inspect the resolved API key for a Token Plan prefix and emit a single informational event before the success event when applicable: - -- If the resolved key starts with `sk-cp-` → emit `OAuthEvent("info", "MiniMax Token Plan key detected; requests are quota-metered (5-hour rolling window for text), not per-token billed.")`. -- Otherwise → no extra event (default pay-as-you-go assumption). - -Detection is purely client-side and informational. The server enforces the actual key-type binding; Pythinker never blocks or rewrites traffic based on the prefix. - -The detection MUST run on the resolved key value (after env-var resolution and trimming) and MUST NOT include the key in the event message or JSON output. - -## Model discovery - -MiniMax exposes `https://api.minimax.io/v1/models` (OpenAI-compatible listing). The setup flow must: - -1. Try to list models from `https://api.minimax.io/v1/models` with `Authorization: Bearer `. -2. If the endpoint returns a usable list, map known IDs (`MiniMax-M2.7`, `MiniMax-M2.7-highspeed`, `MiniMax-M2.5`, `MiniMax-M2.5-highspeed`) into the static catalog above, optionally overriding `max_context_size` from `context_length` and `display_name` from `display_name`. -3. If listing fails with `404`, non-JSON HTML, timeout, or a non-auth server error, fall back to the static official model list and emit an `info` event before success. -4. If listing fails with `401` or `403`, do not save the key. Emit an `error` event with `"Invalid MiniMax API key; the key was not saved."`. - -Discovery is best-effort. Tests must mock all network calls. - -Note: discovery uses the OpenAI-compatible `/v1/models` endpoint even though chat traffic uses the Anthropic-compatible endpoint, because the MiniMax `/v1/models` listing is the documented model-discovery API and returns the same model IDs. - -## Validation - -API key validation must avoid generation requests. - -Preferred validation order: - -1. `GET https://api.minimax.io/v1/models` with `Authorization: Bearer `. -2. If unavailable for non-auth reasons, accept the key and configure the static model list with an info-level event. -3. If the response is `401` or `403`, show a clear error and do not save the key. - -Do not log or display the API key. JSON event output must redact secrets. - -## Data flow - -CLI setup: - -1. User runs `pythinker login --minimax`. -2. Pythinker reads `MINIMAX_API_KEY` from the environment or prompts for one (hidden input). -3. Pythinker attempts model discovery and validates auth failures. -4. Pythinker writes the managed provider entry and the four MiniMax model aliases. -5. Pythinker emits a Token Plan info event if the key has the `sk-cp-` prefix. -6. Pythinker sets `default_model` to `minimax/m2.7` or the first configured MiniMax model. -7. Pythinker reports success with the default model. - -Shell setup: - -1. User runs `/login minimax`. -2. Pythinker prompts for the API key when no environment key exists. -3. Pythinker renders setup events through the existing OAuth event renderer. -4. On success, Pythinker tracks a `login` event with provider `minimax`, clears the console, and reloads the shell. - -## Logout behavior - -Existing OpenAI and OpenCode Go logout behavior must not remove MiniMax providers. Add both user-facing logout options: - -- `pythinker logout --minimax` -- `/logout minimax` - -Logout removes the `managed:minimax-anthropic` provider entry and every model whose `provider` field equals that key. If `default_model` points to a removed MiniMax model, set `default_model` to the first remaining configured model or clear it (`""`). - -## Error handling - -- If the user enters an empty API key and no environment key exists, print `MiniMax API key is required.` -- If model discovery returns `401` or `403`, print `Invalid MiniMax API key; the key was not saved.` -- If model discovery is unavailable for non-auth reasons (timeout, 404, 5xx, malformed JSON), configure the static model list and print an info-level event before the success event. -- The Token Plan info event is emitted regardless of discovery outcome (it is a property of the key itself, not the discovery response). - -## Testing - -Add tests for: - -- Model catalog completeness (four aliases, exact API IDs, all assigned to the anthropic provider key). -- Env precedence: `MINIMAX_API_KEY`. -- Config writes for the single MiniMax managed provider. -- All four current model aliases are created. -- Default model selection prefers `minimax/m2.7`. -- `401` and `403` model discovery failures do not save the key. -- Non-auth model discovery failures fall back to the static official model list with an info event before success. -- API keys are not included in event messages or JSON output. -- Token Plan detection: `sk-cp-` prefix triggers exactly one info event before the success event; non-`sk-cp-` keys do not. -- MiniMax logout removes only MiniMax providers/models and leaves OpenAI and OpenCode Go providers intact. -- CLI parsing for `pythinker login --minimax` and `pythinker logout --minimax`. -- Shell routing for `/login minimax` and `/logout minimax`. - -All tests must mock network calls. No tests should call real MiniMax endpoints. - -Provider construction (`create_llm` for the `anthropic` provider type at a non-default base URL) was verified during the OpenCode Go work in `tests/core/test_openai_provider.py::test_create_llm_supports_opencode_go_anthropic_provider`. That coverage is sufficient; no additional provider-construction test is required for MiniMax. - -## Non-goals - -- Do not add the OpenAI-compatible MiniMax provider (`https://api.minimax.io/v1`) by default. -- Do not add MiniMax legacy models (M2.1, M2, M2-her). -- Do not add MiniMax non-text models (speech, image, video, music). -- Do not add Token Plan quota tracking — Pythinker does not poll usage endpoints. -- Do not change any existing login or logout behavior. -- Do not require any external MiniMax CLI binary. -- Do not integrate with MiniMax's web platform cookie session (out of scope; the docs say it is not API-key authenticated). diff --git a/docs/superpowers/specs/2026-05-06-openrouter-auth-design.md b/docs/superpowers/specs/2026-05-06-openrouter-auth-design.md deleted file mode 100644 index 675e8683..00000000 --- a/docs/superpowers/specs/2026-05-06-openrouter-auth-design.md +++ /dev/null @@ -1,180 +0,0 @@ -# OpenRouter Auth Design - -## Goal - -Add OpenRouter as a managed provider in Pythinker Code so users with an OpenRouter API key can configure the provider from the CLI and use a curated set of popular models routed through OpenRouter's unified API. - -This change is limited to a small starter catalog of six popular models exposed via OpenRouter's OpenAI-compatible chat-completions API. Discovery from OpenRouter's `/api/v1/models` endpoint is used to override metadata for catalog entries but does NOT add new aliases (OpenRouter exposes 500+ models — flooding the user's config is undesirable). - -## Context - -OpenRouter is a meta-provider that aggregates 500+ models from OpenAI, Anthropic, Google, DeepSeek, MiniMax, Meta, Mistral, xAI, and others behind a single OpenAI-compatible endpoint and a single API key. It uses `Authorization: Bearer sk-or-...` and lives at `https://openrouter.ai/api/v1`. - -Pythinker's existing `openai_legacy` provider type covers the wire format. Provider construction at a custom base URL was proven during the OpenCode Go work. - -## User-facing behavior - -CLI: - -```sh -pythinker login --openrouter -pythinker logout --openrouter -``` - -Shell: - -```text -/login openrouter -/logout openrouter -``` - -The command prompts for the OpenRouter API key (hidden input), validates it via best-effort model discovery, writes the managed provider and the curated model aliases, and reloads the shell after successful setup. - -The setup must also support an environment-provided key. If a key is not provided interactively, use: - -1. `OPENROUTER_API_KEY` (de facto convention used by every OpenRouter SDK and example). - -The dedicated OpenRouter mode must not change any existing login or logout flow. - -## Provider configuration - -Provider family ID: - -```text -openrouter -``` - -Configure ONE managed provider entry: - -- `managed:openrouter` — type `openai_legacy`, base URL `https://openrouter.ai/api/v1`. - -Model aliases follow the project convention but include the upstream vendor in the suffix: - -```text -openrouter// -``` - -The `LLMModel.model` field stores the EXACT OpenRouter model slug (`openai/gpt-5.2`, `anthropic/claude-sonnet-4.6`, etc.). The alias prefixes the slug with `openrouter/` (so the alias is `openrouter/openai/gpt-5.2`). - -## Models - -Configure exactly six curated starter models: - -| Alias | OpenRouter slug | Provider type | Display name | Max context | -| --- | --- | --- | --- | --- | -| `openrouter/openai/gpt-5.2` | `openai/gpt-5.2` | `openai_legacy` | GPT-5.2 (OpenRouter) | 400_000 | -| `openrouter/anthropic/claude-sonnet-4.6` | `anthropic/claude-sonnet-4.6` | `openai_legacy` | Claude Sonnet 4.6 (OpenRouter) | 200_000 | -| `openrouter/anthropic/claude-opus-4.7` | `anthropic/claude-opus-4.7` | `openai_legacy` | Claude Opus 4.7 (OpenRouter) | 1_000_000 | -| `openrouter/deepseek/deepseek-v4-pro` | `deepseek/deepseek-v4-pro` | `openai_legacy` | DeepSeek V4 Pro (OpenRouter) | 128_000 | -| `openrouter/google/gemini-2.5-pro` | `google/gemini-2.5-pro` | `openai_legacy` | Gemini 2.5 Pro (OpenRouter) | 1_000_000 | -| `openrouter/openrouter/auto` | `openrouter/auto` | `openai_legacy` | OpenRouter Auto (router) | 1_000_000 | - -Set the default model to `openrouter/openai/gpt-5.2` after successful setup. If that alias is not present in the curated catalog (e.g. user pruned it), fall back to the first configured OpenRouter model. - -The static catalog values above are intentional defaults. Discovery may override `max_context_size` and `display_name` for matching slugs (see next section), but discovery does NOT add new aliases. - -## Model discovery - -OpenRouter exposes `https://openrouter.ai/api/v1/models` returning the full 500+ model catalog. The setup flow must: - -1. Try to list models with `Authorization: Bearer `. -2. If the endpoint returns a usable list, find entries whose `id` matches one of the six curated slugs above. For matches, override `max_context_size` (from the response's `context_length` field) and `display_name` (from `name`). DROP every other discovered model. -3. If listing fails with `404`, non-JSON, timeout, or a non-auth server error, fall back to the static catalog and emit an `info` event before success. -4. If listing fails with `401` or `403`, do not save the key. Emit an `error` event with `"Invalid OpenRouter API key; the key was not saved."`. - -The override-without-add behavior is the key difference from MiniMax/DeepSeek (which replace the catalog with discovered models). The reason: OpenRouter exposes hundreds of models and adding them all would flood `config.toml`. - -Discovery is best-effort. Tests must mock all network calls. - -## Validation - -API-key validation must avoid generation requests. - -Preferred validation order: - -1. `GET https://openrouter.ai/api/v1/models` with `Authorization: Bearer `. -2. If unavailable for non-auth reasons, accept the key and configure the static model list with an info-level event. -3. If the response is `401` or `403`, show a clear error and do not save the key. - -Do not log or display the API key. JSON event output must redact secrets. - -OpenRouter API keys start with the prefix `sk-or-`. We do NOT validate the prefix client-side because users can have legacy or custom-prefixed keys. - -## Data flow - -CLI setup: - -1. User runs `pythinker login --openrouter`. -2. Pythinker reads `OPENROUTER_API_KEY` from the environment or prompts for one (hidden input). -3. Pythinker attempts model discovery and validates auth failures. -4. Pythinker writes the managed provider entry and the six curated model aliases. -5. Pythinker sets `default_model` to `openrouter/openai/gpt-5.2` or the first configured OpenRouter model. -6. Pythinker reports success with the default model. - -Shell setup: - -1. User runs `/login openrouter`. -2. Pythinker prompts for the API key when no environment key exists. -3. Pythinker renders setup events through the existing OAuth event renderer. -4. On success, Pythinker tracks a `login` event with provider `openrouter`, clears the console, and reloads the shell. - -## Logout behavior - -Existing OpenAI, OpenCode Go, MiniMax, DeepSeek, and Anthropic logout behavior must not remove OpenRouter providers. Add both user-facing logout options: - -- `pythinker logout --openrouter` -- `/logout openrouter` - -Logout removes the `managed:openrouter` provider entry and every model whose `provider` field equals that key. If `default_model` points to a removed OpenRouter model, set `default_model` to the first remaining configured model or clear it (`""`). - -## Error handling - -- If the user enters an empty API key and no environment key exists, print `OpenRouter API key is required.` -- If model discovery returns `401` or `403`, print `Invalid OpenRouter API key; the key was not saved.` -- If model discovery is unavailable for non-auth reasons (timeout, 404, 5xx, malformed JSON), configure the static model list and print an info-level event before the success event. - -## Testing - -Add tests for: - -- Model catalog completeness (six curated aliases, exact OpenRouter slugs, all assigned to `managed:openrouter`). -- Env resolution for `OPENROUTER_API_KEY`. -- Config writes for the single OpenRouter managed provider. -- All six curated model aliases are created. -- Default model selection prefers `openrouter/openai/gpt-5.2`. -- `401` and `403` model discovery failures do not save the key. -- Non-auth model discovery failures fall back to the static official model list with an info event before success. -- Override-without-add: a discovery response containing both a known curated slug and an unknown extra slug results in only the curated slug being kept (with overridden context_length / display_name) — the extra slug is dropped. -- API keys are not included in event messages or JSON output. -- OpenRouter logout removes only OpenRouter providers/models. -- CLI parsing for `pythinker login --openrouter` and `pythinker logout --openrouter`. -- Shell routing for `/login openrouter` and `/logout openrouter`. - -All tests must mock network calls. - -Provider construction (`openai_legacy` at custom base URL) is pre-verified by OpenCode Go's Task 6. - -## Provider chooser update - -Append `OpenRouter` as the next sequential option in the `/login` shell chooser. After this change (assuming DeepSeek and Anthropic already landed) the chooser exposes 8 options: - -``` -1. OpenAI ChatGPT (browser) -2. OpenAI ChatGPT (device code) -3. OpenAI API key -4. OpenCode Go -5. MiniMax -6. DeepSeek -7. Anthropic -8. OpenRouter -``` - -## Non-goals - -- Do not auto-add models discovered from `/v1/models` beyond the six curated aliases. Override-only. -- Do not configure OpenRouter-provider-specific routing parameters (`provider.order`, `provider.allow_fallbacks`, etc.) — out of scope; users who need these can edit `config.toml`. -- Do not add OpenRouter ranking/attribution headers (`HTTP-Referer`, `X-OpenRouter-Title`). -- Do not configure OpenRouter's `openrouter/auto` model as the default (users opt in via the catalog; default stays on a deterministic vendor model). -- Do not configure free-tier-only models in the curated catalog. -- Do not change any existing login or logout behavior. -- Do not require the OpenRouter SDK; the existing `openai_legacy` provider wrapper is sufficient. diff --git a/docs/superpowers/specs/2026-05-07-codex-terminal-ui-followup-design.md b/docs/superpowers/specs/2026-05-07-codex-terminal-ui-followup-design.md deleted file mode 100644 index 9c10c5f8..00000000 --- a/docs/superpowers/specs/2026-05-07-codex-terminal-ui-followup-design.md +++ /dev/null @@ -1,37 +0,0 @@ -# Codex Terminal UI Follow-Up Design - -## Summary - -Align the shell prompt, slash command menu, subagent activity rows, and diff summaries more closely with the Codex TUI reference in `.worktrees/codex-main` while preserving the existing prompt-toolkit and Rich architecture. - -## Goals - -- Keep the input surface compact by default, but allow it to grow with text up to five visible rows. -- Scroll input after five visible rows instead of clipping text to two rows. -- Keep slash commands in the Codex order: input area first, command list below it, status footer hidden while the list is active. -- Keep animated dots/particles for running subagent rows. -- Make diff cards compact and summary-first, with clear per-file added/removed counts. - -## Non-Goals - -- Replace prompt-toolkit with a custom TUI engine. -- Port Codex Rust state machines directly. -- Redesign unrelated welcome, history, auth, or modal flows. - -## Design - -The prompt buffer window should use a small preferred height with a maximum of five rows. The compact input background stays on the buffer window, and the prompt marker remains visually aligned with the first editable row. - -The slash menu should remain an HSplit sibling directly below the prompt buffer, not a cursor-anchored float. This mirrors Codex's composer plus popup stack and avoids the clipped-popup bug caused by cursor floats inside a small input window. - -Subagent rows should use a Rich dots spinner as the running indicator. Finished subagent child calls stay compact and dimmed, with errors still visually distinct. - -Diff display should continue to group consecutive same-file diff blocks, but the rendered card should emphasize the file path and added/removed counts before showing hunk details. - -## Testing - -- Unit tests assert the prompt buffer grows to a max of five rows. -- Unit tests assert slash menu placement remains directly after the compact input layout node. -- Worklog tests assert running subagents render a dots spinner. -- Diff tests assert compact per-file summaries include added/removed counts. -- Run `tests/ui_and_conv` and targeted tmux slash completion tests. diff --git a/docs/superpowers/specs/2026-05-07-compact-terminal-input-design.md b/docs/superpowers/specs/2026-05-07-compact-terminal-input-design.md deleted file mode 100644 index 0658093e..00000000 --- a/docs/superpowers/specs/2026-05-07-compact-terminal-input-design.md +++ /dev/null @@ -1,55 +0,0 @@ -# Compact Terminal Input Design - -## Summary - -Update the shell input area to match the compact Claude-style layout supplied by the user. The input block should reserve two visible rows, use thin horizontal separators above and below the editable area, and keep the existing bottom status toolbar behavior. - -## Goals - -- Show a compact two-row input area instead of reserving a large prompt region. -- Use a `>`-style prompt marker visually aligned with the screenshot. -- Keep multiline editing, command handling, modals, and existing prompt-toolkit behaviors intact. -- Preserve the current shell architecture and avoid a broad terminal UI rewrite. - -## Non-Goals - -- Rebuild the shell UI around a new layout engine. -- Change agent event rendering, work-log cards, or interactive visualization behavior unrelated to the input area. -- Change permission, model, or branch semantics beyond their visual placement in the existing toolbar. - -## Design - -The implementation should make a focused change in `src/pythinker_code/ui/shell/prompt.py`: - -- Render compact input chrome without hard-capping the prompt-toolkit buffer window, so cursor-anchored completion menus can still expand normally. -- Render thin separators around the input block, matching the screenshot's simple horizontal-rule treatment. -- Keep the editable prompt line minimal, with a left prompt marker and no heavy bordered panel. -- Leave the bottom toolbar as the source of status information, but style it to remain visually compatible with the compact input block. - -The resulting layout should look like: - -```text -──────────────────────────────────────── -> user input - -──────────────────────────────────────── -model / effort / repo / branch / permission hint -``` - -## Behavior - -- The visible input chrome stays compact and does not reserve the old large titled prompt area. -- Long or multiline input continues to use prompt-toolkit's existing editing behavior within the smaller visible region. -- Modal delegates still suppress the normal input chrome as they do today. -- Existing key bindings and prompt session configuration remain unchanged unless required for the two-row height. - -## Testing - -- Add or update focused shell prompt tests for the configured prompt height and rendered input chrome. -- Run the relevant `tests/ui_and_conv` tests that cover shell prompt/render behavior. -- Run a formatter/check command if practical; note any existing tooling blocker separately. - -## Risks - -- Prompt-toolkit sizing can differ slightly by terminal height, so tests should assert configuration/render intent rather than pixel-perfect terminal behavior. -- Hard-capping the prompt-toolkit input window can clip cursor-anchored slash completion menus, so the compact layout should be achieved through prompt chrome rather than a fixed buffer-window maximum. diff --git a/docs/superpowers/specs/2026-05-07-readable-terminal-reports-design.md b/docs/superpowers/specs/2026-05-07-readable-terminal-reports-design.md deleted file mode 100644 index 6adc3d20..00000000 --- a/docs/superpowers/specs/2026-05-07-readable-terminal-reports-design.md +++ /dev/null @@ -1,60 +0,0 @@ -# Readable Terminal Reports Design - -## Goal - -Improve shell output readability for long audit, search, and report responses, and make completed subagent worklog entries scannable. - -## Current Problems - -- Wide markdown tables with long cells are rendered as terminal columns, which causes words and paths to wrap into narrow vertical fragments. -- Completed subagent entries keep listing many child tool calls, so the completed state looks like an active trace instead of a summary. -- The user needs readable sections for reports and compact completed subagent summaries. - -## Approved Direction - -Use readable sections/cards for dense reports and collapse completed subagent blocks into summaries. - -## Markdown Report Rendering - -When a markdown table is too wide for terminal reading, render each row as a compact record instead of a multi-column table. - -Adaptive table rendering should apply when a table has four or more columns or when its cell content is long enough that normal column layout would be hard to scan. - -Each record should show the row number and then labeled fields, for example: - -```text -1. Accessibility - Issue: Search input relies on placeholder text only. - Why it matters: Placeholder labels disappear during typing and are weak for screen readers. - Suggested improvement: Add aria-label="Search sessions" or a visually hidden label. - Priority: High - Effort: XS -``` - -Small tables should keep the current table renderer. - -## Completed Subagent Rendering - -Running subagents should keep the current activity indicators. - -Completed subagents should switch to a compact completion summary: - -```text -✓ Subagent Audit web frontend completed - explore · a143aa989 · 53 tool calls · 49 hidden - ReadFile: 4 shown · web/src/..., vis/src/... -``` - -Errors should remain visible and should not be hidden behind the compact summary. - -## Testing - -- Add a markdown rendering regression test for wide report tables that verifies labeled records are rendered instead of narrow table columns. -- Add a worklog rendering regression test that verifies completed subagents show a summary and do not dump the full child tool-call list. -- Keep existing compact prompt, slash menu, worklog, and UI tests passing. - -## Out Of Scope - -- Changing model prompts or forcing the assistant to avoid markdown tables. -- Adding interactive expand/collapse controls for historical output. -- Reworking Rich Live architecture beyond the adaptive renderers needed here. diff --git a/docs/superpowers/specs/2026-05-07-selector-family-design.md b/docs/superpowers/specs/2026-05-07-selector-family-design.md deleted file mode 100644 index 3b7b7db9..00000000 --- a/docs/superpowers/specs/2026-05-07-selector-family-design.md +++ /dev/null @@ -1,443 +0,0 @@ -# Selector Family Port — Design Spec - -**Date:** 2026-05-07 -**Branch:** tui-pi-foundation -**Status:** Approved (revised after reading Pi source) - ---- - -## Summary - -Port all 11 Pi-style selector screens to Pythinker using Approach B: `run_selector()` where the UX fits a flat filterable list; focused standalone modules for UX paradigms that fundamentally differ (model with scope-toggle, session with full tree/search/delete UX, settings key/value editor, multi-toggle ordering, grouped resource manager). - -This spec covers three implementation tiers that should ship as three separate plans: - -- **Plan A** — Tier 1–2: theme, thinking, show\_images, extension, oauth + `selector.py` extensions -- **Plan B** — Tier 3: model migration, session migration (complex) -- **Plan C** — Tier 4: settings, scoped\_models, config - ---- - -## File Map - -``` -src/pythinker_code/ui/shell/ - selector.py ← extend: SelectorHeader + on_change callback - selectors/ - __init__.py ← re-exports all run_* functions - theme.py ← run_theme_selector() - thinking.py ← run_thinking_selector() - show_images.py ← run_show_images_selector() - extension.py ← run_extension_selector() - oauth.py ← run_oauth_selector() - model.py ← run_model_selector() (replaces model_picker.py) - session.py ← run_session_selector() (replaces session_picker.py) - session_search.py ← pure search/sort functions (port of session-selector-search.ts) - settings.py ← run_settings_selector() - scoped_models.py ← run_scoped_models_selector() - config.py ← run_config_selector() - # tree.py intentionally deferred — blocked on session tree data model - -tests/ui_and_conv/ - test_selectors_simple.py ← Tier 1 unit tests - test_selector_groups.py ← SelectorHeader nav unit tests - test_settings_selector.py ← SettingItem cycling + cancel - test_scoped_models_selector.py ← toggle / reorder / enable-all / clear-all -``` - -Existing `model_picker.py` and `session_picker.py` become one-line delegation wrappers for one release cycle, then are deleted once all callers are updated. - ---- - -## Section 1: `selector.py` Extensions - -Two small additions to `selector.py` — no behavior changes to existing code: - -### 1a. `SelectorHeader` - -```python -@dataclass(frozen=True, slots=True) -class SelectorHeader: - label: str # rendered as a section divider, not selectable -``` - -`SelectorConfig.items` type changes from `Sequence[SelectorItem[T]]` to -`Sequence[SelectorItem[T] | SelectorHeader]`. - -Render loop: header rows use a distinct style (`class:slash-completion-menu.meta`) -and are skipped by cursor nav (up/down wraps past them). - -Used by: `selectors/config.py` (user-scope / project-scope group dividers). -Not used by `model_selector` — Pi renders models in a flat list sorted by provider. - -### 1b. `on_change` callback - -```python -@dataclass(frozen=True, slots=True) -class SelectorConfig[T]: - ... - on_change: Callable[[T], None] | None = None -``` - -Called whenever the cursor moves to a new `SelectorItem`. Used by `theme_selector` -for live preview. No-op when `None`. - -### 1c. Width note - -`selector.py` currently hardcodes `width = 80` in `items_text()`. Model and oauth -selectors use `truncateToWidth` in Pi. For this port, keep `width = 80` as the -default since prompt_toolkit `FormattedTextControl` doesn't expose render width. -Revisit if long model names truncate badly in practice. - ---- - -## Section 2: Tier 1 — Simple Selectors (Plan A) - -All four call `run_selector()` directly. Each file is ~20–40 lines. - -### `selectors/theme.py` - -```python -run_theme_selector( - current_theme: str, - available_themes: list[str], - on_preview: Callable[[str], None] | None = None, -) -> str | None -``` - -- Items built from `available_themes`; item matching `current_theme` gets `is_current=True`. -- `SelectorConfig.on_change = on_preview` for live preview. - -### `selectors/thinking.py` - -```python -ThinkingLevel = Literal["off", "minimal", "low", "medium", "high", "xhigh"] - -LEVEL_DESCRIPTIONS: dict[ThinkingLevel, str] = { - "off": "No reasoning", - "minimal": "Very brief reasoning (~1k tokens)", - "low": "Light reasoning (~2k tokens)", - "medium": "Moderate reasoning (~8k tokens)", - "high": "Deep reasoning (~16k tokens)", - "xhigh": "Maximum reasoning (~32k tokens)", -} - -run_thinking_selector( - current_level: ThinkingLevel, - available_levels: list[ThinkingLevel], -) -> ThinkingLevel | None -``` - -Also replaces the `ChoiceInput` call for thinking on/off in the `/model` slash command. - -### `selectors/show_images.py` - -```python -run_show_images_selector(current: bool) -> bool | None -``` - -2-item list: `Yes` / `No`. Filter disabled (`enable_filter=False`). - -### `selectors/extension.py` - -```python -run_extension_selector( - title: str, - options: list[str], - *, - current: str | None = None, - timeout: float | None = None, -) -> str | None -``` - -Generic caller-supplied option list. `timeout` is implemented in `extension.py` -using `asyncio.wait_for` — not a `selector.py` concern. - ---- - -## Section 3: Tier 2 — OAuth Selector Migration (Plan A) - -`selectors/oauth.py` extracts the provider-picker from the existing `oauth.py` handler. - -```python -@dataclass(frozen=True, slots=True) -class OAuthProviderEntry: - id: str # platform id (e.g. "anthropic", "openrouter") - name: str # display name - auth_type: Literal["oauth", "api_key"] - -@dataclass(frozen=True, slots=True) -class OAuthProviderStatus: - source: Literal["environment", "runtime", "fallback", "models_json_key", - "models_json_command", "configured", "unconfigured"] - label: str | None = None - -run_oauth_selector( - providers: list[OAuthProviderEntry], - get_status: Callable[[str], OAuthProviderStatus], - *, - action: Literal["login", "logout"] = "login", -) -> str | None # returns provider id -``` - -Each row shows the provider name + status indicator (✓ configured, ✓ env: API key, -• unconfigured). Pi's `formatStatusIndicator()` logic is ported inline. -Auth steps remain in `oauth.py`; only the list picker is extracted. - ---- - -## Section 4: Tier 3 — Existing Picker Migration (Plan B) - -### `selectors/model.py` — migrates `model_picker.py` - -Pi's model selector is a **flat list** (not group-divided), sorted: current model -first, then sorted by provider label. Provider names appear as a `[provider]` badge -on each row, not as group headers. `SelectorHeader` is NOT used here. - -Additional Pi features to port: - -- **Scope toggle** (Tab): all models ↔ scoped models (session-local subset from - `run_scoped_models_selector`). Only shown when a scoped set is active. -- **Async load** with error display (models.json parse errors shown in red). -- Scroll indicator `(N/total)` when list exceeds visible window. -- Model name displayed below the list for the selected item. -- Fuzzy filter (type-to-search, same as current `model_picker.py`). - -The caller passes a flat `list[ModelEntry]` — not `list[ProviderGroup]`. The -existing `ProviderGroup` grouping in `model_picker.py` is a Pythinker-specific -concept not present in Pi; the new selector does not use it. `ModelEntry` already -exists in `model_picker.py` (`name`, `display`, `model_id`). - -```python -@dataclass(frozen=True, slots=True) -class ScopedModelItem: - model_name: str # config key - thinking_level: str | None = None - -run_model_selector( - models: list[ModelEntry], - *, - current_model_name: str | None = None, - scoped_models: list[ScopedModelItem] | None = None, -) -> str | None # returns model config key (ModelEntry.name) -``` - -Custom `Application` (preserves `model_picker.py` pattern; scope-toggle requires -state that doesn't fit `run_selector()`). `model_picker.py` becomes a one-line -wrapper that flattens its `ProviderGroup` list into `ModelEntry` items and delegates. - -### `selectors/session.py` — migrates `session_picker.py` - -Pi's session selector (1023 lines) is the most feature-rich picker. Full feature -list to port: - -**Navigation & display:** -- Scope toggle (Tab): current folder ↔ all sessions -- Sort mode toggle: threaded / recent / fuzzy (separate keybinding) -- Name filter toggle: all / named only (separate keybinding) -- Path toggle: show/hide working directory per session row -- Scroll with `(N/total)` indicator - -**Search:** -- Type-to-filter (fuzzy by default) -- Regex mode: `re:` -- Phrase mode: `"quoted string"` exact match -- Sort switches to "relevance" mode when query is non-empty - -**Tree display (threaded mode, no query):** -- Sessions organized as parent/child tree based on `parentSessionPath` -- ASCII box-drawing connectors (depth + isLast + ancestorContinues tracking) -- Root-level nodes sorted by modified date descending - -**Destructive actions:** -- Delete selected session: dedicated key → confirmation prompt (separate hint line) → execute -- Cannot delete current session (guarded with error message) -- Rename selected session: opens inline input - -**Status:** -- Loading progress indicator while sessions are fetched -- Transient status messages (info / error) with auto-hide - -```python -run_session_selector( - work_dir: HostPath, - current_session: Session, -) -> str | None # returns session ID (not file path) -``` - -**Return value is session ID, not file path.** Both existing callers compare the -result to `current_session.id` (`slash.py:622`) and pass it to -`Reload(session_id=...)` (`slash.py:635`) or treat it as `session_id` -(`cli/__init__.py:979`). File paths are used *internally* only (tree rendering, -delete/rename operations). The session list is loaded with `Session` objects that -carry both `id` and `path`; the selector resolves path → id before returning. - -`session_search.py` — pure functions ported from Pi's `session-selector-search.ts`: - -```python -# session_search.py -class SortMode(Enum): - THREADED = "threaded" - RECENT = "recent" - FUZZY = "fuzzy" - -class NameFilter(Enum): - ALL = "all" - NAMED_ONLY = "named" - -@dataclass(frozen=True, slots=True) -class ParsedSearchQuery: - raw: str - mode: Literal["fuzzy", "regex", "phrase"] - pattern: str # normalized (stripped quotes for phrase, stripped "re:" for regex) - -def parse_search_query(raw: str) -> ParsedSearchQuery: ... -def filter_and_sort_sessions( - sessions: list[Session], - query: ParsedSearchQuery, - sort_mode: SortMode, - name_filter: NameFilter, - work_dir: HostPath, -) -> list[Session]: ... -def has_session_name(session: Session) -> bool: ... -``` - -Custom `Application` — scope/sort/name toggles with async reload don't fit -`run_selector()`. `session_picker.py` becomes a one-line wrapper. - ---- - -## Section 5: Tier 4 — Complex New Selectors (Plan C) - -### `selectors/settings.py` - -Mirrors Pi's `SettingsList`. A key/value editor where Enter cycles a setting's -value; Esc exits. - -```python -@dataclass(frozen=True, slots=True) -class SettingItem: - id: str - label: str - description: str - current_value: str - values: list[str] # options to cycle through - -run_settings_selector(items: list[SettingItem]) -> dict[str, str] | None -# Returns {id: new_value} for changed items, or None on cancel. -``` - -Custom `Application`. No type-to-filter — settings list is short. -Key bindings: ↑↓ navigate, Enter cycle value, Esc cancel, Ctrl+C cancel. - -The `/settings` slash command currently shows a Rich table (read-only). After this -work it gains an interactive editor: builds `SettingItem` rows from `Config` fields, -applies returned diffs to the live config, then shows the updated table. - -### `selectors/scoped_models.py` - -Multi-toggle with ordering. Session-local override of which models are active. - -```python -@dataclass(frozen=True, slots=True) -class ScopedResult: - kind: Literal["unchanged", "all_enabled", "subset"] - ids: list[str] # non-empty only when kind == "subset" - -SCOPED_UNCHANGED = ScopedResult(kind="unchanged", ids=[]) - -run_scoped_models_selector( - all_models: list[ModelEntry], - enabled_ids: list[str] | None, # None = all enabled -) -> ScopedResult -# Returns: -# ScopedResult(kind="unchanged", ...) — user cancelled (Esc) -# ScopedResult(kind="all_enabled", ...)— cleared filter ("all") -# ScopedResult(kind="subset", ids=...) — explicit ordered list -``` - -Key bindings: ↑↓ navigate, Space toggle, Alt+↑/↓ reorder, `a` enable-all, -`A` clear-all, Enter commit, Esc cancel. Custom `Application`. - -### `selectors/config.py` - -Grouped resource manager. Enables/disables extensions, skills, prompts, themes -by source scope (user / project). Uses `SelectorHeader` for scope group dividers. - -```python -@dataclass -class ConfigResource: - path: str - display_name: str - resource_type: Literal["extensions", "skills", "prompts", "themes"] - scope: Literal["user", "project"] - enabled: bool - -run_config_selector(resources: list[ConfigResource]) -> dict[str, bool] | None -# Returns {path: enabled} for changed items, or None on cancel. -``` - -Key bindings: ↑↓ navigate, Space toggle, type to filter, Enter commit, Esc cancel. -Custom `Application` with `SelectorHeader` group rows. - -### `tree_selector` — deferred - -Intentionally not implemented in this spec. Blocked on verifying the session tree -data model (`SessionManager.get_tree()` or equivalent). Will be a separate spec -once the data model is confirmed. Do not create a stub file. - ---- - -## Section 6: Slash-Command Wiring - -Updates to `slash.py`: - -| Command | Calls | Notes | -|---------|-------|-------| -| `/model` | `run_model_selector()` | Replace `ModelPickerApp` call | -| `/theme` | `run_theme_selector()` | New command | -| `/thinking` | `run_thinking_selector()` | New command; also replaces ChoiceInput in `/model` handler | -| `/settings` | `run_settings_selector()` | Upgrades read-only table to interactive editor | -| `/login` | `run_oauth_selector()` | Migrate provider picker from `oauth.py` | -| `/models-scope` | `run_scoped_models_selector()` | New command | -| `/config` | `run_config_selector()` | New command | -| `/session` | `run_session_selector()` | Replace `SessionPickerApp` call | - -Remaining `ChoiceInput` calls in `slash.py` (editor picker at line 356, -undo turn picker at line 957) are out of scope for this spec. - ---- - -## Section 7: Testing - -All tests follow the pattern in `test_tui_card_selector.py` — pure state/render -logic, no TTY. - -| Test file | What it tests | -|-----------|---------------| -| `test_selectors_simple.py` | `SelectorConfig` builds correctly for theme, thinking, show\_images, extension; `is_current` placement; on\_change fires on move | -| `test_selector_groups.py` | `SelectorHeader` rows appear in correct positions; cursor nav skips them; wraps correctly | -| `test_settings_selector.py` | Cycling values; multi-item changes accumulate; cancel returns `None` | -| `test_scoped_models_selector.py` | Toggle, reorder (Alt+↑↓), enable-all, clear-all, cancel returns `ScopedResult(kind="unchanged")` | -| `test_session_search.py` | `parse_search_query` handles fuzzy/regex/phrase; `filter_and_sort_sessions` correct results per sort mode and name filter | - -Session selector UI tests are integration/manual — state machine is async and TTY-bound. - ---- - -## Decisions & Rationale - -| Decision | Rationale | -|----------|-----------| -| Model selector: flat list, not group-divided | Pi renders models flat with `[provider]` badge, current-first. Group headers were assumed from the old `model_picker.py` pattern but not present in Pi source. | -| `SelectorHeader` kept for config\_selector | Grouped resource display (user-scope / project-scope dividers) still benefits from inline headers. | -| Session picker: custom `Application` | 1023-line Pi source with scope, sort, name-filter, tree display, search, delete, rename — far more than a scope toggle. Not reducible to `run_selector()`. | -| OAuth: add `auth_type` + status display | Pi shows ✓ configured / • unconfigured / ✓ env: per-provider. Without this, the picker loses meaningful signal. | -| Settings: new interactive `/settings` | Current command is read-only Rich table. This upgrades it to a proper editor. Not a ChoiceInput replacement — ChoiceInput is not used in `/settings`. | -| `ScopedResult` dataclass vs. `| object` | Typed discriminated result; `| object` defeats Pyright and hides caller bugs. | -| `tree_selector` deferred, no stub file | No stub — a file that always raises `NotImplementedError` is dead code that creates a false import surface. | -| Three implementation plans | Plan B (session) and Plan C (settings/config) are each substantial; splitting avoids one giant plan that can't be reviewed or shipped incrementally. | -| Timeout in `extension.py`, not `selector.py` | `asyncio.wait_for` in the callee keeps `selector.py` clean; timeout is only needed for the extension use case. | -| Session selector returns ID, not path | Pi's session selector returns a file path; Pythinker's callers at `slash.py:622/635` and `cli/__init__.py:979` all treat the return value as a session ID. Translating path → ID inside `run_session_selector` avoids touching all callers in Plan B scope. | -| `session_search.py` as a separate module | Pi separates `session-selector-search.ts` as pure, testable functions. Same split here makes the search/sort logic independently unit-testable without spinning up a full `Application`. | -| Model selector takes `list[ModelEntry]`, not `list[ProviderGroup]` | Pi renders a flat list; `ProviderGroup` is a Pythinker-only concept from the old `model_picker.py`. The one-line delegation wrapper flattens groups → entries so existing callers of `model_picker.py` need no changes in Plan B. | diff --git a/docs/superpowers/specs/2026-05-07-terminal-work-log-refresh-design.md b/docs/superpowers/specs/2026-05-07-terminal-work-log-refresh-design.md deleted file mode 100644 index 80b54e53..00000000 --- a/docs/superpowers/specs/2026-05-07-terminal-work-log-refresh-design.md +++ /dev/null @@ -1,154 +0,0 @@ -# Terminal Work-Log Refresh Design - -## Goal - -Enhance the Pythinker Code terminal UI so users get a more professional, readable view while the agent is working, using tools, applying changes, and showing reports. The design should be inspired by OpenCode's clear session timeline, but adapted to Pythinker's existing Rich and prompt-toolkit shell instead of copying OpenCode's TUI architecture. - -## Scope - -This redesign targets the current shell UI in `src/pythinker_code/ui/shell/visualize/`, primarily `_blocks.py` and `_live_view.py`. - -Included: - -- Live agent activity states for thinking, composing, tool execution, compaction, MCP loading, and side questions. -- Tool call rendering for running, completed, failed, denied, and interrupted states. -- Professional cards for substantial outputs: diffs, shell commands, todos, background tasks, plans, and report-like brief output. -- Clear subagent activity summaries nested under parent agent tool calls. -- Consistent status language and visual hierarchy across live and flushed history output. - -Not included in the first pass: - -- Replacing prompt-toolkit or Rich Live. -- Rebuilding the event protocol. -- Changing agent behavior or tool execution semantics. -- Copying OpenCode's Solid/OpenTUI components directly. - -## Design Direction - -Use an "inspired adaptation" approach. Pythinker keeps its identity, keyboard behavior, approvals, questions, and Rich rendering. The UI adopts OpenCode's stronger work-log structure: concise running rows, polished completed cards, consistent labels, and easier scanning of what changed. - -The default startup copy should use the product name `Pythinker Code`, while internal protocol names and compatibility-facing labels can remain `Pythinker CLI` where changing them would be broader than the UI request. - -## Architecture - -The existing wire flow remains the source of truth: - -- `_LiveView` consumes `TurnBegin`, `StepBegin`, content parts, tool calls, tool results, status updates, approval/question requests, notifications, and `TurnEnd`. -- `_ContentBlock` owns streaming assistant text and thinking indicators. -- `_ToolCallBlock` owns the live and final representation of one tool call. -- `_StatusBlock` owns the compact context/status footer. -- Tool results expose display blocks through `ToolReturnValue.display`. - -The redesign adds small render helpers inside the shell visualize layer rather than adding backend dependencies: - -- `WorkLogEntry`: common layout for one-line or multi-line activity rows. -- `WorkLogCard`: common bordered panel for substantial tool/report output. -- `ToolStyle`: mapping from tool names to label, icon, color, and action wording. -- `DisplayBlockRenderer`: local renderer for display blocks and grouped diff blocks. -- `StatusLine`: clearer status footer backed by `_StatusBlock`. - -These helpers should be private to the shell UI until a concrete reuse need appears. - -## Component Behavior - -### Live Activity - -Thinking and composing stay lightweight, but the labels should read like deliberate work states instead of raw spinner text. Examples: - -- `Thinking ... 4s · 1.2k tokens · 46 tok/s` -- `Composing ... 8s · 520 tokens` -- `Connecting MCP servers ... 2/3 connected · 18 tools` when status data is available. -- `Compacting context ...` - -The existing hidden-thinking behavior remains. If `show_thinking_stream` is enabled, the reasoning preview still works. - -### Tool Entries - -Each tool call should render as a work-log entry with: - -- State icon or spinner. -- Human label: `Read`, `Search`, `Edit`, `Patch`, `Shell`, `Todo`, `Subagent`, `Fetch`, `Ask`, `Skill`, or fallback tool name. -- Target/detail extracted from streamed arguments using existing `extract_key_argument` behavior. -- Status color: running, completed, failed, denied, interrupted. -- Optional child content for display blocks. - -Completed tools should prefer concise rows unless they produced meaningful display blocks. Failed tools should show a concise error line. Denials and user dismissals should be subdued and not styled like crashes. - -### Output Cards - -Substantial tool outputs should render as cards with consistent padding, border style, and titles: - -- Diffs: group consecutive `DiffDisplayBlock` entries by file as today, but wrap them in a clearer card with file path and `+N -N` summary. -- Shell: show command, working directory if known, and output preview. Long output can keep existing pager/expansion behavior where available. -- Todos: show statuses with consistent symbols and muted completed items. -- Background tasks: show task id, state, kind, and description in a compact card. -- Brief reports: render markdown in a report card when the brief is substantial; keep simple brief text inline when short. -- Plans: keep `PlanDisplay` as a bordered panel, but align title, subtitle, padding, and border color with the work-log card language. - -### Subagents - -Subagent activity stays nested under the parent agent tool call. The parent entry should show: - -- Subagent type and id when known. -- Current or latest sub-tool call. -- A compact summary of completed sub-tool calls, capped to avoid overwhelming the main timeline. -- Error state if any sub-tool result failed. - -The first pass does not need multi-level nested subagent visualization beyond the current parent-child relationship. - -### Approvals And Questions - -Approvals and questions remain top-priority interactive panels. The first pass does not need to restructure their behavior, but their border colors and typography should not conflict with the new work-log style. Existing keyboard behavior and pager behavior must be preserved. - -## Data Flow - -No protocol changes are required for the first pass. - -- `ToolCall` creates a running `WorkLogEntry`. -- `ToolCallPart` updates the target/detail as arguments stream. -- `ToolResult` finalizes the entry and renders display blocks. -- `SubagentEvent` updates the parent tool entry with nested activity. -- `StatusUpdate` updates the status footer and any MCP-loading detail. -- `PlanDisplay` flushes active content/tool output before printing a work-log-style plan panel. -- `Notification` remains short-lived in the live area and flushes into history. - -If future UX work needs richer summaries, add optional display block metadata from tools. Do not block the first pass on backend changes. - -## Error Handling - -Failures should be explicit and concise: - -- Tool errors render red work-log entries with the tool label, target, and concise message. -- Permission rejections, dismissed questions, and policy/rule denials use a muted denied/interrupted style. -- Interrupted runs finalize unfinished tools as interrupted, matching existing cleanup behavior. -- Unknown display blocks are ignored as today, but unknown tool names still get a generic work-log entry. - -The UI should avoid dumping large raw payloads into the main timeline unless the user asks for details through existing expansion/pager behavior. - -## Testing - -Add focused tests for pure rendering behavior before implementation changes: - -- Tool label and target formatting for representative tools. -- Running, completed, failed, denied, and interrupted work-log states. -- Grouped diff display blocks with add/remove totals. -- Todo display block rendering. -- Plan panel title/subtitle copy. -- Startup welcome copy using `Pythinker Code`. - -Use Rich console capture for render assertions where practical. Keep tests close to `tests/ui_and_conv/` because this is shell UI behavior. - -Manual smoke test after implementation: - -- Start `pythinker-code` in a sample repository. -- Trigger read, grep, edit, apply patch, shell, todo, plan, subagent, approval, question, compaction, and interrupt flows. -- Confirm the terminal remains readable on narrow and wide widths. - -## Acceptance Criteria - -- The startup welcome says `Welcome to Pythinker Code!`. -- Running tools are easy to distinguish from completed and failed tools. -- Diffs, todos, shell output, plans, and report-like output share a consistent visual language. -- Existing approvals, questions, queued input, steering input, and interrupt behavior continue to work. -- Existing wire protocol compatibility is preserved. -- Focused UI tests pass, and broader `make check` / relevant pytest targets are run before completion. diff --git a/docs/superpowers/specs/2026-05-20-pythinker-review-foundation-design.md b/docs/superpowers/specs/2026-05-20-pythinker-review-foundation-design.md deleted file mode 100644 index af226164..00000000 --- a/docs/superpowers/specs/2026-05-20-pythinker-review-foundation-design.md +++ /dev/null @@ -1,826 +0,0 @@ -# Pythinker Review — Phase 1: Review/Debug/Security Foundation + Diff Gate - -**Date:** 2026-05-20 -**Status:** Implemented in tree — revised after product-direction review, blackbox design review, and blackbox hardening pass -**Scope:** Phase 1 of the shift toward Pythinker as a professional security -reviewer, debugger, and code-reviewer agent. Coding/editing remains available, -but the default product posture is evidence-first diagnosis and review. This -phase ports the three repositories mounted in this workspace's `blackbox/` -folder into Pythinker architecture: `blackbox/clawpatch-main`, -`blackbox/code-review`, and `blackbox/deepsec-main`. Any module, prompt, rule, -or workflow not ported must be documented as an explicit compatibility -exception. - -## 1. Goal - -Ship the substrate and first useful capabilities for an agent-first review, -debugging, and security-analysis product. - -- Product direction: Pythinker's primary workflow becomes read-only professional - analysis first: inspect diffs, logs, stack traces, tests, and code paths; - produce severity-scored findings; explain root cause; and recommend fixes. - It must not apply code changes unless the user explicitly asks for remediation - after reviewing findings. -- Blackbox port: implement a direct Pythinker port of the three repos in - `blackbox/`: review behavior from `blackbox/code-review`, security behavior - from `blackbox/deepsec-main`, and diagnosis/context/fix-plan behavior from - `blackbox/clawpatch-main`. Do not replace blackbox behavior with a weaker - "inspired by" design unless the exception is recorded in the blackbox parity - map with rationale and tests. -- Substrate: a new `packages/pythinker-review` workspace package containing the - reusable review/debug/security engine, findings data model, JSON-on-disk - store, structured diff renderer, deterministic security-signal scanner, - debugger input normalizers, output formatters, and standalone Typer CLIs. -- Pythinker integration: `pythinker-code` owns the root CLI wrappers and built-in - subagent YAML roles (`code-reviewer`, `security-reviewer`, `debugger`). The - package does not auto-register subagents because the current Pythinker - subagent registry is populated from agent YAML at runtime. -- First capabilities: `pythinker review diff`, `pythinker review diff --mode deslopify`, - `pythinker secscan diff`, and `pythinker debug failure` review only the relevant - diff/log/failure evidence, emit findings in pretty / JSON / SARIF where applicable, - preserve evidence/test/minimum-fix-scope metadata, and optionally fail the build on - a configurable severity threshold. Code-reviewr-derived PR assistant commands - (`describe`, `improve`/`suggest`, `ask`, `labels`, `changelog`, `docs`) are read-only - artifact generators over the same bounded diff context. - -Out of scope for Phase 1 (each gets its own future spec): - -- Full whole-repo audit as the default path. Phase 1 may include bounded related - context, but Reviewflow-style full local audit remains a later product surface. -- Full deepsec matcher plugin marketplace and multi-machine worker fan-out. - Phase 1 still carries Deepsec-like rule metadata, validation, and fail-loud - behavior, but keeps the implementation in-process and dependency-light. -- PR-provider write integrations (GitHub / GitLab / Bitbucket / Azure DevOps). -- Auto-fix loops that write source files. Findings may include suggested patches, - but applying them requires an explicit user remediation request. - -## 2. Success criteria - -A change is done when: - -1. `make check-pythinker-review && make test-pythinker-review` pass. -2. A blackbox parity map exists in the spec or package docs covering all three - source repositories, mapping source modules/prompts/rules/workflows to - Pythinker targets, with every intentional deviation called out and covered by - tests where practical. -3. From any git repo on `main`, after creating a branch with a planted bug and - a planted security issue, `pythinker review diff --with-security` produces - at least one finding for each, in pretty / JSON / SARIF format. -4. Given a failing test log or stack trace tied to a changed file, - `pythinker debug failure ` produces a root-cause finding with - reproduction evidence, changed-file correlation, and recommended next action - without modifying files. -5. `pythinker review diff --fail-on high` exits non-zero when a high-or-above - finding is produced and exits zero otherwise. -6. Any chunk timeout, malformed model output after retry, LLM error, or worker - exception makes the run fail non-green by default. A partial run can complete - only when the user passes `--allow-partial`, and the output must make partial - coverage obvious. -7. `--save` writes a complete `runs//` directory; `pythinker review list`, - `pythinker review show `, `pythinker review next`, and - `pythinker review show-finding ` reproduce or inspect findings without - another LLM call. -8. `pythinker review diff` defaults `--model` to the active Pythinker model by - having `pythinker-code` inject a model adapter into `pythinker-review`; the - standalone `pythinker-review` CLI never imports `pythinker-code` to discover - config/auth. -9. Inside an interactive Pythinker session, dispatching to `code-reviewer`, - `security-reviewer`, or `debugger` uses YAML-defined built-in roles and - returns output in the existing `SUMMARY / EVIDENCE / CHANGES / RISKS / - BLOCKERS` shape. -10. The default Pythinker agent prompt and role guidance make review/diagnosis - the first step for ambiguous engineering requests; coding/editing remains - opt-in or delegated after findings are accepted. -11. No regression in existing `pythinker` commands; existing `review` and - `verifier` subagent roles continue to work. -12. Phase 1 adds no unapproved third-party runtime dependencies. Use stdlib - `subprocess` for git, stdlib timestamp-random run IDs instead of GitPython or - `ulid-py`, and char/line budget approximations instead of tokenizer deps. -13. Reviewflow-style evidence validation rejects unsafe paths, out-of-chunk files, - out-of-hunk line ranges, and non-matching evidence snippets before findings are - persisted. -14. DeepSec-style security context includes expanded deterministic signals, - CWE/severity hints, tech detection, and batch-scoped advisor highlights/slug notes. -15. Code-reviewr PR assistant workflows are available as read-only artifact commands - with strict JSON schemas and no provider-side posting or source-file mutation. -16. Reviewflow project/feature/run/finding/patch state models and - mapping/reporting utilities exist as the substrate for later whole-repo audit, - revalidation, and fix orchestration surfaces. - -## 3. Non-goals - -- Auto-applying fixes. Findings may carry a `suggestion.patch` string for the - user to review, but Phase 1 never writes to source files. -- Rebranding Pythinker as a generic coding CLI. The review/debug/security agent - posture is the primary product direction; coding is a controlled remediation - workflow. -- Posting comments to GitHub / GitLab / etc. Pretty / JSON / SARIF only. -- Whole-repo scanning by default. Diff/log/failure scoped analysis plus bounded - current-file / related-file context is the Phase 1 default. -- Package-based subagent discovery. Phase 1 registers subagents through the - existing agent YAML mechanism. -- Parallel worker machines. In-process asyncio fan-out only. -- New telemetry endpoints, hosted services, GitPython, `ulid-py`, or other new - third-party runtime dependencies without explicit maintainer approval. - -## 4. Architecture - -### 4.1 Package layout - -``` -packages/pythinker-review/ -├── pyproject.toml # console_scripts: pythinker-review/secscan/debug -├── README.md -├── src/pythinker_review/ -│ ├── __init__.py -│ ├── cli/ # standalone Typer apps, automation surfaces only -│ ├── engine/ # diff_source, structured_diff, context, runner, dedupe -│ ├── diagnostics/ # failure-log parsing, stack traces, repro evidence -│ ├── llm/ # tiny ReviewLLM protocol + adapters used by tests/standalone CLI -│ ├── reviewers/ # code/security/debug passes, schema, prompt assets -│ ├── signals/ # Deepsec-like deterministic signal/rule registry -│ ├── store/ # findings_store, run, models, gitignore -│ └── output/ # pretty, json, sarif -└── tests/ - ├── unit/ - └── e2e/ -``` - -Pythinker integration edits live in `pythinker-code`: - -``` -src/pythinker_code/cli/review.py # delegates to pythinker_review app -src/pythinker_code/cli/secscan.py # delegates to pythinker_review app -src/pythinker_code/cli/debug.py # delegates to diagnostics app -src/pythinker_code/cli/_lazy_group.py # adds review/secscan/debug lazy commands -src/pythinker_code/agents/default/agent.yaml # registers new built-in subagents -src/pythinker_code/agents/default/code_reviewer.yaml -src/pythinker_code/agents/default/security_reviewer.yaml -src/pythinker_code/agents/default/debugger.yaml -``` - -`make` targets follow the existing package pattern: -`make check-pythinker-review`, `make test-pythinker-review`, plus inclusion from -root `make check` / `make test` after the package is added to the workspace. The -verification matrix in `AGENTS.md` gets one new row. - -### 4.2 Dependency direction and model injection - -``` -pythinker-code ──imports/wraps──▶ pythinker-review ──uses──▶ pythinker-core - └──▶ stdlib subprocess + existing deps -``` - -`pythinker-review` does not import `pythinker-code`. It exposes a small -`ReviewLLM` protocol, for example: - -```python -class ReviewLLM(Protocol): - model_display_name: str - - async def complete_json(self, *, system: str, user: str, timeout_s: float) -> str: ... -``` - -- `pythinker-code` creates the active model using existing Pythinker config, - OAuth, managed-provider, and session code, then passes a `ReviewLLM` adapter - into the review engine. This is what makes `pythinker review diff --model` - default to the active Pythinker provider. -- The standalone `pythinker-review` / `pythinker-secscan` console scripts use - only explicit CLI/env configuration or the fake test LLM. They are useful for - package tests and simple automation, but they do not claim to know the active - Pythinker session model. -- Tests inject a fake `ReviewLLM` returning canned JSON; no real secrets or LLM - calls are required for normal CI. - -### 4.3 Root CLI and standalone CLI - -The shared engine powers agent-first surfaces plus automation wrappers: - -- `pythinker review diff` → Pythinker-integrated code review. -- `pythinker review diff --with-security` → code + security passes in parallel. -- `pythinker secscan diff` → Pythinker-integrated security pass. -- `pythinker debug failure ` → debugger/root-cause pass over logs, - stack traces, failing test output, and the relevant diff/context. -- `pythinker-review diff` / `pythinker-secscan diff` / `pythinker-debug failure` - → standalone package CLIs with explicit/env model configuration. - -The three root commands are added through `_lazy_group.py` so `pythinker --help` -shows them without importing heavy review code during normal startup. - -### 4.4 Blackbox parity gate - -Implementation must start with an intake of the three mounted repos under -`blackbox/`: `blackbox/clawpatch-main`, `blackbox/code-review`, and -`blackbox/deepsec-main`. Engine implementation must not begin until those repos -have been audited and mapped into `packages/pythinker-review/docs/blackbox-parity.md`. - -Initial source-to-target map: - -| Blackbox source | Behavior to preserve | Pythinker target | Phase 1 status | -|---|---|---|---| -| `blackbox/code-review` | Diff-scoped reviewer rubric, strict evidence, no vague findings, line-anchored output | `reviewers/code_review.py`, `reviewers/prompts/code_review.system.md`, `engine/structured_diff.py` | Source mounted; audit and map before implementation | -| `blackbox/deepsec-main` | Direct-mode fail-loud semantics, signal/rule metadata, validation before emitting findings | `signals/models.py`, `signals/scanner.py`, `reviewers/security_review.py`, `engine/runner.py` | Source mounted; audit and map before implementation | -| `blackbox/clawpatch-main` | Structured diff/context shape, local diagnosis, root-cause/fix-plan separation | `engine/context.py`, `diagnostics/`, `reviewers/debug_review.py` | Source mounted; audit and map before implementation | -| `blackbox/code-review` provider concepts | PR provider abstractions and comment formatting | Deferred PR-provider phase | Explicitly out of Phase 1 write integrations | - -Every implementation task that touches these behaviors must update this map (or -`packages/pythinker-review/docs/blackbox-parity.md` after package creation) with -actual source references and test coverage. - -## 5. Components - -All paths in this section are inside `src/pythinker_review/` unless explicitly -marked as `pythinker-code` integration. - -### 5.1 `engine/` - -**`diff_source.py`** — resolves the diff, file list, and SHAs. - -Algorithm: - -1. If `--range A..B` is given, use it directly. -2. Else if `--working-tree`, resolve tracked working-tree changes plus staged - changes; include untracked non-ignored files as added files. -3. Else if `--staged`, use `git diff --cached`. -4. Else (default): resolve base ref by trying `--base` (default `origin/main`), - falling back to `main`, then `master`. Diff is `git merge-base HEAD ` - .. HEAD. -5. Use stdlib `subprocess.run(...)` with bounded timeouts. Do not add GitPython. -6. Reject the run with exit 2 if the diff/file list is empty after filters. - -Output: a `ResolvedDiff` containing raw patch text, base/head SHAs, base ref, -source label, and POSIX repo-relative changed file paths. - -**`structured_diff.py`** — converts unified diff to blackbox-style review input. - -For each file and hunk, render: - -```text -## File: 'src/file.py' - -@@ ... @@ optional section header -__new hunk__ -42 unchanged context -43 + added or changed line -44 unchanged context -__old hunk__ - unchanged context -- removed line - unchanged context -``` - -Rules: - -- New hunk lines are numbered with post-change file line numbers. -- Old hunk lines keep removed content for comparison but are not the primary - location target. -- Reviewers must flag only issues introduced by the diff. A finding location - must point to a changed post-change line when possible. For pure deletions - that introduce risk, anchor to the nearest post-change hunk line and include - the removed line in `evidence_snippet`. -- The renderer keeps enough context to understand scope boundaries and must not - treat an opening brace / block boundary at the end of a hunk as incomplete code. - -**`context.py`** — gathers bounded file context. - -- Include the full current changed file when it fits the chunk budget. -- Otherwise include the structured diff plus bounded current-file sections around - each hunk. -- Include base-file snippets from `git show :` only for the - hunks under review, so the model can compare before/after behavior. -- For security pass only, include small related-file snippets when cheaply - discoverable from imports or obvious local call targets. This is bounded, - best-effort context, not whole-repo scanning. - -**`diagnostics.py` / `diagnostics/`** — normalizes debugger inputs. - -- Accept failing test logs, stack traces, command output, exit codes, and optional - reproduction commands. -- Extract file paths, line numbers, exception names, assertion text, subprocess - commands, and changed-file correlations. -- Keep logs redacted and bounded; never persist secrets from logs verbatim. -- Produce `DiagnosticInput` records consumed by `debug_review.py` and stored with - run metadata for reproducibility. - -**`chunker.py`** — splits review work. - -- Default: one chunk per changed file. -- If a single file exceeds the per-chunk budget, split at hunk boundaries. -- Honors `--include ` / `--exclude ` before chunking. -- Skips binary diffs and vendored/generated paths by default: - `node_modules/`, `.venv/`, `dist/`, `build/`, `.pythinker-review/`, `.git/`, - coverage/build artifacts, and common generated lock/output directories. -- `--no-skip-vendored` disables only the vendored/generated skip list, not git - safety checks. - -Output: `Chunk(file, hunks, structured_diff, current_context, base_context, -security_signals, related_context)`. - -**`runner.py`** — concurrency, retry, and fail-closed behavior. - -- Asyncio worker pool sized by `--jobs` (default 4). -- For each `(chunk, pass)` pair, schedule one LLM call. -- Per-chunk timeout (default 120s). -- Malformed JSON gets one retry with a stricter suffix. A second failure marks - the chunk failed. -- Any timeout, chunk LLM error, worker exception, or malformed output after - retry increments `chunks_failed` and makes the whole run fail non-green by - default. -- `--allow-partial` converts those chunk failures into a completed-with-warnings - result, but output must list the skipped chunks and JSON output must include a - `warnings` array. -- On Ctrl-C: cancel pending, mark run `cancelled`, flush partial findings, exit - 130. - -**`dedupe.py`** — collapses duplicates. - -Key: `(file, max(start_line, 1), min(end_line, +inf), rule_id)`. When two -findings collide, keep higher `severity`, then higher `confidence`, then earlier -`pass` order (`security_review` wins ties on intent). - -### 5.2 `signals/` - -Phase 1 ports the Deepsec direct-mode shape into a lightweight in-process -scanner. Signals are prompt anchors only; they are not emitted as findings unless -the reviewer validates them against code and bounded context. - -Built-in signal families must carry rule metadata, source/sink vocabulary, -confidence reason, exploitability hint, and mitigation expectation where known: - -- secrets/credentials added in the diff, with redaction-safe evidence handling; -- command/process execution fed by changed variables; -- SQL/template/query construction from user-controlled-looking values; -- SSRF-like fetch/request/proxy patterns; -- deserialization/archive extraction of untrusted-looking data; -- crypto misuse and insecure random/token generation; -- authn/authz/permission bypass hints; -- dependency, CI, workflow, or package-manager changes that introduce - supply-chain risk. - -`Signal` fields: `rule_id`, `file`, `line`, `snippet`, `reason`, `confidence`, -`source_kind`, `sink_kind`, `exploitability`, and `mitigation_hint` where -available. The security prompt receives signals grouped by file with an explicit -reminder: "Signals are starting points; verify in code before emitting a -finding. Prefer no finding over unvalidated speculation." - -### 5.3 `reviewers/` - -**`schema.py`** — pydantic models the LLM is asked to produce. - -```python -class ReviewerOutput(BaseModel): - findings: list[RawFinding] - -class RawFinding(BaseModel): - rule_id: str - title: str = Field(max_length=80) - rationale: str - category: Category - severity: Severity - file: str - start_line: int = Field(ge=1) - end_line: int = Field(ge=1) - confidence: float = Field(ge=0.0, le=1.0) - evidence_snippet: str | None = None - suggestion: Suggestion | None = None -``` - -Reviewer post-processing converts `RawFinding` → `Finding` by attaching `id`, -`pass`, `created_at`, `run_id`, and the head SHA for `Location`. It validates -that file paths are repo-relative POSIX paths and that line ranges intersect the -changed post-change file where possible. - -**`code_review.py`** — prompt + caller for the code-review pass. Covers -correctness, design, performance, readability, missing tests, and API breakage. -Prompt rules are ported from `blackbox/code-review` and must include: - -- focus on issues introduced by this diff; -- prefer no finding over vague speculation; -- flag clear bugs/security issues even when trigger scenarios are narrow; -- low-severity concerns require high confidence; -- cite concrete failure modes and changed lines; -- output strict JSON only. - -**`security_review.py`** — prompt + caller for the security pass. Covers -injection, authn/authz, secrets, SSRF, deserialization, crypto misuse, -supply-chain risk, unsafe defaults, and project-specific mitigations found in -bounded context. The prompt receives deterministic `signals` and project context -but must verify findings against code before output. - -**`debug_review.py`** — prompt + caller for the debugger/root-cause pass. It -receives normalized failing test output, stack traces, command logs, exit codes, -structured diff/context, and related files. It must identify likely root cause, -changed-line correlation, reproduction command/evidence, and next action without -patching code. - -All reviewer/debugger passes: - -- Use strict pydantic validation at the model boundary. -- Retry once on malformed JSON with a stricter prompt suffix. -- Treat the second malformed response as a chunk failure. The runner decides - whether that fails the whole run (`default`) or becomes a warning - (`--allow-partial`). - -### 5.4 `store/` - -**`models.py`** — pydantic models defined in §6 below. - -**`findings_store.py`** — append-only JSONL writer. - -- `runs//findings.jsonl` is opened once per run and appended per finding - (`fsync` on close, not per write). -- `runs//meta.json` and `index.json` updates use `.tmp` + atomic rename - to avoid partial writes on crash. -- Run ID uses stdlib only: `YYYYMMDDHHMMSS-`. It sorts - lexicographically by time and avoids an unapproved `ulid-py` dependency. - -**`run.py`** — `RunMeta` lifecycle helpers. Wraps create / update / finalize -transitions and records failed chunk metadata. - -**`gitignore.py`** — idempotent `.gitignore` patcher. - -- Triggers only on first `--save` per repo. -- Only modifies `.gitignore` if it already exists. If it doesn't, the run still - succeeds; it logs an info note that `.pythinker-review/` was not added. -- Adds one line `.pythinker-review/` under a `# pythinker-review` marker comment, - only if not already present. - -### 5.5 `output/` - -**`pretty.py`** — human rendering using existing workspace dependencies only. -Per finding: severity chip, file:line, title, rationale, optional suggestion. -Findings are grouped by file and sorted by `(severity desc, file, start_line)`. -If partial failures occurred, a warning block appears before findings. - -**`json.py`** — emits: - -```json -{"run": {}, "findings": []} -``` - -`run` is the full `RunMeta` (so `run.chunk_failures` carries any partial-coverage -warnings — no separate `warnings` array needed at this layer). The `findings` -schema is the same as on-disk JSONL after re-aggregation. - -**`sarif.py`** — SARIF 2.1.0. Severity mapping: - -| Our severity | SARIF level | -|---|---| -| critical, high | `error` | -| medium | `warning` | -| low, info | `note` | - -`rule_id` maps to `ruleId`; `Location` maps to one `physicalLocation` with -`region.startLine` / `endLine`. Chunk/runtime failures are represented as tool -notifications when SARIF supports them, and remain visible in JSON/pretty. - -### 5.6 `cli/` - -Standalone package CLIs: - -```text -pythinker-review diff [--with-security] [shared flags] -pythinker-review list [--limit N] -pythinker-review show [--format pretty|json|sarif] - -pythinker-secscan diff [shared flags] -pythinker-debug failure [--command ] [shared flags] -``` - -`pythinker-code` exposes root commands by adding lazy wrappers: - -```text -pythinker review diff [--with-security] [shared flags] -pythinker review list [--limit N] -pythinker review show [--format pretty|json|sarif] - -pythinker secscan diff [shared flags] -pythinker debug failure [--command ] [shared flags] -``` - -The wrappers import `pythinker_review`, build a Pythinker-backed `ReviewLLM` -adapter from the active config/session model, and call the shared app/engine. -They do not shell out to `pythinker-review`. - -### 5.7 `pythinker-code` subagent roles - -Phase 1 adds three YAML agent specs, not package auto-registration: - -- `src/pythinker_code/agents/default/code_reviewer.yaml` -- `src/pythinker_code/agents/default/security_reviewer.yaml` -- `src/pythinker_code/agents/default/debugger.yaml` - -`src/pythinker_code/agents/default/agent.yaml` registers them under: - -```yaml -subagents: - code-reviewer: - path: ./code_reviewer.yaml - description: "Diff-focused code review with severity-scored findings." - security-reviewer: - path: ./security_reviewer.yaml - description: "Diff-focused security review with validated findings." - debugger: - path: ./debugger.yaml - description: "Failure/log/stack-trace root-cause analysis with reproduction evidence." -``` - -Each role is read-only by convention and uses existing shell/read/grep tools to -run `pythinker review diff`, `pythinker secscan diff`, or `pythinker debug -failure`, then reformats output into the structured response block below. - -This shell-out path is intentional for Phase 1: it keeps the YAML role free of -new Python wiring, lets the subagent reuse the exact CLI users hit, and lines -up with how `coder` / `review` / `verifier` currently operate. Cost: one -subprocess + re-import of the review engine per dispatch. Acceptable for diff- -sized runs. Future work (deferred, separate spec): an in-process call path that -shares the parent agent's `ReviewLLM` adapter to avoid the subprocess hop. - - -```text -### SUMMARY -### EVIDENCE -### CHANGES -None. -### RISKS -### BLOCKERS -``` - -The existing `review` subagent role is left untouched; the new debugger role is additive and must not weaken `review` or `verifier`. - -## 6. Data model - -```python -class Severity(str, Enum): - critical = "critical" # exploitable now, or correctness break with prod impact - high = "high" # likely bug or vuln; fix before merge - medium = "medium" # real issue, defer-with-issue acceptable - low = "low" # nit, style, micro-perf - info = "info" # FYI, no action required - -class Category(str, Enum): - correctness = "correctness" - security = "security" - debugging = "debugging" - performance = "performance" - readability = "readability" - test_coverage = "test_coverage" - api_design = "api_design" - dependency = "dependency" - secret = "secret" - -class Location(BaseModel): - file: str # repo-relative POSIX path - start_line: int # 1-indexed post-change line, inclusive - end_line: int # 1-indexed post-change line, inclusive - sha: str | None = None # commit SHA the line numbers refer to - -class Suggestion(BaseModel): - summary: str # one-sentence what-to-change - patch: str | None = None # unified diff; validated parseable, not applied - -class Finding(BaseModel): - id: str # sha256(rule_id + file + start_line + title)[:12] - rule_id: str # e.g. "sec.injection.sql", "review.error_handling" - title: str # ≤80 chars - rationale: str # markdown - category: Category - severity: Severity - location: Location - pass_: Literal["code_review", "security_review", "debug_review"] = Field(alias="pass") - suggestion: Suggestion | None = None - evidence_snippet: str | None = None - confidence: float # 0.0–1.0 - confidence_reason: str | None = None - exploitability: str | None = None # security/debug impact narrative - reproduction: str | None = None # debugger reproduction command/evidence - triage: Literal["open", "false_positive", "accepted", "wont_fix"] = "open" - triage_note: str | None = None - created_at: datetime - run_id: str - -class ChunkFailure(BaseModel): - file: str - pass_: Literal["code_review", "security_review", "debug_review"] = Field(alias="pass") - reason: Literal["timeout", "llm_error", "malformed_output", "worker_error"] - message: str - -class RunMeta(BaseModel): - id: str # YYYYMMDDHHMMSS-<8 hex chars> - started_at: datetime - finished_at: datetime | None - status: Literal["running", "completed", "completed_with_warnings", "failed", "cancelled"] - repo_root: str - branch: str | None - head_sha: str - base_ref: str - base_sha: str - source_label: str # e.g. git-diff:origin/main, staged, working-tree - passes: list[Literal["code_review", "security_review", "debug_review"]] - model: str # provider:model-id or standalone model id - chunks_total: int - chunks_done: int - chunks_failed: int - findings_count: int - allow_partial: bool - chunk_failures: list[ChunkFailure] - config_hash: str # hash of reviewer prompts + rule list + signal rules -``` - -### 6.1 On-disk layout - -``` -.pythinker-review/ -├── index.json # {"runs": [{id, started_at, branch, head_sha, status, findings_count}, ...]} -└── runs/ - └── 20260520123045-a1b2c3d4/ - ├── meta.json # RunMeta, including chunk_failures + allow_partial - ├── findings.jsonl # one Finding per line, append-only - └── diff.patch # the diff reviewed (for reproducibility) -``` - -`index.json` is trimmed to the most recent 200 runs; older entries stay on disk -under `runs/` but are not indexed. `pythinker review list` reads `index.json`; -`pythinker review show ` reads `runs//` directly so unindexed older runs -are still recoverable when the user knows the ID. - -## 7. CLI surface - -Shared option group for `review diff` / `secscan diff`: - -| Flag | Default | Purpose | -|---|---|---| -| `--base ` | `origin/main` → `main` → `master` | Base ref for `merge-base HEAD ` | -| `--staged` | off | Diff staged vs HEAD | -| `--working-tree` | off | Diff working tree, staged changes, and untracked non-ignored files | -| `--range A..B` | — | Arbitrary range | -| `--format pretty\|json\|sarif` | `pretty` if TTY else `json` | Output format | -| `--fail-on critical\|high\|medium\|low\|none` | `high` | Exit non-zero when finding ≥ threshold | -| `--allow-partial` | off | Permit completed-with-warnings output when chunks fail; otherwise chunk failures exit 4 | -| `--jobs N` | `4` | Worker pool size | -| `--model ` | active Pythinker provider for `pythinker ...`; explicit/env for standalone CLI | Override LLM | -| `--save / --no-save` | `--save` | Persist to `.pythinker-review/runs//` | -| `--quiet` | off | Suppress progress UI | -| `--include ` | — (repeatable) | Filter to matching files (gitignore-style glob, matched against repo-relative POSIX path) | -| `--exclude ` | — (repeatable) | Skip matching files (same glob semantics as `--include`; `--exclude` wins on conflict) | -| `--no-skip-vendored` | off | Don't auto-skip vendored/generated paths | - -Exit codes: - -- `0` — success, no finding ≥ `--fail-on`, and no chunk/runtime failures. -- `1` — success, at least one finding ≥ `--fail-on`. -- `2` — preflight error (no git, no diff, base ref unresolvable, bad flags). -- `3` — environment/provider error (LLM auth, quota, network, or other failure - that prevents the run from making progress). Rerunning without fixing - credentials/connectivity will fail again. CI should treat this as - "operator-actionable". -- `4` — chunk-level failure without `--allow-partial` (per-chunk timeout, - malformed output after retry, worker exception). The run completed enough to - finalize `meta.json` with `status: failed` and `chunk_failures` populated. - Rerunning may succeed. -- `130` — Ctrl-C (run marked `cancelled`). - -If `--allow-partial` is set and chunks fail, exit code is governed by -`--fail-on` instead of `4`, but output must show -`status: completed_with_warnings`, `chunks_failed > 0`, and the populated -`chunk_failures` list. - -## 8. Error handling - -Fail loud at the boundary and fail closed for CI. - -| Condition | Where caught | Behavior | -|---|---|---| -| `git` missing / not a repo | preflight | exit 2 with clear message | -| base ref unresolvable | preflight | exit 2 | -| empty diff after filters | preflight | exit 2, "no changes to review" | -| LLM auth / quota / network error at runner startup | runner startup | exit 3, surface provider name | -| LLM auth / quota / network error mid-run, affecting all chunks | runner | finalize partials, exit 3 | -| Per-chunk timeout | runner | record `ChunkFailure(reason="timeout")`; default exit 4 | -| Per-chunk LLM error (non-systemic) | runner | record `ChunkFailure(reason="llm_error")`; default exit 4 | -| Malformed JSON | reviewer | one retry; second failure records `ChunkFailure(reason="malformed_output")`; default exit 4 | -| Worker exception | runner | record `ChunkFailure(reason="worker_error")`; default exit 4 | -| Ctrl-C | runner | cancel pending, mark `cancelled`, flush partials, exit 130 | - -`--allow-partial` is the only way for `chunks_failed > 0` to produce a completed -run. Without it, `meta.json` is finalized with `status: failed` and the process -exits 4. This mirrors Deepsec direct mode's fail-loud behavior and prevents a -broken reviewer from masquerading as a green "0 findings" gate. - -Splitting exit 3 (environment/provider) from exit 4 (chunk failure) lets CI -alert differently: exit 3 means "fix your key/network and rerun"; exit 4 means -"the run hit transient or model-quality issues — rerun may succeed, or set -`--allow-partial` and inspect `chunk_failures`". - -## 9. Testing - -Three layers matching existing Pythinker convention: - -**`tests/unit/`** — fake LLM returning canned JSON. Covers: - -- Diff source resolution (range, working-tree, staged, base-ref fallback, empty - diff, untracked files). -- Structured diff rendering with numbered `__new hunk__` / `__old hunk__`, added - files, renamed files, pure deletions, and line-range validation. -- Context gathering budget behavior and bounded related-file snippets. -- Security signal scanner rules and prompt-anchor formatting. -- Chunker boundaries (per-file, per-hunk split, vendored skip, glob filters). -- Runner fail-closed behavior for timeouts, malformed output after retry, worker - exceptions, and `--allow-partial` warnings. -- Dedupe rules (file/line/rule collision, severity tiebreak). -- Store atomicity (mid-write crash leaves no half-files). -- `gitignore.py` idempotency. -- SARIF schema shape, validated against the official SARIF 2.1.0 JSON Schema - using `jsonschema` (already in `uv.lock` transitively; declared as a - `pythinker-review` dev dependency, not a runtime one). -- Severity threshold gate (exit code for each `--fail-on` setting). - -**`tests/e2e/`** — real CLI invocations against fixture git repos with planted -bugs/vulns. Still fake LLM (model selection injected via env var). Verifies exit -codes, file outputs, `--save` persistence, root `pythinker review` / `pythinker secscan` / -`pythinker debug` wrappers, standalone `pythinker-review` / `pythinker-secscan` / -`pythinker-debug`, debugger root-cause output, and lazy CLI command registration. - -**`tests_ai/`** — small set of real-model runs on a curated diff fixture, -asserting recall on planted issues. Gated behind `PYTHINKER_AI_TESTS=1` so -default CI doesn't burn tokens. - -Pythinker integration tests additionally cover: - -- `src/pythinker_code/agents/default/agent.yaml` registers `code-reviewer`, - `security-reviewer`, and `debugger` without changing `review` / `verifier`. -- The new YAML files are included in PyInstaller/package data tests. -- `pythinker review diff` uses the active model adapter path, while standalone - package tests do not import `pythinker_code`. - -## 10. Rollout - -Phase 1 is additive: - -- No behavior change to existing `pythinker` commands. -- Add `packages/pythinker-review` to `[tool.uv.workspace].members` and - `[tool.uv.sources]`. -- Add root dependency from `pythinker-code` to `pythinker-review` so lazy wrapper - modules can import it. -- Add Make targets: `format/check/test/build-pythinker-review`, and include check - / test / build in aggregate targets. -- Update `uv.lock` with `uv sync`; no unapproved new third-party runtime deps. -- Add root CLI wrappers and `_lazy_group.py` entries for `review`, `secscan`, and `debug`. -- Add `code_reviewer.yaml` / `security_reviewer.yaml` / `debugger.yaml` and - register them in default `agent.yaml`. -- Update package/PyInstaller data inclusion tests for the new YAML files and any - prompt/rule assets inside `pythinker-review`. -- AGENTS.md verification matrix gains one row for `packages/pythinker-review`. -- Existing `review` subagent role stays untouched. New roles use distinct - identifiers (`code-reviewer`, `security-reviewer`, `debugger`). -- README's "What's New" gets a Phase 1 entry once shipped. - -Allowed dependencies for Phase 1 are stdlib plus dependencies already present in -the workspace. Specifically: use `subprocess` instead of GitPython and a stdlib -sortable ID instead of `ulid-py`. - -## 11. Phasing (forward look) - -This spec only covers Phase 1. Subsequent phases each get their own spec: - -- **Phase 2 — Local audit (Reviewflow-style)**: semantic slicer for the whole - repo, per-slice review, triage CLI (`pythinker review triage `), - regression diffing between runs, and deeper blackbox parity where full-repo - context is required. -- **Phase 3 — Deep security (deepsec-style)**: external matcher plugin system, - INFO.md project context authoring workflow, FP-cutting revalidation pass, and - multi-machine worker fan-out. Phase 1 already carries the direct-mode rule - metadata shape and fail-loud behavior. -- **Phase 4 — PR provider integrations**: port `blackbox/code-review`'s - `git_providers/` concepts for GitHub / GitLab / Bitbucket / Azure DevOps; add - `pythinker review pr `; ship a GitHub Action. -- **Phase 5 — Fix loop**: `pythinker review fix --finding ` runs an isolated - worktree, validates with configured commands, records a patch attempt. Never - auto-applies. - -Each phase builds on Phase 1's findings store, data model, structured diff -substrate, deterministic signal anchors, and Pythinker subagent roles. - -## 12. Risks and mitigations - -| Risk | Mitigation | -|---|---| -| LLM cost on large diffs | `--jobs` cap, per-chunk size limit, `--exclude`, hard-skip vendored/generated paths by default. Document recommended diff size. | -| False positives erode trust | Phase 1 ships `confidence` and `triage` fields; prompt says low-severity findings require high confidence; Phase 3 revalidation will reduce FP further. | -| False negatives from partial failures | Default fail-closed behavior exits 3 on any chunk failure; `--allow-partial` must be explicit and visibly warns. | -| Prompt regressions silently change output | `config_hash` includes reviewer prompts, rule list, and signal rules. CI `tests_ai/` recall checks on a fixed fixture catch large regressions. | -| SARIF tooling expects specific severities | Mapping documented; SARIF emitter is unit-tested against schema shape. | -| `.gitignore` patcher modifies user files unexpectedly | Only on first `--save`, only if file exists, only adds one line under a marker comment, only if not already present. Documented in README. | -| Active-model integration leaks into standalone package | `pythinker-review` accepts a `ReviewLLM` protocol; only `pythinker-code` builds the active Pythinker adapter. Standalone package tests assert no `pythinker_code` import is required. | -| Subagent registration expectations drift | Phase 1 uses current YAML registration. A separate future design is required for package-discovered subagents. | -| New dependency creep | Use stdlib for git and run IDs; any additional third-party runtime dependency requires explicit maintainer approval. | - -## 13. Open questions - -Implementation planning must choose exact numeric defaults, but the architectural -questions from the blackbox review are resolved above. - -- Exact per-chunk token/character budget for default models. -- Exact list of Phase 1 security signal regexes and their confidence labels. -- Whether `--allow-partial` should be hidden/advanced or documented as a normal - local-development escape hatch. diff --git a/docs/superpowers/specs/2026-05-21-tui-spacing-design.md b/docs/superpowers/specs/2026-05-21-tui-spacing-design.md deleted file mode 100644 index cad337d3..00000000 --- a/docs/superpowers/specs/2026-05-21-tui-spacing-design.md +++ /dev/null @@ -1,128 +0,0 @@ -# TUI Card Spacing Enhancement - -**Date:** 2026-05-21 -**Status:** Approved -**Scope:** Card-style tool call rendering in the Pythinker shell TUI - ---- - -## Problem - -In the current TUI (card style), tool call cards have no vertical padding and no -spacer between the command header and the result body. Consecutive cards also run -together with only a single newline separating them. This makes it hard to visually -group a command with its output or to distinguish where one tool call ends and the -next begins. - ---- - -## Goal - -Add three layers of spacing to make the TUI easier to read: - -1. **Card vertical padding** — breathing room above and below every card -2. **Header-to-results gap** — a blank line separating the command title from the output body -3. **Inter-card gap** — a guaranteed blank line before each card enters scrollback - ---- - -## Design (Option B) - -### Change 1 — Card vertical padding - -**File:** `src/pythinker_code/ui/shell/components/tool_execution.py` -**Location:** `ToolExecutionComponent.render()`, final return statement - -```python -# Before -return Padding(body, (0, 1), style=bg_style) - -# After -return Padding(body, (1, 1), style=bg_style) -``` - -The `(1, 1)` tuple gives Rich's `Padding` 1 blank row top, 1 blank row bottom, 1 column -left, 1 column right. The background tint extends through the padding rows, preserving -the "colored block" appearance while adding vertical air. - -### Change 2 — Header-to-results spacer - -**File:** `src/pythinker_code/ui/shell/components/tool_execution.py` -**Location:** `ToolExecutionComponent.render()`, body assembly block - -```python -# Before -body: RenderableType = children[0] if len(children) == 1 else Group(*children) - -# After -if len(children) <= 1: - body = children[0] if children else Text("") -else: - body = Group(children[0], Text(""), *children[1:]) -``` - -`children` is `[call]`, `[call, result]`, or `[call, result, key_hint]`. The blank -`Text("")` is inserted after `children[0]` (the command header) only when a result or -hint follows. When the tool is still running (only the header exists), behaviour is -unchanged. - -### Change 3 — Explicit inter-card gap - -**File:** `src/pythinker_code/ui/shell/visualize/_live_view.py` -**Locations:** `flush_finished_tool_calls()` and `cleanup()` - -```python -# flush_finished_tool_calls -self._tool_call_blocks.pop(tool_call_id) -console.print() # blank line before card -console.print(block.compose()) - -# cleanup -block = self._tool_call_blocks.pop(tool_call_id) -console.print() # blank line before card -console.print(block.compose()) -``` - -`console.print()` with no arguments emits a single blank line. Combined with the -bottom padding of the previous card and the top padding of the next, this produces a -3-line gap between consecutive cards — clear visual grouping regardless of card size. - ---- - -## Resulting visual rhythm - -``` -[blank — top padding] - find *.md in .agents (limit 100) -[blank — header-to-results spacer] - skills/gen-rust/SKILL.md - skills/release/SKILL.md -[blank — bottom padding] - -[blank — explicit inter-card gap] -[blank — top padding] - find *.md in .claude (limit 100) -[blank — header-to-results spacer] - No files found matching pattern -[blank — bottom padding] -``` - ---- - -## Scope - -- Card style only (`is_card_style() == True`). The worklog style path (`_ToolCallBlock._compose()`) is untouched. -- No changes to individual tool renderers. -- No new abstractions, no new dependencies. - -## Files changed - -| File | Change | -|------|--------| -| `src/pythinker_code/ui/shell/components/tool_execution.py` | Padding + spacer (changes 1 & 2) | -| `src/pythinker_code/ui/shell/visualize/_live_view.py` | Inter-card blank (change 3) | - -## Testing - -- Run existing TUI card renderer tests: `pytest tests/ui_and_conv/test_tui_card_tool_renderers.py` -- Visual spot-check: run Pythinker in card mode and observe a find/read/bash sequence diff --git a/docs/superpowers/specs/2026-05-22-blackbox-src-tui-port-design.md b/docs/superpowers/specs/2026-05-22-blackbox-src-tui-port-design.md deleted file mode 100644 index 0072ac2d..00000000 --- a/docs/superpowers/specs/2026-05-22-blackbox-src-tui-port-design.md +++ /dev/null @@ -1,412 +0,0 @@ -# Blackbox src Big-Bang TUI Port Design - -**Date:** 2026-05-22 -**Status:** Approved -**Scope:** Full Pythinker shell TUI restyle and feature audit using `blackbox/src` as the reference - ---- - -## Problem - -Pythinker's shell TUI works, but its live work display, thinking indicator, subagent rendering, -approval panels, prompt footer, slash-command surfaces, and report-style outputs are not yet using -one consistent terminal design language. Long-running agent work can repaint large cards repeatedly, -subagent activity can dominate the viewport, and footer/context information competes with the live -transcript. - -The requested direction is an aggressive, big-bang restyle: use `blackbox/src` as the comprehensive -reference for terminal UI behavior, including spinner/thinking display, approval windows, prompt -surfaces, agent/task screens, useful prompt patterns, and design-system primitives. - ---- - -## Goal - -Port every useful terminal-facing design and behavior pattern from `blackbox/src` into Pythinker's -Python Rich/prompt_toolkit shell while preserving Pythinker's runtime architecture, wire protocol, -provider model, approval semantics, config compatibility, and no-new-dependency constraint. - -"Full port" means: - -1. Audit all `blackbox/src` terminal UI, command, prompt, permission, task, agent, and tool-display - areas for applicable behavior. -2. Recreate the useful patterns in Pythinker's Python codebase using local Rich and prompt_toolkit - primitives. -3. Standardize Pythinker shell views around one shared design system. -4. Add or adapt user-facing features only when they map cleanly to Pythinker's existing product - model and do not introduce new hosted services or incompatible runtime assumptions. - -It does not mean vendoring the TypeScript/React/Ink renderer, copying product-specific services -verbatim, adding external dependencies, or changing Pythinker's core agent protocol solely to match -Blackbox internals. - ---- - -## Blackbox Areas To Audit And Port - -### Design System - -Reference directories: - -- `blackbox/src/components/design-system/` -- `blackbox/src/components/ui/` -- `blackbox/src/ink/` - -Port into Python render primitives: - -- color roles and status styles -- status icons -- dividers -- keyboard shortcut hints -- bylines and secondary metadata -- panes/dialog shells -- list rows and selected-row states -- compact progress/loading states -- tab-like segmented choices where useful -- width-aware wrapping and truncation rules - -Pythinker should keep Rich as the renderer. Blackbox's Ink renderer should inform layout rules, not -be copied as a runtime. - -### Live Transcript And Messages - -Reference directories: - -- `blackbox/src/components/messages/` -- `blackbox/src/components/Spinner/` -- `blackbox/src/components/tasks/renderToolActivity.tsx` - -Port behavior: - -- one consistent transcript grammar for user, assistant, tool, error, notification, and system rows -- collapsed thinking by default, expanded thinking in verbose/transcript-style modes -- single-line active thinking/composing status with spinner glyph, elapsed time, token counts, and - interrupt hints -- stalled/no-token visual state, reduced-motion behavior, and responsive hiding of secondary status - parts -- grouped tool-use output summaries -- compact active subagent/teammate rows -- detailed finished cards only when a tool result has meaningful content - -### Prompt Input And Footer - -Reference directories: - -- `blackbox/src/components/PromptInput/` -- `blackbox/src/hooks/usePromptSuggestion.ts` -- `blackbox/src/services/PromptSuggestion/` - -Port behavior: - -- stable footer segments for mode, model/provider, context, approvals, background work, and hints -- standardized `esc`, history, slash-command, file mention, shell-mode, and agent-selection hints -- input mode indicator styled consistently with the transcript -- queued/stashed prompt notices when equivalent state exists in Pythinker -- prompt suggestions only where they can be implemented locally and safely -- shimmered/working input state during active agent work - -### Approval, Permission, And Modal Windows - -Reference directories: - -- `blackbox/src/components/permissions/` -- `blackbox/src/components/design-system/Dialog.tsx` -- `blackbox/src/components/TrustDialog/` -- `blackbox/src/components/*Dialog.tsx` -- `blackbox/src/services/mcpServerApproval.tsx` - -Port behavior: - -- a shared modal/dialog shell for approval, question, trust, MCP, config, task, and picker flows -- clear titles, risk explanations, primary/secondary choices, and keyboard hints -- file edit/write/read, shell, web, skill, MCP, plan-mode, sandbox, and fallback permission variants - mapped onto Pythinker's existing approval runtime -- approval decisions must remain backed by Pythinker's current policy and wire events -- no bypass of existing approval enforcement - -### Agents, Tasks, Teams, And Background Work - -Reference directories: - -- `blackbox/src/components/agents/` -- `blackbox/src/components/tasks/` -- `blackbox/src/components/teams/` -- `blackbox/src/tools/AgentTool/` -- `blackbox/src/tasks/` - -Port behavior where it maps to Pythinker: - -- compact agent/task status lines in the live transcript -- detail dialogs for background tasks and subagents -- agent list/detail/editor-style screens for Pythinker agent specs and subagent types -- tool selection affordances for agent creation/editing when compatible with Pythinker specs -- background task stop/detail/output actions using existing Pythinker task APIs -- useful agent prompt patterns and built-in agent taxonomy ideas, adapted to Pythinker's YAML - agent-spec system and skill model - -### Commands And Utility Screens - -Reference directories: - -- `blackbox/src/commands/` -- `blackbox/src/components/HelpV2/` -- `blackbox/src/components/Settings/` -- `blackbox/src/components/mcp/` -- `blackbox/src/components/memory/` -- `blackbox/src/components/diff/` -- `blackbox/src/components/StructuredDiff/` - -Audit and port applicable command-display patterns for: - -- help/keybindings -- model/provider selection -- usage/cost/rate-limit displays -- status/context/compact screens -- MCP server/tool views and approval screens -- plugin/skills screens -- config/settings validation screens -- session/resume/export/share-style views where Pythinker has an equivalent -- diff, review, plan, todos, memory, and task displays - -The work should not add product-specific hosted integrations unless the maintainer separately -approves them. - -### Tools And Tool Result Displays - -Reference directories: - -- `blackbox/src/tools/` -- `blackbox/src/components/messages/AssistantToolUseMessage.tsx` -- `blackbox/src/utils/groupToolUses.ts` -- `blackbox/src/services/toolUseSummary/` - -Port display behavior: - -- human-friendly labels for read/search/edit/write/shell/web/MCP/skill/agent/todo/plan tools -- grouped tool summaries for batches of related work -- compact running rows and richer completed cards -- structured shell output previews with failure emphasis -- structured diffs and file edit summaries -- tool rejection and fallback error messages that are clear but not noisy - -Tool semantics stay in Pythinker. This design targets rendering, grouping, and UI affordances first. - -### Prompts, Output Styles, And Skills - -Reference areas: - -- `blackbox/src/constants/systemPromptSections.ts` -- `blackbox/src/utils/systemPrompt.ts` -- `blackbox/src/outputStyles/` -- `blackbox/src/skills/bundled/` -- `blackbox/src/tools/AgentTool/builtInAgents.ts` -- `blackbox/src/services/autoDream/` -- `blackbox/src/services/MagicDocs/` - -Audit and adapt useful ideas: - -- reusable prompt sections that improve TUI-facing agent behavior -- concise tool-summary prompt patterns -- output style concepts that can map to Pythinker's agent specs or shell settings -- bundled skill ideas that fit Pythinker's skill system -- memory/docs/task prompt patterns only if they do not introduce new services, unsafe automation, or - incompatible persistence models - -Prompt changes must be tested through focused invariants and must preserve Pythinker's existing -provider-aware behavior. - ---- - -## Proposed Pythinker Architecture - -### Shell Design System Layer - -Add a focused Python shell design-system layer under `src/pythinker_code/ui/shell/`, split into -modules such as: - -- `design.py` or `theme.py` for color roles, status styles, icons, and typography helpers -- `motion.py` for spinner frames, reduced motion, elapsed time, token counters, shimmer/stall state -- `layout.py` for responsive truncation, segment hiding, and compact row composition -- `dialogs.py` for shared approval/question/modal shells -- `transcript.py` for user/assistant/tool/message rows if current files become too broad - -These helpers should be private shell UI infrastructure until another package needs them. - -### Live View Restyle - -Refactor the live render path around a consistent transcript model: - -- assistant content remains streamed in the current turn -- active thinking/composing renders as one compact status row -- active tools render as compact rows -- active subagents render as a capped tree of responsive rows -- completed tools flush as compact rows or detailed cards depending on content -- notifications and errors use the same row grammar -- context/model/status moves into a stable footer instead of competing with live output - -This keeps `_LiveView` wired to the existing wire events while reducing repaint height and noise. - -### Prompt And Footer Restyle - -Standardize prompt_toolkit-facing shell components: - -- footer segments use shared style primitives -- mode and context indicators share names/colors with the transcript -- slash autocomplete, suggestions, history search, and queued input states get consistent selected - row, hint, and divider styling -- interrupt and background-agent hints are width-aware - -### Approval And Modal Restyle - -Introduce a shared approval/modal renderer and map existing approval flows onto it: - -- shell command approvals -- file edit/write approvals -- web/MCP/skill/tool approvals -- plan/question approvals -- trust/config/MCP setup and validation screens where Pythinker has matching flows - -The renderer can change presentation only. Approval state and policy decisions remain in -`ApprovalRuntime` and existing wire events. - -### Agent And Task Screens - -Use Blackbox agent/task UX as a reference for Pythinker equivalents: - -- list subagents/background tasks compactly -- open detail views for task output and metadata -- provide stop/return-to-task affordances where existing commands support them -- expose agent specs and tool availability with a consistent list/detail design - -Agent prompt or built-in agent additions should be handled as separate, tested changes inside this -same program of work, not mixed into renderer-only commits. - ---- - -## Compatibility And Boundaries - -The restyle must preserve: - -- CLI flags and command compatibility -- persisted session compatibility -- wire event compatibility unless a migration is explicitly designed -- existing approval behavior -- provider-aware model/usage scoping -- telemetry behavior and opt-out semantics -- existing Pythinker config keys -- no new third-party dependencies without explicit maintainer approval - -Out of scope for this design unless later approved: - -- vendoring Blackbox's TypeScript, React, Ink, or custom renderer code -- adding Blackbox-hosted services, telemetry endpoints, account flows, or unrelated cloud features -- changing Pythinker into a different product model -- porting code that has no Pythinker equivalent and no user-facing terminal value - ---- - -## Data Flow - -No first-order protocol rewrite is required. - -- Existing wire events continue to drive live shell rendering. -- `_LiveView` maps events into transcript rows, active status rows, compact tool rows, cards, and - modal renderables. -- `_StatusBlock` or its replacement maps runtime state into footer segments. -- Existing prompt_toolkit components consume shared shell styling helpers. -- Approval requests continue through `ApprovalRuntime` and current wire projections. -- Tool results continue to use existing display blocks, with richer grouping/rendering layered on - top. - -If a Blackbox-inspired feature needs data Pythinker does not currently expose, prefer a small -optional field on an existing UI-facing data structure. Do not block the entire restyle on protocol -extensions. - ---- - -## Error Handling - -Errors should be explicit, concise, and visually differentiated: - -- failed tools show a red status row plus the shortest actionable message -- denied or dismissed actions use muted denied/interrupted styling -- approval windows show risk context without dumping raw payloads -- unknown tools still render as generic tool rows -- unknown display blocks continue to degrade gracefully -- interrupted turns finalize live rows as interrupted and leave readable scrollback -- long shell/tool output stays previewed or paged rather than flooding the transcript - ---- - -## Testing Strategy - -Use focused tests before broad checks. - -Add or update tests under `tests/ui_and_conv/` for: - -- spinner/motion frames, reduced motion, elapsed/token/stalled-state rendering -- collapsed and expanded thinking display -- compact active tool rows -- compact subagent/task trees with width-aware truncation -- completed tool cards for shell, diff, todos, plans, web/MCP, skill, and agent results -- approval/modal variants for shell, file, web/MCP, skill, plan/question, and fallback approvals -- prompt footer segment rendering -- slash autocomplete/history/search selected-row styling -- usage/model/MCP/plugin/auth/info screen rendering where touched -- cleanup/interrupt behavior -- narrow and wide terminal captures - -Run the smallest relevant gates during implementation, then before completion: - -- focused pytest targets for changed renderers -- `make check-pythinker-code` -- visual smoke test with `uv run pythinker --yolo --prompt "scan code base "` -- targeted CLI smoke tests using `uv run pythinker --yolo --prompt ...` where practical - -If broader command surfaces are changed, add command-level parsing/rendering tests. - ---- - -## Acceptance Criteria - -- The live TUI uses a Blackbox-inspired, Pythinker-native transcript grammar across user, - assistant, thinking, tool, subagent, notification, and error rows. -- Thinking/composing is compact, animated, interruptible, width-aware, and supports reduced motion. -- Active subagents and background tasks render as compact trees or rows instead of tall repainting - cards. -- Approvals, questions, and modal windows use a shared dialog system and preserve existing approval - enforcement. -- Prompt footer, slash autocomplete, history search, shell mode, and shortcut hints use the same - design language as the live transcript. -- Tool results, plans, todos, diffs, shell output, MCP/plugin/auth/info/usage screens, and other - shell views use shared primitives where practical. -- Useful Blackbox agent, prompt, output-style, and skill ideas are audited and adapted only when - compatible with Pythinker's architecture and safety rules. -- No new third-party dependencies, hosted services, telemetry behavior, or provider fan-out are - introduced by the restyle. -- Existing CLI flags, wire events, persisted sessions, approvals, and provider-aware behavior remain - compatible. -- `uv run pythinker --yolo --prompt "scan code base "` runs successfully enough to exercise the - live scan workflow, and the resulting TUI is visually evaluated for readable thinking status, - compact subagent/tool activity, stable footer/context display, and non-overlapping layout. -- Focused UI tests and `make check-pythinker-code` pass before the implementation is called done. - ---- - -## Implementation Notes - -This should be planned as one coordinated restyle with review checkpoints, not as unrelated drive-by -edits. A practical implementation plan should split work by shell surface: - -1. Inventory and mapping from `blackbox/src` to Pythinker shell modules. -2. Shared shell design primitives. -3. Motion/thinking/status row. -4. Transcript, tool, subagent, and task live rendering. -5. Approval/modal rendering. -6. Prompt/footer/autocomplete surfaces. -7. Command/report screens. -8. Agent/prompt/skill audits and compatible adaptations. -9. Focused tests, smoke tests, and final verification. - -Each implementation step should keep behavior testable and avoid mixing renderer-only changes with -prompt/agent behavior changes unless the implementation plan explicitly calls for it. diff --git a/docs/superpowers/specs/2026-05-22-windows-native-installer-design.md b/docs/superpowers/specs/2026-05-22-windows-native-installer-design.md deleted file mode 100644 index ad837f9a..00000000 --- a/docs/superpowers/specs/2026-05-22-windows-native-installer-design.md +++ /dev/null @@ -1,331 +0,0 @@ -# Pythinker Native Windows Installer — Design - -**Status:** Approved (2026-05-22) -**Owner:** Pythoughts-labs -**Tracking:** Brings parity with Claude Code's native installer for the Windows distribution surface. - ---- - -## 1. Goal - -Ship a downloadable `PythinkerSetup-x.y.z.exe` that installs `pythinker` on Windows -with no prerequisites on the user's machine — no Python, no Node, no uv, no shell -script. One double-click → `pythinker` is on PATH and works in a fresh PowerShell. - -The existing PyPI/uv install path (`pip install pythinker-code`, `scripts/install.ps1`) -stays available for developers; the native installer is **additive**, not a -replacement. - -## 2. Non-goals - -- Per-machine (HKLM) install as default. Available as an opt-in via `/ALLUSERS`, - but the default is per-user, no UAC. -- MSI / WiX authoring. Inno Setup is sufficient and lower-overhead. -- Microsoft Store (MSIX) packaging. Possible later; out of scope here. -- macOS/Linux native installers. The existing PyPI + Homebrew (future) routes - cover those platforms. - -## 3. User-visible behavior - -- Download `PythinkerSetup-x.y.z.exe` from the GitHub Releases page. -- Run it. Wizard appears (logo banner, Apache-2.0 EULA, optional - "Add Pythinker to PATH" task pre-checked, optional "Launch pythinker after - install" task). -- No UAC prompt. Files land in `%LOCALAPPDATA%\Programs\Pythinker`. PATH entry - is added to `HKCU\Environment`. `WM_SETTINGCHANGE` is broadcast so already-open - Explorer windows pick the new PATH up for newly-spawned child shells. -- Start Menu entry: *Pythinker → pythinker*. Uninstall entry in Apps & Features. -- A fresh PowerShell window can run `pythinker` immediately. Existing sessions - pick it up once they're restarted. - -Updates: `pythinker update` from a native build downloads the latest -`PythinkerSetup-*.exe` from the configured channel and runs it -`/VERYSILENT /SUPPRESSMSGBOXES`. Same UX as Claude Code's native auto-update. - -## 4. Architecture - -``` -GitHub tag push (pythinker-code-vX.Y.Z) - │ - ▼ -.github/workflows/windows-installer.yml (runs-on: windows-latest) - │ - ├── pyinstaller pythinker.spec → dist/pythinker/ (onedir) - ├── signtool sign dist/pythinker/*.exe (if cert secret present) - ├── iscc installer.iss → dist/PythinkerSetup-X.Y.Z.exe - ├── signtool sign dist/PythinkerSetup-*.exe - └── gh release upload (asset attached to the tag) - -End user Pythinker runtime - │ │ - ▼ ▼ -PythinkerSetup-X.Y.Z.exe `pythinker update` detects native build - │ │ - ▼ ▼ -%LOCALAPPDATA%\Programs\Pythinker\ GET releases JSON → download new setup - pythinker.exe + DLLs → exec /VERYSILENT /UPDATEONLY - .pythinker-native (sentinel) -``` - -## 5. Components - -### 5.1 Repository layout - -``` -packages/ - windows-installer/ - build.ps1 # local + CI orchestrator - pythinker.spec # PyInstaller spec (onedir) - installer.iss # Inno Setup script - versioninfo.txt # PyInstaller --version-file (Company, FileVersion, ProductName) - assets/ - pythinker.ico # 16/32/48/256 icon from docs/media/logo.png - LICENSE.rtf # Apache-2.0 in RTF for the EULA page - pythinker-banner.bmp # 164×314 wizard banner (optional but recommended) - pythinker-header.bmp # 150×57 wizard small image (optional) - sign/ - sign.ps1 # signtool wrapper, no-op if cert env vars unset - README.md # how to build locally - -src/pythinker_code/cli/update.py # extended to handle native-installer path -src/pythinker_code/_native.py # tiny helper: is_native_build(), installer_url() - -.github/workflows/windows-installer.yml # tag-triggered build + sign + release upload - -docs/superpowers/specs/ - 2026-05-22-windows-native-installer-design.md # this file -``` - -### 5.2 PyInstaller spec (`pythinker.spec`) - -- Mode: `--onedir` (NOT `--onefile`). Rationale: - - Onefile extracts to `%TEMP%` on every launch → 0.3–1.5 s startup penalty. - - Onefile is the dominant cause of Defender/SmartScreen false positives - (PyInstaller issue #6754 and many duplicates). - - The wizard hides the `dist/pythinker/` folder under `%LOCALAPPDATA%`; the - user sees only the Start Menu shortcut and the `pythinker` PATH entry. -- Entry point: a thin `cli/__main__.py` that imports `pythinker_code.cli.main` - and calls `main()`. -- `hiddenimports`: enumerate every plugin / subagent module pythinker loads at - runtime via importlib (`pythinker_code.subagents.*`, `pythinker_code.tools.*`, - fastmcp / mcp transitive packages that PyInstaller can't trace). -- `--version-file versioninfo.txt` so the resulting `pythinker.exe` carries - proper Windows version metadata (`CompanyName=Pythinker`, - `ProductName=Pythinker Code`, `FileVersion=X.Y.Z.0`, `OriginalFilename=pythinker.exe`). -- Includes runtime hook that drops a marker file `.pythinker-native` next to - `pythinker.exe` at install time. - -### 5.3 Inno Setup script (`installer.iss`) - -Key directives: - -``` -[Setup] -AppId={{4F4F2EAE-9D55-4E8E-92BC-7C1FA38B6F02}} ; stable GUID across versions; regenerated once in advance -AppName=Pythinker Code -AppVersion={#AppVersion} -AppPublisher=Pythinker -AppPublisherURL=https://pythinker.com -DefaultDirName={localappdata}\Programs\Pythinker -DefaultGroupName=Pythinker -DisableProgramGroupPage=yes -PrivilegesRequired=lowest -PrivilegesRequiredOverridesAllowed=dialog -OutputBaseFilename=PythinkerSetup-{#AppVersion} -Compression=lzma2/ultra64 -SolidCompression=yes -WizardStyle=modern -WizardImageFile=assets\pythinker-banner.bmp -WizardSmallImageFile=assets\pythinker-header.bmp -SetupIconFile=assets\pythinker.ico -UninstallDisplayIcon={app}\pythinker.exe -ArchitecturesAllowed=x64compatible -ArchitecturesInstallIn64BitMode=x64compatible -LicenseFile=assets\LICENSE.rtf -; SignTool directive only used in CI builds where signtool is configured -; SignTool=signtool $f - -[Files] -Source: "..\..\dist\pythinker\*"; DestDir: "{app}"; \ - Flags: ignoreversion recursesubdirs createallsubdirs - -[Tasks] -Name: "modifypath"; Description: "Add Pythinker to your PATH"; GroupDescription: "Shell integration:"; - -[Registry] -; Append {app} to HKCU\Environment\Path (idempotent — see [Code] InitializeWizard) -Root: HKCU; Subkey: "Environment"; ValueType: expandsz; ValueName: "Path"; \ - ValueData: "{olddata};{app}"; \ - Tasks: modifypath; \ - Check: NeedsAddPath('{app}') - -[Icons] -Name: "{group}\Pythinker"; Filename: "{app}\pythinker.exe" - -[Run] -Filename: "{app}\pythinker.exe"; Description: "Launch Pythinker"; \ - Flags: nowait postinstall skipifsilent unchecked - -[Code] -function NeedsAddPath(Param: string): boolean; -var OrigPath: string; -begin - if not RegQueryStringValue(HKCU, 'Environment', 'Path', OrigPath) then begin - Result := True; exit; - end; - Result := Pos(';' + UpperCase(Param) + ';', - ';' + UpperCase(OrigPath) + ';') = 0; -end; - -procedure CurUninstallStepChanged(CurUninstallStep: TUninstallStep); -var Path: string; -begin - if CurUninstallStep = usUninstall then begin - if RegQueryStringValue(HKCU, 'Environment', 'Path', Path) then begin - StringChangeEx(Path, ';' + ExpandConstant('{app}'), '', True); - StringChangeEx(Path, ExpandConstant('{app}') + ';', '', True); - StringChangeEx(Path, ExpandConstant('{app}'), '', True); - RegWriteStringValue(HKCU, 'Environment', 'Path', Path); - end; - end; -end; -``` - -After both install and uninstall PATH edits, the installer broadcasts -`WM_SETTINGCHANGE` via Inno Setup's standard `Environment` parameter so newly -spawned shells see the change without a reboot. (Inno's `expandsz` write + -`Environment` value name triggers this for free.) - -### 5.4 Build pipeline (`.github/workflows/windows-installer.yml`) - -``` -on: - push: - tags: [pythinker-code-v*] - workflow_dispatch: - -jobs: - build: - runs-on: windows-latest - permissions: { contents: write } - steps: - - checkout - - setup-python 3.13 - - install uv; uv sync; uv pip install pyinstaller - - pyinstaller packages/windows-installer/pythinker.spec - - powershell packages/windows-installer/sign/sign.ps1 dist/pythinker/pythinker.exe - - install Inno Setup 6 via choco - - iscc /DAppVersion=${{ github.ref_name }} packages/windows-installer/installer.iss - - powershell packages/windows-installer/sign/sign.ps1 dist/PythinkerSetup-*.exe - - powershell -c "Get-FileHash dist\PythinkerSetup-*.exe -Algorithm SHA256 > dist\PythinkerSetup-*.exe.sha256" - - gh release upload ${{ github.ref_name }} dist/PythinkerSetup-*.exe dist/PythinkerSetup-*.exe.sha256 -``` - -`sign.ps1` reads `WINDOWS_CERT_PFX_BASE64` + `WINDOWS_CERT_PASSWORD` from the -environment. If unset, it logs a warning and exits 0 (unsigned build proceeds). -If set, it decodes the PFX into a temp file, calls - -``` -signtool sign /f $pfx /p $pw \ - /tr http://timestamp.digicert.com /td sha256 /fd sha256 \ - $target -``` - -then deletes the temp PFX. The cert never lands on disk in the repo, and never -appears in logs. - -### 5.5 Update plumbing (`src/pythinker_code/cli/update.py`) - -``` -def update(channel: str = "latest") -> None: - if not _is_native_build(): - return _update_via_uv() # existing path, unchanged - - if os.environ.get("DISABLE_AUTOUPDATER"): - log.info("Auto-update disabled via DISABLE_AUTOUPDATER; skipping") - return - - rel = _gh_releases_lookup(channel) # GET api.github.com/.../releases/{tag} - if not _is_newer(rel.tag, __version__): - log.info("Pythinker is up to date (%s)", __version__) - return - - installer = _download(rel.installer_asset_url) - if not _verify_sha256(installer, rel.sha256): - raise UpdateError("installer checksum mismatch — aborting") - - log.info("Launching native installer (silent)…") - subprocess.Popen([str(installer), "/VERYSILENT", "/SUPPRESSMSGBOXES"]) - sys.exit(0) -``` - -`_is_native_build()` checks for `(Path(sys.executable).parent / ".pythinker-native").exists()`. -This file is dropped by `installer.iss` via: - -``` -[Files] -Source: "..\windows-installer\.pythinker-native"; DestDir: "{app}" -``` - -A file marker is the cheapest, most reliable signal — no env var, no registry -key, survives copy-installs. - -### 5.6 Distribution surfaces - -1. **GitHub Releases** — primary: `PythinkerSetup-x.y.z.exe` + `.sha256` attached to every `pythinker-code-v*` tag. -2. **README** — add a *Windows (native)* section above the current PyPI/uv block: - - Direct download link to the latest Release asset. - - One-line PowerShell installer (downloads + runs setup silently) for users who prefer the terminal: - `irm https://pythinker.com/install/win | iex` *(or the raw GitHub URL until DNS is configured)* -3. **Winget manifest** — auto-publish via `winget-releaser` GitHub Action triggered after a Release publishes. Defer to a follow-up PR; not blocking v1. - -## 6. Risks & mitigations - -| Risk | Mitigation | -|---|---| -| **Pre-cert window** — first releases ship unsigned, SmartScreen will warn. | `sign.ps1` is a no-op without the secret, so CI keeps producing installers; README documents the warning and how to verify SHA-256. Switching on signing is a single GitHub Secret addition, no code change. | -| **AV false positives** on PyInstaller-frozen binaries. | Onedir mode + Authenticode signing eliminates the majority. For the first 1-2 weeks we proactively submit the signed `.exe` to Microsoft Defender via the [false-positive portal](https://www.microsoft.com/wdsi/filesubmission) and run a pre-release VirusTotal scan. | -| **Installer size** — unknown until first freeze; could be 60-120 MB with all of fastmcp/mcp/pythinker-core transitive deps. | Track size in CI as a workflow artifact summary; if it exceeds 150 MB, audit `excludes` in `pythinker.spec` (PyInstaller's `--exclude-module` for unused optional deps). Acceptable target: ≤100 MB compressed installer. | -| **PATH conflict** — a user with both `uv tool install pythinker-code` *and* the native install will have two `pythinker.exe`s on PATH. | Installer's Finished page checks `where.exe pythinker` and shows a notice if more than one is found, suggesting `uv tool uninstall pythinker-code`. | -| **Stale releases JSON during update** — GitHub rate-limit (60 req/hr unanon) or transient 5xx. | `_gh_releases_lookup` retries with backoff once, then surfaces a clear error and points to the manual download URL. No silent failures. | -| **Per-machine vs per-user collision** — admin re-installs as ALLUSERS over an existing per-user install (or vice versa). | `[Code]` `InitializeSetup` probes both `HKCU` and `HKLM` Uninstall keys for our `AppId`; if it finds the other scope, refuses with a friendly "uninstall the existing one first" message. | -| **Frozen-binary update detection on dev installs** — `sys.frozen` is also True for PyInstaller dev builds someone may run locally. | The `.pythinker-native` sentinel is dropped only by the production installer, never by `pyinstaller` directly. Dev frozen builds fall through to the PyPI update path, which is correct. | - -## 7. Acceptance criteria - -A release is considered done when: - -1. CI on a `pythinker-code-vX.Y.Z` tag push produces `PythinkerSetup-X.Y.Z.exe` - attached to the GitHub Release, ≤150 MB. -2. On a fresh Windows 11 VM (no Python, no Node, no uv): - - Double-clicking the `.exe` completes the wizard with **no UAC prompt**. - - A new `pwsh` window shows `pythinker --version` matching the tag. - - `pythinker` launches into the TUI and reaches the prompt without error. -3. `pythinker update` from inside the installed build successfully detects a - newer pre-release, downloads, verifies SHA-256, and re-launches the installer - silently. Post-update, `pythinker --version` reflects the new version. -4. Uninstall via Apps & Features removes `{app}` and the PATH entry; a new - shell no longer finds `pythinker`. -5. Once the signing cert is in place, `signtool verify /pa /v - PythinkerSetup-X.Y.Z.exe` reports a valid Authenticode chain and a valid RFC - 3161 timestamp. - -## 8. Out of scope (deferred) - -- Winget manifest automation (follow-up PR; manifest can be auto-generated by - `winget-releaser` once the Release exists). -- Microsoft Store / MSIX packaging. -- Per-machine install as the default (admin-only enterprise scenarios). -- Localized installer strings beyond English. -- Auto-update channels beyond `latest` / `stable` (no `beta` / `nightly` yet). -- ARM64 Windows native build (PyInstaller support is workable but adds CI complexity; defer until there's demand). - -## 9. Open questions - -None blocking v1. The user has confirmed: -- Per-user install (no admin) is the default. -- In-app `pythinker update` channel backed by GitHub Releases. -- Code signing is required; cert acquisition is in flight and the build - pipeline will accept it via GitHub Secret when ready. -- PyInstaller-onedir over onefile (research-driven: startup speed + AV false - positives). diff --git a/docs/superpowers/specs/2026-05-24-install-downloads-counter-design.md b/docs/superpowers/specs/2026-05-24-install-downloads-counter-design.md deleted file mode 100644 index b7076e29..00000000 --- a/docs/superpowers/specs/2026-05-24-install-downloads-counter-design.md +++ /dev/null @@ -1,213 +0,0 @@ -# Install Downloads Counter — Design - -**Date:** 2026-05-24 -**Status:** Implemented & deployed. - -> **Amendments during/after implementation (read first):** -> 1. **Dropped `dl.pythinker.com`.** The Worker fetches the apex directly via a -> same-zone subrequest (CF routes it to origin, no recursion). See the -> Architecture note. Side effect: install fetches are no longer edge-cached -> (`cf-cache-status: DYNAMIC`). -> 2. **Dropped the User-Agent bot filter (per user request).** The counter now -> increments on **every successful `GET`** of `/install.sh` / `/install.ps1` -> — browsers, crawlers and monitors included. The sections below that describe -> UA bot-filtering (`ua.ts`, "Data flow & bot filter") are **superseded**; the -> `ua.ts` module and its tests were removed. **Recommendation:** relabel the -> badge from "installs" to "downloads", since it now counts all hits, not -> installs. -> 3. **Counter seeded to 5,500** from pepy (PyPI downloads) as a manual baseline, -> then counts forward. The number therefore blends a PyPI-download seed with -> forward all-traffic hit counting. - -## Problem - -Pythinker is installed via `curl -fsSL https://pythinker.com/install.sh | bash` -(and `irm https://pythinker.com/install.ps1 | iex` on Windows). There is no -count of how many times the install script is fetched. We want a **bot-filtered -fetch count**, surfaced as a **README badge** and a **raw JSON API endpoint**. - -## Constraints & key facts - -- **Cloudflare fronts the VPS.** `pythinker.com` is proxied by Cloudflare; the - origin is a self-hosted VPS that serves the script (backed by - `scripts/install-native.sh`). -- **Edge caching makes origin logs useless for counting.** The install endpoint - is served with `Cache-Control: public, max-age=300, s-maxage=900, - stale-if-error=86400`. Most fetches are served from Cloudflare's edge cache - and never reach the VPS, so VPS access logs would massively undercount. The - count **must** happen at the edge, in a Worker that runs on every request - before cache. -- **Vanity metric, not audited.** UA-based bot filtering blocks honest browsers - and crawlers, but not a deliberate `curl -A 'curl/8.5.0'` loop. This counter - is accepted as a vanity/marketing number, not a tamper-resistant metric. This - limitation is intentional and documented. - -## Alternatives considered (hosted platforms) - -There is **no passive "pepy for install scripts"**: pepy works only because PyPI -centralizes downloads in public BigQuery logs. Shell-script installs have no -central registry, so no hosted service can observe the count without us capturing -it at our own edge. Edge caching also means VPS/origin-log-based tools undercount. -So the Worker (edge capture + bot filter) is required regardless; the only thing -a hosted platform could replace is the **storage/badge layer**: - -- **Abacus** (CountAPI successor) / CounterAPI — free counter APIs with native - shields badges. Rejected as the primary store: a free hobby counter - disappearing (as CountAPI did) would silently break a flagship README badge, - and it moves the number off our infra. -- **GoatCounter / Plausible** — full privacy analytics; overkill for one number - and heavier to operate. -- **Cloudflare Workers Analytics Engine** — already considered as "Approach B" - (rolling-window, not cumulative). - -**Decision: own the number in D1.** For a permanent headline metric, data -ownership and no third-party dependency outweigh the zero-ops appeal of a free -hosted counter. D1's hot-row caveat does not bite at install-script volume. - -## Chosen approach - -A Cloudflare Worker bound to the install + API routes increments an atomic D1 -counter on each non-bot fetch, then serves the script bytes from a DNS-only -origin hostname. The badge and JSON API read the same counter. - -### Architecture - -``` -curl/irm ──> CF edge ──> Worker(/install.sh, /install.ps1) - │ GET + 200 ⇒ count (no UA filter — all hits count; see Amendment 2) - │ fetch https://pythinker.com/