🔍 Claude Code User Documentation Review - 2026-03-15

[github-actions[bot]]

This is a daily automated review of the gh-aw documentation from the perspective of a developer who uses Claude Code as their primary AI assistant and does not use GitHub Copilot. The goal is to identify gaps, assumptions, and blockers that would prevent adoption by non-Copilot users.

Executive Summary

The gh-aw documentation continues to support Claude Code users reasonably well. The Quick Start, engines reference, and auth docs clearly present Claude as a first-class option alongside Copilot. There are no critical blockers preventing Claude Code users from getting started. However, six persistent documentation issues — all identified in previous runs but still unresolved after 18+ days — create minor friction. The overall adoption score remains stable at 7/10 for the 18th consecutive day.

Key Finding: Claude Code users can successfully adopt gh-aw. The friction points are real but manageable, consisting primarily of Copilot-centric architecture diagram labels and a handful of missing mentions of Gemini.

---

Persona Context

I reviewed this documentation as a developer who:

• ✅ Uses GitHub for version control
• ✅ Uses Claude Code as primary AI assistant
• ❌ Does NOT use GitHub Copilot
• ❌ Does NOT use Copilot CLI
• ❌ Does NOT have a Copilot subscription

---

Question 1: Onboarding Experience

Can a Claude Code user understand and get started with gh-aw?

Yes. The Quick Start guide (docs/src/content/docs/setup/quick-start.mdx) immediately mentions Claude alongside Copilot in the prerequisites:

"AI Account - GitHub Copilot, Anthropic Claude or OpenAI Codex"

Step 2 of the Quick Start explicitly says the add-wizard command will prompt users to "Select an AI Engine — Choose between Copilot, Claude, or Codex." The secret name ANTHROPIC_API_KEY is called out clearly, with a link to the auth reference.

The landing page (index.mdx) says "Use GitHub Copilot, Claude by Anthropic or OpenAI Codex" — no Copilot-first assumption in the hero text.

Specific Issues Found:

• how-they-work.mdx line 26: States "Workflows support GitHub Copilot (default), Claude by Anthropic, and Codex" — Gemini has been a documented fourth engine for weeks but is still absent here. A Claude Code user reading this file gets an incomplete picture of the engine landscape.
• architecture.mdx lines 177, 248: The Agent Workflow Firewall diagram and MCP Gateway diagram both label the agent container as "Copilot CLI" and "Copilot CLI + MCP client" respectively. Claude Code users may incorrectly assume the firewall only works with Copilot.

Recommended Fixes:

• Update how-they-work.mdx line 26 to include Gemini: "GitHub Copilot (default), Claude by Anthropic, Codex, and Gemini"
• Relabel architecture diagram agent nodes to "AI Engine (Copilot/Claude/Codex)" to be engine-agnostic

---

Question 2: Inaccessible Features for Non-Copilot Users

What features or steps don't work without Copilot?

Features That Require Copilot (by design, correctly documented):

• assign-to-agent safe output — explicitly Copilot-only (assign-to-copilot.mdx), clearly labeled, no confusion
• Copilot custom agent files (.github/agents/) — natively supported only with Copilot; the copilot-custom-agents.md doc correctly explains "Copilot supports agent files natively, while other engines (Claude, Codex) inject the markdown body as a prompt"
• The /agent agentic-workflows command in custom-agent-for-aw.mdx — requires Copilot Chat, but the page includes a "Self-Contained Debugging (Without Copilot)" section using debug.md that works with any AI

Features That Work Without Copilot (engine-agnostic):

• All tools: github:, bash:, edit:, web-fetch:, playwright:, cache-memory:, repo-memory:, agentic-workflows:
• All safe outputs: create-issue, create-pull-request, add-comment, etc.
• All CLI commands: init, compile, add-wizard, run, status, logs, audit, health
• MCP server integration (engine-agnostic)
• Network firewall (despite Copilot-labeled diagrams, it's actually engine-agnostic)

Missing Documentation:

• creating-workflows.mdx GitHub Web Interface section reportedly still gated: "If you have access to GitHub Copilot" — no equivalent path shown for Claude Code users creating workflows via GitHub web UI

---

Question 3: Documentation Gaps and Assumptions

Where does the documentation assume Copilot usage?

Copilot-Centric Language Found In:

• architecture.mdx — AWF diagram (line 177): Agent process box labels only "Copilot CLI"; MCP Gateway diagram (line 248): labels "Copilot CLI + MCP client". Both should say "AI Engine" or list alternatives.
• architecture.mdx — Network firewall configuration example uses engine: copilot as the only illustration. A Claude user may wonder if network: config works differently for Claude.
• how-they-work.mdx line 26 — Lists three engines, missing Gemini despite engines.md and auth.mdx both fully documenting Gemini.
• cli.md line 214 — secrets bootstrap --engine documents options as (copilot, claude, codex), silently omitting gemini.

Missing Alternative Instructions:

• No Claude-specific troubleshooting entries in common-issues.md (only Copilot CLI Not Found and Copilot License Issues)
• auth.mdx line 104: ANTHROPIC_API_KEY setup URL points to (platform.claude.com/redacted) — the correct URL for creating API keys is https://console.anthropic.com` (this has been flagged for 18+ days)

---

Severity-Categorized Findings

🚫 Critical Blockers (Score: 0/10)

None. Claude Code users can successfully follow the Quick Start and get a workflow running.

⚠️ Major Obstacles (Score: 2/10)

Obstacle 1: ANTHROPIC_API_KEY Setup URL May Be Wrong

Impact: Claude users directed to potentially incorrect URL to create their API key

Current State: auth.mdx line 104 reads:

Create an API key at (platform.claude.com/redacted)
```

**Why It's Problematic:** `platform.claude.com/docs` is a documentation page, not where API keys are created. API keys are managed at `console.anthropic.com`. A user following this link may struggle to find the actual API key creation page.

**Suggested Fix:** Update to `https://console.anthropic.com/api-keys`

**Affected Files:** `docs/src/content/docs/reference/auth.mdx` line 104

**Status:** Unresolved for 18+ days

</details>

<details>
<summary><b>Obstacle 2: Web Interface Workflow Creation Gated to Copilot</b></summary>

**Impact:** Claude Code users who prefer creating workflows via GitHub web UI find no documented path

**Current State:** `creating-workflows.mdx` reportedly contains "If you have access to GitHub Copilot" gating for the GitHub Web Interface creation path

**Why It's Problematic:** Claude Code users are excluded from the web-based workflow creation experience with no alternative guidance

**Suggested Fix:** Add an equivalent "Using Claude Code in the Web Interface" section, or reframe the section to be engine-neutral

**Affected Files:** `docs/src/content/docs/setup/creating-workflows.mdx`

**Status:** Unresolved for 18+ days

</details>

### 💡 Minor Confusion Points (Score: 7 total)

- **Architecture diagrams show "Copilot CLI"** — `architecture.mdx` lines 177, 248: AWF and MCP Gateway diagrams label agent nodes "Copilot CLI" — should be engine-agnostic labels
- **`how-they-work.mdx` missing Gemini** — Line 26 lists 3 of 4 supported engines, making the engine landscape appear incomplete
- **`cli.md` secrets bootstrap omits Gemini** — Line 214: `--engine (copilot, claude, codex)` silently excludes Gemini
- **Network config example is Copilot-only** — `architecture.mdx` config snippet shows `engine: copilot` exclusively; could confuse Claude users about engine-agnostic nature of `network:`
- **No Claude troubleshooting entries** — `common-issues.md` has only Copilot-specific troubleshooting (CLI Not Found, License Issues)
- **`custom-agent-for-aw.mdx` page title** — "Copilot Agent Files support for Agentic Workflows" — misleading title for a page that actually contains non-Copilot alternatives too
- **`copilot-custom-agents.md` examples use `engine: copilot`** — All import examples show Copilot-only configurations; a Claude equivalent would help clarify that the same import mechanism works with `engine: claude`

---

### Engine Comparison Analysis

### Available Engines

Based on documentation review:

| Engine | Setup Docs | Examples | Auth Docs | Overall |
|--------|-----------|----------|-----------|---------|
| Copilot | ⭐⭐⭐⭐⭐ | ⭐⭐⭐⭐⭐ | ⭐⭐⭐⭐⭐ | ⭐⭐⭐⭐⭐ |
| Claude | ⭐⭐⭐⭐ | ⭐⭐⭐⭐ | ⭐⭐⭐⭐ | ⭐⭐⭐⭐ |
| Codex | ⭐⭐⭐⭐ | ⭐⭐⭐ | ⭐⭐⭐⭐ | ⭐⭐⭐⭐ |
| Gemini | ⭐⭐⭐⭐ | ⭐ | ⭐⭐⭐⭐ | ⭐⭐⭐ |

**Rating Scale:** ⭐⭐⭐⭐⭐ (Excellent) to ⭐ (Poor/Missing)

**Note:** Gemini has good auth/setup docs but zero example workflows in `.github/workflows/` and is missing from `how-they-work.mdx` and `secrets bootstrap` docs.

---

### Tool Availability Analysis

**Engine-Agnostic Tools (all engines):**
- `github:` — GitHub API MCP server
- `bash:` — Shell command execution
- `edit:` — File editing
- `web-fetch:` — Web content fetching
- `playwright:` — Browser automation
- `cache-memory:` — Persistent memory across runs
- `repo-memory:` — Repository-specific memory
- `agentic-workflows:` — Workflow introspection

**Engine-Specific Tools:**
- `assign-to-agent` safe output — Copilot only (clearly documented)
- `engine.agent` field — Copilot custom agent files (other engines inject as prompt)

**Unclear/Undocumented:**
- `web-search:` — Notes say "some engines require third-party MCP servers"; no clear matrix of which engines have native web search vs MCP-required

---

### Authentication Requirements

#### Current Documentation

| Engine | Auth Secret | Docs Quality | Setup URL Correct? |
|--------|------------|--------------|-------------------|
| Copilot | `COPILOT_GITHUB_TOKEN` | ⭐⭐⭐⭐⭐ | ✅ |
| Claude | `ANTHROPIC_API_KEY` | ⭐⭐⭐⭐ | ⚠️ URL may be wrong |
| Codex | `OPENAI_API_KEY` | ⭐⭐⭐⭐⭐ | ✅ |
| Gemini | `GEMINI_API_KEY` | ⭐⭐⭐⭐⭐ | ✅ |

#### Missing for Claude Users

- `auth.mdx` line 104 links to `(platform.claude.com/redacted) instead of the API key console
- No note that `CLAUDE_CODE_OAUTH_TOKEN` (Claude Code's local auth) is NOT used — only `ANTHROPIC_API_KEY` works (the FAQ reportedly clarifies this, which is good)

---

### Example Workflow Analysis

### Workflow Count by Engine

```
Engine: copilot — 83 workflows (48.3% of 172 total)
Engine: claude  — 37 workflows (21.5% of 172 total, +2 from yesterday)
Engine: codex   — 14 workflows (8.1% of 172 total)
Engine: gemini  —  0 workflows (0%)
No explicit engine — ~38 workflows (use default Copilot)

Quality of Examples

Copilot Examples: Comprehensive — covers all patterns (daily ops, chat ops, issue ops, PR events, multi-repo, etc.)

Claude Examples: Good — 37 real-world workflows covering a wide variety of use cases. Growing steadily (up from 33 on Feb 26, 35 yesterday). Claude users can find relevant examples.

Codex Examples: Adequate — 14 workflows available. Limited but covers major patterns.

Gemini Examples: None — Gemini was added as a supported engine but has zero example workflows in the repository.

---

Recommended Actions

Priority 1: Critical Documentation Fixes

1. Fix ANTHROPIC_API_KEY setup URL — auth.mdx line 104: Change (platform.claude.com/redacted) to https://console.anthropic.com/api-keys`
2. Add Gemini to how-they-work.mdx engine list — Line 26 omits Gemini; update to "GitHub Copilot (default), Claude by Anthropic, Codex, and Gemini"

Priority 2: Major Improvements

1. Relabel AWF architecture diagram — architecture.mdx lines 177 and 248: Replace "Copilot CLI" and "Copilot CLI + MCP client" with engine-agnostic labels like "AI Engine (Copilot/Claude/Codex/Gemini)"
2. Add Gemini to secrets bootstrap --engine docs — cli.md line 214: Expand (copilot, claude, codex) to include gemini
3. Add non-Copilot path to web UI workflow creation — creating-workflows.mdx: Provide Claude Code / generic alternative to the Copilot-gated web interface section

Priority 3: Nice-to-Have Enhancements

1. Add at least one Gemini example workflow to .github/workflows/ to validate Gemini support and provide a reference
2. Add Claude troubleshooting entries to common-issues.md — e.g., "ANTHROPIC_API_KEY not set", "Claude rate limits", "Model not found"
3. Add engine-agnostic network config example in architecture.mdx alongside or instead of the Copilot-only snippet

---

Positive Findings

What Works Well for Claude Code Users

• ✅ Quick Start explicitly lists Claude as a valid engine from the first prerequisites bullet
• ✅ add-wizard wizard guides users through Claude engine selection and ANTHROPIC_API_KEY setup interactively
• ✅ engines.md has comprehensive coverage of all four engines with version pinning examples for each
• ✅ auth.mdx covers all four engines with equal detail (modulo the URL issue)
• ✅ All tools and safe outputs are engine-agnostic — nothing is Copilot-locked except assign-to-agent
• ✅ 37 real-world Claude example workflows in the repo (22% of total), growing steadily
• ✅ copilot-custom-agents.md correctly explains Claude behavior: "other engines (Claude, Codex) inject the markdown body as a prompt"
• ✅ custom-agent-for-aw.mdx includes "Self-Contained Debugging (Without Copilot)" section using debug.md
• ✅ All CLI commands (init, compile, run, status, logs, audit, health) are engine-agnostic
• ✅ Index page hero text explicitly names Claude alongside Copilot

---

Conclusion

Can Claude Code Users Successfully Adopt gh-aw?

Answer: Yes, with minor friction

The gh-aw documentation has made deliberate choices to include Claude Code users: the Quick Start names Claude explicitly, the engines reference is comprehensive, authentication is well-documented (modulo one URL), and 37 Claude example workflows exist in the repository. The core adoption path is clear and unobstructed.

The remaining friction is cosmetic and persistent: architecture diagrams still show "Copilot CLI" labels, Gemini is inconsistently mentioned across docs, one authentication URL is potentially wrong, and one web interface creation path is Copilot-gated. None of these are blockers — they're rough edges that reduce confidence and add minor confusion.

Overall Assessment Score: 7/10

Dimension	Score
Clarity for non-Copilot users	7/10
Claude engine documentation	8/10
Alternative approaches provided	8/10
Engine parity (docs quality)	6/10

Trend

This score has been stable at 7/10 for 18 consecutive days. The same six issues have been reported in every daily run without resolution. This suggests either deliberate product decisions (Copilot-centric diagrams as architectural truth) or low documentation priority for these specific gaps.

Next Steps

The most impactful single change would be fixing the ANTHROPIC_API_KEY setup URL in auth.mdx — a one-line edit that removes a real potential confusion for new Claude users. The second most impactful change is updating the architecture diagram labels to be engine-agnostic, as these appear early in the conceptual documentation and set an incorrect framing.

---

Appendix: Files Reviewed

• README.md
• docs/src/content/docs/index.mdx
• docs/src/content/docs/setup/quick-start.mdx
• docs/src/content/docs/introduction/how-they-work.mdx
• docs/src/content/docs/introduction/architecture.mdx
• docs/src/content/docs/reference/tools.md
• docs/src/content/docs/reference/engines.md
• docs/src/content/docs/reference/auth.mdx
• docs/src/content/docs/setup/cli.md
• docs/src/content/docs/reference/custom-agent-for-aw.mdx
• docs/src/content/docs/reference/copilot-custom-agents.md
• docs/src/content/docs/reference/assign-to-copilot.mdx
• .github/workflows/*.md (sampled engine distribution across 172 files)

---

Report Generated: §23120834763
Workflow: claude-code-user-docs-review
Engine Used: claude (eating our own dog food 🐕)
Days at Score 7/10: 18

AI generated by Claude Code User Documentation Review · history

• expires on Mar 16, 2026, 10:37 PM UTC

[github-actions[bot]]

This discussion has been marked as outdated by Claude Code User Documentation Review.

A newer discussion is available at Discussion #21296.