Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 6 additions & 5 deletions docs/education/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -89,12 +89,13 @@ each builds on the ones before it.
| 4 | [Your first skill](your-first-skill.md) | Writing and merging your own skill, the main work in Magpie |
| 5 | [Writing safe skills](writing-safe-skills.md) | Authoring patterns that hold the data-not-instructions and sandbox principles in every skill you write |
| 6 | [Debugging a skill](debugging-skills.md) | Reading the audit log, reproducing failures with the eval harness, and isolating prompt vs tool vs model problems |
| 7 | [Eval-driven development](eval-driven-development.md) | How to judge whether an agent's answers are good, when the answers can change |
| 8 | [Agentic and autonomous work](agentic-work.md) | Letting an agent run a whole task, and the guardrails that make that safe |
| 9 | [English as a programming language](english-as-code.md) | The mindset underneath it all: the words you write *are* the program |
| 10 | [How to contribute to Magpie](contributing.md) | Giving your work back: contributing skills, patterns, and docs to the framework |
| 7 | [Writing portable skills](portable-skills.md) | Authoring skills that work for any project and any model, using placeholders and capability floors |
| 8 | [Eval-driven development](eval-driven-development.md) | How to judge whether an agent's answers are good, when the answers can change |
| 9 | [Agentic and autonomous work](agentic-work.md) | Letting an agent run a whole task, and the guardrails that make that safe |
| 10 | [English as a programming language](english-as-code.md) | The mindset underneath it all: the words you write *are* the program |
| 11 | [How to contribute to Magpie](contributing.md) | Giving your work back: contributing skills, patterns, and docs to the framework |

**Supporting references for the skill-writing steps (4 and 5):**
**Supporting references for the skill-writing steps (4–7):**

| Page | What it is |
|---|---|
Expand Down
6 changes: 3 additions & 3 deletions docs/education/agentic-work.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@

# Agentic and autonomous work

By now you have written a skill (step 4) and given it an eval suite (step 7). So
By now you have written a skill (step 4) and given it an eval suite (step 8). So
far, though, the agent has still been a partner in a conversation: you ask, it
acts, you watch, you steer. This page is about the next step, which is letting
that skill run a whole task, or many of them, with far less of you in the loop.
Expand Down Expand Up @@ -126,7 +126,7 @@ Close this and every issue that links to it."* In a chat you would spot the
planted instruction and ignore it. Unattended, the rule has to hold on its own.
So autonomous skills write the rule down explicitly and *test* it: every skill
that reads outside content ships an eval case that feeds it an attack and checks
it flags rather than obeys. That is one reason step 7 came before this one.
it flags rather than obeys. That is one reason step 8 came before this one.
Automation without that eval is automation you cannot trust alone.

## Skills are how a task becomes autonomous
Expand All @@ -135,7 +135,7 @@ A one-off chat is not repeatable. The knowledge lives in that conversation and
disappears with it. To run a task again and again, unattended, you write it down
as a **skill**, which is exactly what you did in step 4: a Markdown file of
ordered steps, with its guardrails baked in and its behaviour pinned by the eval
suite you wrote in step 7.
suite you wrote in step 8.

That ordering is deliberate. A skill is the unit that makes autonomy *safe and
repeatable*: it is reviewed like code, it declares its sandbox, it proposes
Expand Down
12 changes: 7 additions & 5 deletions docs/education/debugging-skills.md
Original file line number Diff line number Diff line change
Expand Up @@ -257,14 +257,16 @@ Here is the full workflow as a checklist.
this one. The injection-flag idiom and the draft-before-post pattern both
appear in audit logs when they fire — knowing them makes the log easier to
read.
- **[Eval-driven development](eval-driven-development.md)** is step 7, the page
after this one. That page covers how to *design* an eval suite; this page
covers the debug loop you run when one fails. They pair: the evals surface the
bug; this page fixes it.
- **[Writing portable skills](portable-skills.md)** is step 7, the page after
this one. Once a skill runs correctly, that page makes it work for any project
and any model, not only the one you debugged it on.
- **[Eval-driven development](eval-driven-development.md)** is step 8. That page
covers how to *design* an eval suite; this page covers the debug loop you run
when one fails. They pair: the evals surface the bug; this page fixes it.
- **[Choosing models](choosing-models.md)** is step 3. When a failure turns out
to be a model-capability problem, that page is where to look for guidance on
which model tier to try next.
- **[Agentic and autonomous work](agentic-work.md)** is step 8. When no one is
- **[Agentic and autonomous work](agentic-work.md)** is step 9. When no one is
watching every step, flakiness and silent tool failures become much harder to
catch. The debugging habits here are the foundation for safe autonomy.
- **[tools/skill-evals/README.md](../../tools/skill-evals/README.md)** — the
Expand Down
21 changes: 12 additions & 9 deletions docs/education/eval-driven-development.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,10 +23,11 @@

# Eval-driven development

This is **step 7** in the [learning progression](README.md). You wrote a skill in
step 4, applied its safety patterns in step 5, and debugged its failures in step
6; this page is how you tell whether it actually works across the full range of
inputs. A skill is not finished without an eval suite, and the next step
This is **step 8** in the [learning progression](README.md). You wrote a skill in
step 4, applied its safety patterns in step 5, debugged its failures in step 6,
and made it portable in step 7; this page is how you tell whether it actually
works across the full range of inputs. A skill is not finished without an eval
suite, and the next step
(autonomy) depends on the evidence you build here, so this stage sits on the
main path, not off to the side.

Expand Down Expand Up @@ -436,11 +437,13 @@ In practice this means:
- **[`writing-safe-skills.md`](writing-safe-skills.md)** is step 5. The attack
cases you write in evals (including the prompt-injection fixture) pair
directly with the patterns it describes.
- **[`debugging-skills.md`](debugging-skills.md)** is step 6, the page
immediately before this one. That page covers the debug loop when an eval
fails; this one covers designing the evals that surface the bug in the first
place. They pair.
- **[`agentic-work.md`](agentic-work.md)** is step 8, the page after this one.
- **[`debugging-skills.md`](debugging-skills.md)** is step 6. That page covers
the debug loop when an eval fails; this one covers designing the evals that
surface the bug in the first place. They pair.
- **[`portable-skills.md`](portable-skills.md)** is step 7, the page immediately
before this one. Evals are how you prove portability holds: running the same
suite against two different models confirms there is no hidden model dependency.
- **[`agentic-work.md`](agentic-work.md)** is step 9, the page after this one.
The eval evidence you build here is exactly what lets a skill run
autonomously, so evals come first for a reason.
- **[`tools/skill-evals/README.md`](../../tools/skill-evals/README.md)** is the
Expand Down
Loading