From 642b9bab2f97f9db80cd347db6b74bda369b7d75 Mon Sep 17 00:00:00 2001 From: Dominic Nguyen Date: Fri, 31 Jul 2026 10:24:12 -0700 Subject: [PATCH 1/4] fix bare ? optionals by grouping as ()? MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit On main, postfix ? after a rule reference is a literal token, not a quantifier. Quantifiers only apply after ")". Authors across email, weather, video, code, and calendar test grammars wrote ? / ? intending optional — those patterns never matched. Rewrite all bare ? sites to the supported ()? form. Also add a one-pager RFC comparing: - Design A: keep group-gated quantifiers (this fix) - Design B: reserved postfix ?/*/+ (direction explored in #2765) --- .../docs/rfc-agr-quantifier-semantics.md | 388 ++++++++++++++++++ .../test/calendar-extended.agr | 4 +- .../agentSdkWrapper/test/calendar-new.agr | 4 +- .../agents/code/src/vscode/displaySchema.agr | 8 +- ts/packages/agents/email/src/emailSchema.agr | 8 +- ts/packages/agents/video/src/videoSchema.agr | 2 +- .../agents/weather/src/weatherSchema.agr | 6 +- 7 files changed, 404 insertions(+), 16 deletions(-) create mode 100644 ts/packages/actionGrammar/docs/rfc-agr-quantifier-semantics.md diff --git a/ts/packages/actionGrammar/docs/rfc-agr-quantifier-semantics.md b/ts/packages/actionGrammar/docs/rfc-agr-quantifier-semantics.md new file mode 100644 index 0000000000..1b01da2997 --- /dev/null +++ b/ts/packages/actionGrammar/docs/rfc-agr-quantifier-semantics.md @@ -0,0 +1,388 @@ +# RFC: AGR `?` / `*` / `+` Quantifier Semantics + +**Status:** Open for design review (Curtis) +**Related:** [PR #2765](https://github.com/microsoft/TypeAgent/pull/2765) (parser: bare `?`), companion draft PR (grammar files: wrap as `()?`) +**One-pager** — code > prose. Pick **A** or **B**. + +--- + +## 1. Bug that started this + +Authors write what regex/EBNF muscle memory suggests. Parser does something else. + +```agr +// Author intent: optional polite prefix + = "please" | "could you" | "can you" | "kindly"; + = ? "open outlook" + -> { actionName: "openApp", parameters: { app: "outlook" } }; +``` + +```text +// Actual AST on main today (bare ?): +[ + { type: "ruleReference", name: "Polite", optional: false }, + { type: "string", value: ["?", "open", "outlook"] } // "?" is a LITERAL token +] + +// Match results: +"open outlook" → no match // missing literal "?" +"please open outlook" → no match // missing literal "?" +"please ? open outlook" → match // nonsense +``` + +```agr +// Works today — only after ")": + = ()? "open outlook" -> { ... }; +// Also works: +$(units:)? +(the | a)? +``` + +Repo evidence (pre-fix): ~16 bare `?` sites across email/weather/video/code/calendar test grammars, all author-intent optional, all broken the same way. Built-in category docs already warn: + +```ts +// builtInGrammarCategories.ts +// Usage in patterns: ()? (note: ()? not ? — bare optional not yet supported) +``` + +Meanwhile `sample.agr` documents the *opposite* aspiration: + +```agr +// ts/extensions/agr-language/sample.agr (misleading today) + = + one? two* three+ four // claimed: optional / * / + + | (item)+ + | (prefix)* suffix; +// Reality on main: "one?" is the literal token "one?" +``` + +--- + +## 2. Why the grammar is this way (Curtis) + +```text +Goal: natural-language patterns should accept "?" without escaping. + + what is the time? // "?" is punctuation in the utterance + what is the singer for song ? // same + +So "?" is mostly a *string character*, not a reserved operator. + +Quantifier meaning is deliberately gated: + after ")" → optional / * / + // )?, )*, )+ + after ">)" in $(...)? // already special-cased + after ">" → NOT a quantifier today + after word → NOT a quantifier today +``` + +```ts +// grammarRuleParser.ts — expressionsSpecialChar (main) +// "?" "*" "+" are intentionally ABSENT +export const expressionsSpecialChar = [ + "|", "(", ")", "<", ">", "$", "-", ";", + "{", "}", "[", "]", +]; + +// Only these forms set optional/repeat today: +// ( ... )? ( ... )* ( ... )+ +// $(name:type)? +``` + +--- + +## 3. Two designs + +### Design A — Keep group-gated quantifiers (status quo + fix authors) + +**Rule:** `?` / `*` / `+` are quantifiers **only** immediately after `)`. Everywhere else they are literal characters in string tokens. + +```agr +// ── Canonical optional forms (A) ────────────────────────────────── +()? // optional rule ref +$(units:)? // optional capture (already) +(the | a)? // optional words +("please" | "kindly")? // optional quoted alts + +// ── Literal "?" (no escape needed) ──────────────────────────────── +what is the time? // matches utterance ending in ? +what is the singer for song ? +"really?" // quoted literal including ? + +// ── Illegal / broken author forms under A (must rewrite) ────────── +? // BROKEN → becomes literal "?" +? // BROKEN +one? // BROKEN → token "one?" +``` + +**This PR (grammar-only):** rewrite broken sites to the canonical form. No parser change. + +```diff +- = ? ? | ... ++ = ()? ()? | ... + +- = ? $(location:) $(units:)? ++ = ()? $(location:) $(units:)? + +- = ... ? ? ? -> { ... } ++ = ... ()? ()? ()? -> { ... } + +- = ... ? -> { ... } ++ = ... ()? -> { ... } +``` + +Files touched (6): + +```text +ts/packages/agents/email/src/emailSchema.agr +ts/packages/agents/weather/src/weatherSchema.agr +ts/packages/agents/video/src/videoSchema.agr +ts/packages/agents/code/src/vscode/displaySchema.agr +ts/packages/agentSdkWrapper/test/calendar-new.agr +ts/packages/agentSdkWrapper/test/calendar-extended.agr +``` + +**Make the footgun obvious (A hardening, optional follow-ups):** + +```ts +// Option A1 — lint / compile warning (recommended if we stay on A) +// After parsing a ruleReference, if next non-ws char is '?' | '*' | '+': +// warn: `?` is not optional; did you mean `()`?` +// Literal '?' will be required in the utterance. + +// Option A2 — hard error in strict mode / CI policy check on .agr +// fail on /<[A-Za-z][A-Za-z0-9_]*>[?*+]/ +// fail on bare word quantifiers if we never intend them + +// Option A3 — docs + sample.agr alignment +// delete "one? two* three+" claim; show only ()? forms +``` + +```agr +// After A1 warning, author sees: +// emailSchema.agr(29,19): warning: bare '?' does not make the +// rule optional; '?' is a literal. Use '()?' instead. +``` + +**Pros / cons (A)** + +```text ++ Zero parser/matcher risk; "?" stays free as NL punctuation ++ Matches intentional design + existing $(...)? / ()? special cases ++ Migration is mechanical and already done for known sites +− Authors keep hitting the footgun (regex muscle memory) +− sample.agr / mental model diverge from EBNF/regex +− Asymmetry: $(x)? works, ? does not (unless grouped) +``` + +--- + +### Design B — Reserved quantifiers (opinionated, consistent) + +**Rule:** postfix `?` / `*` / `+` always mean optional / zero-or-more / one-or-more when they follow a complete sub-expression atom. Literal `?` requires a quote or escape. + +```agr +// ── Atoms that accept a postfix quantifier ──────────────────────── +? // == ()? +* // == ()* ++ // == ()+ +$(units:)? // unchanged +(the | a)? // unchanged +the? // optional word "the" +"please"? // optional exact token please + +// ── Literal "?" must be explicit ────────────────────────────────── +"what is the time?" // whole phrase incl. ? +what is the time\? // escaped single char +// bare: what is the time? // PARSE ERROR or "time" optional + stray? +``` + +```ts +// Parser sketch (B) — after every atom, try readQuantifier() +type Quantifier = { optional?: true; repeat?: true }; // ? | * | + + +function readQuantifier(): Quantifier | undefined { + if (isAt("?")) { skip(1); return { optional: true }; } + if (isAt("*")) { skip(1); return { optional: true, repeat: true }; } + if (isAt("+")) { skip(1); return { repeat: true }; } + return undefined; +} + +// Apply after: +// parseRuleName() → + quant? +// parseGroup() → ( ... ) + quant? // already +// parseVariable() → $(...) + quant? // already for ? +// parseStrAtom() → word | "quoted" + quant? // NEW +``` + +```ts +// expressionsSpecialChar (B) — promote quantifiers to special +export const expressionsSpecialChar = [ + "|", "(", ")", "<", ">", "$", "-", ";", + "{", "}", "[", "]", + "?", "*", "+", // NEW — stop string runs before quantifier +]; +``` + +```agr +// Realistic agent patterns under B + = + ? open // polite optional + | what is the time\? // literal ? + | what is the singer for song \? // literal ? + | show ? files // optional owner + | tag