[Schema Consistency] Schema consistency audit - 2026-05-13

[github-actions[bot]]

Overview

Audit of pkg/parser/schemas/main_workflow_schema.json against the canonical frontmatter reference docs and against deprecation prose found inside the schema itself. Three actionable inconsistencies surfaced; all schema fields used in .github/workflows/*.md validated cleanly against the schema (initial "used-but-not-in-schema" hits were false positives from regex matching inside fenced YAML code blocks).

Summary

• Inconsistencies found: 3 documentation gaps + 1 schema-tooling inconsistency (6 paths)
• Categories analyzed: Schema, Parser, Documentation, Workflows
• Strategies used: strategy-field-doc-gap (proven) + strategy-deprecation-keyword-audit (proven)
• New strategy this run: No (day-of-year mod 10 = 3, proven mode)
• Workflow violations: 0 confirmed (all candidates were false positives inside YAML examples)

Critical issues

None. No silent schema/parser mismatches that would let invalid configs through. The issues below are documentation and tooling-marker gaps.

Documentation gaps

1. inline-sub-agents is not in the canonical frontmatter reference

• Schema defines inline-sub-agents: boolean at top level (pkg/parser/schemas/main_workflow_schema.json).
• It has a dedicated page (docs/src/content/docs/reference/inline-sub-agents.md) but is absent from both docs/src/content/docs/reference/frontmatter.md and docs/src/content/docs/reference/frontmatter-full.md.
• Impact: users browsing the canonical frontmatter reference will not discover the field.
• Fix: add an entry under the frontmatter reference (at minimum in frontmatter-full.md) cross-linking to the dedicated page.

2. max-effective-tokens is not in the canonical frontmatter reference

• Schema defines max-effective-tokens at top level with a default of 25000000.
• It is mentioned in docs/src/content/docs/reference/effective-tokens-specification.md, engines.md, and glossary.md, but 0 hits in frontmatter.md and frontmatter-full.md.
• Asymmetric with its sibling max-runs, which has 2 hits in frontmatter-full.md despite both being introduced together (see ADR 31418-move-engine-max-runs-to-top-level-with-awf-enforcement.md).
• Fix: mirror the max-runs section in frontmatter-full.md with a max-effective-tokens entry.

3. disable-model-invocation is missing from the short frontmatter reference

• Schema defines disable-model-invocation: boolean and its description states it is the preferred replacement for deprecated infer.
• Hits: frontmatter.md = 0, frontmatter-full.md = 2.
• Impact: users following the short reference (the default landing page for many) will not discover the canonical replacement for infer and may continue to use the deprecated field.
• Fix: add disable-model-invocation to frontmatter.md alongside related custom-agent fields.

Schema improvements needed

4. Deprecation marker is inconsistent: prose vs JSON-Schema deprecated keyword

The schema currently uses the JSON-Schema "deprecated": true keyword on 6 paths. Another 6 paths declare deprecation only in the description text (DEPRECATED: / Legacy alias for... / Prefer ...). IDEs, schema validators, and editor tooling honor the keyword, not the prose - so these fields are not visually flagged.

Paths described as deprecated but missing the `deprecated: true` keyword

• properties.infer (description: "DEPRECATED: Use 'disable-model-invocation' instead.")
• properties.rate-limit (description: "Legacy alias for 'user-rate-limit'.")
• properties.rate-limit.properties.max (description: "Legacy maximum runs key. Prefer 'max-runs-per-window'.")
• properties.rate-limit.properties.max-runs (description: "Legacy maximum runs key. Prefer 'max-runs-per-window'.")
• properties.on.oneOf.1.properties.command
• properties.tools.properties.github.oneOf.3.properties.mode

Paths already correctly marked with `deprecated: true` (for reference)

• $defs.stdio_mcp_tool.properties.network
• properties.safe-outputs.properties.create-agent-task.oneOf.0
• properties.tools.properties.github.oneOf.3.properties.repos
• properties.tools.properties.github.oneOf.3.properties.toolset
• properties.tools.properties.grep
• properties.tools.properties.serena

Fix: add "deprecated": true to each of the 6 prose-only paths so tooling can render them with strikethrough / warnings.

Parser updates required

None identified this run. The parser handles top-level frontmatter through dynamic map access (frontmatter["<key>"]) and a typed WorkflowFrontmatter struct (pkg/workflow/frontmatter_types.go), so a naive yaml-tag diff against the schema produces noise, not signal. Fields initially flagged as orphaned (e.g. resources) were confirmed to be wired through struct tags.

Workflow violations

None. The pre-computed field_gaps.in_used_not_schema list (affected_sections, independence, model_adequacy, else, https, try, verdict, ...) was sampled and every match landed inside a fenced YAML code block in the markdown body of a workflow file (notably daily-subagent-optimizer.md and contribution-check.md), not in the frontmatter itself. The detection strategy needs a frontmatter-boundary filter to be useful next run.

Recommendations

1. Add inline-sub-agents, max-effective-tokens, and disable-model-invocation entries to docs/src/content/docs/reference/frontmatter-full.md; cross-link from frontmatter.md where space allows.
2. Add "deprecated": true to the 6 schema paths listed above so editor tooling marks them visually.
3. Update the schema-diff generator (/tmp/gh-aw/agent/schema-diff.json) to restrict its workflow-frontmatter scan to the ---...--- frontmatter block only - this will eliminate the false-positive flood in in_used_not_schema.
4. Consider whether rate-limit (legacy alias) should be slated for removal in a future major; the description already directs users to user-rate-limit.

Strategy performance

• Strategies used: strategy-field-doc-gap, strategy-deprecation-keyword-audit
• Findings: 3 doc gaps + 6 schema paths needing deprecated: true
• Effectiveness: HIGH for both - low-cost bulk jq + grep queries produced actionable, specific findings
• Should reuse: YES. Cached at /tmp/gh-aw/cache-memory/strategies.json
• Strategy retired: strategy-go-reference-count produced a false lead on resources; keep with caveat that struct-tag bindings must be cross-checked.

Next steps

• Update frontmatter-full.md with the 3 missing fields (item 1)
• Add "deprecated": true to the 6 schema paths (item 2)
• Fix the schema-diff generator to scope used_in_workflows to frontmatter blocks only (item 3)
• Decide on rate-limit removal timeline (item 4)

References:

• §25782416850

Generated by Schema Consistency Checker · ● 7.1M · ◷

• expires on May 14, 2026, 6:36 AM UTC

[github-actions[bot]]

This discussion was automatically closed because it expired on 2026-05-14T06:36:01.362Z.

Closed by Workflow