Skip to content

Add docs validation script#2

Open
autocarl wants to merge 1 commit into
yllibed:mainfrom
autocarl:ci/docs-validation-script
Open

Add docs validation script#2
autocarl wants to merge 1 commit into
yllibed:mainfrom
autocarl:ci/docs-validation-script

Conversation

@autocarl

@autocarl autocarl commented Jul 4, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Adds npm run docs:validate for Repl/MCP documentation invariants.
  • Validates copy/paste MCP host JSON snippets, including no-build dotnet run host launches and VS Code servers shape requirements.
  • Guards Repl.Mcp wording so docs describe it as the component used to build MCP servers, not as the server users install directly.

Verification

  • npm run docs:validate — passed, 37 files checked.
  • git diff --check — passed.
  • ASTRO_TELEMETRY_DISABLED=1 XDG_CONFIG_HOME=/workspace/.config npm run astro -- build — passed, 36 pages built.

Follow-up needed for CI wiring

I attempted to include the read-only PR workflow and to remove the daily Pages deploy schedule, but the current autocarl token only has the repo scope. GitHub rejected pushes that modify .github/workflows/* without the workflow scope:

refusing to allow a Personal Access Token to create or update workflow `.github/workflows/deploy.yml` without `workflow` scope

So this PR intentionally contains the script only. With a token/account that has the workflow scope, the CI wiring should add this workflow:

name: Validate docs

on:
  pull_request:
  push:
    branches: [main]
  workflow_dispatch:

permissions:
  contents: read

jobs:
  docs:
    runs-on: ubuntu-latest

    steps:
      - name: Checkout website
        uses: actions/checkout@v4

      - name: Setup Node.js
        uses: actions/setup-node@v4
        with:
          node-version: '22'
          cache: npm

      - name: Install npm dependencies
        run: npm ci

      - name: Validate documentation invariants
        run: npm run docs:validate

      - name: Build Astro content
        run: npm run astro -- build
        env:
          ASTRO_TELEMETRY_DISABLED: '1'

Daily deployment note

The daily deployment is caused by .github/workflows/deploy.yml:

schedule:
  - cron: '0 6 * * *'

If daily deploys are not useful, remove that schedule block. Deploys will still happen on push to main, workflow_dispatch, and repository_dispatch from Repl releases.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: c7d4909dee

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread scripts/validate-docs.mjs
if (launchesDotnetProject) {
const projectIndex = args.indexOf('--project');
const noBuildIndex = args.indexOf('--no-build');
if (noBuildIndex === -1 || noBuildIndex > projectIndex) {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Allow valid dotnet option ordering

When a host snippet uses the valid dotnet run --project app.csproj --no-build -- mcp serve ordering, this validator reports a failure even though --no-build is still a dotnet run option before the -- application-argument separator. Microsoft Learn's dotnet run docs list both --no-build and --project as run options and state that recognized options are consumed before -- (https://learn.microsoft.com/en-us/dotnet/core/tools/dotnet-run), so this should check that --no-build appears before -- rather than before --project.

Useful? React with 👍 / 👎.

@carldebilly carldebilly left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good idea overall — this encodes the invariants from the recent MCP docs reviews (--no-build, repl-contacts-sample, VS Code servers shape, Repl.Mcp wording) as executable checks. One logic error and a couple of coordination notes below.

Cross-repo note: this validator consecrates --no-build + a prior dotnet build as the canonical host-config form. The yllibed/repl repo (PR #37) currently documents plain dotnet run in the sample README and configs/*.json — those should be aligned to the same form, otherwise the repo and the website will document two different shapes for the same sample.

Comment thread scripts/validate-docs.mjs
if (launchesDotnetProject) {
const projectIndex = args.indexOf('--project');
const noBuildIndex = args.indexOf('--no-build');
if (noBuildIndex === -1 || noBuildIndex > projectIndex) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This ordering constraint is stricter than dotnet run actually requires: options can appear in any order before the -- separator, so ["run", "--project", "x.csproj", "--no-build", "--", "mcp", "serve"] is perfectly valid yet fails this check with a confusing message. The real invariant is presence before the -- separator (after --, --no-build would be an app argument, not a dotnet option):

const dashDashIndex = args.indexOf('--');
const noBuildIndex = args.indexOf('--no-build');
if (noBuildIndex === -1 || (dashDashIndex !== -1 && noBuildIndex > dashDashIndex)) {
  fail(...);
}

Comment thread scripts/validate-docs.mjs
fail(file, 1, 'Describe Repl.Mcp as the component used to build MCP servers, not as the MCP server itself.');
}

if (file.endsWith('cookbook/mcp-server.mdx') && text.includes('## Agent-host setup')) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sequencing hazard with PR #1: the ## Agent-host setup section this branch validates only exists in PR #1, which as currently written uses "contacts-sample", has no "--no-build", and no dotnet build samples/08-mcp-server/McpServerSample.csproj instruction. Each PR passes in isolation because these checks are conditional, but once both are merged (and CI is wired), docs:validate will fail on main. PR #1 needs to be updated to satisfy these invariants before or together with this one.

Comment thread scripts/validate-docs.mjs

function validateMcpHostDocs(file, text) {
if (/Repl\.Mcp\s+is\s+(?:an?|the)\s+MCP server/i.test(text)) {
fail(file, 1, 'Describe Repl.Mcp as the component used to build MCP servers, not as the MCP server itself.');

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor: this reports the violation at line 1 instead of the offending line, while findLine exists and is used by the other checks — fail(file, findLine(text, 'Repl.Mcp'), ...) would point reviewers at the actual sentence.

Comment thread scripts/validate-docs.mjs
}

if (inJson) buffer.push(line);
}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor: a ```json fence left unclosed at EOF is silently dropped (the buffered block is never pushed), so a malformed doc skips validation instead of failing it. Consider flushing (or failing) when inJson is still true after the loop.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants