Skip to content

fix(cli): make --llms-full complete and accurate - #474

Open
mldangelo-oai wants to merge 19 commits into
mainfrom
mdangelo/codex/fix-llms-full-manifest
Open

fix(cli): make --llms-full complete and accurate#474
mldangelo-oai wants to merge 19 commits into
mainfrom
mdangelo/codex/fix-llms-full-manifest

Conversation

@mldangelo-oai

@mldangelo-oai mldangelo-oai commented Aug 16, 2026

Copy link
Copy Markdown
Collaborator

Summary

Make --llms-full a complete reference for the installed CLI. Incur's Markdown renderer exposed camel-case schema keys as flags and omitted required-option markers, allowed values, and validation constraints. Command-specific output checks also prevented some documentation requests from reaching the renderer.

Changes

  • Render a documentation-only view of the live command schemas with canonical kebab-case flags, required options, allowed values, defaults, and constraints, including constraints on repeated values.
  • Include versioned runtime metadata and reuse the packaged README as the source for authentication, configuration, scan results, exit codes, saved scans, and publication guidance.
  • Preserve command and group discovery, use scoped headings, and include the long operating guide only in the root manifest.
  • Share command-result restrictions between CLI enforcement and every command reference, while documenting supported discovery formats and command-result-only filtering.
  • Normalize supported global-value option spellings and share their consumption across command validation and discovery, while preserving the structured Incur manifest and shell-completion protocol.
  • Keep built-in option diagnostics value-free, recover static guidance from the selected command schema, and reuse the existing safe framework-error rendering without changing successful structured output.
  • Check the public command contracts against their live schemas and shared result policies, avoid assertions on generated English prose, cover built-in argument ownership with a pure parser regression, and exercise the installed manifest and packaged guide.

Testing

  • Current-head manifest tests: 13 passed.
  • Previous-head error, CLI, manifest, workbench, and skill tests on 6bfcdbb6: 187 passed.
  • Current-head randomized full suite on beedc8ee, Bun 1.3.14, --no-env-file, and seed 12345: 1,307 passed, 11 skipped, zero failures. Live integrations and machine-policy opt-ins were disabled; inherited credential, state, and runtime overrides were removed.
  • Current-head build, pack, check:package, and standalone test:package: passed. The package check validated 222 archive entries and the installed CLI, including the actual nested worker without a global Codex installation. The archive is byte-identical to the previous published-head archive because the follow-up changes only a test assertion.
  • Previous-head standard full suite on 0ce81229: 1,304 passed, 11 skipped, zero failures.
  • pnpm run types, pnpm run format, and git diff --check: passed.
  • The initial sandboxed full run hit a macOS nested-sandbox setup error. The unchanged affected test and the complete suite passed when rerun outside the outer sandbox.
  • Three fresh native review passes and independent verification on beedc8ee found no remaining blocking or advisory findings. Exact-head local full/package validation is complete; fresh pushed-head standard/cross-platform CI remains required before merge.
  • The previous pushed-head Windows Node 24 shard timed out in the unchanged credential-home authentication test. The same test and ACL implementation match recorded main, whose Windows 24 shard passed; the same published head also passed Windows 22's corresponding shard. The existing timeout, credential protections, and CI workflow remain unchanged.

Risk and rollout

The change adapts Markdown discovery, shares existing result-output policy, and retains Incur's live parser schemas and argument semantics. Persisted scan state and publication behavior are unchanged. Framework failures reuse the existing safe error renderer and trusted schema constraints. The installed-package nested-worker check remains intact. A prior Windows Node 24 verifier hit its worker-startup timer; the same helper passed on recorded main and on the PR's Windows Node 22 verifier. Fresh exact-head CI, including Windows Node 24, is still required. No dependency upgrade or feature-specific release-version change is included.

Public disclosure review

Newly authored material was reviewed for public disclosure. Existing automated review comments contain access-restricted report references; maintainer cleanup remains outstanding.

  • No customer, partner, prospect, or user identities, data, or identifying details are included.
  • No credentials, personal data, private source, scan findings, or nonpublic links or tickets are included.
  • I reviewed the branch name, title, description, commits, changes, comments, logs, screenshots, attachments, and links for public disclosure.

@github-actions github-actions Bot added the bug Something isn't working label Aug 16, 2026

Copy link
Copy Markdown
Collaborator Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Security review completed. No security issues were found in this pull request.

Reviewed commit: 5789da8956

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Nice work!

Reviewed commit: 5789da8956

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@mldangelo-oai
mldangelo-oai marked this pull request as ready for review August 16, 2026 03:54

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 5789da8956

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread sdk/typescript/tests-ts/cli-manifest.test.ts Outdated

Copy link
Copy Markdown
Collaborator Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Security review completed. No security issues were found in this pull request.

Reviewed commit: d2e63a0a1d

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d2e63a0a1d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +45 to +46
INCUR_VALUE_OPTIONS.has(argument) &&
argv[index + 1] !== undefined

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Strip equals-form global options before deriving scope

When an accepted equals-form global option precedes the command path, such as codex-security --filter-output=scan scans show --llms-full, this exact lookup does not consume it. renderFullMarkdownManifest consequently treats the option as the first command argument and leaves the scope empty, producing a root heading and root operating guide even though Incur selected only scans show; handle equals forms for --filter-output, --token-limit, and --token-offset before deriving the scope.

Useful? React with 👍 / 👎.

Comment on lines +173 to +177
if (options && property.type === "array") {
details.push("Repeat this flag for multiple values.");
if (Array.isArray(property.default)) {
details.push(`Default: ${codeValue(property.default)}.`);
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Document constraints on repeatable option values

Every repeatable option such as --path, --knowledge-base, and --codex is an array whose string items have a minimum length, but Zod places that constraint under property.items and this branch only describes repetition and the outer default. Because Incur does not render these constraints itself, the full manifest omits that empty repeated values are rejected; include applicable item-level constraints when documenting arrays.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Collaborator Author

@codex review

Please review the current head, 59ae515c1f81f982a0edeaaa0b07574653e899db.

@chatgpt-codex-connector

Copy link
Copy Markdown

Security review completed. No security issues were found in this pull request.

Reviewed commit: 59ae515c1f

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 59ae515c1f

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

{ commands: [{ name: command }] },
[command],
);
expect(scoped).toMatch(/--filter-output.*only to command results/u);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Test filtering semantics without pinning renderer prose

The latest commit reintroduces an assertion on exact generated English by requiring the words “only to command results,” so a harmless copy edit will fail this test even when filtering behavior and documented restrictions remain correct. Assert a stable semantic property or source-of-truth value instead; this is fresh evidence after the earlier wording assertions were removed.

AGENTS.md reference: sdk/typescript/AGENTS.md:L29-L32

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Collaborator Author

@codex review

Please review the current head, beedc8eee5352c29787ba0882c2564a75ad1277e.

@chatgpt-codex-connector

Copy link
Copy Markdown

Security review completed. No security issues were found in this pull request.

Reviewed commit: beedc8eee5

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Nice work!

Reviewed commit: beedc8eee5

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant