Skip to content

🐛 FIX: report accurate source lines for directives and text#1164

Draft
chrisjsewell wants to merge 1 commit into
masterfrom
claude/myst-parser-triage-u28jxh
Draft

🐛 FIX: report accurate source lines for directives and text#1164
chrisjsewell wants to merge 1 commit into
masterfrom
claude/myst-parser-triage-u28jxh

Conversation

@chrisjsewell

Copy link
Copy Markdown
Member

Fixes three classes of wrong source-line attribution, plus two rST-parity alignments.

Directive option warnings point at the offending option

Previously all option warnings pointed at the directive's opening line (:opt: style) or the opening --- (YAML style), and unknown keys were aggregated into one warning:

```{note}
---
class: tip
name: mynote
align: whatever   <- line 7
---
```

reported <string>:4: 'note': Unknown option keys: ['align'] .... Now each warning (unknown key, invalid value, bad YAML, # comments) carries the offending line — <string>:7: 'note': Unknown option key: 'align' ... — for both option styles, resolving the long-standing TODO advance line number:

  • parsers/options.py gains a public options_to_tokens (per-key Positions; options_to_items is now a thin wrapper), and its State records comment lines.
  • _parse_directive_options computes an options_position per block style and per-key absolute lines; the --- block split is now line-based: text trailing a closing delimiter still becomes body content (as previously), but is kept as its own line, so it — and every body line after it — maps exactly (the old character-based split shifted all subsequent body lines by one).
  • Directives synthesized from non-source content (HTML images/admonitions) attribute option warnings to the element's own line.

First-line body content was off by one

For no-argument directives with content on the opening line (```{note} text), every body warning reported one line too low. body_offset is now -1 for the merged first line (documented in DirectiveParsingResult; all internal consumers verified, and myst-nb's code-cell cannot hit this branch since it takes an argument).

Stale document.current_line (closes #546)

run_directive set document.current_line and never reset it, so docutils' Node.setup_child stamped the most recent directive's line (or 0) onto every subsequently-created line-less node — every Text node in the document. This is exactly the "errors attributed to the most recent directive" / "line 0" behaviour reported for sphinxcontrib-spelling, and visible in gettext output (table cells extracted at index.md:0). The renderer now keeps current_line in sync as nodes are created; the regenerated .pot fixtures show cells at their real lines.

rST-parity alignments

  • block_text (supersedes Aligns directive's block_text content to rST's #1048, with thanks to @kdeldycke): directives now receive the full directive text — opening fence, options block, unparsed body, closing fence — matching rST, including the header/closing lines that Aligns directive's block_text content to rST's #1048 noted it could not reconstruct. Third-party directives that compute absolute lines from block_text (e.g. click-extra) now get rST-consistent input. Note: erroring directives that embed self.block_text in their error output (list-table, figure, etc.) now show the full fence rather than only the body.
  • include with :literal:: the literal block's line is now start-line + 1 (exact docutils behaviour), not a hardcoded 1.

Behaviour changes to be aware of

  • Unknown-key warnings changed shape: one per key (Unknown option key: 'x' (allowed: [...])) instead of one aggregate — the myst.directive_option subtype is unchanged, so suppression configs keep working.
  • Old-vs-new probes were run against master for: the full docutils rendering suite, a sphinx + sphinx-design project (HTML and searchindex.js byte-identical), myst-nb notebook reading (byte-identical, including malformed metadata blocks), gettext, includes, and a ~30k-input fuzz of the option-parsing rewrite — no rendering differences beyond the intended warning lines/messages and the two rST-parity items above.

Known limitations (documented, out of scope)

  • Warnings for inline syntax report the enclosing paragraph's first line (markdown-it inline tokens carry no per-line maps).
  • Warnings inside multi-line substitution output can map past the substitution's location.
  • Line attribution inside {include}d markdown is off by one (pre-existing, both before and after this PR) — to be tracked separately.

All changed fixture numbers were hand-verified against their source snippets; new conformance cases in reporter_warnings.md lock in the corrected lines (markdown in → warning+line out, reusable as a language-agnostic corpus).


Generated by Claude Code

Fix three classes of wrong source-line attribution:

- Directive option warnings (unknown key, invalid value, bad YAML,
  comments) now point at the offending option's own source line,
  for both `---` and `:option:` block styles, instead of the
  directive's opening line.
  Unknown option keys are now warned once per key, at its own line.
  The `---` options block is now split line-based: any text trailing
  a closing delimiter still becomes body content (as previously),
  but is kept as its own line, so it and all subsequent body lines
  map exactly to their source lines.
  A new `options_to_tokens` API in `myst_parser.parsers.options`
  exposes the tokenizer's per-key positions, and its `State` now
  records the lines of `#` comments
  (`options_to_items` is now a thin wrapper around it).
  Directives run from synthesized, non-source content (HTML images
  and admonitions) attribute option warnings to the element's line.

- Directives with body content on the opening line (no-argument
  directives such as `{note}`) reported every body warning one line
  too low; `body_offset` is now `-1` for the merged first line.

- `document.current_line` was set once per directive and never
  reset, so docutils' `Node.setup_child` stamped the most recent
  directive's line (or 0) onto every subsequently-created line-less
  node, e.g. every `Text` node - the root cause of tools such as
  sphinxcontrib-spelling and gettext reporting 'line 0' or the last
  directive's line. The renderer now keeps `current_line` in sync
  as nodes are created.

Also:

- `block_text` passed to directives is now the full directive text
  (opening fence, options, body, closing fence), matching what an
  equivalent rST directive receives, so third-party directives that
  inspect it see rST-consistent content.
- `include` with `:literal:` now sets the literal block's line to
  `start-line + 1` (docutils parity) instead of a hardcoded 1.

Closes #546
@chrisjsewell chrisjsewell force-pushed the claude/myst-parser-triage-u28jxh branch from f7d6408 to 59480c2 Compare July 15, 2026 02:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Incorrect line number being shown

1 participant