🐛 FIX: report accurate source lines for directives and text#1164
Draft
chrisjsewell wants to merge 1 commit into
Draft
🐛 FIX: report accurate source lines for directives and text#1164chrisjsewell wants to merge 1 commit into
chrisjsewell wants to merge 1 commit into
Conversation
This was referenced Jul 14, 2026
Fix three classes of wrong source-line attribution:
- Directive option warnings (unknown key, invalid value, bad YAML,
comments) now point at the offending option's own source line,
for both `---` and `:option:` block styles, instead of the
directive's opening line.
Unknown option keys are now warned once per key, at its own line.
The `---` options block is now split line-based: any text trailing
a closing delimiter still becomes body content (as previously),
but is kept as its own line, so it and all subsequent body lines
map exactly to their source lines.
A new `options_to_tokens` API in `myst_parser.parsers.options`
exposes the tokenizer's per-key positions, and its `State` now
records the lines of `#` comments
(`options_to_items` is now a thin wrapper around it).
Directives run from synthesized, non-source content (HTML images
and admonitions) attribute option warnings to the element's line.
- Directives with body content on the opening line (no-argument
directives such as `{note}`) reported every body warning one line
too low; `body_offset` is now `-1` for the merged first line.
- `document.current_line` was set once per directive and never
reset, so docutils' `Node.setup_child` stamped the most recent
directive's line (or 0) onto every subsequently-created line-less
node, e.g. every `Text` node - the root cause of tools such as
sphinxcontrib-spelling and gettext reporting 'line 0' or the last
directive's line. The renderer now keeps `current_line` in sync
as nodes are created.
Also:
- `block_text` passed to directives is now the full directive text
(opening fence, options, body, closing fence), matching what an
equivalent rST directive receives, so third-party directives that
inspect it see rST-consistent content.
- `include` with `:literal:` now sets the literal block's line to
`start-line + 1` (docutils parity) instead of a hardcoded 1.
Closes #546
f7d6408 to
59480c2
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes three classes of wrong source-line attribution, plus two rST-parity alignments.
Directive option warnings point at the offending option
Previously all option warnings pointed at the directive's opening line (
:opt:style) or the opening---(YAML style), and unknown keys were aggregated into one warning:reported
<string>:4: 'note': Unknown option keys: ['align'] .... Now each warning (unknown key, invalid value, bad YAML,#comments) carries the offending line —<string>:7: 'note': Unknown option key: 'align' ...— for both option styles, resolving the long-standingTODO advance line number:parsers/options.pygains a publicoptions_to_tokens(per-keyPositions;options_to_itemsis now a thin wrapper), and itsStaterecords comment lines._parse_directive_optionscomputes anoptions_positionper block style and per-key absolute lines; the---block split is now line-based: text trailing a closing delimiter still becomes body content (as previously), but is kept as its own line, so it — and every body line after it — maps exactly (the old character-based split shifted all subsequent body lines by one).First-line body content was off by one
For no-argument directives with content on the opening line (
```{note} text), every body warning reported one line too low.body_offsetis now-1for the merged first line (documented inDirectiveParsingResult; all internal consumers verified, and myst-nb'scode-cellcannot hit this branch since it takes an argument).Stale
document.current_line(closes #546)run_directivesetdocument.current_lineand never reset it, so docutils'Node.setup_childstamped the most recent directive's line (or 0) onto every subsequently-created line-less node — everyTextnode in the document. This is exactly the "errors attributed to the most recent directive" / "line 0" behaviour reported for sphinxcontrib-spelling, and visible in gettext output (table cells extracted atindex.md:0). The renderer now keepscurrent_linein sync as nodes are created; the regenerated.potfixtures show cells at their real lines.rST-parity alignments
block_text(supersedes Aligns directive'sblock_textcontent to rST's #1048, with thanks to @kdeldycke): directives now receive the full directive text — opening fence, options block, unparsed body, closing fence — matching rST, including the header/closing lines that Aligns directive'sblock_textcontent to rST's #1048 noted it could not reconstruct. Third-party directives that compute absolute lines fromblock_text(e.g. click-extra) now get rST-consistent input. Note: erroring directives that embedself.block_textin their error output (list-table,figure, etc.) now show the full fence rather than only the body.includewith:literal:: the literal block's line is nowstart-line + 1(exact docutils behaviour), not a hardcoded 1.Behaviour changes to be aware of
Unknown option key: 'x' (allowed: [...])) instead of one aggregate — themyst.directive_optionsubtype is unchanged, so suppression configs keep working.searchindex.jsbyte-identical), myst-nb notebook reading (byte-identical, including malformed metadata blocks), gettext, includes, and a ~30k-input fuzz of the option-parsing rewrite — no rendering differences beyond the intended warning lines/messages and the two rST-parity items above.Known limitations (documented, out of scope)
{include}d markdown is off by one (pre-existing, both before and after this PR) — to be tracked separately.All changed fixture numbers were hand-verified against their source snippets; new conformance cases in
reporter_warnings.mdlock in the corrected lines (markdown in → warning+line out, reusable as a language-agnostic corpus).Generated by Claude Code