chore: salvage /tmp doc tools + reflow the v2.3.1 release notes - #349
chore: salvage /tmp doc tools + reflow the v2.3.1 release notes#349doublegate wants to merge 2 commits into
Conversation
…reboot
An NVIDIA driver/kernel update needed a reboot, which wipes /tmp. The
`tmp-salvage` dry run offered 305 candidates totalling ~600 MB; three files were
taken and everything else was deliberately left to be wiped.
The three exist NOWHERE ELSE on disk (checked across all of ~/Code) and were
written during the v2.2.5-v2.3.0 documentation work:
reflow.py stdlib-only. Unwraps hard-wrapped markdown into single
full-width lines while preserving blank lines, ATX
headings, horizontal rules, fenced code blocks, tables,
blockquotes and raw HTML -- the constructs that break
when naively joined. This is the tool that fixed the
GitHub release-notes formatting: notes published from
v2.2.5 onward had been hard-wrapped at ~80 columns,
which GitHub renders as artificially narrow text
instead of using the full width available.
assemble.py bs4. Rendered HTML fragment -> standalone document.
guardrails_assemble.py bs4. Same, for the provenance-guardrails doc: injects a
title block and reddens a curated set of hard takeaways.
WHAT WAS LEFT BEHIND, and why -- recorded in docs/SALVAGE_MANIFEST.md so the
curation is auditable rather than implicit:
* perf.data (586 MB), probe.perf, fp.data, di.data, bolt.log -- perf recordings
and CI logs, regenerable with one command;
* ~100 .md/.json PR-review files -- already posted to PR #348;
* fisk-*.png / v2p-*.png (8 MB) -- screenshots that may be commercial-ROM
framebuffers, not swept into the tree unexamined;
* /tmp/accuracycoin-build/ -- third-party MIT source plus a Windows nesasm.exe.
Vendoring third-party source is a deliberate provenance decision under the
project's firewall rules, not a salvage side effect;
* agy-review.sh + selftest -- duplicates of files already in scripts/ and
Local_Only-Projects/antigravity-pr-review/;
* p3.sh/p4.sh/probe*.sh/mi.rs -- extraction artifacts (workflow YAML pulled out
for shellcheck) and a temp probe already reverted on purpose.
Everything built during the session was already committed (frame_probe.rs,
ab_check.sh, frame_breakdown.sh, the perf_log_check.py p99 gate), so the
scratchpad held no unique source beyond these three.
Verification: SHA-256 checked identical to the /tmp originals after copying; all
three parse; reflow.py is ruff-clean and was round-trip tested against a case
covering paragraphs, list continuations, fenced code, tables and headings.
The two bs4 scripts are preserved VERBATIM and are not ruff-clean (SIM115,
UP031). Deliberate: bs4 is not installed here and no sample fragment survives, so
a lint rewrite could not be executed to prove it still behaved. Rewriting code
that cannot be run is a worse trade than a style nit; the README says so and says
to clean them up when someone next has real input to test against.
README-doc-tools.md is added alongside them, because three unexplained scripts in
release-automation/ six months from now are only marginally better than losing
them.
The v2.3.1 release notes were published hard-wrapped at ~80 columns, which
GitHub renders as artificially narrow text instead of using the full width
available. Every other recent release uses the wide format; measured line-length
profiles of the published bodies:
v2.2.4 median=86 p90=329 max=563 wide
v2.2.5 median=165 p90=383 max=1006 wide
v2.2.9 median=491 p90=820 max=1025 wide
v2.3.0 median=245 p90=639 max=725 wide
v2.3.1 median=75 p90=81 max=136 HARD-WRAPPED <-- the regression
v2.3.1 was the only one out of style: the earlier pass that fixed v2.2.5 onward
held, and this was a fresh regression introduced when the notes were authored.
Reflowed with scripts/release-automation/reflow.py -- the tool written for
exactly this and salvaged from /tmp one commit earlier, which makes forgetting to
run it a particularly avoidable miss. Result: median 75 -> 93, p90 81 -> 240,
max 136 -> 463, in line with the rest.
Content is unchanged, which was verified rather than assumed: word count
identical at 719 before and after, 13 table rows and 6 headings preserved on both
sides. reflow.py only joins wrapped continuation lines within a paragraph or list
item and explicitly preserves blank lines, ATX headings, horizontal rules, fenced
code blocks, tables, blockquotes and raw HTML.
The published release body was updated to match via `gh release edit
--notes-file`, so the source file and GitHub agree; all three platform assets
remain attached.
📝 WalkthroughWalkthroughAdded three release-automation scripts for Markdown reflow and HTML assembly. Documented their recovery and usage. Reformatted the v2.3.1 release notes without changing their content. ChangesRelease automation
Estimated code review effort: 3 (Moderate) | ~20 minutes Possibly related PRs
Suggested reviewers: 🚥 Pre-merge checks | ✅ 8 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (8 passed)
✨ Finishing Touches 💡 1📝 Generate docstrings 💡
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Antigravity review (Gemini via Ultra)This PR reflows Blocking issuesNone found. Suggestions
Nitpicks
Automated first-pass review by |
There was a problem hiding this comment.
Pull request overview
Adds salvaged documentation/release-note helper scripts to the repository and reformats the v2.3.1 GitHub release notes to remove hard-wrapping so they render at full width.
Changes:
- Add
reflow.pyto unwrap hard-wrapped Markdown while preserving common structural Markdown constructs. - Add two BeautifulSoup-based HTML fragment “assembler” scripts plus a short README describing intended usage.
- Reflow
.github/release-notes/v2.3.1.mdand record the/tmpsalvage decision indocs/SALVAGE_MANIFEST.md.
Reviewed changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| scripts/release-automation/reflow.py | New stdlib-only Markdown reflow tool used to unwrap hard-wrapped release notes. |
| scripts/release-automation/assemble.py | New BeautifulSoup-based HTML fragment → document assembler (one-off tooling). |
| scripts/release-automation/guardrails_assemble.py | New variant of the assembler for the guardrails document with custom title block and highlighting. |
| scripts/release-automation/README-doc-tools.md | Documents the purpose and usage of the salvaged doc/release-note helpers. |
| docs/SALVAGE_MANIFEST.md | Records what was (and was not) salvaged from /tmp and why. |
| .github/release-notes/v2.3.1.md | Reflows the v2.3.1 release notes to remove hard wrapping while keeping content intact. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| frag_path, out_path = sys.argv[1], sys.argv[2] | ||
| html = open(frag_path, encoding="utf-8").read() | ||
| soup = BeautifulSoup(html, "html.parser") |
| frag_path, out_path = sys.argv[1], sys.argv[2] | ||
| html = open(frag_path, encoding="utf-8").read() | ||
| soup = BeautifulSoup(html, "html.parser") |
| content = ln.lstrip()[1:] | ||
| content = content.removeprefix(" ") |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 6
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@docs/SALVAGE_MANIFEST.md`:
- Around line 131-132: Clarify the SHA-256 verification statement in the
recovery evidence near the referenced release-automation documentation: state
whether hashes were captured before subsequent edits to reflow.py or other
files, and record the per-file hashes if they constitute release evidence.
Ensure the wording does not imply that the current files remain byte-identical
unless that is verified, while preserving provenance for recovered and
externally sourced artifacts.
In `@scripts/release-automation/assemble.py`:
- Around line 6-7: Validate that exactly two command-line arguments are provided
before assigning paths or reading files in assemble.py at lines 6-7 and
guardrails_assemble.py at lines 8-9; otherwise exit with a clear usage message.
Apply the same argument-count validation and message consistently in both
scripts.
- Around line 27-33: Validate the required table and NOTE paragraph structure
before slicing in the surrounding fragment assembly logic: handle missing
matches with a clear rejection error, and reject any fragment where the NOTE
index is not after the table index. Preserve the existing before, table, and
middle extraction only after confirming both elements exist and table_i <
note_i.
In `@scripts/release-automation/README-doc-tools.md`:
- Around line 33-41: Update the documentation for assemble.py and
guardrails_assemble.py to state that the rendered HTML input must contain a
top-level table and a top-level paragraph whose text starts with “NOTE”. Keep
the existing usage example, and either document that missing elements cause
failure or add explicit validation with an actionable error in assemble.py.
In `@scripts/release-automation/reflow.py`:
- Around line 46-52: Update flush() to join buffered lines without stripping
trailing spaces, preserving Markdown hard-break markers and each line’s
whitespace boundaries. Revise the blank-line handling around the logic at lines
92-98 so authored runs of blank lines are retained rather than collapsed, and
add round-trip coverage for hard breaks and multiple blank lines.
- Around line 56-78: Update the fence handling in the reflow loop to parse
container prefixes first, record the opening fence character and length, and
only close a code block with a matching delimiter of sufficient length. Ensure
fenced blocks inside blockquotes are recognized correctly, while non-fence code
lines remain unchanged; replace the current in_code toggle logic around flush()
and out.append().
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: 72a90697-26bf-471f-a909-a9c06126a940
📒 Files selected for processing (6)
.github/release-notes/v2.3.1.mddocs/SALVAGE_MANIFEST.mdscripts/release-automation/README-doc-tools.mdscripts/release-automation/assemble.pyscripts/release-automation/guardrails_assemble.pyscripts/release-automation/reflow.py
| SHA-256 verified identical to source after copy. See | ||
| `scripts/release-automation/README-doc-tools.md` for usage. |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win
Clarify the SHA-256 verification timeline.
The README states that reflow.py was fixed after recovery, while this line says the files were SHA-256 identical to the /tmp sources after copying. State whether the hashes were captured before later edits, and record per-file hashes if this is the release evidence. Otherwise, readers may treat the current files as byte-identical recovered artifacts.
As per coding guidelines, preserve provenance and clearly document recovered, discarded, or externally sourced artifacts.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@docs/SALVAGE_MANIFEST.md` around lines 131 - 132, Clarify the SHA-256
verification statement in the recovery evidence near the referenced
release-automation documentation: state whether hashes were captured before
subsequent edits to reflow.py or other files, and record the per-file hashes if
they constitute release evidence. Ensure the wording does not imply that the
current files remain byte-identical unless that is verified, while preserving
provenance for recovered and externally sourced artifacts.
Source: Coding guidelines
| frag_path, out_path = sys.argv[1], sys.argv[2] | ||
| html = open(frag_path, encoding="utf-8").read() |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
Validate the two required command-line arguments.
Missing arguments raise IndexError. Extra arguments are ignored. Exit with a usage message unless exactly two paths are supplied.
scripts/release-automation/assemble.py#L6-L7: validate the argument count before reading the input and output paths.scripts/release-automation/guardrails_assemble.py#L8-L9: apply the same validation and usage message.
Proposed fix
-frag_path, out_path = sys.argv[1], sys.argv[2]
+if len(sys.argv) != 3:
+ raise SystemExit(f"usage: {sys.argv[0]} INPUT_FRAGMENT OUTPUT_HTML")
+frag_path, out_path = sys.argv[1:]As per path instructions, prioritize correctness and clear error messages over style.
📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| frag_path, out_path = sys.argv[1], sys.argv[2] | |
| html = open(frag_path, encoding="utf-8").read() | |
| if len(sys.argv) != 3: | |
| raise SystemExit(f"usage: {sys.argv[0]} INPUT_FRAGMENT OUTPUT_HTML") | |
| frag_path, out_path = sys.argv[1:] |
🧰 Tools
🪛 ast-grep (0.45.0)
[warning] 6-6: File path is request-/variable-derived; validate and normalize to prevent path traversal.
Context: open(frag_path, encoding="utf-8")
Note: [CWE-22] Improper Limitation of a Pathname to a Restricted Directory ('Path Traversal').
(open-filename-from-request)
🪛 Ruff (0.16.1)
[warning] 7-7: Use a context manager for opening files
(SIM115)
📍 Affects 2 files
scripts/release-automation/assemble.py#L6-L7(this comment)scripts/release-automation/guardrails_assemble.py#L8-L9
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@scripts/release-automation/assemble.py` around lines 6 - 7, Validate that
exactly two command-line arguments are provided before assigning paths or
reading files in assemble.py at lines 6-7 and guardrails_assemble.py at lines
8-9; otherwise exit with a clear usage message. Apply the same argument-count
validation and message consistently in both scripts.
Source: Path instructions
| table_i = next(i for i, e in enumerate(els) if e.name == "table") | ||
| note_i = next(i for i, e in enumerate(els) | ||
| if e.name == "p" and e.get_text().lstrip().startswith("NOTE")) | ||
|
|
||
| before = "".join(str(e) for e in els[:table_i]) | ||
| table = str(els[table_i]) | ||
| middle = "".join(str(e) for e in els[table_i + 1:note_i]) |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
Validate the required fragment structure before slicing.
If the table or NOTE paragraph is absent, next() raises StopIteration. If the NOTE paragraph precedes the table, this code emits the NOTE twice. Reject fragments unless both elements exist and table_i < note_i.
Proposed fix
-table_i = next(i for i, e in enumerate(els) if e.name == "table")
-note_i = next(i for i, e in enumerate(els)
- if e.name == "p" and e.get_text().lstrip().startswith("NOTE"))
+table_i = next((i for i, e in enumerate(els) if e.name == "table"), None)
+note_i = next(
+ (i for i, e in enumerate(els)
+ if e.name == "p" and e.get_text().lstrip().startswith("NOTE")),
+ None,
+)
+if table_i is None or note_i is None or table_i >= note_i:
+ raise ValueError("expected a top-level table followed by a NOTE paragraph")As per path instructions, prioritize correctness and clear error messages over style.
📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| table_i = next(i for i, e in enumerate(els) if e.name == "table") | |
| note_i = next(i for i, e in enumerate(els) | |
| if e.name == "p" and e.get_text().lstrip().startswith("NOTE")) | |
| before = "".join(str(e) for e in els[:table_i]) | |
| table = str(els[table_i]) | |
| middle = "".join(str(e) for e in els[table_i + 1:note_i]) | |
| table_i = next((i for i, e in enumerate(els) if e.name == "table"), None) | |
| note_i = next( | |
| (i for i, e in enumerate(els) | |
| if e.name == "p" and e.get_text().lstrip().startswith("NOTE")), | |
| None, | |
| ) | |
| if table_i is None or note_i is None or table_i >= note_i: | |
| raise ValueError("expected a top-level table followed by a NOTE paragraph") | |
| before = "".join(str(e) for e in els[:table_i]) | |
| table = str(els[table_i]) | |
| middle = "".join(str(e) for e in els[table_i + 1:note_i]) |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@scripts/release-automation/assemble.py` around lines 27 - 33, Validate the
required table and NOTE paragraph structure before slicing in the surrounding
fragment assembly logic: handle missing matches with a clear rejection error,
and reject any fragment where the NOTE index is not after the table index.
Preserve the existing before, table, and middle extraction only after confirming
both elements exist and table_i < note_i.
Source: Path instructions
| `assemble.py` and `guardrails_assemble.py` take a rendered HTML fragment and | ||
| produce a standalone document. They need BeautifulSoup, which is **not** a | ||
| project dependency — install it in a throwaway venv rather than adding it to the | ||
| repo: | ||
|
|
||
| ```bash | ||
| python3 -m venv /tmp/venv && /tmp/venv/bin/pip install beautifulsoup4 | ||
| /tmp/venv/bin/python scripts/release-automation/assemble.py frag.html out.html | ||
| ``` |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
Document the required assemble.py input structure.
Line 33 describes both tools as accepting any rendered HTML fragment. The supplied scripts/release-automation/assemble.py implementation searches for a top-level <table> and a top-level paragraph whose text starts with NOTE; missing either element raises StopIteration. Document these preconditions, or add explicit validation with an actionable error.
As per path instructions, flag documentation that drifts from the code it describes rather than just prose nits.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@scripts/release-automation/README-doc-tools.md` around lines 33 - 41, Update
the documentation for assemble.py and guardrails_assemble.py to state that the
rendered HTML input must contain a top-level table and a top-level paragraph
whose text starts with “NOTE”. Keep the existing usage example, and either
document that missing elements cause failure or add explicit validation with an
actionable error in assemble.py.
Source: Path instructions
| def flush(): | ||
| if buf: | ||
| first = buf[0] | ||
| indent = first[: len(first) - len(first.lstrip())] | ||
| joined = indent + " ".join(p.strip() for p in buf) | ||
| out.append(joined) | ||
| buf.clear() |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | 🏗️ Heavy lift
Preserve Markdown hard breaks and blank-line runs.
p.strip() removes the two trailing spaces that encode a Markdown hard break. Lines 92-97 also collapse authored blank-line runs. Both behaviors conflict with the documented preservation guarantee and can change the rendered release body.
Preserve hard-break boundaries and retain each blank line instead of normalizing this whitespace. Add round-trip cases for first··\nsecond and paragraphs separated by multiple blank lines.
Also applies to: 92-98
🧰 Tools
🪛 Ruff (0.16.1)
[warning] 46-46: Missing return type annotation for private function flush
Add return type annotation: None
(ANN202)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@scripts/release-automation/reflow.py` around lines 46 - 52, Update flush() to
join buffered lines without stripping trailing spaces, preserving Markdown
hard-break markers and each line’s whitespace boundaries. Revise the blank-line
handling around the logic at lines 92-98 so authored runs of blank lines are
retained rather than collapsed, and add round-trip coverage for hard breaks and
multiple blank lines.
| # Fenced code block toggle. | ||
| if stripped.startswith(("```", "~~~")): | ||
| flush() | ||
| out.append(ln) | ||
| in_code = not in_code | ||
| continue | ||
| if in_code: | ||
| out.append(ln) | ||
| continue | ||
| if stripped == "": | ||
| flush() | ||
| out.append("") | ||
| continue | ||
| # Blockquote: join consecutive `>` lines into one full-width `> ...` line. | ||
| if stripped.startswith(">"): | ||
| content = ln.lstrip()[1:] | ||
| content = content.removeprefix(" ") | ||
| if buf and buf[0].lstrip().startswith(">"): | ||
| buf.append(content) # continuation of the current blockquote | ||
| else: | ||
| flush() | ||
| buf.append(ln.rstrip()) # seed with the full `> ...` line | ||
| continue |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | 🏗️ Heavy lift
Track the opening fence before closing a code block.
startswith(("```", "~~~")) toggles state for any fence-looking line. A three-backtick line closes a four-backtick block. A tilde fence can also close a backtick fence. Fences inside blockquotes are not recognized and their code lines are joined into one blockquote line.
Record the opening fence character and length. Only close on a matching delimiter. Parse container prefixes before applying fence handling.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@scripts/release-automation/reflow.py` around lines 56 - 78, Update the fence
handling in the reflow loop to parse container prefixes first, record the
opening fence character and length, and only close a code block with a matching
delimiter of sufficient length. Ensure fenced blocks inside blockquotes are
recognized correctly, while non-fence code lines remain unchanged; replace the
current in_code toggle logic around flush() and out.append().
Two small chores, no code changes.
1. Salvage three doc tools from
/tmpbefore a rebootAn NVIDIA driver/kernel update needed a reboot, which wipes
/tmp. Thetmp-salvagedry run offered 305 candidates / ~600 MB; three files were taken. All three exist nowhere else on disk (checked across~/Code).reflow.pyassemble.pybs4guardrails_assemble.pybs4Left behind deliberately, recorded in
docs/SALVAGE_MANIFEST.mdso the curation is auditable: a 586 MBperf.dataand other perf recordings (regenerable), ~100 PR-review files already posted to #348, screenshots that may be commercial-ROM framebuffers, and/tmp/accuracycoin-build/— third-party MIT source plus a Windowsnesasm.exe, where vendoring is a deliberate provenance decision rather than a salvage side effect.The two
bs4scripts are preserved verbatim and are not ruff-clean (SIM115,UP031). Deliberate:bs4isn't installed and no sample fragment survives, so a lint rewrite couldn't be executed to prove it still behaved.reflow.py— testable, being stdlib-only — was fixed and round-trip verified.2. Reflow the v2.3.1 release notes
They were published hard-wrapped at ~80 columns, which GitHub renders as artificially narrow. Measured across published bodies:
v2.3.1 was the only one out of style — a fresh regression, not a return of the earlier problem. Fixed with
reflow.py(salvaged one commit earlier, which makes forgetting to run it a particularly avoidable miss): median 75 → 93, p90 81 → 240, max 136 → 463.Content verified unchanged rather than assumed: word count identical at 719, 13 table rows and 6 headings preserved. The published release body was updated to match via
gh release edit, and all three platform assets remain attached.Verification
pre-commitclean (markdownlint, EOF, whitespace, line endings);reflow.pyruff-clean and round-trip tested against paragraphs, list continuations, fenced code, tables and headings; SHA-256 verified identical to the/tmporiginals after copying. No Rust touched — AccuracyCoin and nestest unaffected by construction.Summary by CodeRabbit
Documentation
New Tools