Validate python tri-comparison ledger; fix Cython func_start recall gap - #2001
Merged
Conversation
…investigation Adds the args-granularity taxonomy (per_function/program_level/none/proxy), the direct-dispatch-when-obvious refinement to step 1, and the confirmed fix-and-ship-in-same-pass extension to step 4.3 bucket 1, all learned during prior sweep sessions but never committed. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…on ledger tri-comparison-ledger-sweep on python: all 4 unvalidated shapes were investigated directly (no dispatch needed, corpus evidence was conclusive) and are now status:validated, closing python's ledger backlog entirely. Two real, fixed defects found: - #1999: python's func_start regex only matched the literal `def` keyword, so Cython's `cdef`/`cpdef` module-level function definitions (no `def` at all) were a complete recall gap -- 68 real functions across cython/MemoryView.pyx (52) and .pxd (16) in the crucible corpus, ctags-corroborated, 0 found by GitGalaxy. Added a second func_start alternative for cdef/cpdef signatures, excluding cdef class/struct/enum/union/extern/packed/fused (declarations, not functions). One narrow residual gap left undone: a Tempita `{{template}}` return-type placeholder in one codegen-artifact function. - #2000: tri_comparison_gatherer.py's ctags args counter mis-split a comma inside a quoted string-literal default value (numpy/crackfortran.py's markoutercomma(line, comma=','):). Fixed with quote-tracking, tightly bounded for single-quotes specifically because Rust's lifetime syntax (`&'a str`, `Context<'_>`) is also a bare apostrophe with no closing quote -- an initial looser bound silently regressed a previously-clean rust shape by treating two unrelated lifetimes as one fake string, caught by re-running the full multi-language reconciliation before trusting the fix. Two shapes confirmed as non-defects: tree-sitter-python has no concept of Cython's `cdef class` syntax at either the class or method level (documented as Claim 2 in docs/why_gitgalaxy_beats_ast_here.md, now extended with the class-level evidence too). Verified: new extraction-gauntlet cases in test_python.py (valid/invalid/ pathological + ReDoS sweep), crucible_check.py against the full ~80-repo corpus (31 diffs, all scoped to python/cython, both golden masters re-blessed), ruff/mypy audits clean. Ledger, chart, and points-of-interest regenerated; docs/language_status/python.md gets a new tri-comparison findings section (backlog capstone). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Contributor
4 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
tri-comparison-ledger-sweepskill on python: all 4 previously-unvalidatedledger shapes are now
status: validated, clearing python's backlog entirely.func_startfor python only matched the literaldefkeyword, so Cython'scdef/cpdefmodule-level function definitions (.pyx/.pxd/.pxiare deliberately routedunder python's extension set) were a complete recall gap -- 68 real functions, ctags-corroborated,
0 found by GitGalaxy. Added a second
func_startalternative, excludingcdef class/struct/enum/union/extern/packed/fused(declarations, not functions).tri_comparison_gatherer.py's ctags args counter mis-split a comma inside a quotedstring-literal default value. Fixed with quote-tracking -- caught and fixed a self-inflicted
regression on a real rust shape mid-fix (Rust lifetimes are a bare apostrophe with no closing
quote), verified by re-running the full multi-language reconciliation, not just python's, before
landing.
docs/why_gitgalaxy_beats_ast_here.md's existing Claim 2(tree-sitter-python has no concept of Cython's
cdef classat the class level either, not justmethod-level scope loss).
docs/language_status/python.mdas thesweep's capstone.
Verification
tests/extraction/languages/test_python.py(valid/invalid/pathological + a dedicated ReDoS sweep for the new branch) -- all pass.
python tests/tools/crucible_check.pyagainst the full ~80-repo corpus: 31 diffs, all scoped topython/cython(plus expected timestamp churn), both golden masters re-blessed(
update_golden_master.py --yes).ruff_audit.py --ci/mypy_audit.py --ci: no new findings beyond baseline.tri_comparison_chart.py --all --write,tri_comparison_report.py --write) diffed to confirm only python (and a transient rustshape that resolved back to non-reproducing) changed -- no other language's real data moved.
Test plan
pytest tests/extraction/languages/test_python.py tests/extraction/languages/test_python_strict.py tests/extraction/languages/test_embedded_python.py tests/extraction/languages/test_embedded_python_strict.pycrucible_check.py(full corpus, both venvs) -- PASS after reblessingruff_audit.py --ci/mypy_audit.py --ci-- cleanFixes #1999, Fixes #2000
🤖 Generated with Claude Code