Close the last python Cython func_start recall gap (get_slice_from_memview) - #2007
Merged
Merged
Conversation
…mview) Follow-up to the tri-comparison-ledger-sweep fix in #1999: one function was still missing after that fix -- get_slice_from_memview's return type is a Cython Tempita codegen placeholder (cdef {{memviewslice_name}} *get_slice_ from_memview(...)), which the plain-identifier return-type token couldn't match. Closing it took two fixes, not one: - Widened func_start's return-type token to also accept a bounded {{identifier}} placeholder form. - That alone wasn't enough: it surfaced a second, independent bug in the shared _extract_name name-normalizer (detector.py, used by every language). Its generic split("{") truncation mistook the placeholder's own {{/}} for a C-style body-opening brace, truncating the match down to the bogus name "cdef" instead of the real name -- invisible from the isolated regex alone, only visible once checked against the real scan pipeline's actual DB output. GitGalaxy now finds all 84 real functions in cython/MemoryView.pyx (100% recall vs. ctags, up from 83/84) -- the python/function/existence/agree[ctags]_vs[gitgalaxy,tree_sitter] ledger shape no longer reproduces at all. Verified: new test cases in test_python.py (func_start regex + ReDoS sweep) and test_detector.py (_extract_name, since that fix is shared infrastructure); crucible_check.py against the full ~80-repo corpus (diffs scoped only to python/cython, both golden masters re-blessed again); ruff/mypy clean; ledger/ chart/report regenerated and diffed to confirm only python's real data moved. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Contributor
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Follow-up to #2001 (merged). That PR closed 68/69 missing Cython
cdef/cpdeffunctionoccurrences but left one documented residual gap:
get_slice_from_memviewincython/MemoryView.pyx, whose return type is a Cython Tempita codegen placeholder(
cdef {{memviewslice_name}} *get_slice_from_memview(...)).Closing it took two fixes, not one:
func_start's return-type token to also accept a bounded{{identifier}}placeholderform.
_extract_namename-normalizer (detector.py, used by every language, not just python): itsgeneric
split("{")truncation (meant to cut a C-style body-opening brace off the raw match)mistook the placeholder's own
{{/}}for that brace, truncating the match down to the bogusname
"cdef"instead of the real name. This was invisible from the isolated regex alone -- itonly surfaced once checked against the real scan pipeline's actual DB output (the DB was
recording a phantom function literally named
cdefat that line).GitGalaxy now finds all 84 real functions in
cython/MemoryView.pyx-- 100% recall vs. ctags,no residual gap. The
python/function/existence/agree[ctags]_vs[gitgalaxy,tree_sitter]ledgershape no longer reproduces at all.
Verification
tests/extraction/languages/test_python.py(func_start regex, valid case +dedicated ReDoS sweep) and
tests/core_engine/test_detector.py(_extract_name, since that fixis shared infrastructure across every language).
python tests/tools/crucible_check.pyagainst the full ~80-repo corpus: diffs scoped only topython/cython/MemoryView.pyx's Function Analysis, both golden masters re-blessed.ruff_audit.py --ci/mypy_audit.py --ci: no new findings beyond baseline.moved -- no other language affected by the shared
_extract_namechange.docs/language_status/python.md§10 and filed a follow-up comment on python func_start regex misses Cython cdef/cpdef module-level function definitions #1999.Test plan
pytest tests/extraction/languages/test_python.py tests/core_engine/test_detector.pycrucible_check.py(full corpus, both venvs) -- PASS after reblessingruff_audit.py --ci/mypy_audit.py --ci-- clean🤖 Generated with Claude Code