Bound Mode A's per-function args search to the label's own statement - #1985
Merged
Conversation
_calculate_block_metrics's generic args-count derivation searched the WHOLE greedy block when no args_search_text was passed -- for Mode A (_slice_by_labels), that block can span many unrelated statements past the matched label's own signature, since Mode A's body legitimately runs to the next func_start match, not to a brace-bounded end. For cobol/fortran/dockerfile (the only 3 Mode A languages with no dedicated args_count_override), this let an unrelated, unbounded later occurrence get misattributed as the current label's own parameter count: a dockerfile RUN could pick up an unrelated ARG line swept into its greedy span, and cobol paragraphs could pick up an unrelated CALL...USING statement or the file's own PROCEDURE DIVISION USING/RETURNING header -- confirmed on real corpus data (a cobol paragraph literally named TIMESTAMP was showing 13 "arguments"). Adds _mode_a_args_window_end, which bounds the search to the label's own statement span by following real line-continuation syntax (backslash for dockerfile, ampersand for fortran, including Dockerfile heredocs and Fortran's own inline-comment/blank-line/cpp-directive interruptions -- WRF's real subroutines can have 200+ parameters across 90+ continuation lines) rather than a blind character count. Explicitly scoped to primary_lang_id in (cobol, fortran, dockerfile) -- NOT "args_count_override is None", which also fires for abap/agc_assembly/assembly whenever their own override legitimately resolves to None (e.g. an ABAP interface-implemented method with no DEFINITION-section entry). Verified this distinction matters: an earlier gate on the override alone silently applied an untuned bound to real ABAP methods and regressed their correct args counts -- caught by this fix's own crucible_check verification before it shipped. One known, narrow, separately-tracked residual: fortran's init_domain (module_initialize_real.F) drops from an accidental 1 to 0, because its own func_start match.start() lands on an unrelated earlier statement due to a separate, pre-existing func_start regex bug (filed as #1982) -- the old unbounded search happened to still reach the real signature further into an inflated span; a correctly bounded search anchored at the same wrong start can't. Will self-resolve once #1982 lands. Fixes #1973. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Contributor
🐦⬛ Muninn Security Scan
ℹ️ Info Findings[checkov] Ensure that a user for the container has been createdFile: [checkov] Ensure that HEALTHCHECK instructions have been added to container imagesFile: [checkov] Ensure the base image uses a non latest version tagFile: |
squid-protocol
added a commit
that referenced
this pull request
Aug 21, 2026
squid-protocol
added a commit
that referenced
this pull request
Aug 21, 2026
…1987) * Fix jcl and m4 func_start recall: route to Mode A, not brace search Same bug shape as dockerfile (#1976): jcl and m4 have no ScopeParsingRegistry entry and no brace-delimited bodies at all, so they silently fell through to Mode_B_Braces in _function_slice -- which only produces a named function when a literal { happens to appear by coincidence within its search window. jcl's func_start matches "// <name> EXEC ..." job-step lines -- JCL is fixed-column mainframe syntax with no brace concept anywhere. m4's func_start matches define(...)/m4_define(...)/AC_DEFUN(...) etc. -- macro definitions are parenthesis-delimited with backtick/bracket quoting, not brace-delimited. Confirmed via direct measurement (struct_func_start, the raw signal, vs. function_count, the named list actually reaching consumers): jcl: 3 raw matches, 0 reached the named list (0% recall) m4: 39 raw matches, 1 reached the named list (2.6% recall) Routes both to Mode A (_slice_by_labels), the same greedy-to-next- match heuristic already proven for abap/cobol/fortran/assembly/ dockerfile. Post-fix: both hit 100% recall in the local corpus (3/3, 39/39). makefile was also flagged as a candidate in the same investigation but its corpus sample was inconclusive (1 match total) -- deliberately left untouched, not part of this fix's scope. Verified against the full ~80-repo crucible corpus (both golden masters re-blessed; drift is real function-count/structural-mass changes in jcl/m4 files plus expected global spatial-layout ripple across the corpus, not corruption elsewhere). Fixes #1975. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Co-Authored-By: Gemini 3.1 Pro <noreply@google.com> * Regenerate golden masters against merged main (#1985 + jcl/m4 fix) --------- Co-authored-by: Joe Esquibel <squid-protocol@users.noreply.github.com> Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com> Co-authored-by: Gemini 3.1 Pro <noreply@google.com>
squid-protocol
added a commit
that referenced
this pull request
Aug 21, 2026
squid-protocol
added a commit
that referenced
this pull request
Aug 21, 2026
…-shift Second conflict round: #1987 (jcl/m4) landed on main right after the first resolution, requiring another regenerate-and-rebless pass. Also fixes a real, separate issue caught while investigating why PR #1985/#1987 merged despite a failing ruff-audit CI check: #1972 (record_keeper.py's class_count fix, merged earlier and unrelated to this PR) shifted 7 pre-existing baselined findings by a couple of lines each -- tests/ruff_audit_baseline.json was never regenerated to absorb that shift, so every PR's ruff-audit CI run since has been reporting those 7 as spurious 'new' findings. Confirmed via audit_check.py that all 7 are pure line-shifts (same file/code/ message, just moved), not real regressions, and regenerated the baseline to absorb them. Verified against the exact CI-pinned ruff version (0.16.0, not whatever floats on PATH locally).
squid-protocol
added a commit
that referenced
this pull request
Aug 21, 2026
* Add named class (build-stage) extraction for dockerfile Dockerfile had no named class extraction at all: it wasn't in _CLASS_START_NAMED_EXTRACTION_LANGS, so it fell back to the legacy generic regex (class|struct|interface|trait|enum), which never matches Dockerfile's FROM syntax at all -- correctly, since Dockerfile has none of those keywords. Result: class_data stayed completely empty for every Dockerfile scanned, even though struct_class_start (the raw signal) correctly counted every real FROM line. class_start's own regex previously captured only the literal keyword FROM itself, not the build stage's real name -- extended it to an alternation shape (matching the existing Fortran/Lua/ABAP convention _resolve_class_start_match already documents): group 1 captures the AS <alias> name when present, group 2 falls back to the bare base-image reference for a stage with no alias (the file's final/default stage). --platform=$VAR-style flags between FROM and the image reference are skipped over. Added dockerfile to _CLASS_START_NAMED_EXTRACTION_LANGS, and to the boundary-resolution skip condition alongside abap (same #1907 rationale -- Dockerfile stages are never nested, and the brace search would mistake ${VAR} template-substitution braces for a real body opener). Verified: class_data row counts now match struct_class_start exactly (69/6/1/1 across the moby test corpus), with real stage names (base, criu, xx, binary-dummy, ...) instead of the literal string "FROM" repeated. ReDoS-probed the new regex directly (5000-char runs, deeply nested continuations, huge flag/image tokens) -- all sub-millisecond. One known, accepted side effect: THE LINEAGE EXTRACTOR (detector.py's generic "any class_start match with 2+ groups treats group 2 as an inheritance parent") doesn't know groups 1/2 here are alternation- exclusive, not name-then-parent -- a bare FROM <image> (no alias) sweeps the image reference into that file's parent_entity metadata. This is a pre-existing, already-shipped pattern (fortran's own class_start has the identical alternation shape and triggers the same behavior for bare TYPE declarations today), not something this fix introduces -- tracked as its own issue (#1983) for a future, more alternation-aware fix to the shared extractor, not blocking here. Fixes #1974. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Co-Authored-By: Gemini 3.1 Pro <noreply@google.com> * Regenerate golden masters against merged main (#1985 + dockerfile class fix) * Regenerate golden masters (2nd merge-race) and fix ruff baseline line-shift Second conflict round: #1987 (jcl/m4) landed on main right after the first resolution, requiring another regenerate-and-rebless pass. Also fixes a real, separate issue caught while investigating why PR #1985/#1987 merged despite a failing ruff-audit CI check: #1972 (record_keeper.py's class_count fix, merged earlier and unrelated to this PR) shifted 7 pre-existing baselined findings by a couple of lines each -- tests/ruff_audit_baseline.json was never regenerated to absorb that shift, so every PR's ruff-audit CI run since has been reporting those 7 as spurious 'new' findings. Confirmed via audit_check.py that all 7 are pure line-shifts (same file/code/ message, just moved), not real regressions, and regenerated the baseline to absorb them. Verified against the exact CI-pinned ruff version (0.16.0, not whatever floats on PATH locally). --------- Co-authored-by: Joe Esquibel <squid-protocol@users.noreply.github.com> Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com> Co-authored-by: Gemini 3.1 Pro <noreply@google.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
_calculate_block_metrics's generic per-functionargs_countderivation searched the entire greedyblockwhen noargs_search_textwas passed. For Mode A (_slice_by_labels), that block legitimately spans from onefunc_startmatch to the next -- often many unrelated statements -- since Mode A has no brace-bounded body the way Mode B does.For the 3 Mode A languages with no dedicated
args_count_override(cobol, fortran, dockerfile), this let an unrelated, later occurrence get misattributed as the current label's own parameter count:RUNcould pick up an unrelatedARGline swept into its greedy span.CALL ... USINGstatement, or the file's ownPROCEDURE DIVISION USING/RETURNINGheader -- confirmed on real corpus data: a paragraph literally namedTIMESTAMPwas showing 13 "arguments."Fix
Adds
_mode_a_args_window_end, which bounds the args search to the matched label's own statement span by following real line-continuation syntax (\for dockerfile,&for fortran -- including Dockerfile heredocs and Fortran's own inline-comment/blank-line/cpp-directive interruptions, since WRF's real subroutines can have 200+ parameters spread across 90+ continuation lines) rather than a blind character-count window.Scoped explicitly to
primary_lang_id in (cobol, fortran, dockerfile)-- deliberately NOTargs_count_override is None, which also fires for abap/agc_assembly/assembly whenever their own override legitimately resolves toNone(e.g. an ABAP interface-implemented method with no DEFINITION-section entry). This distinction turned out to matter in practice: an earlier version of this fix gated on the override alone and silently applied an untuned bound to real ABAP methods, regressing their correct args counts (zcl_abapgit_http_client.clas.abap'scheck_http_200/send_receive: real 1, regressed to 0) -- caught by this PR's owncrucible_check.pyrun before it shipped, then fixed to gate on language explicitly instead.One known, narrow, separately-tracked residual
fortran's
init_domain(module_initialize_real.F) drops from an accidental1to0. Root cause: its ownfunc_startmatch'smatch.start()lands on an unrelated earlier statement (INTEGER :: internal_time_loop, several blank lines before the realSUBROUTINE init_domainline) due to a separate, pre-existing bug in fortran'sfunc_startregex -- filed as #1982. The old unbounded search happened to still reach the real signature further into an inflated span; a correctly bounded search anchored at the same (wrong) match start can't. Will self-resolve once #1982 lands.Verification
pytestacross dockerfile/cobol/fortran/abap extraction + strict tests, plustest_detector.py: 672 passed.ruff_audit.py --ci/mypy_audit.py --ci: clean against baseline.galaxyscope --db-onlyagainstlanguage-crucible/data/{dockerfile,cobol,fortran,abap}):RUNs showing spurious 2).TIMESTAMP=13,ACCOUNT-OVERDRAFT-COUNT=1,LOCAL-STORAGE=1; all clearly bogus for paragraphs with no formal parameter concept).lsm/lsm_mosaic, 247/291 real dummy arguments correctly preserved); 1 known residual traced to fortran func_start's optional TYPE-prefix can bridge across an unrelated statement to a distant SUBROUTINE/FUNCTION #1982.check_http_200=1,send_receive=1,set_header=4, matching the pre-existing golden master).crucible_check.py(full_precision+zero_dependency): real, expected drift only in dockerfile/cobol/fortran files plus global-aggregate ripple; both golden masters re-blessed; final re-run: PASS/PASS.This is #1973 in the same dockerfile tri-comparison manual-verification session as #1976 (merged) -- found while re-verifying the dockerfile func_start fix's own downstream effects, generalized once cobol/fortran turned out to share the exact same unbounded-search code path.
Fixes #1973.
🤖 Generated with Claude Code