ci+local: upgrade AI PR reviewer model gpt-5.5 -> gpt-5.6-sol (effort xhigh)

[igerber]

Summary

• Swaps the AI PR reviewer to gpt-5.6-sol at xhigh effort on BOTH surfaces simultaneously: the CI codex-action (ai_pr_review.yml model:) and the local review script (openai_review.py DEFAULT_MODEL), which share the same _build_codex_cmd invocation - keeping local review predictive of CI review.
• Validated empirically before the swap via the tools/reviewer-eval/ 4-arm blinded campaign (feat(reviewer-eval): N-arm matrix, blinded grading, corpus 2->11 for the GPT-5.6 evaluation #700 infrastructure): 66 runs over 11 cases, three independent identity-blind graders, pre-registered gates in DECISION_RULE.md. Results: gpt-5.6-sol@xhigh reliably caught 7/10 ground-truth bugs vs gpt-5.5@xhigh's 2/10 (+2 unstable), with zero catch regressions and three strict improvements (including an S4 missed-bug probe gpt-5.5 historically passed over). gpt-5.6-sol@max showed no decisive gain over xhigh at ~1.5-2x latency, so effort stays xhigh; gpt-5.6-terra tracked ~gpt-5.5.
• Supporting pins: _is_reasoning_model prefix tuple += gpt-5.6, PRICING += gpt-5.6-sol (5/30), -terra (2.50/15), -luna (1/6) per 1M tokens, ai-review-local.md doc refs, and configs.json control-arm flip (gpt-5.6-sol@xhigh becomes role=control; gpt-5.5 retained as the previous-production arm for future back-comparisons).
• Harness follow-through: bare smoke now derives its default arm from the sole role=control entry at run time (a control flip can never drift the default; regression-tested), and the s3-wcr-pfloor-comment corpus fixture's comment overclaim - which the winning arms correctly flagged during the campaign - is corrected to purpose-phrasing, restoring it as a genuinely clean negative control.
• Built-in canary: this PR's own auto-review runs from the merge ref, i.e. under gpt-5.6-sol - the review below this description is the new model operating in the production action environment.

Methodology references (required if estimator / math changes)

• Method name(s): N/A - no estimator/methodology changes. The only diff_diff/-adjacent content is inside a frozen reviewer-eval corpus fixture (comment-only wording fix in s3-wcr-pfloor-comment/inject.diff); the live tree's diff_diff/utils.py is untouched.
• Paper / source link(s): N/A
• Any intentional deviations from the source (and why): None

Validation

• Tests added/updated: tests/test_evals_runtime.py +1 (test_smoke_default_selects_control_arm). Full touched suites green: 372 tests across test_evals_{runtime,engine,adapters}.py + test_openai_review.py; verify-corpus 11/11 (regenerated fixture applies cleanly at its pinned base).
• Backtest / simulation / notebook evidence (if applicable): campaign artifacts local-only under tools/reviewer-eval/runs/ (gitignored); verdict summarized in the CHANGELOG entry.
• Model IDs, max effort acceptance, and codex-action model:/effort: passthrough verified live against codex-cli 0.144.5 on 2026-07-18.

Security / privacy

• Confirm no secrets/PII in this PR: Yes

---

Generated with Claude Code

https://claude.ai/code/session_019xnguzh8asX9dY54djvR41

[github-actions]

Overall Assessment

✅ Looks good — no unmitigated P0 or P1 findings.

Executive Summary

• No live estimator, identification, weighting, variance, or inference code changed.
• CI and local Codex defaults consistently use gpt-5.6-sol with xhigh.
• Bare smoke tests now correctly derive the sole control arm dynamically.
• Only minor test and auditability gaps remain.

Methodology

• Severity: P3 — Informational
  Impact: None. The only methodology-adjacent change is a corpus fixture comment, not production code. Its conditional p-value-floor wording matches the documented deviation in docs/methodology/REGISTRY.md:L143-L152.
  Concrete fix: None required. See tools/reviewer-eval/corpus/cases/s3_negative/s3-wcr-pfloor-comment/inject.diff:L17-L27.

Code Quality

No findings. Control selection remains fail-closed and explicit in tools/reviewer-eval/run_eval.py:L154-L169.

Performance

No findings.

Maintainability

No findings. Workflow, local defaults, documentation, and evaluator control metadata are synchronized.

Tech Debt

No findings. No correctness limitation requiring TODO.md tracking was introduced.

Security

No findings. The changed configuration does not expose credentials or broaden sandbox permissions.

Documentation/Tests

• Severity: P3 — Missing direct model-family regression tests
  Impact: Tests do not directly pin gpt-5.6-sol reasoning classification, its 900-second timeout, or the new pricing entries, so future refactoring could regress these supporting behaviors.
  Concrete fix: Add parameterized assertions for the three GPT-5.6 pricing entries, _is_reasoning_model("gpt-5.6-sol"), and _resolve_timeout(None, "gpt-5.6-sol") in tests/test_openai_review.py:L1655-L1728.

• Severity: P3 — Evaluation verdict is local-only
  Impact: The exact campaign claims in CHANGELOG.md:L11-L26 cannot be independently audited from the repository because tools/reviewer-eval/runs/ is ignored.
  Concrete fix: Optionally commit a sanitized aggregate VERDICT.md containing gate results and artifact hashes, without raw model outputs.

[github-actions]

🔁 AI review rerun (requested by @igerber)

Head SHA: 3b1059aead12df28bb5814b9bee9d76e1f233257

---

Overall Assessment

✅ Looks good — no unmitigated P0 or P1 findings.

Executive Summary

• No estimator, weighting, variance, inference, or identification code changed.
• Prior GPT-5.6 regression-test and auditability gaps are resolved.
• CI, local defaults, and dynamic smoke-control selection are consistent.
• One minor documentation reference is stale.

Methodology

• Severity: P3 — Informational
  Impact: None. The fixture-only WCR comment accurately describes the Registry-documented conditional p-value floor.
  Concrete fix: None required. See docs/methodology/REGISTRY.md:L143-L152 and tools/reviewer-eval/corpus/cases/s3_negative/s3-wcr-pfloor-comment/inject.diff:L17-L27.

Code Quality

No findings. Bare smoke correctly derives and validates the sole control arm in tools/reviewer-eval/run_eval.py:L154-L169 and L285-L291.

Performance

No findings.

Maintainability

No findings.

Tech Debt

No findings.

Security

No findings. The workflow retains its read-only sandbox and existing safety strategy.

Documentation/Tests

• Severity: P3 — Stale verdict path
  Impact: The configuration metadata references the ignored/local runs/gpt56/VERDICT.md, making the new committed audit record harder to discover.
  Concrete fix: Change the reference to tools/reviewer-eval/campaigns/2026-07-gpt56-VERDICT.md in tools/reviewer-eval/config/configs.json:L2.

The prior P3 gaps are otherwise resolved by tests/test_openai_review.py:L1655-L1674 and the committed campaign verdict. Tests could not be executed in this review environment because pytest and the corpus’s pinned commits were unavailable; Python AST, JSON parsing, and direct configuration-resolution checks passed.

[github-actions]

🔁 AI review rerun (requested by @igerber)

Head SHA: e2f1b7843a3aac3db630960d2beefcac78b84d29

---

Overall Assessment

✅ Looks good — no unmitigated P0 or P1 findings.

Executive Summary

• No estimator, weighting, variance, inference, or identification behavior changed.
• CI and local reviewer defaults consistently use gpt-5.6-sol at xhigh.
• Bare smoke correctly derives the current control arm at runtime.
• The prior stale campaign-verdict reference is resolved.
• No security or performance regressions identified.

Methodology

• Severity: P3 — Informational
  Impact: None. The fixture-only WCR comment matches the Registry-documented, alpha-conditional p-value-floor deviation.
  Concrete fix: None required.
  docs/methodology/REGISTRY.md:L143-L152
tools/reviewer-eval/corpus/cases/s3_negative/s3-wcr-pfloor-comment/inject.diff:L17-L27

Code Quality

No findings. Control resolution fails closed and delegates to existing configuration validation.
tools/reviewer-eval/run_eval.py:L154-L169
tools/reviewer-eval/run_eval.py:L271-L291

Performance

No findings.

Maintainability

No findings. CI, local defaults, reasoning classification, and configuration metadata are aligned.

Tech Debt

No findings.

Security

No findings. The workflow retains its read-only sandbox and fork-safety controls.

Documentation/Tests

No findings. The previous stale verdict-path issue is fixed at tools/reviewer-eval/config/configs.json:L2, and regression coverage exists at:

• tests/test_evals_runtime.py:L2103-L2140
• tests/test_openai_review.py:L1655-L1674

pytest was unavailable in the review environment; AST/JSON parsing and direct control-resolution, timeout, pricing, and Codex-command assertions passed.