Skip to content

feat(rdd): RDPlot - optimal data-driven RD plots (CCT 2015, rdplot parity)#695

Merged
igerber merged 2 commits into
mainfrom
rdd-rdplot
Jul 19, 2026
Merged

feat(rdd): RDPlot - optimal data-driven RD plots (CCT 2015, rdplot parity)#695
igerber merged 2 commits into
mainfrom
rdd-rdplot

Conversation

@igerber

@igerber igerber commented Jul 18, 2026

Copy link
Copy Markdown
Owner

Summary

  • PR-B of the rdplot arc (paper review merged in docs: add rdplot paper review (CCT 2015 JASA) #694): new RDPlot class + RDPlotResult (diff_diff/rdplot.py), the CCT 2015 exploratory RD-plot diagnostic with end-to-end parity against rdplot() in CRAN rdrobust 4.0.0 (same sha-pinned tarball as the estimator port).
  • All 8 binselect bin-count selectors ({evenly, quantile spaced} x {IMSE-optimal, mimicking variance} x {spacings, polynomial-regression variance estimators}), manual nbins/scale/h/support/ci, masspoints detection with the spacings-to-pr adjust remap, per-bin means/SEs/CIs (vars_bins with R's column names), 500-point-per-side global-fit curves, and implied-scale + Supplement-S.1 WIMSE weights in summary() (matching R's summary.rdplot).
  • Covariate-adjusted plots (fit(..., covariates=...), R's covs= with covs_eval="mean") reuse the feat(rdd): covariate-adjusted RD - covariates= on fit() with rdrobust 4.0.0 parity (CCFT 2019) #691 partialled-gamma machinery including the collinearity pipeline, degenerate-adjustment guards, and the shared covariate-name validation contract.
  • Rendering is an optional lazy-matplotlib RDPlotResult.plot(ax=...) (matplotlib is NOT a dependency; the numbers are the parity surface).
  • Golden parity on 24 configs (benchmarks/R/generate_rdplot_golden.R -> benchmarks/data/rdplot_golden.json) incl. the vendored Senate data; the paper supplement's own Figure SA-1/SA-2 selector outputs (esmv 15/35, es 8/9, qs 21/16, qsmv 28/49) are asserted as JSON-independent anchors.
  • Docs sweep: REGISTRY RDPlot section (lifted from the merged docs: add rdplot paper review (CCT 2015 JASA) #694 review per its CI note), api rst, references.rst, llms.txt + llms-full.txt, README Diagnostics one-liner, doc-deps.yaml entry, CHANGELOG [Unreleased].

Methodology references (required if estimator / math changes)

  • Method name(s): RDPlot (RD plot bin-count selection diagnostic; NOT a treatment-effect estimator - no inference surface beyond per-bin display CIs)
  • Paper / source link(s): Calonico, S., Cattaneo, M. D., & Titiunik, R. (2015). Optimal Data-Driven Regression Discontinuity Plots. JASA, 110(512), 1753-1769. https://doi.org/10.1080/01621459.2015.1017578 (+ Supplemental Appendix); R source of record: CRAN rdrobust 4.0.0 rdplot.R.
  • Any intentional deviations from the source (and why): all REGISTRY-labeled - fractional/pair scale products take CCT 2015 Eq 2's ceiling (R 4.0.0 crashes there by accident: vector-indexing row mismatch / vectorized-if error on R >= 4.2); missing-row and discrete-outcome (spacings-selector) warnings where R is silent; clear errors for zero-effective-h, single-support-point, and near-empty sides where R degenerates opaquely; R's unconditional covs message() moved to the docstring. R quirks REPLICATED and documented: left-side bin-edge slot reflection under empty bins, qs+nbins "manually evenly spaced" label, Inf J_IMSE on zero-variance sides, negative-variance floor, QS type-7 cutpoints (R itself deviates from the paper's inf-form inverse there).

Validation

  • Tests added/updated: tests/test_rdplot.py (59 tests: 24-config golden parity at rtol=1e-9, paper anchors, R-quirk locks, deviation locks, API contracts incl. transactional set_params and the matplotlib ImportError path; golden suite skips if the JSON is absent)
  • Backtest / simulation / notebook evidence (if applicable): R smoke on the vendored Senate CSV reproduces the paper's supplement figure numbers exactly; rendered plot verified visually
  • black/ruff clean; mypy zero errors on the pinned CI toolchain

Security / privacy

  • Confirm no secrets/PII in this PR: Yes

🤖 Generated with Claude Code

https://claude.ai/code/session_01VwhFMnFQGBumYfwbeQaUjm

@github-actions

Copy link
Copy Markdown

Overall Assessment

Looks good — no unmitigated P0/P1 findings identified.

Executive Summary

  • RDPlot affects the new RD plot diagnostic, not treatment-effect estimation.
  • Selector math, WIMSE scale reporting, mass-point handling, covariate adjustment, and R-parity quirks are documented in docs/methodology/REGISTRY.md:L3959-L4103.
  • The implementation matches the cited CCT 2015 formulas reviewed in docs/methodology/papers/calonico-cattaneo-titiunik-2015-review.md:L81-L167.
  • Documented deviations from R, including fractional/pair scale, QS type-7 cutpoints, and left-bin edge reflection, are properly registry-labeled and tested.
  • I could not run the test suite here: the environment lacks pytest and pandas.

Methodology

No unmitigated findings.

P3 informational: Documented RDPlot deviations/quirks reviewed.
Impact: The PR intentionally deviates from R on surfaces where R crashes or is silent, and replicates some R quirks for parity.
Concrete fix: None required. The deviations are labeled in docs/methodology/REGISTRY.md:L4015-L4066 and locked by tests in tests/test_rdplot.py:L360-L389.

Code Quality

No findings.

The core numerical paths are localized in diff_diff/rdplot.py:L884-L1006, with validation and edge guards in diff_diff/rdplot.py:L586-L662 and diff_diff/rdplot.py:L673-L735.

Performance

No blocking findings.

P3 informational: Very large manual nbins can allocate large jump grids and dense bin-dummy matrices with covariates.
Impact: Potential high memory/time use for extreme user input.
Concrete fix: None required for this PR; this is documented as a scope seam in docs/methodology/REGISTRY.md:L4086-L4093.

Maintainability

No findings.

The new public API is exported in diff_diff/__init__.py:L215-L218 and diff_diff/__init__.py:L526-L528, and documentation drift tracking was added in docs/doc-deps.yaml:L426-L445.

Tech Debt

No untracked blocking debt found.

Documented scope seams, including no sampling weights, no subset=, and limited covs_eval, are recorded in docs/methodology/REGISTRY.md:L4086-L4096.

Security

No findings.

I found no secrets or unsafe execution paths in the added RDPlot implementation, tests, benchmark generator, or golden data.

Documentation/Tests

No blocking findings.

Coverage is broad: golden R parity in tests/test_rdplot.py:L105-L189, paper anchors in tests/test_rdplot.py:L213-L239, R-quirk and edge-case locks in tests/test_rdplot.py:L242-L357, documented deviation tests in tests/test_rdplot.py:L360-L389, and API/plot contracts in tests/test_rdplot.py:L505-L606.

Verification not run: python -m pytest tests/test_rdplot.py -q -p no:cacheprovider failed because pytest is not installed; a plain import smoke check also failed because pandas is not installed.

@igerber igerber added the ready-for-ci Triggers CI test workflows label Jul 18, 2026
@github-actions

Copy link
Copy Markdown

🔁 AI review rerun (requested by @igerber)

Head SHA: f3bd3f2f7fef3512c3558a053bb7322cf5820564


Overall Assessment

Looks good — no unmitigated P0/P1 findings identified.

Executive Summary

  • RDPlot is a new RD plot diagnostic, not a treatment-effect estimator.
  • Selector formulas, WIMSE weights, mass-point handling, covariate adjustment, and R-parity quirks are documented in docs/methodology/REGISTRY.md:L3959-L4103.
  • The implementation matches the cited CCT 2015 selector formulas in the paper review and registry.
  • Edge-case coverage includes missing rows, empty bins, zero-variance sides, mass points, discrete outcomes, invalid bandwidths, and single-support sides.
  • I could not run tests in this sandbox because pytest, numpy, pandas, and scipy are not installed.

Methodology

No unmitigated findings.

Severity: P3 informational
Impact: Documented deviations from R include fractional/pair scale, QS type-7 cutpoints, left-edge slot reflection, missing-row warnings, defensive support guards, and covariate message behavior. These are registry-labeled notes/deviations, so they are not defects.
Concrete fix: None required. See docs/methodology/REGISTRY.md:L4015-L4066 and implementation locks in diff_diff/rdplot.py:L939-L997.

Code Quality

No findings.

Constructor validation and parameter propagation are covered in diff_diff/rdplot.py:L369-L481, with API contract tests in tests/test_rdplot.py:L511-L540.

Performance

Severity: P3 informational
Impact: Very large manual nbins can allocate large jump grids and dense covariate/bin-dummy matrices. This is an extreme-input performance seam, not a correctness issue.
Concrete fix: None required for this PR; documented in docs/methodology/REGISTRY.md:L4086-L4093 and noted in diff_diff/rdplot.py:L326-L331.

Maintainability

No findings.

The implementation is localized in diff_diff/rdplot.py, exported via diff_diff/__init__.py:L215-L218 and diff_diff/__init__.py:L526-L528, and tied into docs drift tracking in docs/doc-deps.yaml:L426-L445.

Tech Debt

No untracked blocking debt found.

Documented scope seams include no sampling weights, no subset=, limited covs_eval, and no ggplot cosmetic parity. These are tracked in docs/methodology/REGISTRY.md:L4086-L4096.

Security

No findings.

I found no secrets, unsafe execution paths, or new security-sensitive behavior in the changed implementation, docs, tests, benchmark generator, or golden data.

Documentation/Tests

No blocking findings.

Coverage is broad: R golden parity in tests/test_rdplot.py:L105-L195, paper anchors in tests/test_rdplot.py:L219-L245, edge/quirk tests in tests/test_rdplot.py:L248-L363, documented deviation tests in tests/test_rdplot.py:L366-L395, and API/rendering contracts in tests/test_rdplot.py:L511-L612.

Verification not run: python -m pytest tests/test_rdplot.py -q -p no:cacheprovider failed because pytest is not installed; import checks also showed numpy, pandas, and scipy are unavailable in this sandbox.

…rity)

PR-B of the rdplot arc (paper review merged in #694). New RDPlot class +
RDPlotResult in diff_diff/rdplot.py, parity-targeting rdplot() in CRAN
rdrobust 4.0.0: all 8 binselect bin-count selectors (ES/QS x
IMSE-optimal/mimicking-variance x spacings/polynomial-regression),
manual nbins/scale/h/support/ci, masspoints detection with the
spacings-to-pr adjust remap, per-bin means/SEs/CIs (vars_bins, R column
names), 500-point-per-side global-fit curves (vars_poly), implied scale
+ Supplement S.1 WIMSE weights in summary(), covariate-adjusted plots
reusing the #691 partialled-gamma machinery, and an optional lazy
matplotlib RDPlotResult.plot() (matplotlib not a dependency).

Golden parity on 24 configs (benchmarks/R/generate_rdplot_golden.R ->
rdplot_golden.json) incl. the vendored Senate data; supplement Figures
SA-1/SA-2 selector outputs asserted as JSON-independent paper anchors.
R quirks replicated and REGISTRY-documented (left-edge slot reflection
under empty bins, qs+nbins label, Inf J_IMSE on zero-variance sides,
negative-sigma2 floor, k=4->3->2 raises-only fallback ladder, QS type-7
cutpoints). Documented deviations/defensive guards: fractional/pair
scale takes CCT 2015 Eq 2's ceiling (R crashes there), missing-row and
discrete-outcome warnings, zero-effective-h / single-support-point /
tiny-side clear errors, covariate-name validation shared with the
estimator. Docs sweep: REGISTRY RDPlot section, api rst, references,
llms.txt/llms-full.txt, README one-liner, doc-deps entry, CHANGELOG.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VwhFMnFQGBumYfwbeQaUjm
@github-actions

Copy link
Copy Markdown

🔁 AI review rerun (requested by @igerber)

Head SHA: 4af6867ede1d70d82b71a78ed6ffd14dfeaaea95


Overall Assessment

⚠️ Needs changes — one unmitigated P1 edge-case/methodology mismatch found.

Executive Summary

  • RDPlot is correctly documented as a diagnostic, not a treatment-effect estimator.
  • Selector formulas, R-parity quirks, covariate adjustment, and documented deviations are covered in docs/methodology/REGISTRY.md.
  • [Newly identified] One zero-variance-side rescue can be missed for constant floating outcomes because RDPlot uses np.var(...) == 0.
  • The focused test suite could not be run in this sandbox because pytest and numpy are not installed.
  • No security issues or secret-like strings found in the changed RDPlot files.

Methodology

Severity: P1 [Newly identified]
Impact: The registry says zero outcome variance on a side should be rescued to J = 1 with a warning. RDPlot computes var_y_l/var_y_r via np.var(..., ddof=1) and only rescues on exact var_y == 0. The existing rdrobust port documents that NumPy can return tiny nonzero variance for constant floats like 0.7, so a constant side can skip the rescue, produce J_MV = Inf, then divide by zero during bin-grid construction. This violates the documented zero-variance contract and can error in production.
Location: diff_diff/rdplot.py:L857-L858, diff_diff/rdplot.py:L963-L984; supporting context in diff_diff/_rdrobust_port.py:L318-L328.
Concrete fix: Use exact constancy for the rescue, e.g. const_y_l = np.all(y_l == y_l[0]), set var_y_l = 0.0 when true, and trigger the warning/J=1 rescue from that flag. Add a regression test with one side constant at 0.7.

Severity: P3 informational
Impact: Fractional/pair scale, QS type-7 cutpoints, left-edge slot reflection, missing-row warnings, and covariate message behavior are documented as notes/deviations in the registry, so they are not defects.
Concrete fix: None required.

Code Quality

No additional findings. Constructor validation and parameter propagation appear complete through get_params()/set_params() and the result surface.

Performance

Severity: P3 informational
Impact: Very large manual nbins/scale can allocate large grids and dense covariate/bin-dummy matrices. This is documented in the registry and docstring.
Concrete fix: None required for this PR.

Maintainability

No findings. The implementation is localized in diff_diff/rdplot.py, exported through diff_diff/__init__.py, and tied into docs/doc-deps.yaml.

Tech Debt

No untracked blocking debt found. The documented RDPlot scope seams are registry-tracked.

Security

No findings. I did not find secret-like tokens in the changed RDPlot implementation, tests, benchmark generator, docs, or golden fixture.

Documentation/Tests

Severity: P3 informational
Impact: Tests are broad, but I could not execute them here: python -m pytest tests/test_rdplot.py -q -p no:cacheprovider failed because pytest is missing, and import checks fail because numpy is missing.
Concrete fix: CI should run the committed RDPlot suite; add the 0.7 constant-side regression above.

Path to Approval

  1. Replace the zero-variance rescue condition with an exact-constancy check for each side’s outcome.
  2. Add a test where one side has constant floating outcome 0.7, asserting RDPlot().fit() warns, returns J[side] == 1.0, and builds vars_bins without error.

@igerber igerber removed the ready-for-ci Triggers CI test workflows label Jul 18, 2026
…sides

np.var of a constant 0.7 vector is ~1.3e-32 (single-pass mean roundoff)
while R's two-pass var() is exactly 0, so the var==0 rescue was skipped
for constant non-representable-float outcomes and the resulting Inf J_MV
crashed bin construction on a zero jump. Route the side variances through
the port's _var0 exact-constancy helper (the R semantics already ported
for the fuzzy identification stop), reproducing R's J=1 rescue, its Inf
J_IMSE / NaN J_MV echoes, and the variability warning. Regression test
with a constant-0.7 side added.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VwhFMnFQGBumYfwbeQaUjm
@github-actions

Copy link
Copy Markdown

🔁 AI review rerun (requested by @igerber)

Head SHA: a2f9cd5c8d7022d16e9d2de65119333564fab84a


Overall Assessment

Looks good — the prior P1 finding is resolved, and I found no unmitigated P0/P1 issues.

Executive Summary

  • The previous constant-float zero-variance-side issue is fixed via _var0() and covered by a regression test.
  • RDPlot remains documented and implemented as a diagnostic, not a treatment-effect estimator.
  • The main R-parity deviations I checked are labeled in REGISTRY.md, so they are informational, not blockers.
  • No inline statistical-inference anti-patterns were introduced in RDPlot.
  • I could not run the focused test suite locally because pytest is not installed.

Methodology

Severity: P3 informational
Impact: Prior P1 resolved. Constant floating outcomes like 0.7 now hit the exact-constancy variance path before selector resolution, matching the registry’s zero-variance rescue contract.
Location: diff_diff/rdplot.py:L858-L866, diff_diff/rdplot.py:L971-L984, tests/test_rdplot.py:L353-L372, docs/methodology/REGISTRY.md:L4004-L4006
Concrete fix: None required.

Severity: P3 informational
Impact: Fractional/pair scale, QS type-7 cutpoints, left-edge slot reflection, missing-row warnings, noncontinuous-outcome warnings, and covariate message behavior are documented with Note/Deviation labels, so they are not methodology defects.
Location: docs/methodology/REGISTRY.md:L4015-L4066
Concrete fix: None required.

Code Quality

No findings. Constructor validation, get_params()/set_params(), and result exposure appear consistently threaded for the new RDPlot parameters.

Performance

Severity: P3 informational
Impact: Very large manual nbins/scale values can allocate large grids and dense covariate/bin-dummy matrices, but this is documented as a scope seam.
Location: docs/methodology/REGISTRY.md:L4086-L4093
Concrete fix: None required.

Maintainability

No findings. The implementation is localized in diff_diff/rdplot.py and exported through diff_diff/__init__.py.

Tech Debt

No untracked blocking tech debt found. The documented RDPlot scope seams are registry-tracked.

Security

No findings. I scanned the changed files for common secret/token patterns and found no relevant matches.

Documentation/Tests

Severity: P3 informational
Impact: The committed tests cover R golden parity, paper anchors, R quirks, documented deviations, validation, API contracts, and the prior constant-float rescue regression. I could not execute them locally because /usr/bin/python: No module named pytest.
Location: tests/test_rdplot.py:L105-L195, tests/test_rdplot.py:L219-L245, tests/test_rdplot.py:L310-L372, tests/test_rdplot.py:L387-L416, docs/methodology/REGISTRY.md:L4098-L4103
Concrete fix: Ensure CI runs python -m pytest tests/test_rdplot.py -q -p no:cacheprovider.

@igerber igerber added the ready-for-ci Triggers CI test workflows label Jul 18, 2026
@igerber
igerber merged commit 2f8e2ea into main Jul 19, 2026
38 of 39 checks passed
@igerber
igerber deleted the rdd-rdplot branch July 19, 2026 00:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-for-ci Triggers CI test workflows

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant