From 598e3a37cde3289b73629c3608ccce2b7902fca5 Mon Sep 17 00:00:00 2001 From: igerber Date: Sat, 4 Apr 2026 15:13:13 -0400 Subject: [PATCH 1/3] Add label-gated CI and documentation dependency map Two process improvements: 1. Label-gated CI: rust-test.yml and notebooks.yml jobs now require the `ready-for-ci` label on PRs before running. AI review still runs on every push. This eliminates wasted CI runs during the iterative AI review phase. The `unlabeled` event type ensures removing the label clears passing CI status. 2. Documentation dependency map: docs/doc-deps.yaml maps all 47 source files to their dependent documentation. New /docs-impact skill surfaces impacted docs. Wired into /pre-merge-check, /submit-pr, and /push-pr-update for automatic enforcement. /docs-check gains a `map` validation mode. Co-Authored-By: Claude Opus 4.6 (1M context) --- .claude/commands/docs-check.md | 39 +- .claude/commands/docs-impact.md | 122 +++++++ .claude/commands/pre-merge-check.md | 27 +- .claude/commands/push-pr-update.md | 34 +- .claude/commands/submit-pr.md | 23 +- .github/workflows/notebooks.yml | 5 + .github/workflows/rust-test.yml | 10 + CLAUDE.md | 2 + CONTRIBUTING.md | 2 + docs/doc-deps.yaml | 535 ++++++++++++++++++++++++++++ 10 files changed, 762 insertions(+), 37 deletions(-) create mode 100644 .claude/commands/docs-impact.md create mode 100644 docs/doc-deps.yaml diff --git a/.claude/commands/docs-check.md b/.claude/commands/docs-check.md index c3f58e03..0e43d72b 100644 --- a/.claude/commands/docs-check.md +++ b/.claude/commands/docs-check.md @@ -1,6 +1,6 @@ --- description: Verify documentation completeness including scholarly references -argument-hint: "[all | readme | refs | api | tutorials]" +argument-hint: "[all | readme | refs | api | tutorials | map]" --- # Documentation Completeness Check @@ -11,11 +11,12 @@ Verify that documentation is complete and includes appropriate scholarly referen The user may provide an optional argument: `$ARGUMENTS` -- If empty or "all": Run all checks +- If empty or "all": Run all checks (including map validation) - If "readme": Check README.md sections only - If "refs" or "references": Check scholarly references only - If "api": Check API documentation (RST files) only - If "tutorials": Check tutorial coverage only +- If "map": Validate docs/doc-deps.yaml integrity only ## Estimators and Required Documentation @@ -197,9 +198,41 @@ Tutorial Coverage: Summary: 15/18 checks passed, 3 issues found ``` +### 8. Dependency Map Validation (if "map" or "all") + +Validate the integrity of `docs/doc-deps.yaml`: + +1. **Read and parse** `docs/doc-deps.yaml`. If missing or malformed YAML, report error. + +2. **Check all doc paths exist**: For every `path` in every `sources` entry, verify the file + exists on disk. Report missing files: + ``` + [FAIL] docs/doc-deps.yaml references non-existent: docs/old_name.rst + ``` + +3. **Check all source files have entries**: List all `diff_diff/*.py` and + `diff_diff/visualization/*.py` files. For each, verify it appears either as a key in + `sources:` or as a member of a `groups:` entry. Report missing: + ``` + [WARN] diff_diff/new_module.py has no entry in docs/doc-deps.yaml + ``` + +4. **Check for orphan doc paths**: Collect all unique doc paths from the map. Check if any + doc file referenced in the map no longer exists or has been renamed. + +5. **Report summary**: + ``` + Dependency Map (docs/doc-deps.yaml): + Sources mapped: 28 + Groups defined: 9 + Doc paths referenced: 45 + [PASS/FAIL] All doc paths exist + [PASS/WARN] All source files have entries + ``` + ## Notes - This check is especially important after adding new estimators - The CONTRIBUTING.md file documents what documentation is required for new features - Missing references should cite the original methodology paper, not textbooks -- When adding new estimators, update this skill's tables accordingly +- When adding new estimators, update this skill's tables and docs/doc-deps.yaml accordingly diff --git a/.claude/commands/docs-impact.md b/.claude/commands/docs-impact.md new file mode 100644 index 00000000..2731f046 --- /dev/null +++ b/.claude/commands/docs-impact.md @@ -0,0 +1,122 @@ +--- +description: Show which documentation files may need updating based on changed source files +argument-hint: "[file1.py file2.py ...]" +--- + +# Documentation Impact Report + +Identify which documentation files may need updating when source files in `diff_diff/` change. +Uses the dependency map at `docs/doc-deps.yaml`. + +## Arguments + +`$ARGUMENTS` is an optional space-separated list of source file paths (e.g., `diff_diff/staggered.py`). +If empty, auto-detect changed files from git. + +## Instructions + +### 1. Load Dependency Map + +Read `docs/doc-deps.yaml` using the Read tool. + +If the file does not exist or cannot be parsed, display: +``` +Error: docs/doc-deps.yaml not found or malformed. Cannot generate impact report. +``` +And stop. + +### 2. Identify Changed Source Files + +**If `$ARGUMENTS` is non-empty**: Use those file paths directly as the changed files list. + +**If `$ARGUMENTS` is empty**: Auto-detect from git: + +```bash +# Unstaged changes +git diff --name-only HEAD 2>/dev/null || true +# Staged changes +git diff --cached --name-only 2>/dev/null || true +# Untracked files +git ls-files --others --exclude-standard 2>/dev/null || true +``` + +Filter results to only files matching `diff_diff/**/*.py`. Deduplicate. + +If no source files found, display: +``` +No changed source files in diff_diff/ detected. Nothing to report. +``` +And stop. + +### 3. Resolve Group Membership + +For each changed file, check if it appears in any `groups:` list in the YAML. +If it does, resolve it to the **first entry** in that group (the primary module). +This is the key used for doc lookup in the `sources:` section. + +Example: if `diff_diff/staggered_bootstrap.py` changed, it resolves to `diff_diff/staggered.py` +because it is in the `staggered` group. + +### 4. Look Up Impacted Docs + +For each resolved source entry in the `sources:` section: +1. Get the `drift_risk` level +2. Get the list of `docs` entries (path, type, section, note) + +Collect all impacted docs across all changed files. Deduplicate by path, but merge +section hints from different sources (e.g., REGISTRY.md may be referenced by both +staggered.py and survey.py with different section hints). + +### 5. Validate Doc Paths + +For each unique doc `path` in the results, verify the file exists on disk using the +Read tool (or Glob). If a path does not exist, flag it: +``` +[STALE] docs/doc-deps.yaml references non-existent file: +``` + +### 6. Display Report + +Group results by drift risk level (HIGH first, then MEDIUM, then LOW). +Within each group, show the type label and path, with section hints where available. + +**Output format:** + +``` +=== Documentation Impact Report === +Changed: + +HIGH DRIFT RISK: + [methodology] docs/methodology/REGISTRY.md --
+ [roadmap] docs/survey-roadmap.md + +MEDIUM DRIFT RISK: + [user_guide] README.md --
+ [tutorial] docs/tutorials/02_staggered_did.ipynb + [performance] docs/benchmarks.rst + +LOW DRIFT RISK: + [api_reference] docs/api/staggered.rst + +No map entry: +Stale references: +Always update: CHANGELOG.md +``` + +### 7. Flag Missing Entries + +List any changed source files that had no entry in the `sources:` section and were not +members of any group: +``` +No map entry: diff_diff/new_module.py (consider adding to docs/doc-deps.yaml) +``` + +## Examples + +```bash +# Auto-detect from git status +/docs-impact + +# Check specific files +/docs-impact diff_diff/staggered.py diff_diff/survey.py +``` diff --git a/.claude/commands/pre-merge-check.md b/.claude/commands/pre-merge-check.md index aec8f435..0f397d2e 100644 --- a/.claude/commands/pre-merge-check.md +++ b/.claude/commands/pre-merge-check.md @@ -109,19 +109,26 @@ git diff HEAD -- | grep "^+.*def " | head -10 For each changed function, flag: "Verify docstring Parameters section matches updated signature for: ``" -#### 2.5 Methodology Documentation Check +#### 2.5 Documentation Impact Check -If any methodology files changed, check whether `docs/methodology/REGISTRY.md` was also -modified in the changed file set (from Section 1). +If any source files in `diff_diff/` changed, read `docs/doc-deps.yaml` and identify which +dependent documentation files are NOT also in the changed file set (from Section 1). -If methodology files changed but REGISTRY.md was NOT modified, flag: -"Methodology files changed but `docs/methodology/REGISTRY.md` was not updated. If your -changes deviate from reference implementations, document them using a reviewer-recognized -label (`**Note:**`, `**Deviation from R:**`, or `**Note (deviation from R):**`) — -undocumented deviations are flagged as P1 by the AI reviewer and cannot be mitigated -by TODO.md." +For each changed source file: +1. Look up its entry in `docs/doc-deps.yaml` (resolving group membership for multi-file modules) +2. Check each dependent doc's `path` against the changed file set +3. Report HIGH drift risk docs that were NOT changed as warnings -This is a WARNING, not a blocker — not every methodology change involves a deviation. +**Report format**: +``` +Documentation impact: source files changed but related docs were not updated: + [HIGH] docs/methodology/REGISTRY.md --
+ [HIGH] docs/survey-roadmap.md + [MEDIUM] README.md --
(3 more MEDIUM/LOW docs — run /docs-impact for details) +``` + +This is a WARNING, not a blocker — not every source change requires a doc update. +For full details, run `/docs-impact`. #### 2.6 Secret Scanning Patterns (Canonical Definitions) diff --git a/.claude/commands/push-pr-update.md b/.claude/commands/push-pr-update.md index b0656871..5416e288 100644 --- a/.claude/commands/push-pr-update.md +++ b/.claude/commands/push-pr-update.md @@ -128,13 +128,15 @@ When the working tree is clean but commits are ahead, check for methodology issu If warnings are found, display them as warnings (non-blocking) since changes are already committed. -3. **REGISTRY.md check**: Check whether `docs/methodology/REGISTRY.md` is also in the committed changes (`git diff --name-only ..HEAD`). - If methodology files changed but REGISTRY.md was NOT modified, warn: - "Methodology files changed but `docs/methodology/REGISTRY.md` was not updated. - If your changes deviate from reference implementations, document them using a - reviewer-recognized label (`**Note:**`, `**Deviation from R:**`, or - `**Note (deviation from R):**`) — undocumented deviations are flagged as P1 - by the AI reviewer." +3. **Documentation impact check**: Check which source files in `diff_diff/` are in the committed changes. + If source files are present, read `docs/doc-deps.yaml` and check which dependent + documentation files are NOT also in the committed changes. For HIGH drift risk docs, warn: + ``` + Documentation impact: source files changed but related docs were not updated: + [HIGH] docs/methodology/REGISTRY.md —
+ [HIGH] docs/survey-roadmap.md + Run /docs-impact for full details. + ``` This is a WARNING, not a blocker. Note: Section 3b checks are informational warnings only — no AskUserQuestion prompt, since changes are already committed and cannot be unstaged. This differs from the staged-changes path (Section 3) which offers a "fix vs continue" choice. @@ -167,14 +169,14 @@ Note: Section 3b checks are informational warnings only — no AskUserQuestion p ``` Use AskUserQuestion. If user chooses to fix, abort the commit flow. - **REGISTRY.md check** (if methodology files are staged): - Check whether `docs/methodology/REGISTRY.md` is also in the staged file set. - If methodology files changed but REGISTRY.md was NOT staged, warn: - "Methodology files changed but `docs/methodology/REGISTRY.md` was not updated. - If your changes deviate from reference implementations, document them using a - reviewer-recognized label (`**Note:**`, `**Deviation from R:**`, or - `**Note (deviation from R):**`) — undocumented deviations are flagged as P1 - by the AI reviewer." + **Documentation impact check** (if source files are staged): + If source files in `diff_diff/` are present, read `docs/doc-deps.yaml` and check which + dependent documentation files are NOT also in the staged set. For HIGH drift risk docs, warn: + ``` + Documentation impact: source files changed but related docs were not updated: + [HIGH] docs/methodology/REGISTRY.md —
+ Run /docs-impact for full details. + ``` This is a WARNING, not a blocker. 3. **Capture file count for reporting**: @@ -277,6 +279,7 @@ Commit: - Files changed: AI code review triggered. Results will appear shortly. +When AI review is green, add the `ready-for-ci` label to trigger CI tests (if not already added). PR URL: ``` @@ -291,6 +294,7 @@ Files changed: PR URL: Tip: Run /ai-review to request AI code review. +Note: CI tests require the `ready-for-ci` label on the PR. ``` ## Error Handling diff --git a/.claude/commands/submit-pr.md b/.claude/commands/submit-pr.md index 040778b5..a46edb67 100644 --- a/.claude/commands/submit-pr.md +++ b/.claude/commands/submit-pr.md @@ -154,14 +154,18 @@ Determine if this is a fork-based workflow: ``` Use AskUserQuestion. If user chooses to fix, abort the commit flow and let them address the issues. -3. **REGISTRY.md check** (if methodology files are staged): - Check whether `docs/methodology/REGISTRY.md` is also in the staged file set (`git diff --cached --name-only`). - If methodology files changed but REGISTRY.md was NOT staged, warn: - "Methodology files changed but `docs/methodology/REGISTRY.md` was not updated. - If your changes deviate from reference implementations, document them using a - reviewer-recognized label (`**Note:**`, `**Deviation from R:**`, or - `**Note (deviation from R):**`) — undocumented deviations are flagged as P1 - by the AI reviewer." +3. **Documentation impact check** (if source files are staged): + ```bash + git diff --cached --name-only | grep "^diff_diff/.*\.py$" + ``` + If source files are present, read `docs/doc-deps.yaml` and check which dependent + documentation files are NOT also in the staged set. For HIGH drift risk docs, warn: + ``` + Documentation impact: source files changed but related docs were not updated: + [HIGH] docs/methodology/REGISTRY.md —
+ [HIGH] docs/survey-roadmap.md + Run /docs-impact for full details. + ``` This is a WARNING, not a blocker. ### 6. Commit Changes @@ -352,7 +356,8 @@ Changes included: Next steps: - Review the PR at the URL above -- Request reviewers if needed +- AI code review runs automatically on PR open +- When AI review is green, add the `ready-for-ci` label to trigger CI tests - Run /review-pr to get AI review ``` diff --git a/.github/workflows/notebooks.yml b/.github/workflows/notebooks.yml index 97c20446..6696c133 100644 --- a/.github/workflows/notebooks.yml +++ b/.github/workflows/notebooks.yml @@ -10,6 +10,7 @@ on: - '.github/workflows/notebooks.yml' pull_request: branches: [main] + types: [opened, synchronize, reopened, labeled, unlabeled] paths: - 'docs/tutorials/**' - 'diff_diff/**' @@ -22,6 +23,10 @@ on: jobs: execute-notebooks: name: Execute tutorial notebooks + if: >- + github.event_name == 'push' + || github.event_name == 'schedule' + || contains(github.event.pull_request.labels.*.name, 'ready-for-ci') runs-on: ubuntu-latest steps: diff --git a/.github/workflows/rust-test.yml b/.github/workflows/rust-test.yml index 03166c2f..96683a12 100644 --- a/.github/workflows/rust-test.yml +++ b/.github/workflows/rust-test.yml @@ -11,6 +11,7 @@ on: - '.github/workflows/rust-test.yml' pull_request: branches: [main] + types: [opened, synchronize, reopened, labeled, unlabeled] paths: - 'rust/**' - 'diff_diff/**' @@ -25,6 +26,9 @@ jobs: # Run Rust unit tests on all platforms rust-tests: name: Rust Unit Tests (${{ matrix.os }}) + if: >- + github.event_name == 'push' + || contains(github.event.pull_request.labels.*.name, 'ready-for-ci') runs-on: ${{ matrix.os }} env: PYO3_USE_ABI3_FORWARD_COMPATIBILITY: 1 @@ -58,6 +62,9 @@ jobs: # Build and test with Python on multiple platforms python-tests: name: Python Tests (${{ matrix.os }}, py${{ matrix.python-version }}) + if: >- + github.event_name == 'push' + || contains(github.event.pull_request.labels.*.name, 'ready-for-ci') runs-on: ${{ matrix.os }} strategy: fail-fast: false @@ -158,6 +165,9 @@ jobs: # Test pure Python fallback (without Rust extension) python-fallback: name: Pure Python Fallback + if: >- + github.event_name == 'push' + || contains(github.event.pull_request.labels.*.name, 'ready-for-ci') runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 diff --git a/CLAUDE.md b/CLAUDE.md index 70eb23ef..08231f11 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -135,6 +135,7 @@ category (`Methodology/Correctness`, `Performance`, or `Testing/Docs`): | File | Contains | |------|----------| | `docs/methodology/REGISTRY.md` | Academic foundations, equations, edge cases — **consult before methodology changes** | +| `docs/doc-deps.yaml` | Source-to-documentation dependency map — **consult when any source file changes** | | `CONTRIBUTING.md` | Documentation requirements, test writing guidelines | | `.claude/commands/dev-checklists.md` | Checklists for params, methodology, warnings, reviews, bugs (run `/dev-checklists`) | | `.claude/memory.md` | Debugging patterns, tolerances, API conventions (git-tracked) | @@ -145,6 +146,7 @@ category (`Methodology/Correctness`, `Performance`, or `Testing/Docs`): ## Workflow - For non-trivial tasks, use `EnterPlanMode`. Consult `docs/methodology/REGISTRY.md` for methodology changes. +- When modifying source files in `diff_diff/`, consult `docs/doc-deps.yaml` to identify impacted documentation. Run `/docs-impact` to see the full list. - For bug fixes, grep for the pattern across all files before fixing. - Follow the relevant development checklists (run `/dev-checklists`). - Before submitting: run `/pre-merge-check`, then `/ai-review-local` for pre-PR AI review. diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 5c158eb2..c6802369 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -29,6 +29,8 @@ When implementing new functionality, **always include accompanying documentation - Move implemented features from planned to current status - Update version numbers +6. **docs/doc-deps.yaml** - Add source-to-doc mappings for the new module + ### For Bug Fixes or Minor Enhancements - Update relevant docstrings diff --git a/docs/doc-deps.yaml b/docs/doc-deps.yaml new file mode 100644 index 00000000..53a209cf --- /dev/null +++ b/docs/doc-deps.yaml @@ -0,0 +1,535 @@ +# Documentation Dependency Map +# Maps source files to documentation that may need updating when the source changes. +# Consumed by /docs-impact skill and referenced in CLAUDE.md. +# +# Relationship types: +# methodology - REGISTRY.md section (update if algorithm/edge-case behavior changes) +# tutorial - Jupyter notebook demonstrating the feature +# api_reference - Sphinx RST auto-generated from docstrings (update if signature/docstring changes) +# user_guide - README.md, llms*.txt, choosing_estimator.rst sections +# roadmap - ROADMAP.md, survey-roadmap.md status tracking +# performance - benchmarks.rst, performance-plan.md +# internal - CLAUDE.md, CONTRIBUTING.md (developer-facing) +# +# Drift risk levels: +# high - historically drifts or has dense cross-references +# medium - moderate cross-references, changes occasionally +# low - stable relationship, rarely needs updating + +# ────────────────────────────────────────────────────────────────────── +# Groups: multi-file modules that share documentation dependencies. +# Members resolve to the first entry (the primary module) for doc lookup. +# ────────────���───────────────────────────────────────────────────────── +groups: + staggered: + - diff_diff/staggered.py + - diff_diff/staggered_aggregation.py + - diff_diff/staggered_bootstrap.py + - diff_diff/staggered_results.py + staggered_triple_diff: + - diff_diff/staggered_triple_diff.py + - diff_diff/staggered_triple_diff_results.py + trop: + - diff_diff/trop.py + - diff_diff/trop_global.py + - diff_diff/trop_local.py + - diff_diff/trop_results.py + imputation: + - diff_diff/imputation.py + - diff_diff/imputation_bootstrap.py + - diff_diff/imputation_results.py + two_stage: + - diff_diff/two_stage.py + - diff_diff/two_stage_bootstrap.py + - diff_diff/two_stage_results.py + efficient_did: + - diff_diff/efficient_did.py + - diff_diff/efficient_did_bootstrap.py + - diff_diff/efficient_did_covariates.py + - diff_diff/efficient_did_results.py + - diff_diff/efficient_did_weights.py + continuous_did: + - diff_diff/continuous_did.py + - diff_diff/continuous_did_bspline.py + - diff_diff/continuous_did_results.py + stacked_did: + - diff_diff/stacked_did.py + - diff_diff/stacked_did_results.py + visualization: + - diff_diff/visualization/__init__.py + - diff_diff/visualization/_common.py + - diff_diff/visualization/_continuous.py + - diff_diff/visualization/_diagnostic.py + - diff_diff/visualization/_event_study.py + - diff_diff/visualization/_power.py + - diff_diff/visualization/_staggered.py + - diff_diff/visualization/_synthetic.py + +# ────────────────────────────────────────────────────────────────────── +# Source-to-docs mappings +# ────────���─────────────────────────��─────────────────────────────────── +sources: + + # ── Base estimators ──────��─────────────────────────────────────────── + + diff_diff/estimators.py: + drift_risk: medium + docs: + - path: docs/methodology/REGISTRY.md + section: "DifferenceInDifferences, TwoWayFixedEffects, MultiPeriodDiD" + type: methodology + - path: docs/api/estimators.rst + type: api_reference + - path: docs/tutorials/01_basic_did.ipynb + type: tutorial + - path: docs/tutorials/04_parallel_trends.ipynb + type: tutorial + - path: docs/tutorials/09_real_world_examples.ipynb + type: tutorial + - path: README.md + section: "DifferenceInDifferences" + type: user_guide + - path: docs/llms-full.txt + section: "DifferenceInDifferences" + type: user_guide + - path: docs/llms.txt + type: user_guide + - path: docs/choosing_estimator.rst + type: user_guide + - path: docs/benchmarks.rst + type: performance + + diff_diff/twfe.py: + drift_risk: low + docs: + - path: docs/methodology/REGISTRY.md + section: "TwoWayFixedEffects" + type: methodology + - path: docs/api/estimators.rst + type: api_reference + + # ── CallawaySantAnna (staggered group) ─────��──────────────────────── + + diff_diff/staggered.py: + drift_risk: high + docs: + - path: docs/methodology/REGISTRY.md + section: "CallawaySantAnna" + type: methodology + - path: docs/api/staggered.rst + type: api_reference + - path: docs/tutorials/02_staggered_did.ipynb + type: tutorial + - path: docs/tutorials/16_survey_did.ipynb + type: tutorial + note: "CallawaySantAnna survey examples" + - path: README.md + section: "CallawaySantAnna" + type: user_guide + - path: docs/llms-full.txt + section: "CallawaySantAnna" + type: user_guide + - path: docs/llms.txt + type: user_guide + - path: docs/choosing_estimator.rst + type: user_guide + - path: docs/benchmarks.rst + type: performance + - path: docs/survey-roadmap.md + type: roadmap + note: "Phase 4/6/7 CS survey sections" + + # ── StaggeredTripleDifference (staggered_triple_diff group) ───────── + + diff_diff/staggered_triple_diff.py: + drift_risk: low + docs: + - path: docs/methodology/REGISTRY.md + section: "StaggeredTripleDifference" + type: methodology + - path: docs/api/staggered.rst + type: api_reference + + # ── SunAbraham ────────────────────────────────────────────────────── + + diff_diff/sun_abraham.py: + drift_risk: medium + docs: + - path: docs/methodology/REGISTRY.md + section: "SunAbraham" + type: methodology + - path: docs/api/staggered.rst + type: api_reference + - path: docs/tutorials/02_staggered_did.ipynb + type: tutorial + note: "SunAbraham comparison section" + - path: README.md + section: "SunAbraham" + type: user_guide + - path: docs/llms-full.txt + section: "SunAbraham" + type: user_guide + - path: docs/choosing_estimator.rst + type: user_guide + - path: docs/benchmarks.rst + type: performance + + # ── ImputationDiD (imputation group) ───��──────────────────────────── + + diff_diff/imputation.py: + drift_risk: medium + docs: + - path: docs/methodology/REGISTRY.md + section: "ImputationDiD" + type: methodology + - path: docs/api/imputation.rst + type: api_reference + - path: docs/tutorials/11_imputation_did.ipynb + type: tutorial + - path: README.md + section: "ImputationDiD" + type: user_guide + - path: docs/llms-full.txt + section: "ImputationDiD" + type: user_guide + - path: docs/choosing_estimator.rst + type: user_guide + + # ── TwoStageDiD (two_stage group) ────────────���───────────────────── + + diff_diff/two_stage.py: + drift_risk: medium + docs: + - path: docs/methodology/REGISTRY.md + section: "TwoStageDiD" + type: methodology + - path: docs/api/two_stage.rst + type: api_reference + - path: docs/tutorials/12_two_stage_did.ipynb + type: tutorial + - path: README.md + section: "TwoStageDiD" + type: user_guide + - path: docs/llms-full.txt + section: "TwoStageDiD" + type: user_guide + - path: docs/choosing_estimator.rst + type: user_guide + + # ── EfficientDiD (efficient_did group) ────────────────────────────── + + diff_diff/efficient_did.py: + drift_risk: medium + docs: + - path: docs/methodology/REGISTRY.md + section: "EfficientDiD" + type: methodology + - path: docs/api/efficient_did.rst + type: api_reference + - path: docs/tutorials/15_efficient_did.ipynb + type: tutorial + - path: README.md + section: "EfficientDiD" + type: user_guide + - path: docs/llms-full.txt + section: "EfficientDiD" + type: user_guide + - path: docs/choosing_estimator.rst + type: user_guide + - path: docs/benchmarks.rst + type: performance + + # ── ContinuousDiD (continuous_did group) ────────────────────��─────── + + diff_diff/continuous_did.py: + drift_risk: medium + docs: + - path: docs/methodology/REGISTRY.md + section: "ContinuousDiD" + type: methodology + - path: docs/methodology/continuous-did.md + type: methodology + note: "Detailed CDiD implementation docs" + - path: docs/api/continuous_did.rst + type: api_reference + - path: docs/tutorials/14_continuous_did.ipynb + type: tutorial + - path: README.md + section: "ContinuousDiD" + type: user_guide + - path: docs/llms-full.txt + section: "ContinuousDiD" + type: user_guide + - path: docs/choosing_estimator.rst + type: user_guide + + # ── SyntheticDiD ───────────��─────────────────────────────────────── + + diff_diff/synthetic_did.py: + drift_risk: medium + docs: + - path: docs/methodology/REGISTRY.md + section: "SyntheticDiD" + type: methodology + - path: docs/api/estimators.rst + type: api_reference + - path: docs/tutorials/03_synthetic_did.ipynb + type: tutorial + - path: README.md + section: "SyntheticDiD" + type: user_guide + - path: docs/llms-full.txt + section: "SyntheticDiD" + type: user_guide + - path: docs/choosing_estimator.rst + type: user_guide + - path: docs/benchmarks.rst + type: performance + + # ── TripleDifference ─────────────────────────────────────────────── + + diff_diff/triple_diff.py: + drift_risk: medium + docs: + - path: docs/methodology/REGISTRY.md + section: "TripleDifference" + type: methodology + - path: docs/api/triple_diff.rst + type: api_reference + - path: docs/tutorials/08_triple_diff.ipynb + type: tutorial + - path: README.md + section: "TripleDifference" + type: user_guide + - path: docs/llms-full.txt + section: "TripleDifference" + type: user_guide + - path: docs/choosing_estimator.rst + type: user_guide + + # ── StackedDiD (stacked_did group) ───────────────────────────────── + + diff_diff/stacked_did.py: + drift_risk: medium + docs: + - path: docs/methodology/REGISTRY.md + section: "StackedDiD" + type: methodology + - path: docs/api/stacked_did.rst + type: api_reference + - path: docs/tutorials/13_stacked_did.ipynb + type: tutorial + - path: README.md + section: "StackedDiD" + type: user_guide + - path: docs/llms-full.txt + section: "StackedDiD" + type: user_guide + - path: docs/choosing_estimator.rst + type: user_guide + + # ── TROP (trop group) ──────────────���─────────────────────────────── + + diff_diff/trop.py: + drift_risk: medium + docs: + - path: docs/methodology/REGISTRY.md + section: "TROP" + type: methodology + - path: docs/api/trop.rst + type: api_reference + - path: docs/tutorials/10_trop.ipynb + type: tutorial + - path: README.md + section: "TROP" + type: user_guide + - path: docs/llms-full.txt + section: "TROP" + type: user_guide + - path: docs/choosing_estimator.rst + type: user_guide + - path: docs/performance-plan.md + type: performance + + # ── HonestDiD ───────��────────────────────────────────────────────── + + diff_diff/honest_did.py: + drift_risk: medium + docs: + - path: docs/methodology/REGISTRY.md + section: "HonestDiD" + type: methodology + - path: docs/api/honest_did.rst + type: api_reference + - path: docs/tutorials/05_honest_did.ipynb + type: tutorial + - path: README.md + section: "HonestDiD" + type: user_guide + - path: docs/llms-full.txt + section: "HonestDiD" + type: user_guide + + # ── BaconDecomposition ───���───────────────────────────────────────── + + diff_diff/bacon.py: + drift_risk: low + docs: + - path: docs/methodology/REGISTRY.md + section: "BaconDecomposition" + type: methodology + - path: docs/api/bacon.rst + type: api_reference + - path: README.md + section: "BaconDecomposition" + type: user_guide + + # ── Diagnostics & analysis ───────���───────────────────────────────── + + diff_diff/diagnostics.py: + drift_risk: low + docs: + - path: docs/methodology/REGISTRY.md + section: "Diagnostics" + type: methodology + - path: docs/api/diagnostics.rst + type: api_reference + - path: docs/tutorials/04_parallel_trends.ipynb + type: tutorial + + diff_diff/pretrends.py: + drift_risk: low + docs: + - path: docs/methodology/REGISTRY.md + section: "Pre-trends" + type: methodology + - path: docs/api/pretrends.rst + type: api_reference + - path: docs/tutorials/04_parallel_trends.ipynb + type: tutorial + - path: docs/tutorials/07_pretrends_power.ipynb + type: tutorial + + diff_diff/power.py: + drift_risk: low + docs: + - path: docs/methodology/REGISTRY.md + section: "Power Analysis" + type: methodology + - path: docs/api/power.rst + type: api_reference + - path: docs/tutorials/06_power_analysis.ipynb + type: tutorial + - path: docs/tutorials/07_pretrends_power.ipynb + type: tutorial + + # ── Survey support ───────────────────────────────────────────────── + + diff_diff/survey.py: + drift_risk: high + docs: + - path: docs/methodology/REGISTRY.md + section: "Survey" + type: methodology + - path: docs/survey-roadmap.md + type: roadmap + - path: docs/tutorials/16_survey_did.ipynb + type: tutorial + - path: README.md + section: "Survey" + type: user_guide + - path: docs/llms-full.txt + section: "Survey" + type: user_guide + - path: docs/choosing_estimator.rst + section: "Survey Design Support" + type: user_guide + + # ── Infrastructure ───────────────────────────────────────────────── + + diff_diff/linalg.py: + drift_risk: medium + docs: + - path: docs/performance-plan.md + type: performance + - path: docs/benchmarks.rst + type: performance + - path: docs/api/utils.rst + type: api_reference + + diff_diff/utils.py: + drift_risk: medium + docs: + - path: CLAUDE.md + section: "safe_inference pattern" + type: internal + - path: docs/api/utils.rst + type: api_reference + + diff_diff/results.py: + drift_risk: medium + docs: + - path: docs/api/results.rst + type: api_reference + - path: docs/llms-full.txt + section: "Results API" + type: user_guide + - path: docs/llms.txt + type: user_guide + + diff_diff/bootstrap_utils.py: + drift_risk: low + docs: + - path: docs/survey-roadmap.md + type: roadmap + note: "Phase 6 bootstrap+survey interaction" + + diff_diff/prep.py: + drift_risk: low + docs: + - path: docs/api/prep.rst + type: api_reference + + diff_diff/prep_dgp.py: + drift_risk: low + docs: + - path: docs/api/prep.rst + type: api_reference + + diff_diff/datasets.py: + drift_risk: low + docs: + - path: docs/api/datasets.rst + type: api_reference + - path: docs/tutorials/09_real_world_examples.ipynb + type: tutorial + + diff_diff/practitioner.py: + drift_risk: low + docs: + - path: docs/llms-practitioner.txt + type: user_guide + + # ── Visualization (visualization group) ──────────────────────────── + + diff_diff/visualization/__init__.py: + drift_risk: low + docs: + - path: docs/api/visualization.rst + type: api_reference + + # ── Minimal entries (infrastructure files) ───────────────────────── + + diff_diff/__init__.py: + drift_risk: low + docs: + - path: docs/llms.txt + type: user_guide + note: "Public API surface" + + diff_diff/_backend.py: + drift_risk: low + docs: + - path: docs/performance-plan.md + type: performance + note: "Rust backend detection" From d4d9f659e6f520e728f4ad143d46d5908f8225c6 Mon Sep 17 00:00:00 2001 From: igerber Date: Sat, 4 Apr 2026 15:45:45 -0400 Subject: [PATCH 2/3] Address AI review: gate job enforcement and methodology warning fix P1: Replace job-level if: conditions with a ci-gate job that fails when the ready-for-ci label is absent. Expensive jobs depend on the gate via needs:, so they cannot run (or skip-as-passing) without the label. Resolves the skipped-satisfies-required-checks concern. P1: Restore REGISTRY.md warning coverage by always warning about type: methodology dependencies regardless of drift_risk level. Medium-drift estimators now trigger methodology doc warnings again. P3: Add ROADMAP.md to docs-impact "always check" list, narrow the doc-deps.yaml roadmap type description, and defer unrelated-label and CI-validation items to TODO.md. Co-Authored-By: Claude Opus 4.6 (1M context) --- .claude/commands/docs-impact.md | 15 ++++++++++++--- .claude/commands/pre-merge-check.md | 9 ++++++--- .claude/commands/push-pr-update.md | 12 ++++++++---- .claude/commands/submit-pr.md | 6 ++++-- .github/workflows/notebooks.yml | 19 +++++++++++++++---- .github/workflows/rust-test.yml | 27 ++++++++++++++++++--------- TODO.md | 2 ++ docs/doc-deps.yaml | 2 +- 8 files changed, 66 insertions(+), 26 deletions(-) diff --git a/.claude/commands/docs-impact.md b/.claude/commands/docs-impact.md index 2731f046..6c8fa5fa 100644 --- a/.claude/commands/docs-impact.md +++ b/.claude/commands/docs-impact.md @@ -77,7 +77,14 @@ Read tool (or Glob). If a path does not exist, flag it: ### 6. Display Report -Group results by drift risk level (HIGH first, then MEDIUM, then LOW). +Display results in priority order: + +1. **METHODOLOGY (always warn)**: All docs with `type: methodology`, regardless of `drift_risk`. + These are shown first because undocumented methodology deviations are P1 in AI review. +2. **HIGH DRIFT RISK**: Remaining docs (non-methodology) with `drift_risk: high`. +3. **MEDIUM DRIFT RISK**: Docs with `drift_risk: medium` (excluding methodology, already shown). +4. **LOW DRIFT RISK**: Docs with `drift_risk: low` (excluding methodology, already shown). + Within each group, show the type label and path, with section hints where available. **Output format:** @@ -86,8 +93,10 @@ Within each group, show the type label and path, with section hints where availa === Documentation Impact Report === Changed: +METHODOLOGY (always warn): + docs/methodology/REGISTRY.md --
+ HIGH DRIFT RISK: - [methodology] docs/methodology/REGISTRY.md --
[roadmap] docs/survey-roadmap.md MEDIUM DRIFT RISK: @@ -100,7 +109,7 @@ LOW DRIFT RISK: No map entry: Stale references: -Always update: CHANGELOG.md +Always check: CHANGELOG.md, ROADMAP.md ``` ### 7. Flag Missing Entries diff --git a/.claude/commands/pre-merge-check.md b/.claude/commands/pre-merge-check.md index 0f397d2e..57c07e75 100644 --- a/.claude/commands/pre-merge-check.md +++ b/.claude/commands/pre-merge-check.md @@ -117,14 +117,17 @@ dependent documentation files are NOT also in the changed file set (from Section For each changed source file: 1. Look up its entry in `docs/doc-deps.yaml` (resolving group membership for multi-file modules) 2. Check each dependent doc's `path` against the changed file set -3. Report HIGH drift risk docs that were NOT changed as warnings +3. Report docs that were NOT changed as warnings: + - ALL docs with `type: methodology` (regardless of `drift_risk`) — methodology deviations + are P1 in AI review, so this warning must always fire + - All HIGH `drift_risk` docs (any type) **Report format**: ``` Documentation impact: source files changed but related docs were not updated: - [HIGH] docs/methodology/REGISTRY.md --
+ [METHODOLOGY] docs/methodology/REGISTRY.md --
[HIGH] docs/survey-roadmap.md - [MEDIUM] README.md --
(3 more MEDIUM/LOW docs — run /docs-impact for details) + [MEDIUM] README.md --
(N more -- run /docs-impact for details) ``` This is a WARNING, not a blocker — not every source change requires a doc update. diff --git a/.claude/commands/push-pr-update.md b/.claude/commands/push-pr-update.md index 5416e288..863a101e 100644 --- a/.claude/commands/push-pr-update.md +++ b/.claude/commands/push-pr-update.md @@ -130,10 +130,12 @@ When the working tree is clean but commits are ahead, check for methodology issu 3. **Documentation impact check**: Check which source files in `diff_diff/` are in the committed changes. If source files are present, read `docs/doc-deps.yaml` and check which dependent - documentation files are NOT also in the committed changes. For HIGH drift risk docs, warn: + documentation files are NOT also in the committed changes. Warn about: + - ALL docs with `type: methodology` (regardless of `drift_risk`) + - All HIGH `drift_risk` docs (any type) ``` Documentation impact: source files changed but related docs were not updated: - [HIGH] docs/methodology/REGISTRY.md —
+ [METHODOLOGY] docs/methodology/REGISTRY.md —
[HIGH] docs/survey-roadmap.md Run /docs-impact for full details. ``` @@ -171,10 +173,12 @@ Note: Section 3b checks are informational warnings only — no AskUserQuestion p **Documentation impact check** (if source files are staged): If source files in `diff_diff/` are present, read `docs/doc-deps.yaml` and check which - dependent documentation files are NOT also in the staged set. For HIGH drift risk docs, warn: + dependent documentation files are NOT also in the staged set. Warn about: + - ALL docs with `type: methodology` (regardless of `drift_risk`) + - All HIGH `drift_risk` docs (any type) ``` Documentation impact: source files changed but related docs were not updated: - [HIGH] docs/methodology/REGISTRY.md —
+ [METHODOLOGY] docs/methodology/REGISTRY.md —
Run /docs-impact for full details. ``` This is a WARNING, not a blocker. diff --git a/.claude/commands/submit-pr.md b/.claude/commands/submit-pr.md index a46edb67..d71b8807 100644 --- a/.claude/commands/submit-pr.md +++ b/.claude/commands/submit-pr.md @@ -159,10 +159,12 @@ Determine if this is a fork-based workflow: git diff --cached --name-only | grep "^diff_diff/.*\.py$" ``` If source files are present, read `docs/doc-deps.yaml` and check which dependent - documentation files are NOT also in the staged set. For HIGH drift risk docs, warn: + documentation files are NOT also in the staged set. Warn about: + - ALL docs with `type: methodology` (regardless of `drift_risk`) + - All HIGH `drift_risk` docs (any type) ``` Documentation impact: source files changed but related docs were not updated: - [HIGH] docs/methodology/REGISTRY.md —
+ [METHODOLOGY] docs/methodology/REGISTRY.md —
[HIGH] docs/survey-roadmap.md Run /docs-impact for full details. ``` diff --git a/.github/workflows/notebooks.yml b/.github/workflows/notebooks.yml index 6696c133..9b2ff1ea 100644 --- a/.github/workflows/notebooks.yml +++ b/.github/workflows/notebooks.yml @@ -21,12 +21,23 @@ on: - cron: '0 6 * * 0' jobs: + # Lightweight gate: fails on PRs without the ready-for-ci label. + ci-gate: + name: CI Gate + runs-on: ubuntu-latest + steps: + - name: Require ready-for-ci label on PRs + if: >- + github.event_name == 'pull_request' + && !contains(github.event.pull_request.labels.*.name, 'ready-for-ci') + run: | + echo "::error::The 'ready-for-ci' label is required to run CI tests." + echo "Add the label to trigger CI." + exit 1 + execute-notebooks: name: Execute tutorial notebooks - if: >- - github.event_name == 'push' - || github.event_name == 'schedule' - || contains(github.event.pull_request.labels.*.name, 'ready-for-ci') + needs: [ci-gate] runs-on: ubuntu-latest steps: diff --git a/.github/workflows/rust-test.yml b/.github/workflows/rust-test.yml index 96683a12..d22c1d85 100644 --- a/.github/workflows/rust-test.yml +++ b/.github/workflows/rust-test.yml @@ -23,12 +23,25 @@ env: CARGO_TERM_COLOR: always jobs: + # Lightweight gate: fails on PRs without the ready-for-ci label. + # Expensive jobs depend on this via needs: so they won't start if the gate fails. + ci-gate: + name: CI Gate + runs-on: ubuntu-latest + steps: + - name: Require ready-for-ci label on PRs + if: >- + github.event_name == 'pull_request' + && !contains(github.event.pull_request.labels.*.name, 'ready-for-ci') + run: | + echo "::error::The 'ready-for-ci' label is required to run CI tests." + echo "Add the label to trigger CI." + exit 1 + # Run Rust unit tests on all platforms rust-tests: name: Rust Unit Tests (${{ matrix.os }}) - if: >- - github.event_name == 'push' - || contains(github.event.pull_request.labels.*.name, 'ready-for-ci') + needs: [ci-gate] runs-on: ${{ matrix.os }} env: PYO3_USE_ABI3_FORWARD_COMPATIBILITY: 1 @@ -62,9 +75,7 @@ jobs: # Build and test with Python on multiple platforms python-tests: name: Python Tests (${{ matrix.os }}, py${{ matrix.python-version }}) - if: >- - github.event_name == 'push' - || contains(github.event.pull_request.labels.*.name, 'ready-for-ci') + needs: [ci-gate] runs-on: ${{ matrix.os }} strategy: fail-fast: false @@ -165,9 +176,7 @@ jobs: # Test pure Python fallback (without Rust extension) python-fallback: name: Pure Python Fallback - if: >- - github.event_name == 'push' - || contains(github.event.pull_request.labels.*.name, 'ready-for-ci') + needs: [ci-gate] runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 diff --git a/TODO.md b/TODO.md index 9ea2f232..1ddd661d 100644 --- a/TODO.md +++ b/TODO.md @@ -84,6 +84,7 @@ Deferred items from PR reviews that were not addressed before merge. |-------|----------|----|----------| | ImputationDiD event-study SEs recompute full conservative variance per horizon (should cache A0/A1 factorization) | `imputation.py` | #141 | Low | | Rust faer SVD ndarray-to-faer conversion overhead (minimal vs SVD cost) | `rust/src/linalg.rs:67` | #115 | Low | +| Unrelated label events (e.g., adding `bug` label) re-trigger CI workflows when `ready-for-ci` is already present; filter `labeled`/`unlabeled` events to only `ready-for-ci` transitions | `.github/workflows/rust-test.yml`, `notebooks.yml` | #269 | Low | #### Testing/Docs @@ -93,6 +94,7 @@ Deferred items from PR reviews that were not addressed before merge. | CS R helpers hard-code `xformla = ~ 1`; no covariate-adjusted R benchmark for IRLS path | `tests/test_methodology_callaway.py` | #202 | Low | | ~376 `duplicate object description` Sphinx warnings — restructure `docs/api/*.rst` to avoid duplicate `:members:` + `autosummary` | `docs/api/*.rst` | — | Low | | Doc-snippet smoke tests only cover `.rst` files; `.txt` AI guides outside CI validation | `tests/test_doc_snippets.py` | #239 | Low | +| Add CI validation for `docs/doc-deps.yaml` integrity (stale paths, unmapped source files) | `docs/doc-deps.yaml` | #269 | Low | --- diff --git a/docs/doc-deps.yaml b/docs/doc-deps.yaml index 53a209cf..b9d5d10e 100644 --- a/docs/doc-deps.yaml +++ b/docs/doc-deps.yaml @@ -7,7 +7,7 @@ # tutorial - Jupyter notebook demonstrating the feature # api_reference - Sphinx RST auto-generated from docstrings (update if signature/docstring changes) # user_guide - README.md, llms*.txt, choosing_estimator.rst sections -# roadmap - ROADMAP.md, survey-roadmap.md status tracking +# roadmap - survey-roadmap.md (ROADMAP.md is surfaced globally by /docs-impact) # performance - benchmarks.rst, performance-plan.md # internal - CLAUDE.md, CONTRIBUTING.md (developer-facing) # From f7157f4d6e0c7d7b0322f9a543397a0529110a4c Mon Sep 17 00:00:00 2001 From: igerber Date: Sat, 4 Apr 2026 16:05:18 -0400 Subject: [PATCH 3/3] Add methodology deps for shared helpers, document CI gate rollout Add type: methodology entries to doc-deps.yaml for linalg.py (variance/cluster-robust SE), utils.py (safe_inference NaN gating), and bootstrap_utils.py (bootstrap/survey bootstrap). These shared helpers implement REGISTRY.md-documented behavior and must trigger methodology warnings when changed. Document label-gated CI workflow in CLAUDE.md. Configure CI Gate as a required status check on main via branch protection. Co-Authored-By: Claude Opus 4.6 (1M context) --- CLAUDE.md | 3 +++ docs/doc-deps.yaml | 11 ++++++++++- 2 files changed, 13 insertions(+), 1 deletion(-) diff --git a/CLAUDE.md b/CLAUDE.md index 08231f11..d7bf561d 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -145,6 +145,9 @@ category (`Methodology/Correctness`, `Performance`, or `Testing/Docs`): ## Workflow +- CI tests are gated behind the `ready-for-ci` label. The `CI Gate` required status check + enforces this — PRs cannot merge until the label is added. Tests run automatically once + the label is present. - For non-trivial tasks, use `EnterPlanMode`. Consult `docs/methodology/REGISTRY.md` for methodology changes. - When modifying source files in `diff_diff/`, consult `docs/doc-deps.yaml` to identify impacted documentation. Run `/docs-impact` to see the full list. - For bug fixes, grep for the pattern across all files before fixing. diff --git a/docs/doc-deps.yaml b/docs/doc-deps.yaml index b9d5d10e..b89c06a7 100644 --- a/docs/doc-deps.yaml +++ b/docs/doc-deps.yaml @@ -450,6 +450,9 @@ sources: diff_diff/linalg.py: drift_risk: medium docs: + - path: docs/methodology/REGISTRY.md + section: "Variance Estimation, Cluster-Robust SE" + type: methodology - path: docs/performance-plan.md type: performance - path: docs/benchmarks.rst @@ -460,6 +463,9 @@ sources: diff_diff/utils.py: drift_risk: medium docs: + - path: docs/methodology/REGISTRY.md + section: "Inference, safe_inference NaN gating" + type: methodology - path: CLAUDE.md section: "safe_inference pattern" type: internal @@ -478,8 +484,11 @@ sources: type: user_guide diff_diff/bootstrap_utils.py: - drift_risk: low + drift_risk: medium docs: + - path: docs/methodology/REGISTRY.md + section: "Bootstrap, Survey Bootstrap" + type: methodology - path: docs/survey-roadmap.md type: roadmap note: "Phase 6 bootstrap+survey interaction"