Skip to content

examples/TESTING.md documents a per-rung MORPH_LADDER_RUNGS computed from changed paths; CI hardcodes 'all' #255

Description

@Yaraslaut

Found while fixing #179 (PR #254). Filed separately rather than folded in, per
AGENTS.md#179 is about the rung list, this is about a scoping mechanism
that was documented and never built.

Verification status

Reproduced by reading the workflow, on dfb238b (PR #254's branch, whose
changes do not touch this).
Every claim below is a grep of the checked-in
.github/workflows/ci.yml. I did not measure CI minutes, so the cost
argument in the last section is inferred, not measured.

The claim

examples/TESTING.md, "CI tiers", states that a rung's CI cost is managed by
computing MORPH_LADDER_RUNGS from the changed paths:

a rung's cost is managed by path-filtering (MORPH_LADDER_RUNGS computed
from changed paths: examples/<rung>/** → that rung; examples/common/**
or include/morph/** → all rungs), not by deferring work to an off-hours
run

and item 1 of the same list says the ladder-tests job is "path-filtered per
the MORPH_LADDER_RUNGS rule above", and later that "ASan is scoped to
changed rungs".

The section introduces itself as "grounded in the existing workflows", and
item 1 is written in the present tense about the job as it stands — it even
corrects itself about -LE stress being a no-op today. So it reads as a
description of current CI, not as a plan.

What CI actually does

MORPH_LADDER_RUNGS is never computed from anything. Every occurrence in
.github/workflows/ci.yml is a literal:

255:            EXTRA_ARGS+=(-DMORPH_BUILD_QT=ON -DMORPH_BUILD_LADDER=ON -DMORPH_LADDER_RUNGS=all)
447:            -DMORPH_LADDER_RUNGS=kanban \
683:            -DMORPH_LADDER_RUNGS=all \
843:            -DMORPH_LADDER_RUNGS=all \
1034:            -DMORPH_LADDER_RUNGS=all \

Line 683 is ladder-tests, line 843 is ladder-sanitizers. Both are
unconditional all. Line 447 is kanban-tsan, deliberately pinned to one
rung and unrelated to changed paths.

The changed-paths filter those two jobs do have is a single boolean: it sets
run=true or run=false for the whole job. It cannot select rungs, and
nothing downstream of it tries to. So:

  • A PR touching only examples/pastebin/** builds and tests all six
    rungs, not pastebin.
  • "ASan is scoped to changed rungs" is false in both directions:
    ladder-sanitizers either instruments every rung or runs no rung at all.

Why this matters beyond tidiness

This is the same shape as #179 — a control that reads as if it gates something
finer than it does — with the failure pointing the other way. #179's filter
silently ran less than the docs promised; this one silently runs more. The
practical consequence is the cost the section exists to manage: the two most
expensive jobs in ci.yml do a full six-rung Qt build on any qualifying
change, and the document that would tell a reader that says the opposite.

It also matters for planning. examples/LADDER.md scopes the ladder at eight
rungs; the per-rung selection this text describes is the stated reason the
ladder does not "dominate CI minutes by rung 3". If nobody notices it was
never implemented, that projection is resting on a mechanism that does not
exist.

Resolution

Either is fine, but they are opposite fixes and the choice should be
deliberate:

  1. Correct the prose to describe the boolean run/skip filter and
    MORPH_LADDER_RUNGS=all that CI actually uses, and drop the "ASan is
    scoped to changed rungs" claim. Cheapest; concedes the cost story.
  2. Implement the described scoping: derive MORPH_LADDER_RUNGS in the
    filter step from the changed paths (the rung list is now readable there
    via scripts/ladder_rungs.sh, added in ci: make examples/rungs.txt the single rung list, derive or guard every consumer #254, so the mechanical part is
    already available), falling back to all for examples/common/**,
    include/morph/**, cmake/, and the workflow itself.

If option 2 is taken, note the trap #179 is about: a computed
MORPH_LADDER_RUNGS that comes out empty would configure no rung and pass
silently. It needs the same treatment — an explicit failure when the
computation yields nothing while the job decided to run.

What would close this

examples/TESTING.md's "CI tiers" section and ci.yml agreeing about how
MORPH_LADDER_RUNGS is chosen — whichever way that agreement is reached.

What would reopen it

MORPH_LADDER_RUNGS becoming a computed value in one place and a literal in
another again, or the "scoped to changed rungs" phrasing returning without a
job that implements it.

Metadata

Metadata

Assignees

No one assigned

    Labels

    area: ciSubsystem: ciarea: ladderSubsystem: ladderdocumentationImprovements or additions to documentation

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions