Skip to content

feat(docs): self-host the repo-derived README badges - #243

Open
cdeust wants to merge 2 commits into
mainfrom
docs/self-hosted-badges
Open

feat(docs): self-host the repo-derived README badges#243
cdeust wants to merge 2 commits into
mainfrom
docs/self-hosted-badges

Conversation

@cdeust

@cdeust cdeust commented Jul 28, 2026

Copy link
Copy Markdown
Owner

Stacked on docs/mcp-toplist-badge (#242) — review that one first; GitHub retargets this to main when #242 merges.

The README's five repo-derived badges were remote images: a third-party request on every view, and a claim about this repository its host could restate with no commit here. Every one of those figures is determined by the working tree, so a committed SVG can always be made true again by regenerating it — with no network at all.

Two commits

  1. 93dd60e refactor(scripts): extract the shared badge SVG renderer — behavior-preserving. assets/badge-mcp-toplist.svg is byte-identical before and after; its 51 tests pass unchanged.
  2. f22fa65 feat(docs): self-host the repo-derived README badges — the change itself.

What it does

  • scripts/generate_repo_badges.py renders license / python / tests / references / version from the canonical readers check_doc_claims.py already uses — not reimplementations, since a badge that disagreed with the gate would be the exact drift this exists to prevent.
  • Blocking CI on every push and PR: --check in the static-claims job (four file-derived badges, no network, no suite) and --check --test-count in the test job, where the collected count is known.
  • The gates were rewired, not just the README. check_doc_claims.py matched the version and test figures by regex over the shields.io URLs. Against the new README those patterns match nothing and both checks would have passed quietly — a gate that reports success while reading nothing. They now read each figure from the committed SVG's <title> via check_badge(), which fails closed on a missing file and on an unmatched title; check_no_hotlinked_badges() rejects any reintroduced img.shields.io so reverting this is loud.
  • badge_render.render() now parses its own output and raises BadgeMarkupError on anything not well-formed. A provenance comment containing -- is illegal inside an XML comment and produced five unparseable badges during this work; the guard is on the finished artifact rather than on any one way to break it.
  • Advertised suite size synced 6348 → 6373 across 11 sites in 6 files (the 51 new tests).

Two badges deliberately NOT converted

Badge Why it stays live
CI status Reflects the last run on main. A committed copy would assert "passing" while main was broken — false by construction, not merely stale. Also GitHub-hosted, so not a third-party beacon: the reader already loaded the page from that origin.
OpenSSF Best Practices An external body's live assessment, downgradable with no commit here; freezing it could over-claim a standing we no longer hold. .bestpractices.json additionally justifies displaying their image.

The MCP Toplist badge sits between the two and is committed, because it carries an explicit "as of <month>" stamp: a stale copy is still a true statement about a point in time.

Say the word if you want either of the two converted anyway.


Completion Ledger (§13.2)

Path enumeration — every branch and error arm in the diff

Path Evidence
canonical_licence() — found test_licence_comes_from_the_packaging_metadata
canonical_licence() — both patterns miss → ClaimError test_a_missing_licence_is_an_error_not_a_guess
canonical_python_floor() — found test_python_floor_comes_from_requires_python
canonical_python_floor() — miss → ClaimError test_a_missing_python_floor_is_an_error_not_a_guess
build_badges()test_count present → 5 badges test_every_badge_states_its_canonical_value
build_badges()test_count is None → tests badge omitted, not guessed test_without_a_live_count_no_test_badge_is_built
RepoBadge.render() — provenance/derivation embedded in the file test_every_badge_carries_its_derivation_in_the_file
RepoBadge.render()<title> states the claim test_the_title_states_the_claim_for_screen_readers
write() — target absent or differs → written, returns True test_a_drifted_count_is_rewritten_on_a_real_run
write() — target identical → no write, returns False test_generation_then_check_is_clean
stale() — file missing test_a_missing_badge_is_reported_and_nothing_is_written
stale() — content differs test_a_hand_edited_badge_is_reported_as_stale
stale() — current → None test_generation_then_check_is_clean
main()--check with failures → exit 1 test_a_hand_edited_badge_is_reported_as_stale
main()--check clean → exit 0 test_generation_then_check_is_clean
main() — unreadable canonical source → exit 2, distinct from drift test_an_unreadable_canonical_source_exits_two_not_one
main()--check must not repair what it rejects test_check_does_not_repair_the_file_it_rejects
main() — write mode, changed/unchanged tallies test_a_drifted_count_is_rewritten_on_a_real_run
check_badge() — file missing → fails closed test_a_missing_badge_file_fails_closed
check_badge() — title unmatched → fails closed test_a_badge_without_a_version_title_fails_closed
check_badge() — figure disagrees test_stale_committed_badge_is_reported
check_badge() — agrees → no failure full check_doc_claims.py run, clean
check_no_hotlinked_badges() — hotlink found, with line number test_a_reintroduced_shields_hotlink_is_reported
check_no_hotlinked_badges() — none test_the_readme_hotlinks_no_shields_badge
badge_render.render() — well-formed → returned test_every_badge_is_well_formed_xml
badge_render.render()-- in provenance → BadgeMarkupError test_a_double_hyphen_in_provenance_is_refused
badge_render.render() — markup-bearing text escaped test_markup_bearing_text_is_escaped
Committed artifacts on disk agree with the tree test_committed_badges_agree_with_the_repository, test_every_committed_badge_on_disk_is_well_formed
README references the committed files, not URLs test_the_readme_shows_the_committed_files
CI-status + OpenSSF stay live (negative assertion) test_the_live_status_badges_stay_live

§13.1 checklist

A. Correctness & behavior

  • A1 Happy paths tested end-to-end: 6373 passed, 30 subtests passed in 174.77s locally; 108/108 across the three badge test files.
  • A2 Edge cases: missing file, hand-edited file, unmatched title, absent canonical key, None test count, markup-bearing text, -- in a comment, drifted figure — each row above.
  • A3 Every failure path asserted on its observable effect: ClaimError messages, exit 1 vs exit 2, the specific gate message. No error arm is unexercised.
  • A4 Inputs are repo files at a trust boundary; unreadable/unparseable inputs raise rather than default.
  • A5 Invariants: --check never writes (test_check_does_not_repair_the_file_it_rejects); generation is idempotent (test_generation_then_check_is_clean).
  • A6 Idempotent by construction — write() no-ops when content matches.

B. Concurrency — N/A: single-shot synchronous scripts, no shared mutable state, no async, no external waits.

C. Resources & performance

  • C1 Loops are over a fixed 5-badge list and the README's lines. No unbounded growth.
  • C2 All I/O via Path.read_text/write_text; no handles held, no temp artifacts. Tests use per-test temp dirs.
  • C3 N/A: not a hot path — one static CI step, no measurable budget.

D. Security

  • D1 N/A: no SQL/shell/path construction from data; badge text is XML-escaped through saxutils.escape and the result is re-parsed.
  • D2 Inputs are in-repo files, not untrusted; the escape+reparse guard holds regardless.
  • D3 No secrets touched. Net effect is fewer outbound requests — five third-party beacons removed from every README view.

E. Interfaces & compatibility

  • E1 Additive: one new script, one new flag pair. check_doc_claims.py's CLI is unchanged.
  • E2 Downstream consumers identified by name: .github/workflows/ci.yml (both jobs, wired in this PR), CONTRIBUTING.md and CLAUDE.md (doc-claims workflow, updated), refresh_mcp_toplist_badge.py (shares badge_render, output byte-identical — verified in commit 1).
  • E3 N/A: no persisted data format.
  • E4 N/A: pure text I/O, no path/encoding/process assumptions beyond UTF-8, which is stated explicitly at every call.

F. Observability & operations

  • F1 Every failure prints an actionable message naming the file and the remedy (run scripts/generate_repo_badges.py); asserted by the check-mode tests. Success prints badges OK (N checked).
  • F2 No degraded mode: the tests badge is omitted rather than written from a guess when no count is supplied — the explicit alternative to a silent default, asserted by test_without_a_live_count_no_test_badge_is_built.

G. Tests

  • G1 Ledger above maps every diff path.
  • G2 Both bugs found en route have regression tests: unparseable -- comment → test_a_double_hyphen_in_provenance_is_refused; silently-disabled gate → test_a_missing_badge_file_fails_closed + test_a_badge_without_a_version_title_fails_closed.
  • G3 Deterministic and isolated: _FakeRepo + per-test temp dirs, no fixed paths, no sleeps, no ordering dependence. Full suite green at default parallelism.
  • G4 Negative assertions present: test_the_readme_hotlinks_no_shields_badge, test_the_live_status_badges_stay_live, test_check_does_not_repair_the_file_it_rejects, test_without_a_live_count_no_test_badge_is_built.
  • G5 Full suite run locally, quoted in A1. ruff check + ruff format --check clean on all touched files.

H. Code quality & delivery

  • H1 SOLID: RepoBadge is data + its own rendering, one badge per instance; adding a badge is one list entry, no edit to write/stale/main (OCP). Layering: the generator depends on the canonical readers, never the reverse. Sizes: generator 261 ln, largest function 24 ln, ≤2 params, ≤2 nesting levels. No magic numbers — the one measured constant (5.79 px/char) is sourced in badge_render.py with its measurement date.
  • H2 Each function readable from itself plus its docstring; no dead code, no debug leftovers, no commented-out code.
  • H3 Matches neighboring script conventions (check_doc_claims.py, refresh_mcp_toplist_badge.py); one language per file.
  • H4 CHANGELOG entry added. README.md, CONTRIBUTING.md, CLAUDE.md, docs/ASSURANCE-CASE.md, .bestpractices.json updated in the same PR.
  • H5 Two commits: the behavior-preserving extraction separate from the feature.
  • H6 CI green on the exact pushed tree — pending at time of writing; will not be merged before it is.
  • H7 Boy-scout (§14) — defects seen in touched material, all fixed here, none deferred: the shields-URL regexes that would have gone silent (fixed, not noted); badge_render.render() returning malformed XML (guard + regression test); the stale 6348 count in 11 sites across 6 files (synced). No unfixed, un-issued seen defect remains.

Standing caveat (unchanged from #242)

BADGE_REFRESH_TOKEN is not set, so the monthly MCP-toplist refresh PRs will not trigger required checks until a PAT with repo scope is added. Unrelated to this diff; restated so it does not get lost.

🤖 Generated with Claude Code

https://claude.ai/code/session_01BJt9KN23rUNQzwoJv4jRWw

Base automatically changed from docs/mcp-toplist-badge to main July 28, 2026 20:41
cdeust and others added 2 commits July 28, 2026 23:38
Behavior-preserving. A second generator (generate_repo_badges.py, next
commit) needs the same shields-style geometry the MCP Toplist refresher
already had, so the geometry moves to a sibling module rather than being
copied. What stays in the refresher is what makes that badge THAT badge:
the bar icon, the banner palette, and the wording of its claim.

Path-based sibling import, following the launcher.py/launcher_deps*.py
idiom already in scripts/: it resolves identically whether the file runs as
a script (script dir already on sys.path) or is loaded through
importlib.util.spec_from_file_location by a test, which is how tests_py
loads every script gate.

XML escaping moves INTO the renderer rather than staying at each call site.
The one bug this code has ever shipped was an unescaped '<' from the
"Top <0.1%" tier making the badge unparseable XML; centralising the escape
means a new generator cannot reintroduce it by forgetting.

Proof of preservation: render_badge(964/81919, 2026-07-28) is byte-identical
to the committed assets/badge-mcp-toplist.svg, and the existing 51 tests pass
unchanged (51 passed in 0.82s).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BJt9KN23rUNQzwoJv4jRWw
The README's five repo-derived badges were remote images: a third-party
request fired on every view, and a claim about this repository that its
host could restate at any time with no commit here. Each of those five
figures is determined by the working tree, so a committed SVG can always
be made true again by regenerating it, with no network at all.

scripts/generate_repo_badges.py renders them from the canonical readers
already used by the doc-claims gate — not reimplementations of them, since
a badge that disagreed with the gate is the exact drift this prevents. It
runs as a blocking check on every push and PR: `--check` in the static
job for the four file-derived badges, and `--check --test-count` in the
test job, where the collected count is known.

Converting the badges alone would have silently disabled two gates.
check_doc_claims.py matched the version and test figures by regex over
the shields.io URLs; against the new README those patterns find nothing
and both checks pass quietly. They now read each figure from the
committed SVG's <title> via check_badge(), which fails closed on a
missing file and on a title it cannot match, and check_no_hotlinked_badges()
rejects any reintroduced img.shields.io so the reversion is loud.

Two badges are deliberately left live. The CI status badge reflects the
last run on main; a committed copy would assert "passing" while main was
broken, which is false by construction rather than merely stale, and it
is GitHub-hosted so it is not a third-party beacon. The OpenSSF badge is
an external body's live assessment that can be downgraded without a
commit here, and .bestpractices.json justifies displaying their image.

badge_render.render() now parses its own output and raises
BadgeMarkupError on anything not well-formed: a provenance comment
containing "--" is illegal inside an XML comment and produced five
unparseable badges during this work. The guard sits on the finished
artifact rather than on any one way to break it.

Also syncs the advertised suite size 6348 -> 6373 across 11 sites in 6
files, the count the 51 new tests move it to.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BJt9KN23rUNQzwoJv4jRWw
@cdeust
cdeust force-pushed the docs/self-hosted-badges branch from f22fa65 to 18d4505 Compare July 28, 2026 21:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant