ci: add Apache RAT licence-header check with SPDX stamping for Markdown#777
Merged
Conversation
Adds an Apache RAT (Release Audit Tool) gate that runs on every PR and push to main, verifying every source file carries an approved licence header — the same check ASF Infra runs on release artefacts, now caught before merge instead of at release time. RAT treats a file with no recognised header as unapproved. Python, shell, YAML and other comment-bearing sources already carry the full ASF header; Markdown carries none by convention, so it needs an SPDX identifier RAT recognises as approved. Changes: - .github/workflows/rat.yml — downloads Apache RAT 0.18 from archive.apache.org (permanent, version-pinned) with a pinned sha512, runs it over the tree honouring .gitignore, and drives pass/fail off the unapproved-licenses report so a failure lists the offending files rather than a Java stack trace. `actions/*` are implicitly on the ASF allowlist; actions are SHA-pinned per repo convention. - .rat-excludes — intentional header-less files: generated lock files, empty package markers, PEP 561 markers, sync-state, and the third-party-licence detection fixtures that deliberately contain non-Apache (GPL / EPL / …) licence text as eval input. - tools/dev/add-license-headers.py + a prek hook — stamps an SPDX Apache-2.0 identifier and licence URL into Markdown: a two-line HTML comment for plain Markdown, and the same two lines as YAML comments inside the front matter for files that open with `---` (so line 1 stays `---` and no key is added for a validator to reject). Idempotent: a file that already carries any SPDX-License-Identifier declaration is left untouched. The hook fixes files in place and fails the commit on change (the end-of-file-fixer convention). - Stamps the 1767 Markdown files that lacked a header (271 already had one and were skipped). - Adds the ASF header to six comment-bearing config files (.gitattributes, .lychee.toml, .typos.toml, .zizmor.yml, .rat-excludes, default-config.yaml) so RAT approves them too. Verified: RAT reports 0 unapproved; prek add-license-headers, skill-and-tool-validate, markdownlint, and doctoc all pass; the new script passes ruff / mypy. Generated-by: Claude Code (Claude Opus 4.8)
added 2 commits
July 8, 2026 01:06
The list-workspace-members job exported the PR's changed-file list into
the CHANGED environment variable, then exec'd python3. On a repo-wide PR
(the licence-header sweep in this change touches ~1776 files, ~160 KB of
paths) that env var pushes the process environment past the exec size
limit, so python3 dies with "Argument list too long" (exit 126) before
the matrix is computed.
Write the list to ${RUNNER_TEMP} and pass only the file path in the
environment; python reads the file. The env stays small regardless of PR
size. No behaviour change for normal PRs.
Generated-by: Claude Code (Claude Opus 4.8)
…acts The download step unpacked the RAT tarball into the checkout and the run step wrote rat-report.txt / rat-stderr.log there too, so RAT scanned its own working files — rat-stderr.log has no licence header, tripping Unapproved:1 and failing the job. Do all RAT work under RUNNER_TEMP, outside the scanned tree. Also gitignore the artefacts so a local in-checkout run is excluded via --input-exclude-parsed-scm GIT. Generated-by: Claude Code (Claude Opus 4.8)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
main,verifying every source file carries an approved licence header — the ASF
Infra release check, caught before merge instead of at release time.
tools/dev/add-license-headers.py(+ a prek hook) that stamps an SPDX Apache-2.0 identifier into Markdown, and
stamp the 1767 files that lacked one (271 already had a header, skipped).
.rat-excludes; add the ASF header tosix comment-bearing config files.
Type of change
prek, workflows, validators)docs/,README.md,CONTRIBUTING.md) — SPDX header stampingTest plan
prek run --all-filespasses (add-license-headers, markdownlint, doctoc,skill-and-tool-validate, check-placeholders, vendor-neutrality all green)
ruff check/ruff format --check/mypydouble-stamps a pre-existing header)
RFC-AI-0004 compliance
check-placeholdersprek hook passesLinked issues
Notes for reviewers
.github/workflows/rat.yml,.rat-excludes,tools/dev/add-license-headers.py,.pre-commit-config.yaml, and 5 configfiles given ASF headers. The remaining ~1767 files are the mechanical SPDX
stamp (
<!-- SPDX-License-Identifier: Apache-2.0 / URL -->, or the same as aYAML comment inside front matter so line 1 stays
---).archive.apache.org(permanent, version-pinned) witha pinned sha512; bump instructions are in the workflow header.
.rat-excludescovers generated lockfiles, empty package/PEP-561 markers,sync-state, and the third-party-licence detection fixtures under
step-4-third-party-license/— those deliberately contain GPL/EPL text as evalinput, so they are neither stamped nor RAT-scanned.
Generated-by: Claude Code (Claude Opus 4.8)