Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions .github/workflows/compatibility.yml
Original file line number Diff line number Diff line change
Expand Up @@ -66,3 +66,12 @@ jobs:
for tests in compatibility/consumers/*/tests; do
python -m pytest "$tests"
done
- name: Record dated compatibility evidence
run: python scripts/record_compatibility_evidence.py > compatibility-evidence.json
- name: Upload compatibility evidence
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: base-cli-compatibility-evidence-${{ github.run_id }}
path: compatibility-evidence.json
if-no-files-found: error
retention-days: 90
4 changes: 4 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,10 @@ and versions are tracked in the repo-root `VERSION` file.
attestations to protected release workflows.
- Add a generated public API reference and a strict CI drift check so every
stable facade export remains searchable and documented.
- Add a permissioned-adopter evidence policy and dated compatibility-run
artifacts without presenting maintained fixtures as customer adoption.
- Add a permissioned-adopter evidence policy and dated compatibility-run
artifacts without presenting maintained fixtures as customer adoption.

### Changed

Expand Down
3 changes: 3 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,9 @@ python -m pip install base-cli
Read the complete documentation at
<https://basefoundry.github.io/base-cli/>.

Compatibility results and the permissioned-adopter policy are documented in
the [adoption and compatibility evidence guide](https://basefoundry.github.io/base-cli/adoption-evidence/).

## Quick start

Create a small command with a consistent context, logging, and cleanup
Expand Down
17 changes: 12 additions & 5 deletions compatibility/README.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,10 @@
# Downstream compatibility consumers

These are three independent, maintainable consumer fixtures used as adoption
evidence until a non-Base team grants permission for a public case study. They
are deliberately separate packages with separate names, entry points, and test
suites; none imports another fixture or Base product code.
These are three maintained reference consumer fixtures used as compatibility
evidence. They are deliberately separate packages with separate names, entry
points, and test suites; none imports another fixture or Base product code.
They are not independent adopters and must not be presented as customer
outcomes.

| Consumer | Shape | Use case | Compatibility outcome |
| --- | --- | --- | --- |
Expand All @@ -20,6 +21,12 @@ own dependencies, and runs each consumer's tests. This catches import,
packaging, adapter, and contract regressions without relying on repository
source imports.

The downstream job records a dated JSON result as the
`base-cli-compatibility-evidence-<run-id>` artifact. It binds the result to the
framework revision and version and lists the exact fixture and matrix that
passed. See [`adoption-evidence.md`](../docs/adoption-evidence.md) for the
claim and permission boundary.

The same workflow runs the Typer adapter and Beacon fixture against Typer
0.25.1, 0.26.0, and 0.27.1 on Python 3.10 through 3.14. This matrix covers the
transition from Click's public command classes to Typer's vendored Click fork.
Expand All @@ -35,4 +42,4 @@ for tests in compatibility/consumers/*/tests; do python -m pytest "$tests"; done
```

The fixtures are not customer claims. A permissioned public adopter can be
substituted in the manifest while retaining the same downstream contract tests.
added as a separate record while retaining the same downstream contract tests.
12 changes: 6 additions & 6 deletions docs/adopter-readiness.md
Original file line number Diff line number Diff line change
Expand Up @@ -73,18 +73,18 @@ release-blocking finding back to the compatibility register.

## Evidence and downstream compatibility

The repository does not claim a customer identity without permission. Until an
external team authorizes a public case study, three independent consumer
fixtures provide the reviewable evidence:
The repository does not claim a customer identity without permission. Three
maintained reference fixtures provide compatibility evidence, but they are not
independent adopters or customer outcomes:

- **Atlas** — a Click inventory command migrated without rebuilding its tree;
- **Beacon** — a typed Typer deployment command using the optional adapter; and
- **Cinder** — a scheduled reconciliation command with dry-run and JSON output.

Each fixture has its own package metadata and tests, is installed against the
published framework wheel in CI, and records a stable invocation outcome. A
permissioned adopter can replace a fixture with a public case study without
changing the compatibility test contract.
published framework wheel in CI, and records a stable invocation outcome. The
[adoption and compatibility evidence](adoption-evidence.md) guide defines the
permissioned adopter program and the public-claim boundary.

## Adoption friction and release gate

Expand Down
67 changes: 67 additions & 0 deletions docs/adoption-evidence.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
# Adoption and compatibility evidence

This page separates verifiable framework evidence from adoption claims. It is
part of the v1.0 release gate: every public statement must be traceable to a
repository file, a dated CI run, or a permissioned adopter record.

## Current status

As of the latest review, **zero independent external adopters are publicly
confirmed**. The repository maintains three reference consumer fixtures—Atlas,
Beacon, and Cinder—to exercise the compatibility boundary. They are useful
engineering evidence, but they are not customer outcomes and are not counted
as adoption.

The independent-adopter target is three permissioned outcomes. Recruitment is
**not started** until a team opts in through the support channel. No name,
metric, logo, download count, or case-study language is published without
written permission from that team.

## Evidence categories

| Category | What it proves | Source | Public claim allowed |
| --- | --- | --- | --- |
| Reference fixture | A maintained consumer shape remains compatible | [`compatibility/README.md`](https://github.com/basefoundry/base-cli/blob/main/compatibility/README.md) and CI | “The fixture passed.” |
| Compatibility run | A specific revision/version passed a dated matrix | `base-cli-compatibility-evidence-<run-id>` artifact | “Version X passed run Y.” |
| External adopter | A real independent team completed an agreed outcome | Permissioned adopter record | Only the approved case-study facts |

Reference fixtures must remain separate packages with their own metadata and
tests. Base, base-demo, and other adjacent repositories are explicitly
excluded from the independent-adopter count.

## Adopter program

1. **Recruit:** offer a short evaluation against the [adopter readiness
checklist](adopter-readiness.md); record only a consented contact and the
intended CLI shape.
2. **Onboard:** pin a supported minor release, run the installed-wheel smoke
test, and capture migration friction without receiving private credentials
or production data.
3. **Verify:** run the adopter's agreed compatibility and contract tests; keep
the framework revision, package versions, platform, and dated result.
4. **Publish (optional):** request written approval for an anonymized metric or
named case study. A refusal or incomplete pilot remains a valid private
outcome and is never presented as adoption.
5. **Review quarterly:** re-confirm consent, compatibility status, and the
exact wording of every public claim.

The support issue template is the intake path. Security reports and private
reproductions must follow [`SECURITY.md`](https://github.com/basefoundry/base-cli/blob/main/SECURITY.md).

## Reproducible compatibility records

Every successful `Reference consumers` workflow can emit a JSON record named
`base-cli-compatibility-evidence-<run-id>`. It contains the source revision,
framework version, fixture names, Python/Typer matrix, timestamp, and the
commands needed to repeat the check. Download a record from a run with:

```bash
gh run download <run-id> \
--repo basefoundry/base-cli \
--name base-cli-compatibility-evidence-<run-id> \
--dir compatibility-evidence
```

The record is an engineering result, not a customer claim. Before publishing
an external outcome, maintainers must link its permissioned source record and
make the approved scope clear.
2 changes: 2 additions & 0 deletions docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,8 @@ application.
with the underlying parser and decide whether its lifecycle boundary fits.
- Start with the [adopter readiness guide](adopter-readiness.md) for a
production evaluation.
- See the [adoption and compatibility evidence guide](adoption-evidence.md)
for dated CI results and the permissioned-adopter program.
- Read [API stability](api-stability.md) and the [migration guide](migrations.md)
before upgrading across a compatibility boundary.
- Browse the [complete public API reference](api-reference.md) for exported
Expand Down
1 change: 1 addition & 0 deletions mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@ nav:
- Getting started:
- Framework choice: framework-choice.md
- Adopter readiness: adopter-readiness.md
- Adoption evidence: adoption-evidence.md
- Platform support: platform-support.md
- Consumer profiles: consumer-profiles.md
- API and contracts:
Expand Down
58 changes: 58 additions & 0 deletions scripts/record_compatibility_evidence.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
#!/usr/bin/env python3
"""Write a dated, reproducible record for a compatibility workflow run."""

from __future__ import annotations

import json
import os
import subprocess
from datetime import datetime, timezone
from pathlib import Path
from typing import Any

ROOT = Path(__file__).resolve().parents[1]
FIXTURES = ("atlas_click", "beacon_typer", "cinder_automation")
PYTHON_VERSIONS = ("3.10", "3.11", "3.12", "3.13", "3.14")
TYPER_VERSIONS = ("0.25.1", "0.26.0", "0.27.1")


def _revision() -> str:
value = os.environ.get("GITHUB_SHA")
if value:
return value
try:
return subprocess.check_output(["git", "rev-parse", "HEAD"], cwd=ROOT, text=True).strip()
except (OSError, subprocess.CalledProcessError):
return "unknown"


def _version() -> str:
return (ROOT / "VERSION").read_text(encoding="utf-8").splitlines()[0].strip()


def main() -> None:
now = datetime.now(timezone.utc).replace(microsecond=0).isoformat().replace("+00:00", "Z")
record: dict[str, Any] = {
"schema_version": 1,
"status": "passed",
"recorded_at": now,
"source_revision": _revision(),
"framework_version": _version(),
"workflow": os.environ.get("GITHUB_WORKFLOW", "local"),
"run_id": os.environ.get("GITHUB_RUN_ID"),
"run_attempt": os.environ.get("GITHUB_RUN_ATTEMPT"),
"ref": os.environ.get("GITHUB_REF_NAME", "local"),
"fixtures": list(FIXTURES),
"matrix": {"python": list(PYTHON_VERSIONS), "typer": list(TYPER_VERSIONS)},
"reproduce": [
"python scripts/validate_consumers.py",
"python -m build --wheel",
"python -m pytest compatibility/consumers/*/tests",
],
"claim_policy": "Reference fixtures demonstrate compatibility only; they are not external adoption claims.",
}
print(json.dumps(record, indent=2, sort_keys=True))


if __name__ == "__main__":
main()
4 changes: 2 additions & 2 deletions scripts/validate_consumers.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
#!/usr/bin/env python3
"""Validate the three independent downstream consumer fixtures."""
"""Validate the three maintained downstream reference consumer fixtures."""

from __future__ import annotations

Expand Down Expand Up @@ -65,7 +65,7 @@ def main() -> None:
seen.add(str(record["slug"]))
if seen != set(EXPECTED_SLUGS):
fail(f"manifest slugs must be {EXPECTED_SLUGS!r}")
print(f"Validated {len(records)} independent downstream consumer fixtures.")
print(f"Validated {len(records)} downstream reference consumer fixtures.")


if __name__ == "__main__":
Expand Down
2 changes: 2 additions & 0 deletions tests/validate.sh
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@ required_files=(
docs/security-threat-model.md
docs/security-review.md
docs/adopter-readiness.md
docs/adoption-evidence.md
MANIFEST.in
scripts/validate_package_artifact.py
scripts/validate_installed_package.py
Expand All @@ -39,6 +40,7 @@ required_files=(
scripts/validate_consumers.py
scripts/generate_release_metadata.py
scripts/validate_release_metadata.py
scripts/record_compatibility_evidence.py
scripts/benchmark_runtime.py
tests/conftest.py
compatibility/README.md
Expand Down
Loading