test(cluster): envtest coverage for verb-layer round-trips (up + down + list)

[bdchatham]

Problem

Today's envtest only covers the kube/ primitives (Apply, List, Delete) at cluster/internal/kube/{apply,list_delete}_envtest_test.go. The verb layer (runBenchUp, runBenchDown, runChainDown, runRPCDown, runBenchList) is only unit-tested with stubbed deps — meaning a real apiserver never sees rendered manifests, and the bug classes envtest is best at catching slip through.

Specifically uncovered:

• Template → SSA round-trip bugs (invalid manifests slip past unit tests)
• Label/selector contract drift between up (emitter) and down/list (consumer)
• Job-immutability guard's actual SSA behavior
• still-terminating propagation through the verb's Hint field
• Selector-scoping regressions in down (catastrophic class — wrong-engineer destruction)

Impact

Pre-MVP regression risk. The recent sei.io/image-sha label→annotation move and the sei.io/component rename in PR #106 are exactly the bug class round-trip envtest catches and unit tests cannot. Without this, every label/selector edit ships unguarded for at least one PR cycle.

Relevant experts

• kubernetes-specialist — owned the design in this session; owns CRD vendoring decisions, test shape, envtest mechanics.

Proposed approach

Ten-test MVP slate, ~12-15s runtime, follows existing patterns at cluster/internal/kube/{apply,list_delete}_envtest_test.go.

runBenchUp (4):

1. TestBenchUp_AppliesAllFourDocsInOrder — locks template→SSA + label/annotation contract.
2. TestBenchUp_IdempotentReapplyMarksUnchanged — SSA happy path.
3. TestBenchUp_JobImmutabilityGuardBlocksImageChange — pre-seed Job, mutate digest, expect typed error before SNDs touched.
4. TestBenchUp_JobImmutabilityGuardBlocksDurationChange — second axis of guard.

runBenchDown (3):

5. TestBenchDown_DryRunListsAllFourKinds — dry-run, no Delete calls.
6. TestBenchDown_AppliesCascadingDelete — round-trip via real runBenchUp; seed sei.io/engineer=alice decoy, verify untouched.
7. TestBenchDown_StillTerminatingHintWhenFinalizerHolds — finalizer-stuck → Hint reports it, DeletedAt nil.

Selector-scoping for chain/rpc down (2):

8. TestChainDown_DoesNotDeleteBenchJobs — chain + bench in same namespace, run chain down, bench resources untouched.
9. TestRPCDown_DoesNotDeleteValidator — rpc + validator in same namespace, run rpc down, validator untouched.

runBenchList (1):

10. TestBenchList_OnlyShowsCallerOwnedResources — seed bdc + alice, list as bdc, alice's bench filtered.

Infrastructure additions

• Vendor SND CRD snapshot at cluster/internal/kube/testdata/crds/seinodedeployment.yaml, sourced from sei-k8s-controller main HEAD at vendor time. Record upstream commit SHA in cluster/internal/kube/testdata/crds/.source for traceability.
• make sync-crds target — pulls fresh snapshot from pinned commit.
• cluster/envtest_main_test.go — package TestMain mirroring kube/apply_envtest_test.go; realBenchDeps helper wires real apply paths against testenv.WriteKubeconfig.
• Extend make test-envtest to include ./cluster/....
• One-line guard in CRD-touching test files: // Use unstructured.Unstructured here — do not import sei-k8s-controller/api/* (would close the dep cycle with sidecar/client).

Architectural constraints

• All test code uses unstructured.Unstructured, never typed seiv1alpha1.SeiNodeDeployment{} Go structs. Importing the controller's typed API would create a circular Go module dependency: sei-k8s-controller already imports github.com/sei-protocol/seictl/sidecar/client (per CLAUDE.md), so adding the controller's typed API to seictl would close the cycle.
• internal/testenv stays seictl-agnostic per its existing docstring (internal/testenv/testenv.go:1-9). Anything seictl-shaped lives in cluster/envtest_main_test.go. This preserves the sei-protocol/platform#118 extraction path to a shared sei-k8s-testkit module.
• Tests live in package cluster (not cluster_test) for unexported access to runBenchUp etc. — avoids exporting a test-only public surface.
• Tests use the verb's actual renderManifests rather than golden fixtures — coupling to render details is the feature; catches template bugs.

One-way doors

• CRD snapshot path cluster/internal/kube/testdata/crds/seinodedeployment.yaml — locks every test to this path. Implementer may push back if cluster/testdata/crds/ (verb-scope rather than kube-scope) makes more sense.
• Test file naming <verb>_envtest_test.go — sibling to <verb>_test.go.
• In-package residence (package cluster).

Acceptance criteria

• All 10 tests pass under make test-envtest locally and in CI.
• Vendored CRD at cluster/internal/kube/testdata/crds/seinodedeployment.yaml with .source recording upstream commit SHA.
• make sync-crds target works against sei-k8s-controller main.
• CI envtest job (.github/workflows/envtest.yml) includes ./cluster/....
• No new imports of github.com/sei-protocol/sei-k8s-controller/... in seictl's go.mod.
• internal/testenv/testenv.go remains seictl-agnostic (no seictl imports added).
• Header guard comment present on the CRD-touching test files.

Out of scope (deferred with triggers)

• runChainUp / runRPCUp envtest coverage entirely — defer until chain/rpc grow logic beyond SND wrappers (scaling, migration). runBenchUp exercises the same render+apply path.
• Status-injection tests for bench list aggregation — defer until a regression slips through the existing aggregateBenchSummaries unit test. Avoids needing a PatchSubresource helper.
• CRD-validation-fidelity tests — defer until upstream CRD tightens and our templates drift.
• All-namespaces aggregation tests — defer until a multi-namespace operational story emerges.
• Foreign-field-manager conflict tests at the verb level — already covered at the kube layer.
• Profile-CM endpoint-count test — defer until seiload profile rendering grows complexity.
• WaitForCRDEstablished and PatchSubresource helpers in internal/testenv — defer along with their tests.
• Migration of internal/testenv → pkg/testenv (or extraction to sei-protocol/sei-k8s-testkit) — separate workstream tracked in sei-protocol/platform#118; deferred until controller-side envtest needs the extraction.

References

• Coral synthesis with kubernetes-specialist produced the test slate and CRD strategy.
• Existing patterns: cluster/internal/kube/apply_envtest_test.go, cluster/internal/kube/list_delete_envtest_test.go.
• Harness: internal/testenv/testenv.go (already exposes WithCRDPaths, RESTConfig, WriteKubeconfig, UniqueNamespace).
• CI: .github/workflows/envtest.yml, Makefile test-envtest target.
• Related: sei-protocol/platform#118 (testkit extraction).
• Locked-contract drift this would catch: PR fix(rpc): cross-review followups — envelope rename, default→primary, brittle-test guard #106 (label→annotation move on sei.io/image-sha, sei.io/component rename).