Skip to content

Add E2E integration tests with testdata fixtures #52

@Ilyes512

Description

@Ilyes512

Parent: #42 (architectural review)

Problem

Test coverage is broad in unit-test breadth (~37 *_test.go files) but thin on end-to-end flows. The existing tests in pkg/cmd/*_test.go exercise individual commands, but:

  • There is no integration test that drives specs use github:foo/bar ./out against a fixture template through the actual cobra command tree.
  • There is no canonical testdata/ of sample templates shared across packages. Each test rolls its own ad-hoc template via t.TempDir() and os.WriteFile.
  • pkg/util/git/ integration tests are gated with //go:build integration and require network access — appropriate, but the rest of the surface has no equivalent harness.
  • Build tag conventions are fine; the gap is in fixture sharing and CLI-level coverage.

Proposed direction

  1. Create testdata/ at the repo root (or internal/testdata/) with at least these canonical fixtures:

    • minimal/ — one file, two variables, no conditionals.
    • conditional/ — variables that gate other variables, plus directory-level conditionals via empty path segments.
    • computed/ — at least three computed values with an interdependency graph that exercises the topo sort.
    • hooks/ — pre-use and post-use hooks, both inline (project.yaml) and file-based (hooks/*.sh).
    • verbatim/.specsverbatim file plus a binary asset and a file with literal {{ that must not be parsed.
    • delimiters/ — uses __delimiters: { left: "[[", right: "]]" } to verify custom delimiters end-to-end.
  2. Add pkg/cmd/integration_test.go (build tag //go:build integration or just standard tests if they don't need network) that drives the cobra command tree:

    • Execute against an in-process fixture (using the local-path source) with stdin/stdout/stderr pipes.
    • Assert on the produced filesystem tree (via fs.WalkDir + golden files or cmp.Diff).
    • Cover: template download (against a local path), template use, specs use (one-shot), template list, template upgrade, template validate.
  3. Refactor existing unit tests to consume the shared fixtures where it reduces churn. Don't force a single style — unit tests that exercise one focused branch should stay narrow.

  4. CI: keep the existing go test -race ./... lane. Add an integration lane (no network — uses local-path fixtures only) so CI catches end-to-end breakage.

Acceptance criteria

  • testdata/ exists with the six fixtures listed above (or a documented subset, justified).
  • At least one E2E test per command in pkg/cmd that drives the cobra tree end-to-end against a local-path fixture.
  • Integration tests run without network access (no real GitHub clones).
  • CI runs both unit and integration lanes on every PR.
  • Existing tests still pass; intermittent or filesystem-leaky tests are flagged or fixed.

Out of scope

  • Replacing the unit test suite.
  • Network-dependent E2E tests against real GitHub (that lane already exists for pkg/util/git).

References

  • pkg/cmd/testhelpers_test.go — current shared helpers (small)
  • pkg/util/git/ — example of build-tag-gated integration tests
  • Architecture docs sample patterns: docs/content/docs/architecture/template-engine.md, computed-values.md

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestgoPull requests that update go code

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions