Skip to content

Go Edition: CI integration — acceptance test matrix (both editions x all platforms) #504

@Zordrak

Description

@Zordrak

Summary

Integrate the acceptance test suite into CI with a matrix that tests both Bash and Go editions across all supported platforms.

Parent Epic

Part of #488 — Go Edition: Full Feature Parity Implementation

Motivation

The acceptance test matrix is the enforcement mechanism for feature parity. CI must run the shared acceptance test suite against both editions on every PR and push. If either edition fails on any platform, the build is red. This is the quality gate that ensures the Go edition achieves and maintains parity with the Bash edition.

Clean-Room Constraint

This is a clean-room implementation. Contributors MUST NOT read, reference, copy, or adapt source code from tofuutils/tenv, hashicorp/hc-install, or any other third-party tfenv-like tool. The sole reference is tfenv's own Bash source code, documentation, and test suite.

Proposed Design

CI Matrix

strategy:
  matrix:
    edition: [bash, go]
    os: [ubuntu-latest, macos-latest, windows-latest]

For push to master, the expanded matrix includes:

  • ubuntu-24.04, ubuntu-22.04, macos-14, macos-13, windows-2025, windows-2022

Workflow Steps

  1. Checkout repository
  2. Set up Go (for test runner and Go edition build)
  3. Build Go edition (if testing Go edition): go build -o tfenv-go ./cmd/tfenv
  4. Set TFENV_TEST_BINARY:
    • Bash edition: ${{ github.workspace }}/bin/tfenv
    • Go edition: ${{ github.workspace }}/go/tfenv-go
  5. Run acceptance tests: go test -v -race -count=1 ./go/test/acceptance/...
  6. Run Go unit tests (Go edition only): go test -v -race -count=1 ./go/internal/...
  7. Upload test results as artifacts

Integration Tests

A separate job (or matrix dimension) for integration tests tagged //go:build integration that hit real releases.hashicorp.com. These run on a schedule (nightly) or on-demand, not on every PR.

Existing Tests

The existing Bash test suite in test/ continues to run as before — it is not removed or replaced. The new acceptance suite runs alongside it. Over time, the acceptance suite subsumes the legacy tests.

Go Build Cache

Use GitHub Actions Go build cache to speed up repeated builds.

Acceptance Criteria

  • CI workflow runs acceptance tests against Bash edition on all platforms
  • CI workflow runs acceptance tests against Go edition on all platforms
  • CI workflow runs Go unit tests with -race flag
  • Both editions must pass for the build to be green
  • Test results are visible in GitHub Actions output (not just pass/fail)
  • Integration tests (real network) run on schedule, not on every PR
  • Existing Bash test suite continues to run (not replaced)
  • Workflow uses Go build cache for performance
  • Windows tests handle .exe extension correctly
  • Workflow is defined in .github/workflows/ (new file or extension of existing)
  • Failed tests produce clear, actionable output in CI logs

Dependencies

Implementation Notes

  • Reference .github/workflows/test.yml for the existing CI configuration
  • The ADR specifies this is a NEW workflow or extension — coordinate with maintainer before modifying existing workflows (per AGENTS.md: "Do not modify .github/workflows/ without explicit approval")
  • Consider whether to add a new workflow file (e.g., acceptance-test.yml) or extend the existing test.yml
  • TFENV_TEST_BINARY must be an absolute path in CI
  • On Windows, the Go binary will be tfenv-go.exe — tests must handle this
  • The acceptance tests use TFENV_REMOTE to point at the mock server for most tests — CI does not need special network access for these
  • Consider adding a badge to README.md for the acceptance test status

Labels

type:feature, priority:high, complexity:medium

Metadata

Metadata

Assignees

No one assigned

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions