Skip to content

Update e2e tests#400

Closed
nikatza wants to merge 24 commits into
mainfrom
integration-tests
Closed

Update e2e tests#400
nikatza wants to merge 24 commits into
mainfrom
integration-tests

Conversation

@nikatza

@nikatza nikatza commented Jun 10, 2026

Copy link
Copy Markdown
Contributor

E2e tests update

Update tests to validate the full reconciliation pipeline from Kubernetes CRD to gNMI JSON output.

Rationale

Unit tests verify individual functions but miss the gaps between layers: CRD validation, controller logic, provider field mapping, and gNMI path construction. For example, a controller passing the wrong value to a provider.

The current e2e tests cover this but rely on Kind clusters with a full operator installation. This PR refactors both test modes to share the same testdata-driven approach:

  • Envtest mode — runs without infrastructure using envtest + in-process gnmi-test-server. Tests run sequentially but complete in seconds.
  • Cluster mode — runs against Kind with full operator installation. Validates RBAC, webhooks, metrics. Each test deploys its own gnmi-test-server pod in its own namespace.

Test Format

Each test is a txtar file in test/e2e/testdata/<provider>/:

-- state/preload --              # OPTIONAL: initial gNMI state
{"System": {"procsys-items": {"bootTime": "1700000000"}}}

-- Interface/lo0 --
apiVersion: networking.metal.ironcore.dev/v1alpha1
kind: Interface
spec:
  deviceRef:
    name: device                 # substituted at runtime
  ...

-- state/expect --
{"System": {"intf-items": ...}}

Resources are created sequentially, waiting for Configured (or Ready if the former does not exist) condition before proceeding. This handles dependencies between resources (e.g., BGPPeer requires BGP).

Note: The JSON stored in gnmi-test-server may differ from actual provider output. For example, NXOS DME_UNSET_PROPERTY_MARKER values are filtered out by the test server.

Usage

# Fast, no cluster required
make test-e2e-envtest PROVIDER=cisco-nxos-gnmi

# Full cluster mode (requires Kind)
make test-e2e PROVIDER=cisco-nxos-gnmi

Key Changes

  • Unified TestEnvironment interface in testutil/ — both modes implement the same contract
  • Cluster mode now deploys a gnmi-test-server pod per test for isolation
  • Shared testdata discovery — both modes auto-discover test/e2e/testdata/<provider>/*.txt
  • Added PreloadGNMIState() for setting initial device state before reconciliation
  • Added state/preload txtar section for tests that need pre-existing gNMI state
  • Add more samples

@nikatza nikatza force-pushed the integration-tests branch 3 times, most recently from 9393d5b to 2e3e8bb Compare June 10, 2026 23:31
@nikatza nikatza changed the title Integration tests Integration tests via envtest and fake gNMI server Jun 10, 2026
@nikatza nikatza force-pushed the integration-tests branch 4 times, most recently from ad16ab5 to d46ae90 Compare June 11, 2026 11:47
@nikatza nikatza changed the title Integration tests via envtest and fake gNMI server Update e2e tests Jun 16, 2026
@nikatza nikatza force-pushed the integration-tests branch from c3069ef to e6a30bf Compare June 16, 2026 13:36
nikatza added 19 commits June 16, 2026 17:02
Integration tests validate the full reconciliation pipeline from Kubernetes CRD
to gNMI JSON output. Uses envtest with an in-process gNMI server that accumulates
state, allowing tests to run in seconds without real network infrastructure.

Tests auto-discover from testdata/<provider>/ directories, enabling multi-provider
coverage. Each test uses txtar format: K8s YAML resources followed by expected
gNMI JSON state. Resources are created sequentially, waiting for readiness
conditions before proceeding to handle dependencies between resources.

JSON comparison uses normalizeJSON() to sort arrays recursively, avoiding flaky
tests from non-deterministic field ordering in gNMI responses.

Cleanup removes finalizers in two passes before deletion. This prevents
controllers from re-reconciling during teardown, which would cause conflicts
with the test's expected clean state.

Add WithBindAddress option to allow binding to 0.0.0.0 for container deployments.
The server defaults to 127.0.0.1 for safe in-process tests, but when running as
a standalone server in a pod, it needs to listen on all interfaces.

Signed-off-by: Pujol <enric.pujol@sap.com>
Explain test architecture, txtar format, and how to add new tests.

Signed-off-by: Pujol <enric.pujol@sap.com>
Signed-off-by: Pujol <enric.pujol@sap.com>
Signed-off-by: Pujol <enric.pujol@sap.com>
Signed-off-by: Pujol <enric.pujol@sap.com>
Tests three interface types: Loopback, Physical L3 with BFD, Aggregate L2 with vPC.

Signed-off-by: Pujol <enric.pujol@sap.com>
Signed-off-by: Pujol <enric.pujol@sap.com>
Signed-off-by: Pujol <enric.pujol@sap.com>
Signed-off-by: Pujol <enric.pujol@sap.com>
Signed-off-by: Pujol <enric.pujol@sap.com>
Signed-off-by: Pujol <enric.pujol@sap.com>
Signed-off-by: Pujol <enric.pujol@sap.com>
OSPF and PIM require interfaces to be configured first. Test files include
the interface dependency.

Signed-off-by: Pujol <enric.pujol@sap.com>
Signed-off-by: Pujol <enric.pujol@sap.com>
Signed-off-by: Pujol <enric.pujol@sap.com>
Signed-off-by: Pujol <enric.pujol@sap.com>
Tests BGP router + peer configuration with loopback interface and
BGPConfig.  Resources packed together since BGPPeer requires BGP to
exist first.

Signed-off-by: Pujol <enric.pujol@sap.com>
Signed-off-by: Pujol <enric.pujol@sap.com>
Signed-off-by: Pujol <enric.pujol@sap.com>
nikatza added 5 commits June 16, 2026 17:02
Signed-off-by: Pujol <enric.pujol@sap.com>
Signed-off-by: Pujol <enric.pujol@sap.com>
Signed-off-by: Pujol <enric.pujol@sap.com>
Signed-off-by: Pujol <enric.pujol@sap.com>
@nikatza nikatza force-pushed the integration-tests branch from e6a30bf to 7390aa7 Compare June 16, 2026 15:02
@github-actions

Copy link
Copy Markdown

Merging this branch will not change overall coverage

Impacted Packages Coverage Δ 🤖
github.com/ironcore-dev/network-operator/internal/transport/gnmiext 90.21% (ø)
github.com/ironcore-dev/network-operator/test/e2e 0.00% (ø)
github.com/ironcore-dev/network-operator/test/e2e/testutil 0.00% (ø)
github.com/ironcore-dev/network-operator/test/gnmi 0.00% (ø)
github.com/ironcore-dev/network-operator/test/gnmi/testserver 0.00% (ø)

Coverage by file

Changed files (no unit tests)

Changed File Coverage Δ Total Covered Missed 🤖
github.com/ironcore-dev/network-operator/test/e2e/testutil/cluster.go 0.00% (ø) 0 0 0
github.com/ironcore-dev/network-operator/test/e2e/testutil/doc.go 0.00% (ø) 0 0 0
github.com/ironcore-dev/network-operator/test/e2e/testutil/envtest.go 0.00% (ø) 0 0 0
github.com/ironcore-dev/network-operator/test/e2e/testutil/helpers.go 0.00% (ø) 0 0 0
github.com/ironcore-dev/network-operator/test/e2e/testutil/provider.go 0.00% (ø) 0 0 0
github.com/ironcore-dev/network-operator/test/gnmi/main.go 0.00% (ø) 0 0 0
github.com/ironcore-dev/network-operator/test/gnmi/testserver/server.go 0.00% (ø) 0 0 0

Please note that the "Total", "Covered", and "Missed" counts above refer to code statements instead of lines of code. The value in brackets refers to the test coverage of that file in the old version of the code.

Changed unit test files

  • github.com/ironcore-dev/network-operator/internal/transport/gnmiext/client_test.go
  • github.com/ironcore-dev/network-operator/test/e2e/cluster_suite_test.go
  • github.com/ironcore-dev/network-operator/test/e2e/cluster_test.go
  • github.com/ironcore-dev/network-operator/test/e2e/e2e_test.go
  • github.com/ironcore-dev/network-operator/test/e2e/envtest_suite_test.go
  • github.com/ironcore-dev/network-operator/test/e2e/envtest_test.go
  • github.com/ironcore-dev/network-operator/test/e2e/suite_test.go
  • github.com/ironcore-dev/network-operator/test/e2e/util_test.go

@nikatza

nikatza commented Jun 16, 2026

Copy link
Copy Markdown
Contributor Author

Closing this PR as it is replaced by smaller ones.

@nikatza nikatza closed this Jun 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant