Skip to content

Reduce K8s test resource usage with single-node Kind clusters#69459

Draft
nailo2c wants to merge 1 commit into
apache:mainfrom
nailo2c:single-node-kind-k8s-tests
Draft

Reduce K8s test resource usage with single-node Kind clusters#69459
nailo2c wants to merge 1 commit into
apache:mainfrom
nailo2c:single-node-kind-k8s-tests

Conversation

@nailo2c

@nailo2c nailo2c commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

The K8s test Kind clusters have used a control-plane + worker topology since the original Kind migration in 2019 (#5837), and it was never revisited. Nothing uses the second node: workload pods can never schedule on the control-plane (NoSchedule taint), yet kind load copies the ~1.8GB Airflow image and the pinned test images into every node — the control-plane copy is pure overhead. No test depends on the topology.

This switches the cluster to a single control-plane node (Kind's own default), which runs the workloads too — Kind removes the taint on single-node clusters.

Changes

  • kind-cluster-conf.yaml: drop the worker node, move extraPortMappings to the control-plane
  • get_kubernetes_port_numbers(): the forwarded port was read from the hard-coded nodes[1], which raises IndexError on the new config; it now looks the mapping up by containerPort, so rendered configs of clusters created before this change keep working too
  • unit tests for the port extraction (single-node / legacy two-node / missing mapping)
  • doc sample outputs and workflow comments updated to match

Local benchmark

Single run, Apple-silicon macOS + Docker Desktop, same image on both sides, KubernetesExecutor, python 3.10 / K8s v1.30.13:

metric two-node (main) single-node (this PR) delta
upload-k8s-image 114.1s 66.4s −42%
containerd disk, all nodes 15.1 GB 7.8 GB −48%
memory after deploy, all nodes ~3.39 GiB ~2.65 GiB −22%
kind create cluster ¹ 50.5s 16.8s −67%
deploy-airflow 97.0s 83.9s −14%
test results 57 passed / 3 skipped 57 passed / 3 skipped identical

¹ measured with plain kind create cluster and the node image cached on both sides, isolating the topology effect — the worker join is the expensive part.

Verification

  • KubernetesExecutor suite on the single-node cluster: 57 passed / 3 skipped — identical to the two-node baseline
  • CeleryExecutor (largest pod count: redis + celery workers): 53 passed / 7 skipped, including the two Celery-specific tests that are skipped under KubernetesExecutor

Every job in the CI K8s matrix (36 K8S System jobs per canary run, ~500 job-minutes) pays the double image load today. I'll post a per-job duration comparison against recent canary runs once CI has run on this PR.

Validation

  • uv run --project dev/breeze pytest dev/breeze/tests/test_kubernetes_utils.py -xvs
  • prek run --from-ref main --stage pre-commit
  • full manual breeze k8s flow (create / configure / upload / deploy / tests / delete) for both executors above

Was generative AI tooling used to co-author this PR?
  • Yes — Claude Code (Fable 5)

Generated-by: Claude Code (Fable 5) following the guidelines

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant