Reduce K8s test resource usage with single-node Kind clusters#69459
Draft
nailo2c wants to merge 1 commit into
Draft
Reduce K8s test resource usage with single-node Kind clusters#69459nailo2c wants to merge 1 commit into
nailo2c wants to merge 1 commit into
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The K8s test Kind clusters have used a control-plane + worker topology since the original Kind migration in 2019 (#5837), and it was never revisited. Nothing uses the second node: workload pods can never schedule on the control-plane (
NoScheduletaint), yetkind loadcopies the ~1.8GB Airflow image and the pinned test images into every node — the control-plane copy is pure overhead. No test depends on the topology.This switches the cluster to a single control-plane node (Kind's own default), which runs the workloads too — Kind removes the taint on single-node clusters.
Changes
kind-cluster-conf.yaml: drop the worker node, moveextraPortMappingsto the control-planeget_kubernetes_port_numbers(): the forwarded port was read from the hard-codednodes[1], which raisesIndexErroron the new config; it now looks the mapping up bycontainerPort, so rendered configs of clusters created before this change keep working tooLocal benchmark
Single run, Apple-silicon macOS + Docker Desktop, same image on both sides, KubernetesExecutor, python 3.10 / K8s v1.30.13:
upload-k8s-imagekind create cluster¹deploy-airflow¹ measured with plain
kind create clusterand the node image cached on both sides, isolating the topology effect — the worker join is the expensive part.Verification
Every job in the CI K8s matrix (36
K8S Systemjobs per canary run, ~500 job-minutes) pays the double image load today. I'll post a per-job duration comparison against recent canary runs once CI has run on this PR.Validation
uv run --project dev/breeze pytest dev/breeze/tests/test_kubernetes_utils.py -xvsprek run --from-ref main --stage pre-commitbreeze k8sflow (create / configure / upload / deploy / tests / delete) for both executors aboveWas generative AI tooling used to co-author this PR?
Generated-by: Claude Code (Fable 5) following the guidelines