Carry the cloud CLIs in the image the bgp-cloud-connector e2e jobs run in by frobware · Pull Request #85398 · openshift/release

frobware · 2026-09-17T13:27:03Z

Every e2e job in this config talks to a cloud, and until now each one found its tools a different way. The Azure job ran from an image with az baked in; the AWS and ROSA jobs fetched an aws CLI from the internet partway through a run, after a cluster was already up. That last shape is the expensive one to get wrong, and it is what this removes.

There is now a single e2e-runner image, built FROM src so the Go suite still has its toolchain and its repository, carrying aws, gcloud, az and jq. Every e2e step runs from it.

Where each tool comes from, and why:

aws -- copied out of ocp/4.23:upi-installer, which is where the rest of CI gets its cloud tooling. v2 bundles its own Python, so the tree under /usr/local/aws-cli is self-contained and travels.
az and gcloud -- installed from Microsoft's and Google's own repositories. Neither survives being copied out of upi-installer: its az is a venv with a baked interpreter path, and its gcloud is pinned to 563 where the repository gives 585.
jq -- already in the build root at 1.6, installed explicitly so the image states its contract rather than inheriting it.

oc is deliberately not in the image. Each step gets one from cli: latest, built from the release under test, and entrypoint-wrapper appends $CLI_DIR to PATH rather than prepending it:

// ci-tools cmd/entrypoint-wrapper/main.go
proc.Env = append(proc.Env, fmt.Sprintf("PATH=%s:%s", os.Getenv("PATH"), cliDir))

An oc in the image would therefore shadow it, and every step would quietly run upi-installer's oc -- taken from ocp/4.22:cli -- against a 4.23 cluster.

The image test

azure-cli-image becomes e2e-runner-image and proves the whole set rather than just az:

a command -v roll call naming the resolved path of each tool, walking the whole list so one missing tool does not hide the next
aws --version, gcloud version, az version, jq --version, because resolving on PATH is not the same as working
an aws floor of 2.34.7, the version at which hack/aws/ensure-cli.sh stops fetching one of its own. aws comes from upi-installer, which is rebuilt on its own schedule, so the floor is asserted rather than assumed
the nine az route-server flags the scripts actually pass, unchanged

It needs no cluster and runs in seconds, so a missing tool costs a minute instead of forty.

What this means for openshift/bgp-cloud-connector

hack/aws/ensure-cli.sh and hack/gcp/ensure-cli.sh both short-circuit when a new enough CLI is already on PATH, and both now find one: 2.35.24 against a 2.34.7 floor, and 585.0.0 against a 450.0.0 floor. Neither downloads anything in CI any more. The credentials step in e2e-rosa-operator no longer prepends hack/aws/ensure-cli.sh's output to PATH, because there is nothing left for it to do. Removing the scripts is a separate change over there.

Test plan

Built and exercised locally, substituting the build root for src (which does not exist outside a job) and podman for in-cluster buildah, run as a uid the image does not know:

ok: aws    /usr/bin/aws       aws-cli/2.35.24 Python/3.14.6
ok: gcloud /usr/bin/gcloud    Google Cloud SDK 585.0.0
ok: az     /usr/bin/az        azure-cli 2.90.0
ok: jq     /usr/bin/jq        jq-1.6
ok: aws 2.35.24
+ the nine az route-server flags
exit 0

The Go toolchain survives the dnf transaction: go1.26.7, GNU Make 4.3, git, gcc all intact.

Two things that local build cannot cover, and that the rehearsal is for: the build itself under buildah on src, and whether the build cluster can reach packages.cloud.google.com. Microsoft's repository is already proven reachable by the az install this replaces; Google's is not.

Summary by CodeRabbit

The bgp-cloud-connector e2e jobs now use a shared e2e-runner image. The image includes aws, gcloud, az, and jq, while oc remains supplied by cli: latest.

The CI validation now checks tool availability, versions, the AWS minimum version, and required Azure route-server flags.

The ROSA credentials step no longer changes PATH for AWS CLI installer output.

Every e2e job here talks to a cloud, and each found its tools a different way: the Azure job had an image with az in it, while the AWS and ROSA jobs fetched an aws CLI from the internet partway through a run. A download that happens after a cluster is already up is the most expensive possible thing to get wrong. Replace azure-e2e-runner with a single e2e-runner carrying aws, gcloud, az and jq, and run every e2e step from it. aws is copied out of upi-installer, which is where the rest of CI gets its cloud tooling; az and gcloud come from the vendors' own repositories, because upi-installer's az is a venv with a baked interpreter path and its gcloud is pinned two releases back. oc is deliberately absent. Each step gets one from cli: latest, built from the release under test, and the entrypoint wrapper appends that directory to PATH rather than prepending it, so an oc in the image would shadow it and every step would quietly run an oc older than the cluster it is talking to. azure-cli-image becomes e2e-runner-image and checks all four tools, their versions, and the aws floor that hack/aws/ensure-cli.sh uses to decide whether to fetch one. It needs no cluster, so a missing tool costs a minute instead of forty.

coderabbitai · 2026-09-17T13:27:40Z

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: CHILL

Plan: Enterprise

Run ID: 0f7ef7ec-f067-4a09-89b7-aa9744cc2c6f

📥 Commits

Reviewing files that changed from the base of the PR and between bfc1cf7 and 7ecd331.

⛔ Files ignored due to path filters (1)

ci-operator/jobs/openshift/bgp-cloud-connector/openshift-bgp-cloud-connector-main-presubmits.yaml is excluded by !ci-operator/jobs/**

📒 Files selected for processing (1)

ci-operator/config/openshift/bgp-cloud-connector/openshift-bgp-cloud-connector-main.yaml

Included review availability: Your plan provides up to 12 included reviews per hour; 11 remain after this review.

Walkthrough

The CI configuration builds and validates a shared e2e-runner image with AWS CLI, Azure CLI, Google Cloud CLI, and jq. AWS, Azure, GCP, and ROSA steps now use this image.

Changes

Shared E2E runner

Layer / File(s)	Summary
Build and validate shared runner `ci-operator/config/openshift/bgp-cloud-connector/openshift-bgp-cloud-connector-main.yaml`	The configuration builds `e2e-runner` from `src`, adds the cloud CLIs and `jq`, and validates tool availability plus the required AWS CLI version.
Use shared runner in E2E steps `ci-operator/config/openshift/bgp-cloud-connector/openshift-bgp-cloud-connector-main.yaml`	AWS, Azure, GCP, and ROSA steps use `e2e-runner`. The ROSA credentials step no longer installs AWS CLI with `hack/aws/ensure-cli.sh`.

Priority: ⬇️ Low

Estimated code review effort: 3 (Moderate) | ~20 minutes

Change: Other

Suggested reviewers: alebedev87

Merge Risk: ⚪ Minimal · up to 7ecd3

The shared runner migration preserves required CLI availability for the migrated E2E steps. No actionable merge risk was identified.

🚥 Pre-merge checks | ✅ 15

✅ Passed checks (15 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly and concisely summarizes the main change: adding the cloud CLIs to the image used by the bgp-cloud-connector e2e jobs.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 0…
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Stable And Deterministic Test Names	✅ Passed	The PR changes only CI configuration and generated presubmit YAML. The diff introduces no Ginkgo `It`, `Describe`, `Context`, or `When` declarations. The new CI identifiers, including `e2e-runner-imag…
Test Structure And Quality	✅ Passed	PASS: The pull request changes only CI YAML and generated presubmit configuration. The added `e2e-runner-image` check is a shell command sequence, not Ginkgo test code. The diff contains no `Describe`…
Microshift Test Compatibility	✅ Passed	PASS — The pull request changes only two CI YAML files. The authoritative diff adds or updates image-build and CI shell commands, and it adds no Go files, Ginkgo declarations, or new e2e tests using I…
Single Node Openshift (Sno) Test Compatibility	✅ Passed	PASS: The reviewed range changes only two CI YAML files. It adds or updates image-build, tool-validation, e2e step, and presubmit configuration, but adds no Ginkgo tests or new test bodies using It(),…
Topology-Aware Scheduling Compatibility	✅ Passed	PASS: The pull request changes only CI image/test configuration and generated presubmit configuration. The authoritative diff contains no deployment manifests, operator code, controllers, or topology-…
Ote Binary Stdout Contract	✅ Passed	PASS: The pull request changes only two CI YAML files; the authoritative diff contains no Go files or OTE process-level code. The added `echo`/`printf` and cloud CLI output are shell commands in the i…
Ipv6 And Disconnected Network Test Compatibility	✅ Passed	PASS: The reviewed range changes only CI configuration and generated presubmit YAML. It adds no Ginkgo test code or constructs such as `It`, `Describe`, `Context`, or `When`, and it adds no IPv4 liter…
No-Weak-Crypto	✅ Passed	The PR changes only CI YAML for the shared runner image and presubmit job. The added content installs cloud CLIs, imports repository signing keys, checks CLI versions, and compares the AWS version num…
Container-Privileges	✅ Passed	The pull request changes only the CI image definition, CI test configuration, and generated presubmit job. The authoritative diff contains no `privileged`, `hostPID`, `hostNetwork`, `hostIPC`, `SYS_AD…
No-Sensitive-Data-In-Logs	✅ Passed	PASS. The added test logs only executable paths, CLI versions, and Azure flag checks. The image build adds public package-repository URLs, not secrets. The credential-step logging of decoded token cla…

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create a new PR

_{Comment @coderabbitai help to get the list of available commands.}

openshift-merge-bot · 2026-09-17T13:32:05Z

@frobware: pj-rehearse could not automatically process this event because the request waited in queue for longer than 5 minutes. Use /pj-rehearse to trigger rehearsals manually.

frobware · 2026-09-17T13:32:44Z

/pj-rehearse pull-ci-openshift-bgp-cloud-connector-main-e2e-runner-image

openshift-ci · 2026-09-17T13:34:44Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: frobware

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

~~ci-operator/config/openshift/bgp-cloud-connector/OWNERS~~ [frobware]
~~ci-operator/jobs/openshift/bgp-cloud-connector/OWNERS~~ [frobware]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

frobware · 2026-09-17T13:34:47Z

/retest

openshift-merge-bot · 2026-09-17T13:36:07Z

@frobware: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

frobware · 2026-09-17T13:46:19Z

/test all

frobware · 2026-09-17T13:49:00Z

/retest

frobware · 2026-09-17T13:49:01Z

/pj-rehearse pull-ci-openshift-bgp-cloud-connector-main-e2e-runner-image

openshift-merge-bot · 2026-09-17T13:51:12Z

@frobware: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

frobware · 2026-09-17T13:59:04Z

/retest

frobware · 2026-09-17T13:59:05Z

/pj-rehearse pull-ci-openshift-bgp-cloud-connector-main-e2e-runner-image

openshift-merge-bot · 2026-09-17T13:59:08Z

@frobware: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

frobware · 2026-09-17T14:09:07Z

/retest

frobware · 2026-09-17T14:09:09Z

/pj-rehearse pull-ci-openshift-bgp-cloud-connector-main-e2e-runner-image

openshift-merge-bot · 2026-09-17T14:13:51Z

@frobware: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

frobware · 2026-09-17T14:24:29Z

/pj-rehearse pull-ci-openshift-bgp-cloud-connector-main-e2e-runner-image

frobware · 2026-09-17T14:27:55Z

/retest

openshift-merge-bot · 2026-09-17T14:29:31Z

@frobware: your /pj-rehearse request was not processed because the request waited in queue for longer than 5 minutes. Please retry in a few minutes.

frobware · 2026-09-17T14:36:21Z

/retest

frobware · 2026-09-17T14:36:22Z

/pj-rehearse pull-ci-openshift-bgp-cloud-connector-main-e2e-runner-image

openshift-merge-bot · 2026-09-17T14:38:17Z

@frobware: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

frobware · 2026-09-17T14:54:13Z

/test core-valid

frobware · 2026-09-17T14:55:26Z

/pj-rehearse pull-ci-openshift-bgp-cloud-connector-main-e2e-runner-image

frobware · 2026-09-17T14:56:11Z

/test core-valid

frobware · 2026-09-17T14:57:45Z

/retest

frobware · 2026-09-17T14:57:46Z

/pj-rehearse pull-ci-openshift-bgp-cloud-connector-main-e2e-runner-image

openshift-merge-bot · 2026-09-17T14:58:18Z

@frobware: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

openshift-merge-bot · 2026-09-17T15:02:49Z

@frobware: your /pj-rehearse request was not processed because the request waited in queue for longer than 5 minutes. Please retry in a few minutes.

openshift-ci · 2026-09-17T15:10:36Z

@frobware: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/agent-model-policy	`7ecd331`	link	true	`/test agent-model-policy`
ci/prow/core-valid	`7ecd331`	link	true	`/test core-valid`
ci/rehearse/openshift/bgp-cloud-connector/main/e2e-runner-image	`7ecd331`	link	unknown	`/pj-rehearse pull-ci-openshift-bgp-cloud-connector-main-e2e-runner-image`

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

openshift-ci Bot requested review from alebedev87 and jpinsonneau September 17, 2026 13:34

openshift-ci Bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Sep 17, 2026

frobware closed this Sep 17, 2026

frobware reopened this Sep 17, 2026

Conversation

frobware commented Sep 17, 2026 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

The image test

What this means for openshift/bgp-cloud-connector

Test plan

Summary by CodeRabbit

Uh oh!

coderabbitai Bot commented Sep 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

Uh oh!

openshift-merge-bot Bot commented Sep 17, 2026

Uh oh!

frobware commented Sep 17, 2026

Uh oh!

openshift-ci Bot commented Sep 17, 2026

Uh oh!

frobware commented Sep 17, 2026

Uh oh!

openshift-merge-bot Bot commented Sep 17, 2026

Uh oh!

frobware commented Sep 17, 2026

Uh oh!

frobware commented Sep 17, 2026

Uh oh!

frobware commented Sep 17, 2026

Uh oh!

openshift-merge-bot Bot commented Sep 17, 2026

Uh oh!

frobware commented Sep 17, 2026

Uh oh!

frobware commented Sep 17, 2026

Uh oh!

openshift-merge-bot Bot commented Sep 17, 2026

Uh oh!

frobware commented Sep 17, 2026

Uh oh!

frobware commented Sep 17, 2026

Uh oh!

openshift-merge-bot Bot commented Sep 17, 2026

Uh oh!

frobware commented Sep 17, 2026

Uh oh!

frobware commented Sep 17, 2026

Uh oh!

openshift-merge-bot Bot commented Sep 17, 2026

Uh oh!

frobware commented Sep 17, 2026

Uh oh!

frobware commented Sep 17, 2026

Uh oh!

openshift-merge-bot Bot commented Sep 17, 2026

Uh oh!

frobware commented Sep 17, 2026

Uh oh!

frobware commented Sep 17, 2026

Uh oh!

frobware commented Sep 17, 2026

Uh oh!

frobware commented Sep 17, 2026

Uh oh!

frobware commented Sep 17, 2026

Uh oh!

openshift-merge-bot Bot commented Sep 17, 2026

Uh oh!

openshift-merge-bot Bot commented Sep 17, 2026

Uh oh!

openshift-ci Bot commented Sep 17, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

frobware commented Sep 17, 2026 •

edited by coderabbitai Bot

Loading

coderabbitai Bot commented Sep 17, 2026 •

edited

Loading