Skip to content

ci(publish-bake): drop the interim setup-python — the runner image bakes venv now - #4594

Draft
rbuergi wants to merge 2 commits into
mainfrom
ci/publish-bake-drop-interim-setup-python
Draft

rbuergi wants to merge 2 commits into
mainfrom
ci/publish-bake-drop-interim-setup-python

Conversation

@rbuergi

@rbuergi rbuergi commented Sep 17, 2026

Copy link
Copy Markdown
Contributor

Draft until the roll is proven. Do not merge on the reasoning below — merge it on the proof. A satellite that cannot publish is a delivery outage, which is exactly what #4583 was opened for this morning.

What this removes

#4583 added actions/setup-python to publish-bake on 2026-09-17 because the self-hosted ARC image shipped python3 without ensurepip, so publish-bake-bundles.sh could not build the venv it installs its pinned Azure SDKs into. Every satellite's publish step went red with the rest of the run green — MeshWeaver.Crm run 35199593887, MeshWeaver.Manufacturing run 35205340142. Its own comment says to delete it once the runner image bakes python3-venv.

Why it is safe — both runners this job can land on

runs-on: ${{ vars.MW_RUNNER_DOCKER || 'ubuntu-latest' }}, so exactly two:

runner evidence that python3 -m venv works
aks-silos-dind (every private repo) Systemorph/Memex#396 bakes python3-venv. Its build asserts, as uid 1001: python3 -m venv: works as uid 1001, pip present inside the venv. Run against the pushed image itself, as uid 1001: /usr/bin/python3 -m venvpip install azure-storage-file-share==12.26.0 azure-identity==1.25.3 → both import — this script's own SDK_PINS, not a substitute
ubuntu-latest (fallback where the org variable is invisible) always had it — actions/runner-images images/ubuntu/scripts/build/install-python.sh: apt-get install --no-install-recommends python3 python3-dev python3-pip python3-venv

What it does NOT do

It does not delete the requirement — it states it where the step used to be. The next person to meet an ensurepip failure should learn there that this is the image's job, not this workflow's, rather than find a blank and re-add a step that hides the requirement.

Merge condition

  1. ci-runners-apply.yml has rolled @sha256:acdbefbc… onto both sets;
  2. Systemorph/Memex#400 (the proof lane) is green — it asserts /usr/bin/python3 builds a venv whose pip installs these exact pins, on a real runner;
  3. a real publishing run has published with this file (I will link it here).

Then mark ready. Until all three, the interim step stays: it costs ~0 s on a hosted runner and a few seconds on ours, which is nothing against a fleet that cannot deliver.

🤖 Generated with Claude Code

…kes venv now

#4583 added `actions/setup-python` to this job on 2026-09-17 because the self-hosted ARC
image shipped python3 WITHOUT ensurepip, so publish-bake-bundles.sh could not build the
venv it installs its pinned Azure SDKs into — and every satellite's publish step went red
with the rest of the run green (MeshWeaver.Crm 35199593887, Manufacturing 35205340142).
Its own comment said to delete it once the image baked python3-venv.

The image does: meshweaver-ci-runner 20260917-3c61f83 (Systemorph/Memex#396), rolled onto
both scale sets, its build asserting `python3 -m venv: works as uid 1001, pip present
inside the venv` and runner-proof*.yml asserting on a real runner that /usr/bin/python3
builds a venv whose pip installs THIS script's own SDK pins from PyPI.

Both runners this job can land on are covered — the `ubuntu-latest` fallback always was:
runner-images' install-python.sh does `apt-get install python3 python3-dev python3-pip
python3-venv`. The requirement is stated where the step used to be rather than deleted
silently, because the next person to see an ensurepip failure should find out here that
it is the image's job, not this workflow's.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Copilot AI lite review requested due to automatic review settings September 17, 2026 11:39

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The workflow can still select runner images without verified venv support, risking publication failures.

Get a fresh assessment by requesting another Copilot review.

Pull request overview

Removes the interim Python setup step from the publish-bake workflow, relying on runner images to provide venv and pip.

Changes:

  • Deletes actions/setup-python.
  • Documents the runner-image prerequisite.
File summaries
File Description
.github/workflows/node-repo-publish-bake.yml Removes Python environment provisioning before bundle publication.
Review details
  • Files reviewed: 1/1 changed files
  • Comments generated: 1
  • Review effort level: Lite

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment on lines +1790 to +1794
# It is gone because the image bakes `python3-venv` (Systemorph/Memex
# deployments/aks/ci-runners/runner-image/Dockerfile, meshweaver-ci-runner 20260917-3c61f83),
# and the `ubuntu-latest` fallback above has always had it (runner-images'
# install-python.sh: `apt-get install python3 python3-dev python3-pip python3-venv`). Both
# runners this job can land on therefore satisfy it — if a third ever appears, assert it

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verified REAL, and load-bearing. Fixed in 19093c0.

You are right on both halves of the mechanism. runs-on here is ${{ vars.MW_RUNNER_DOCKER || 'ubuntu-latest' }} — an ORG VARIABLE resolved from the CALLER at queue time — and check-reusable-workflow-runners.py validates only the expression SHAPE. Its own docstring says so: the rule is which operands may appear and that the last one is the 'ubuntu-latest' fallback. It cannot see which image a label selects, so a re-pointed scale set, a rolled-back runner image or a third label reaches this job with nothing red, and the run dies at publish-bake-bundles.sh:480 (python3 -m venv "$VENV") with its real cause several steps upstream. That is precisely the 2026-09-17 outage shape.

What I did NOT do is add an assertion that only fails earlier — that replaces the diagnosis, not the guarantee, and the publication would still be lost. The three steps that now stand where setup-python stood replace the guarantee itself:

  1. PROVEpython3 -m venv is actually executed into $RUNNER_TEMP and bin/pip checked. A CAPABILITY probe, deliberately not a version check: python3 --version was green on the runner that took the fleet down, because ensurepip ships as a separate package.
  2. HEALactions/setup-python@v7 runs ONLY when the probe failed, with a ::warning:: naming $RUNNER_NAME, so the image defect is loud rather than hidden by a step that runs unconditionally. A correct runner pays a sub-second probe and no interpreter download, which was the point of the PR.
  3. ASSERT — the healed interpreter is re-probed and the job fails RED with ::error:: if it still cannot build a venv. This sits before the publish step, so nothing has been written and the existing publication stays sealed.

So all three of the cases you named are covered — stale image, private image, third label — and the requirement is asserted at the point of use instead of being documented in a comment. Gates run on the change: check-reusable-workflow-runners 0 violations over 24 jobs, check-workflow-shell 0 live findings, check-workflow-yaml-keys / -timeouts / -permission-pairing clean.

@github-actions

github-actions Bot commented Sep 17, 2026

Copy link
Copy Markdown
Contributor

Test Results (shard 3)

431 tests  +6   431 ✅ +6   57s ⏱️ -3s
  3 suites +1     0 💤 ±0 
  3 files   +1     0 ❌ ±0 

Results for commit 19093c0. ± Comparison against base commit 5672869.

♻️ This comment has been updated with latest results.

@github-actions

github-actions Bot commented Sep 17, 2026

Copy link
Copy Markdown
Contributor

Test Results (shard 0)

265 tests  +1   265 ✅ +1   2m 45s ⏱️ -1s
  1 suites ±0     0 💤 ±0 
  1 files   ±0     0 ❌ ±0 

Results for commit 19093c0. ± Comparison against base commit 5672869.

♻️ This comment has been updated with latest results.

@github-actions

github-actions Bot commented Sep 17, 2026

Copy link
Copy Markdown
Contributor

Test Results (shard 1)

1 551 tests  +1 018   1 551 ✅ +1 018   2m 56s ⏱️ + 1m 50s
    2 suites +    1       0 💤 ±    0 
    2 files   +    1       0 ❌ ±    0 

Results for commit 19093c0. ± Comparison against base commit 5672869.

♻️ This comment has been updated with latest results.

@github-actions

github-actions Bot commented Sep 17, 2026

Copy link
Copy Markdown
Contributor

Test Results (shard 4)

    3 files  ±  0      3 suites  ±0   5m 33s ⏱️ +6s
1 857 tests  - 429  1 857 ✅  - 429  0 💤 ±0  0 ❌ ±0 
1 858 runs   - 429  1 858 ✅  - 429  0 💤 ±0  0 ❌ ±0 

Results for commit 19093c0. ± Comparison against base commit 5672869.

♻️ This comment has been updated with latest results.

@github-actions

github-actions Bot commented Sep 17, 2026

Copy link
Copy Markdown
Contributor

Test Results (shard 2)

710 tests   - 781   518 ✅  - 971   5m 34s ⏱️ - 2m 1s
  3 suites  -   1   192 💤 +190 
  3 files    -   1     0 ❌ ±  0 

Results for commit 19093c0. ± Comparison against base commit 5672869.

♻️ This comment has been updated with latest results.

@github-actions

github-actions Bot commented Sep 17, 2026

Copy link
Copy Markdown
Contributor

Test Results (shard 5)

    5 files   -   1      5 suites   - 1   12m 7s ⏱️ +36s
3 661 tests +489  3 659 ✅ +679  2 💤  - 190  0 ❌ ±0 
3 665 runs  +489  3 663 ✅ +679  2 💤  - 190  0 ❌ ±0 

Results for commit 19093c0. ± Comparison against base commit 5672869.

♻️ This comment has been updated with latest results.

@github-actions

github-actions Bot commented Sep 17, 2026

Copy link
Copy Markdown
Contributor

Test Results

   17 files  ±  0     17 suites  ±0   29m 54s ⏱️ +29s
8 475 tests +304  8 281 ✅ +304  194 💤 ±0  0 ❌ ±0 
8 480 runs  +304  8 286 ✅ +304  194 💤 ±0  0 ❌ ±0 

Results for commit 19093c0. ± Comparison against base commit 5672869.

♻️ This comment has been updated with latest results.

Copilot's review of #4594 is right and the point is load-bearing: dropping the
unconditional `actions/setup-python` removed the only RUNTIME guarantee that
`python3 -m venv` can succeed, and nothing else supplies one. `runs-on` here is
`${{ vars.MW_RUNNER_DOCKER || 'ubuntu-latest' }}` — an ORG VARIABLE resolved from
the caller at queue time — and check-reusable-workflow-runners.py validates only
the EXPRESSION SHAPE (its own docstring: the rule is about which operands appear,
never which image a label selects). So a re-pointed scale set, a rolled-back
runner image or a third label reaches this job with nothing red, and the failure
lands at publish-bake-bundles.sh's own `python3 -m venv` — which is exactly the
2026-09-17 fleet-wide publication outage (MeshWeaver.Crm run 35199593887,
MeshWeaver.Manufacturing run 35205340142).

An assertion that merely fails earlier would replace the DIAGNOSIS, not the
guarantee, so this replaces the guarantee instead — three steps where the step
used to be:

  1. PROVE  — actually create a venv on the interpreter this job will use. A
     CAPABILITY probe, never a version check: `python3 --version` was green on
     the runner that took the fleet down, because ensurepip ships separately.
  2. HEAL   — `actions/setup-python` ONLY when the probe failed, with a
     `::warning::` naming the runner, so the image defect is loud instead of
     hidden. A good runner pays a sub-second probe and no download.
  3. ASSERT — re-probe the healed interpreter and fail RED before anything is
     written. Nothing has been published at that point, so the existing
     publication stays sealed.

Gates: check-reusable-workflow-runners 0 violations (24 jobs), check-workflow-shell
0 live findings, check-workflow-yaml-keys / -timeouts / -permission-pairing clean.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@rbuergi rbuergi added the release-stream Required for the 3.0.0-ci release stream label Sep 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

release-stream Required for the 3.0.0-ci release stream

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants