Conversation
buildUser: 'worker' + buildUserCreate: true is a leftover from the pre-container (v1 agent) pipeline. It's incompatible with the v2 yap-rocky-8 container: sudoers there only whitelists NOPASSWD for yap/tee/yum/dnf, so `useradd -m worker` fails with "Permission denied" (build #1-3 of PR-21). Tested against the pinned m0rf30/yap-rocky-8:2.6.2 image directly: - `sudo useradd -m worker` (no NOPASSWD entry) -> "a password is required" - `sudo -u worker dnf install ...` (non-root) -> "has to be run with superuser privileges" -- so even creating the user wouldn't have fixed the preBuildScript step - plain `sudo dnf install ...` as root -> succeeds (dnf is whitelisted) - `bazel build` as root in this container -> succeeds Drop the buildUser override entirely; rocky-8 now builds as root, same as rocky-9 and ubuntu already do successfully.
M0Rf30
requested changes
Sep 18, 2026
Per review feedback.
M0Rf30
approved these changes
Sep 18, 2026
Bazel's rules_python refuses to run as root, so rocky-8 must build unprivileged. The old override created a 'worker' user, which fails on v2 nodes: sudoers only grants NOPASSWD for yap/tee/yum/dnf, so useradd is denied. The v2 image already ships a non-root user (yap, uid 1000), so reuse it via buildUser instead of creating one. Run the dnf pre-build script with explicit sudo (dnf is whitelisted) and set preBuildScriptSudo false so it is not rewritten to 'sudo -u yap dnf', which lacks root. Add -s since yap's dnf cache sync also needs root. Verified against m0rf30/yap-rocky-8:2.6.2: bazel clears the root gate and reaches target analysis.
Bazel sizes --jobs from the host /proc core count, not the cgroup quota, so it spawned far more concurrent compiles than the 4-CPU/10Gi yap container could hold. Every one of those held its own RSS while being time-sliced onto 4 CPUs, which OOMKilled the container ~35 min in. Pin --jobs to the CPU limit. No wall-clock cost: 4 CPUs was already the real throughput ceiling.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
rocky-8build branch fails on every PR/build:Root cause
buildUser: 'worker'+buildUserCreate: trueis a leftover from the pre-container (v1 agent) pipeline. It's structurally incompatible with the v2yap-rocky-8container: its sudoers only whitelists NOPASSWD foryap, tee, yum, dnf—useraddisn't there, so anysudo useradd(with or without-u <user>) fails.Verified directly against the pinned
m0rf30/yap-rocky-8:2.6.2image:sudo useradd -m worker→sudo: a password is required(no NOPASSWD entry)sudo -u worker dnf install ...(non-root) →Error: This command has to be run with superuser privileges— so even if the user existed, thepreBuildScriptstep would still failsudo dnf install ...as root → succeeds (dnf is whitelisted)bazel buildas root in this container → succeedsFix
Drop the
buildUser/buildUserCreateoverride entirely.rocky-8now builds as root, same asrocky-9andubuntualready do successfully in this same pipeline.Related
buildStage.groovy'suseraddcall with plainsudoalso fails the same way once tested (useradd still isn't NOPASSWD-whitelisted).