Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 9 additions & 1 deletion .github/build.sh
Original file line number Diff line number Diff line change
Expand Up @@ -150,7 +150,15 @@ if [ "$build_rc" -ne 0 ]; then
echo "build.sh: build failed via an sccache cache error — retrying WITHOUT cache (clean reconfigure)."
rm -f "$build_log"
rm -rf build && mkdir -p build
cmake -Bbuild $@ || exit 1
# -DGGML_CCACHE=OFF is required here, not just dropping our own $LAUNCH flags: ggml's own
# CMakeLists.txt self-enables ccache/sccache (GGML_CCACHE, default ON) whenever it finds one
# on PATH and CMAKE_C_COMPILER_LAUNCHER/CMAKE_CXX_COMPILER_LAUNCHER are unset — which is exactly
# this retry's state, since sccache is still on PATH from the failed attempt. It wires itself in
# via the global RULE_LAUNCH_COMPILE property (wraps nvcc too, not just C/C++), so without this
# flag the "uncached" retry silently re-enables the very launcher it's trying to avoid and fails
# identically. Confirmed live: the b10333/CUDA-13.3 retry hit the same fatbinary/.ptx failure
# because of this (see CLAUDE.md's sccache rollout section for the incident).
cmake -Bbuild -DGGML_CCACHE=OFF $@ || exit 1
cmake --build build --config Release -j"${JOBS}" || exit 1
LAUNCH="" # cache disabled for this run; skip the stats query below
else
Expand Down
18 changes: 14 additions & 4 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -531,10 +531,20 @@ v0.16.0 + the probe this is no longer a risk.) Job-by-job status:
`fatbinary fatal: Could not open input file 'acc.compute_75.ptx'` immediately followed by
`sccache: Compiler killed by signal 1`, on the very first `.cu` TU (a cold-cache miss, not a hit
issue) — an sccache/nvcc incompatibility for `-virtual` architecture targets (e.g. `75-virtual`),
not a real compile error. The existing mid-build retry-without-cache mechanism (see below) didn't
catch it because its trigger regex didn't include this failure's wording; `build.sh`'s regex now
also matches `Compiler killed by signal`, so this failure mode falls back to an uncached, green
`-O3` build like every other sccache/nvcc incompatibility instead of redding the job.
not a real compile error. **Two fixes were needed, not one.** (1) The existing mid-build
retry-without-cache mechanism (see below) didn't catch it because its trigger regex didn't include
this failure's wording; `build.sh`'s regex now also matches `Compiler killed by signal`. (2) That
alone still wasn't enough — verified live on the very next dispatch (run 31340386884): the retry
fired but failed **identically**, because ggml's own `ggml/src/CMakeLists.txt` self-enables
ccache/sccache (`GGML_CCACHE`, default `ON`) whenever it finds one on `PATH` and
`CMAKE_C_COMPILER_LAUNCHER`/`CMAKE_CXX_COMPILER_LAUNCHER` are unset — exactly the retry's state,
since sccache is still on `PATH` from the failed attempt (build.sh only clears its own `$LAUNCH`
flags, not ggml's independent detection). ggml wires itself in via the **global**
`RULE_LAUNCH_COMPILE` CMake property, which wraps nvcc too, not just C/C++ — so the "uncached"
retry silently re-enabled the very launcher it was trying to avoid. Fix: the retry's `cmake
-Bbuild` now also passes `-DGGML_CCACHE=OFF`, so a genuinely uncached build is guaranteed
regardless of what's left on `PATH`. This fallback now falls back to a real, green `-O3` build
like every other sccache/nvcc incompatibility instead of redding the job.
3. `crosscompile-linux-aarch64` — ✅ **enabled**, now a **native `ubuntu-24.04-arm` build** (not
dockcross): `build.sh` self-fetches the aarch64 static-musl sccache (the fetch block in
`build.sh` maps `uname -m` → `x86_64`/`aarch64`) and the probe guards it. See "Linux aarch64:
Expand Down
Loading