Upgrade llama.cpp to b10333 and CUDA to 13.3

.github/build_cuda_linux.sh

@@ -5,15 +5,15 @@
 #
 # SPDX-License-Identifier: MIT
 
-# A Cuda 13.2 install script for RHEL8/Rocky8/Manylinux_2.28
+# A Cuda 13.3 install script for RHEL8/Rocky8/Manylinux_2.28
 # Available versions can be found at:
 # https://developer.download.nvidia.com/compute/cuda/repos/rhel8/x86_64/
 
 sudo dnf install -y kernel-devel kernel-headers
 sudo dnf install -y https://dl.fedoraproject.org/pub/epel/epel-release-latest-8.noarch.rpm
 sudo dnf config-manager --add-repo https://developer.download.nvidia.com/compute/cuda/repos/rhel8/x86_64/cuda-rhel8.repo
 
-sudo dnf install -y cuda-toolkit-13-2
+sudo dnf install -y cuda-toolkit-13-3
 
 # CUDA target architectures — LOCAL-dev build-speed knob.
 #
@@ -38,4 +38,4 @@ case "${CUDA_FAST_BUILD:-}" in
     ;;
 esac
 
-exec .github/build.sh $@ -DGGML_CUDA=1 -DCMAKE_CUDA_COMPILER=/usr/local/cuda-13.2/bin/nvcc $CUDA_ARCH_ARGS
+exec .github/build.sh $@ -DGGML_CUDA=1 -DCMAKE_CUDA_COMPILER=/usr/local/cuda-13.3/bin/nvcc $CUDA_ARCH_ARGS

.github/workflows/publish.yml

@@ -1671,7 +1671,7 @@ jobs:
         uses: Jimver/cuda-toolkit@v0.2.36
         id: cuda-toolkit
         with:
-          cuda: '13.2.0'
+          cuda: '13.3.1'
       - name: Install sccache (shared compiler cache)
         if: env.USE_CACHE == 'true' && env.SCCACHE_WEBDAV_TOKEN != ''
         continue-on-error: true

CLAUDE.md

@@ -6,16 +6,16 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co
 
 Java bindings for [llama.cpp](https://github.com/ggerganov/llama.cpp) via JNI, providing a high-level API for LLM inference in Java. The Java layer communicates with a native C++ library through JNI.
 
-Current llama.cpp pinned version: **b10327**
+Current llama.cpp pinned version: **b10333**
 
 ## Upgrading CUDA Version
 
-Current CUDA version: **13.2**
+Current CUDA version: **13.3**
 
 To change the CUDA version, update the following **three** places:
 
-1. **`.github/build_cuda_linux.sh`** — Line 16: `sudo dnf install -y cuda-toolkit-13-2`
-2. **`.github/build_cuda_linux.sh`** — Line 41: `-DCMAKE_CUDA_COMPILER=/usr/local/cuda-13.2/bin/nvcc`
+1. **`.github/build_cuda_linux.sh`** — Line 16: `sudo dnf install -y cuda-toolkit-13-3`
+2. **`.github/build_cuda_linux.sh`** — Line 41: `-DCMAKE_CUDA_COMPILER=/usr/local/cuda-13.3/bin/nvcc`
 3. **`llama/pom.xml`** — The `<classifier>` tag in the `cuda` jar execution: `cuda13-linux-x86-64`
 
 Also update the header comment in `build_cuda_linux.sh` and the job name in `.github/workflows/release.yaml` for clarity.
@@ -27,15 +27,15 @@ https://developer.download.nvidia.com/compute/cuda/repos/rhel8/x86_64/
 
 **Note:** Each CUDA version supports only certain GCC versions. If the dockcross container uses a newer GCC than CUDA supports, the build will fail with `unsupported GNU version`. Check NVIDIA's compatibility table before downgrading CUDA.
 
-Example: To upgrade from 13.2 to a hypothetical 13.3:
+Example: To upgrade from 13.3 to a hypothetical 13.4:
 ```bash
 # Edit .github/build_cuda_linux.sh:
-#   line 10: cuda-toolkit-13-2 -> cuda-toolkit-13-3
-#   line 12: /usr/local/cuda-13.2/bin/nvcc -> /usr/local/cuda-13.3/bin/nvcc
+#   line 10: cuda-toolkit-13-3 -> cuda-toolkit-13-4
+#   line 12: /usr/local/cuda-13.3/bin/nvcc -> /usr/local/cuda-13.4/bin/nvcc
 # Edit llama/pom.xml classifier: cuda13-linux-x86-64 (major version only, no need to change for minor bumps)
-# Edit CLAUDE.md line: Current CUDA version: **13.2** -> **13.3**
+# Edit CLAUDE.md line: Current CUDA version: **13.3** -> **13.4**
 git add .github/build_cuda_linux.sh llama/pom.xml CLAUDE.md
-git commit -m "Upgrade CUDA from 13.2 to 13.3"
+git commit -m "Upgrade CUDA from 13.3 to 13.4"
 ```
 
 ### Fast local CUDA builds (`CUDA_FAST_BUILD`) — single-arch speed knob
@@ -217,7 +217,7 @@ Wiring (mirrors the CUDA-Linux / OpenCL-Android classifier pattern):
    - `build-windows-x86_64` / `build-windows-x86` — **Ninja CPU**, artifacts `Windows-{arch}-libraries`
      → picked up by the `package` job's `pattern: "*-libraries"` into the **default** tree.
    - `build-windows-x86_64-msvc` / `build-windows-x86-msvc` — **MSVC CPU**, artifacts `Windows-{arch}-msvc`.
-   - `build-windows-x86_64-cuda` — `Jimver/cuda-toolkit@v0.2.35` (CUDA `13.2.0`) + `-DGGML_CUDA=ON`,
+   - `build-windows-x86_64-cuda` — `Jimver/cuda-toolkit@v0.2.36` (CUDA `13.3.1`) + `-DGGML_CUDA=ON`,
      artifact `Windows-x86_64-cuda`.
    - `build-windows-x86_64-vulkan` — `jakoch/install-vulkan-sdk-action` + `-DGGML_VULKAN=ON`, artifact
      `Windows-x86_64-vulkan`.
@@ -429,7 +429,7 @@ needs no extra step here, `build-webui` re-reads the tag and rebuilds the matchi
 ships no UI):
 ```bash
 # needs node/npm + network; embed.cpp is plain C++17 (no npm)
-git clone --depth 1 --branch b10327 https://github.com/ggml-org/llama.cpp /tmp/lc
+git clone --depth 1 --branch b10333 https://github.com/ggml-org/llama.cpp /tmp/lc
 ( cd /tmp/lc/tools/ui && npm ci && npm run build \
   && ( cd dist && find . -type f -not -path './_gzip/*' \
        | while read -r f; do mkdir -p "_gzip/$(dirname "$f")"; gzip -9 -c "$f" > "_gzip/$f"; done ) \
@@ -469,7 +469,7 @@ cache lives in **Depot Cache** over sccache's **WebDAV** backend:
 - `SCCACHE_WEBDAV_TOKEN: ${{ secrets.DEPOT_TOKEN }}` — a Depot **organization** token, stored
   as the repo secret **`DEPOT_TOKEN`**.
 
-Because `sccache` is **content-addressed** and llama.cpp is pinned (`GIT_TAG b10327`), the
+Because `sccache` is **content-addressed** and llama.cpp is pinned (`GIT_TAG b10333`), the
 ~280 upstream object files are byte-identical every run, so a warm cache recompiles only the
 *changed* files. Depot's cache is **shared across all branches** (unlike GitHub's
 per-branch `actions/cache`), so every branch builds incrementally; a `b<nnnn>` version bump
@@ -1278,7 +1278,7 @@ ctest --test-dir build --output-on-failure -R "ResultsToJson"
 
 #### Upstream source location (in CMake build tree)
 
-llama.cpp is fetched via CMake FetchContent, pinned to `GIT_TAG b10327`.
+llama.cpp is fetched via CMake FetchContent, pinned to `GIT_TAG b10333`.
 
 **GoogleTest** is a separate `BUILD_TESTING`-only FetchContent (`GIT_TAG v1.17.0`), used solely
 by the `jllama_test` C++ unit-test binary — not by the shipped library, and not coupled to the

README.md

@@ -11,7 +11,7 @@
 **Build:**  
 ![Java 8+](https://img.shields.io/badge/Java-8%2B-informational)  
 ![Platform](https://img.shields.io/badge/Platform-Linux%20%7C%20macOS%20%7C%20Windows%20%7C%20Android-lightgrey)  
-[![llama.cpp b10327](https://img.shields.io/badge/llama.cpp-%23b10327-informational)](https://github.com/ggml-org/llama.cpp/releases/tag/b10327)  
+[![llama.cpp b10333](https://img.shields.io/badge/llama.cpp-%23b10333-informational)](https://github.com/ggml-org/llama.cpp/releases/tag/b10333)  
 [![JPMS](https://img.shields.io/badge/JPMS-modular%20JAR-25A162)](https://openjdk.org/projects/jigsaw/)  
 ![JUnit](https://img.shields.io/badge/tested%20with-JUnit6-25A162)  
 [![JSpecify](https://img.shields.io/badge/JSpecify-1.0.0%20%40NullMarked-25A162)](https://jspecify.dev)  

docs/history/llama-cpp-breaking-changes.md

@@ -583,3 +583,5 @@ Used during `llama.cpp` version bumps: when upgrading, scan this file from the r
 | b10319–b10322 | upstream verification (sandbox) | Remaining **6** patches re-verified against a clean b10322 checkout (ggml/llama.cpp commit `f8e30266`): fail-loud `PATCH_COMMAND` clean. **Configure-only verification per this step's process** (full build deferred to the b10327 final target). |
 | b10322–b10327 | `tools/tts/tts.cpp` (**reference-tool timing/logging tweak — now also reports vocoder time via `get_output`'s existing optional `out_n_samples` param**), `ggml/src/ggml-cuda/cpy.cu` (**GPU backend internal**), `tools/ui/**` (**large contenteditable chat-form rewrite, dominates the diff at 4950 lines**) | **No project-source change (final step of this b10319→b10327 walk).** `tools/mtmd/mtmd-helper.h` has **zero diff** across this whole range — confirmed via a direct diff of just that file. `mtmd_helper::gen_audio::get_output`'s 4th parameter (`int64_t * out_n_samples = nullptr`) already existed before this range; `tools/tts/tts.cpp` merely started passing it (for better timing logs), it did not become newly required. `tts_engine.cpp`'s existing 3-arg `gen.get_output(&sample_rate, &data, &data_len)` call stays valid unchanged. Zero diff under `common/`, `include/`, `tools/mtmd/mtmd.h`, `tools/server/*.h`, `tools/server/*.cpp`, `CMakeLists.txt`, `vendor/sheredom/subprocess.h`. |
 | b10322–b10327 | upstream verification (sandbox, final target) | Remaining **6** patches re-verified against a clean b10327 checkout (ggml/llama.cpp commit `69bf6437`): fail-loud `PATCH_COMMAND` clean. **Full local verification (mandatory for the final target):** fresh configure + full `cmake --build` (`libjllama.so` + `jllama_test` compile and link, `-O3`, no undefined references) + `ctest` **482/482 passing**. Closes the b10319→b10327 walk (2 steps: b10319→b10322→b10327), on a new branch (`claude/java-llama-cpp-b10327`) following on from the b10310→b10319 walk merged as PR #381. |
+| b10327–b10333 | `common/{arg.cpp,common.h}` (**purely additive: new `--tools-runtime` CLI flag + `common_params::server_tools_runtime` field**), `tools/server/{server-tools.h,server-tools.cpp,server.cpp}` (**new opt-in Docker-backed runtime for built-in tools**), `ggml/src/ggml-cuda/{ggml-cuda.cu,rope.cu,rope.cuh}` (**GPU backend internals — CUDA rope kernel rework**) | **No project-source change (single-step bump, diff under the 100 KiB chunking threshold).** `common_params::server_tools_runtime` is a new field appended after the existing `server_tools` member — purely additive, no existing field renamed/removed/reordered. `server_tools::setup(...)` gained a third parameter (`const std::string & tools_runtime`) at its one call site in `server.cpp`'s `llama_server()`, which is inside the region `patches/0006` touches only earlier in the function (embedded-mode flag/argv handling) — verified the patch still applies cleanly (see next row). `jllama.cpp`/`jni_helpers.hpp`/`json_helpers.hpp` reference neither `server_tools` nor `server-tools.h` directly (confirmed via grep), so the 584-line `server-tools.cpp` rewrite (adding an opt-in `--tools-runtime docker:<image>` execution backend) is entirely internal to `server.cpp`'s built-in-tools wiring. CUDA rope-kernel changes are internal to `ggml-cuda`; zero diff under `common/chat.h`, `common/speculative.h`, `tools/mtmd/mtmd.h`, `include/llama-cpp.h`, `common/arg.h` (beyond the additive flag above), `include/llama.h`, `common/download.h`, `tools/server/server-schema.h`. |
+| b10327–b10333 | upstream verification (sandbox, final target) | All **6** patches re-verified against a clean b10333 checkout (ggml/llama.cpp commit `08659901c`): sequential `git apply` (filename order, mirroring the `PATCH_COMMAND` applier) succeeded for every patch, including `0006`/`0007` across the touched `server.cpp` region. **Full local verification (mandatory for the final target):** fresh configure + full `cmake --build` (`libjllama.so` + `jllama_test` compile and link, `-O3`, no undefined references) + `ctest` **482/482 passing**. Single-step bump (76 KiB / 71 KiB excl. WebUI, 6 commits — under the 100 KiB chunking threshold), on branch `claude/cuda-toolkit-action-update-0675dk` (continuing the CUDA-toolkit-action-bump branch). |

llama/CMakeLists.txt

@@ -173,7 +173,7 @@ set(LLAMA_BUILD_APP OFF CACHE BOOL "" FORCE)
 FetchContent_Declare(
 	llama.cpp
 	GIT_REPOSITORY https://github.com/ggerganov/llama.cpp.git
-	GIT_TAG        b10327
+	GIT_TAG        b10333
 	PATCH_COMMAND  ${CMAKE_COMMAND}
 		-DPATCH_DIR=${CMAKE_CURRENT_SOURCE_DIR}/patches
 		-DLLAMA_SRC=<SOURCE_DIR>

llama/src/main/java/net/ladenthin/llama/value/LlamaCppVersion.java

@@ -10,28 +10,28 @@
  * library was compiled against, exposed as a compile-time constant so callers can render a badge or
  * emit a startup log line without loading the native library.
  *
- * <p>{@link #LLAMA_CPP_VERSION} is a pure-Java string ({@code "b10327"}) that mirrors the
+ * <p>{@link #LLAMA_CPP_VERSION} is a pure-Java string ({@code "b10333"}) that mirrors the
  * {@code GIT_TAG} in {@code llama/CMakeLists.txt}. It is available even when {@code libjllama} is
  * absent (pure-Java checkout, before {@code System.load}), which is what makes it suitable for a
  * lightweight version badge in Android or other UIs.</p>
  *
  * <p>For the <em>authoritative</em> value that is baked into the native binary — the build number
- * plus the resolved upstream commit, e.g. {@code "b10327-<commit>"} — call
+ * plus the resolved upstream commit, e.g. {@code "b10333-<commit>"} — call
  * {@link net.ladenthin.llama.LlamaModel#getLlamaCppBuildInfo()} instead; that reads llama.cpp's own
  * {@code build-info} through JNI and therefore cannot drift from the compiled library (but requires
  * the native library to be loaded).</p>
  */
 public final class LlamaCppVersion {
 
     /**
-     * The pinned llama.cpp release tag this library was built against, e.g. {@code "b10327"}.
+     * The pinned llama.cpp release tag this library was built against, e.g. {@code "b10333"}.
      *
      * <p>Kept in lockstep with {@code GIT_TAG} in {@code llama/CMakeLists.txt} — see the
      * "Upgrading/Downgrading llama.cpp Version" checklist in {@code CLAUDE.md}. This is the
      * compile-time pin; use {@link net.ladenthin.llama.LlamaModel#getLlamaCppBuildInfo()} for the
      * value actually linked into the native binary.</p>
      */
-    public static final String LLAMA_CPP_VERSION = "b10327";
+    public static final String LLAMA_CPP_VERSION = "b10333";
 
     // Constants holder — not instantiable.
     private LlamaCppVersion() {}