diff --git a/.github/build_cuda_linux.sh b/.github/build_cuda_linux.sh index d1c2bc31..e2c39c03 100755 --- a/.github/build_cuda_linux.sh +++ b/.github/build_cuda_linux.sh @@ -5,7 +5,7 @@ # # SPDX-License-Identifier: MIT -# A Cuda 13.2 install script for RHEL8/Rocky8/Manylinux_2.28 +# A Cuda 13.3 install script for RHEL8/Rocky8/Manylinux_2.28 # Available versions can be found at: # https://developer.download.nvidia.com/compute/cuda/repos/rhel8/x86_64/ @@ -13,7 +13,7 @@ sudo dnf install -y kernel-devel kernel-headers sudo dnf install -y https://dl.fedoraproject.org/pub/epel/epel-release-latest-8.noarch.rpm sudo dnf config-manager --add-repo https://developer.download.nvidia.com/compute/cuda/repos/rhel8/x86_64/cuda-rhel8.repo -sudo dnf install -y cuda-toolkit-13-2 +sudo dnf install -y cuda-toolkit-13-3 # CUDA target architectures — LOCAL-dev build-speed knob. # @@ -38,4 +38,4 @@ case "${CUDA_FAST_BUILD:-}" in ;; esac -exec .github/build.sh $@ -DGGML_CUDA=1 -DCMAKE_CUDA_COMPILER=/usr/local/cuda-13.2/bin/nvcc $CUDA_ARCH_ARGS +exec .github/build.sh $@ -DGGML_CUDA=1 -DCMAKE_CUDA_COMPILER=/usr/local/cuda-13.3/bin/nvcc $CUDA_ARCH_ARGS diff --git a/.github/workflows/publish.yml b/.github/workflows/publish.yml index 5e7b24c9..35efdfbc 100644 --- a/.github/workflows/publish.yml +++ b/.github/workflows/publish.yml @@ -1671,7 +1671,7 @@ jobs: uses: Jimver/cuda-toolkit@v0.2.36 id: cuda-toolkit with: - cuda: '13.2.0' + cuda: '13.3.1' - name: Install sccache (shared compiler cache) if: env.USE_CACHE == 'true' && env.SCCACHE_WEBDAV_TOKEN != '' continue-on-error: true diff --git a/CLAUDE.md b/CLAUDE.md index 187f21bd..5fef2328 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -6,16 +6,16 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co Java bindings for [llama.cpp](https://github.com/ggerganov/llama.cpp) via JNI, providing a high-level API for LLM inference in Java. The Java layer communicates with a native C++ library through JNI. -Current llama.cpp pinned version: **b10327** +Current llama.cpp pinned version: **b10333** ## Upgrading CUDA Version -Current CUDA version: **13.2** +Current CUDA version: **13.3** To change the CUDA version, update the following **three** places: -1. **`.github/build_cuda_linux.sh`** — Line 16: `sudo dnf install -y cuda-toolkit-13-2` -2. **`.github/build_cuda_linux.sh`** — Line 41: `-DCMAKE_CUDA_COMPILER=/usr/local/cuda-13.2/bin/nvcc` +1. **`.github/build_cuda_linux.sh`** — Line 16: `sudo dnf install -y cuda-toolkit-13-3` +2. **`.github/build_cuda_linux.sh`** — Line 41: `-DCMAKE_CUDA_COMPILER=/usr/local/cuda-13.3/bin/nvcc` 3. **`llama/pom.xml`** — The `` tag in the `cuda` jar execution: `cuda13-linux-x86-64` Also update the header comment in `build_cuda_linux.sh` and the job name in `.github/workflows/release.yaml` for clarity. @@ -27,15 +27,15 @@ https://developer.download.nvidia.com/compute/cuda/repos/rhel8/x86_64/ **Note:** Each CUDA version supports only certain GCC versions. If the dockcross container uses a newer GCC than CUDA supports, the build will fail with `unsupported GNU version`. Check NVIDIA's compatibility table before downgrading CUDA. -Example: To upgrade from 13.2 to a hypothetical 13.3: +Example: To upgrade from 13.3 to a hypothetical 13.4: ```bash # Edit .github/build_cuda_linux.sh: -# line 10: cuda-toolkit-13-2 -> cuda-toolkit-13-3 -# line 12: /usr/local/cuda-13.2/bin/nvcc -> /usr/local/cuda-13.3/bin/nvcc +# line 10: cuda-toolkit-13-3 -> cuda-toolkit-13-4 +# line 12: /usr/local/cuda-13.3/bin/nvcc -> /usr/local/cuda-13.4/bin/nvcc # Edit llama/pom.xml classifier: cuda13-linux-x86-64 (major version only, no need to change for minor bumps) -# Edit CLAUDE.md line: Current CUDA version: **13.2** -> **13.3** +# Edit CLAUDE.md line: Current CUDA version: **13.3** -> **13.4** git add .github/build_cuda_linux.sh llama/pom.xml CLAUDE.md -git commit -m "Upgrade CUDA from 13.2 to 13.3" +git commit -m "Upgrade CUDA from 13.3 to 13.4" ``` ### Fast local CUDA builds (`CUDA_FAST_BUILD`) — single-arch speed knob @@ -217,7 +217,7 @@ Wiring (mirrors the CUDA-Linux / OpenCL-Android classifier pattern): - `build-windows-x86_64` / `build-windows-x86` — **Ninja CPU**, artifacts `Windows-{arch}-libraries` → picked up by the `package` job's `pattern: "*-libraries"` into the **default** tree. - `build-windows-x86_64-msvc` / `build-windows-x86-msvc` — **MSVC CPU**, artifacts `Windows-{arch}-msvc`. - - `build-windows-x86_64-cuda` — `Jimver/cuda-toolkit@v0.2.35` (CUDA `13.2.0`) + `-DGGML_CUDA=ON`, + - `build-windows-x86_64-cuda` — `Jimver/cuda-toolkit@v0.2.36` (CUDA `13.3.1`) + `-DGGML_CUDA=ON`, artifact `Windows-x86_64-cuda`. - `build-windows-x86_64-vulkan` — `jakoch/install-vulkan-sdk-action` + `-DGGML_VULKAN=ON`, artifact `Windows-x86_64-vulkan`. @@ -429,7 +429,7 @@ needs no extra step here, `build-webui` re-reads the tag and rebuilds the matchi ships no UI): ```bash # needs node/npm + network; embed.cpp is plain C++17 (no npm) -git clone --depth 1 --branch b10327 https://github.com/ggml-org/llama.cpp /tmp/lc +git clone --depth 1 --branch b10333 https://github.com/ggml-org/llama.cpp /tmp/lc ( cd /tmp/lc/tools/ui && npm ci && npm run build \ && ( cd dist && find . -type f -not -path './_gzip/*' \ | while read -r f; do mkdir -p "_gzip/$(dirname "$f")"; gzip -9 -c "$f" > "_gzip/$f"; done ) \ @@ -469,7 +469,7 @@ cache lives in **Depot Cache** over sccache's **WebDAV** backend: - `SCCACHE_WEBDAV_TOKEN: ${{ secrets.DEPOT_TOKEN }}` — a Depot **organization** token, stored as the repo secret **`DEPOT_TOKEN`**. -Because `sccache` is **content-addressed** and llama.cpp is pinned (`GIT_TAG b10327`), the +Because `sccache` is **content-addressed** and llama.cpp is pinned (`GIT_TAG b10333`), the ~280 upstream object files are byte-identical every run, so a warm cache recompiles only the *changed* files. Depot's cache is **shared across all branches** (unlike GitHub's per-branch `actions/cache`), so every branch builds incrementally; a `b` version bump @@ -1278,7 +1278,7 @@ ctest --test-dir build --output-on-failure -R "ResultsToJson" #### Upstream source location (in CMake build tree) -llama.cpp is fetched via CMake FetchContent, pinned to `GIT_TAG b10327`. +llama.cpp is fetched via CMake FetchContent, pinned to `GIT_TAG b10333`. **GoogleTest** is a separate `BUILD_TESTING`-only FetchContent (`GIT_TAG v1.17.0`), used solely by the `jllama_test` C++ unit-test binary — not by the shipped library, and not coupled to the diff --git a/README.md b/README.md index 3083e362..30b981fe 100644 --- a/README.md +++ b/README.md @@ -11,7 +11,7 @@ **Build:** ![Java 8+](https://img.shields.io/badge/Java-8%2B-informational) ![Platform](https://img.shields.io/badge/Platform-Linux%20%7C%20macOS%20%7C%20Windows%20%7C%20Android-lightgrey) -[![llama.cpp b10327](https://img.shields.io/badge/llama.cpp-%23b10327-informational)](https://github.com/ggml-org/llama.cpp/releases/tag/b10327) +[![llama.cpp b10333](https://img.shields.io/badge/llama.cpp-%23b10333-informational)](https://github.com/ggml-org/llama.cpp/releases/tag/b10333) [![JPMS](https://img.shields.io/badge/JPMS-modular%20JAR-25A162)](https://openjdk.org/projects/jigsaw/) ![JUnit](https://img.shields.io/badge/tested%20with-JUnit6-25A162) [![JSpecify](https://img.shields.io/badge/JSpecify-1.0.0%20%40NullMarked-25A162)](https://jspecify.dev) diff --git a/docs/history/llama-cpp-breaking-changes.md b/docs/history/llama-cpp-breaking-changes.md index b13bdced..d35d2d08 100644 --- a/docs/history/llama-cpp-breaking-changes.md +++ b/docs/history/llama-cpp-breaking-changes.md @@ -583,3 +583,5 @@ Used during `llama.cpp` version bumps: when upgrading, scan this file from the r | b10319–b10322 | upstream verification (sandbox) | Remaining **6** patches re-verified against a clean b10322 checkout (ggml/llama.cpp commit `f8e30266`): fail-loud `PATCH_COMMAND` clean. **Configure-only verification per this step's process** (full build deferred to the b10327 final target). | | b10322–b10327 | `tools/tts/tts.cpp` (**reference-tool timing/logging tweak — now also reports vocoder time via `get_output`'s existing optional `out_n_samples` param**), `ggml/src/ggml-cuda/cpy.cu` (**GPU backend internal**), `tools/ui/**` (**large contenteditable chat-form rewrite, dominates the diff at 4950 lines**) | **No project-source change (final step of this b10319→b10327 walk).** `tools/mtmd/mtmd-helper.h` has **zero diff** across this whole range — confirmed via a direct diff of just that file. `mtmd_helper::gen_audio::get_output`'s 4th parameter (`int64_t * out_n_samples = nullptr`) already existed before this range; `tools/tts/tts.cpp` merely started passing it (for better timing logs), it did not become newly required. `tts_engine.cpp`'s existing 3-arg `gen.get_output(&sample_rate, &data, &data_len)` call stays valid unchanged. Zero diff under `common/`, `include/`, `tools/mtmd/mtmd.h`, `tools/server/*.h`, `tools/server/*.cpp`, `CMakeLists.txt`, `vendor/sheredom/subprocess.h`. | | b10322–b10327 | upstream verification (sandbox, final target) | Remaining **6** patches re-verified against a clean b10327 checkout (ggml/llama.cpp commit `69bf6437`): fail-loud `PATCH_COMMAND` clean. **Full local verification (mandatory for the final target):** fresh configure + full `cmake --build` (`libjllama.so` + `jllama_test` compile and link, `-O3`, no undefined references) + `ctest` **482/482 passing**. Closes the b10319→b10327 walk (2 steps: b10319→b10322→b10327), on a new branch (`claude/java-llama-cpp-b10327`) following on from the b10310→b10319 walk merged as PR #381. | +| b10327–b10333 | `common/{arg.cpp,common.h}` (**purely additive: new `--tools-runtime` CLI flag + `common_params::server_tools_runtime` field**), `tools/server/{server-tools.h,server-tools.cpp,server.cpp}` (**new opt-in Docker-backed runtime for built-in tools**), `ggml/src/ggml-cuda/{ggml-cuda.cu,rope.cu,rope.cuh}` (**GPU backend internals — CUDA rope kernel rework**) | **No project-source change (single-step bump, diff under the 100 KiB chunking threshold).** `common_params::server_tools_runtime` is a new field appended after the existing `server_tools` member — purely additive, no existing field renamed/removed/reordered. `server_tools::setup(...)` gained a third parameter (`const std::string & tools_runtime`) at its one call site in `server.cpp`'s `llama_server()`, which is inside the region `patches/0006` touches only earlier in the function (embedded-mode flag/argv handling) — verified the patch still applies cleanly (see next row). `jllama.cpp`/`jni_helpers.hpp`/`json_helpers.hpp` reference neither `server_tools` nor `server-tools.h` directly (confirmed via grep), so the 584-line `server-tools.cpp` rewrite (adding an opt-in `--tools-runtime docker:` execution backend) is entirely internal to `server.cpp`'s built-in-tools wiring. CUDA rope-kernel changes are internal to `ggml-cuda`; zero diff under `common/chat.h`, `common/speculative.h`, `tools/mtmd/mtmd.h`, `include/llama-cpp.h`, `common/arg.h` (beyond the additive flag above), `include/llama.h`, `common/download.h`, `tools/server/server-schema.h`. | +| b10327–b10333 | upstream verification (sandbox, final target) | All **6** patches re-verified against a clean b10333 checkout (ggml/llama.cpp commit `08659901c`): sequential `git apply` (filename order, mirroring the `PATCH_COMMAND` applier) succeeded for every patch, including `0006`/`0007` across the touched `server.cpp` region. **Full local verification (mandatory for the final target):** fresh configure + full `cmake --build` (`libjllama.so` + `jllama_test` compile and link, `-O3`, no undefined references) + `ctest` **482/482 passing**. Single-step bump (76 KiB / 71 KiB excl. WebUI, 6 commits — under the 100 KiB chunking threshold), on branch `claude/cuda-toolkit-action-update-0675dk` (continuing the CUDA-toolkit-action-bump branch). | diff --git a/llama/CMakeLists.txt b/llama/CMakeLists.txt index 4dca8b35..c144ec41 100644 --- a/llama/CMakeLists.txt +++ b/llama/CMakeLists.txt @@ -173,7 +173,7 @@ set(LLAMA_BUILD_APP OFF CACHE BOOL "" FORCE) FetchContent_Declare( llama.cpp GIT_REPOSITORY https://github.com/ggerganov/llama.cpp.git - GIT_TAG b10327 + GIT_TAG b10333 PATCH_COMMAND ${CMAKE_COMMAND} -DPATCH_DIR=${CMAKE_CURRENT_SOURCE_DIR}/patches -DLLAMA_SRC= diff --git a/llama/src/main/java/net/ladenthin/llama/value/LlamaCppVersion.java b/llama/src/main/java/net/ladenthin/llama/value/LlamaCppVersion.java index d3fd05c6..6e297018 100644 --- a/llama/src/main/java/net/ladenthin/llama/value/LlamaCppVersion.java +++ b/llama/src/main/java/net/ladenthin/llama/value/LlamaCppVersion.java @@ -10,13 +10,13 @@ * library was compiled against, exposed as a compile-time constant so callers can render a badge or * emit a startup log line without loading the native library. * - *

{@link #LLAMA_CPP_VERSION} is a pure-Java string ({@code "b10327"}) that mirrors the + *

{@link #LLAMA_CPP_VERSION} is a pure-Java string ({@code "b10333"}) that mirrors the * {@code GIT_TAG} in {@code llama/CMakeLists.txt}. It is available even when {@code libjllama} is * absent (pure-Java checkout, before {@code System.load}), which is what makes it suitable for a * lightweight version badge in Android or other UIs.

* *

For the authoritative value that is baked into the native binary — the build number - * plus the resolved upstream commit, e.g. {@code "b10327-"} — call + * plus the resolved upstream commit, e.g. {@code "b10333-"} — call * {@link net.ladenthin.llama.LlamaModel#getLlamaCppBuildInfo()} instead; that reads llama.cpp's own * {@code build-info} through JNI and therefore cannot drift from the compiled library (but requires * the native library to be loaded).

@@ -24,14 +24,14 @@ public final class LlamaCppVersion { /** - * The pinned llama.cpp release tag this library was built against, e.g. {@code "b10327"}. + * The pinned llama.cpp release tag this library was built against, e.g. {@code "b10333"}. * *

Kept in lockstep with {@code GIT_TAG} in {@code llama/CMakeLists.txt} — see the * "Upgrading/Downgrading llama.cpp Version" checklist in {@code CLAUDE.md}. This is the * compile-time pin; use {@link net.ladenthin.llama.LlamaModel#getLlamaCppBuildInfo()} for the * value actually linked into the native binary.

*/ - public static final String LLAMA_CPP_VERSION = "b10327"; + public static final String LLAMA_CPP_VERSION = "b10333"; // Constants holder — not instantiable. private LlamaCppVersion() {}