Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co

Java bindings for [llama.cpp](https://github.com/ggerganov/llama.cpp) via JNI, providing a high-level API for LLM inference in Java. The Java layer communicates with a native C++ library through JNI.

Current llama.cpp pinned version: **b10154**
Current llama.cpp pinned version: **b10173**

## Upgrading CUDA Version

Expand Down Expand Up @@ -429,7 +429,7 @@ needs no extra step here, `build-webui` re-reads the tag and rebuilds the matchi
ships no UI):
```bash
# needs node/npm + network; embed.cpp is plain C++17 (no npm)
git clone --depth 1 --branch b10154 https://github.com/ggml-org/llama.cpp /tmp/lc
git clone --depth 1 --branch b10173 https://github.com/ggml-org/llama.cpp /tmp/lc
( cd /tmp/lc/tools/ui && npm ci && npm run build \
&& ( cd dist && find . -type f -not -path './_gzip/*' \
| while read -r f; do mkdir -p "_gzip/$(dirname "$f")"; gzip -9 -c "$f" > "_gzip/$f"; done ) \
Expand Down Expand Up @@ -469,7 +469,7 @@ cache lives in **Depot Cache** over sccache's **WebDAV** backend:
- `SCCACHE_WEBDAV_TOKEN: ${{ secrets.DEPOT_TOKEN }}` — a Depot **organization** token, stored
as the repo secret **`DEPOT_TOKEN`**.

Because `sccache` is **content-addressed** and llama.cpp is pinned (`GIT_TAG b10154`), the
Because `sccache` is **content-addressed** and llama.cpp is pinned (`GIT_TAG b10173`), the
~280 upstream object files are byte-identical every run, so a warm cache recompiles only the
*changed* files. Depot's cache is **shared across all branches** (unlike GitHub's
per-branch `actions/cache`), so every branch builds incrementally; a `b<nnnn>` version bump
Expand Down Expand Up @@ -1263,7 +1263,7 @@ ctest --test-dir build --output-on-failure -R "ResultsToJson"

#### Upstream source location (in CMake build tree)

llama.cpp is fetched via CMake FetchContent, pinned to `GIT_TAG b10154`.
llama.cpp is fetched via CMake FetchContent, pinned to `GIT_TAG b10173`.

**GoogleTest** is a separate `BUILD_TESTING`-only FetchContent (`GIT_TAG v1.17.0`), used solely
by the `jllama_test` C++ unit-test binary — not by the shipped library, and not coupled to the
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@
**Build:**
![Java 8+](https://img.shields.io/badge/Java-8%2B-informational)
![Platform](https://img.shields.io/badge/Platform-Linux%20%7C%20macOS%20%7C%20Windows%20%7C%20Android-lightgrey)
[![llama.cpp b10154](https://img.shields.io/badge/llama.cpp-%23b10154-informational)](https://github.com/ggml-org/llama.cpp/releases/tag/b10154)
[![llama.cpp b10173](https://img.shields.io/badge/llama.cpp-%23b10173-informational)](https://github.com/ggml-org/llama.cpp/releases/tag/b10173)
[![JPMS](https://img.shields.io/badge/JPMS-modular%20JAR-25A162)](https://openjdk.org/projects/jigsaw/)
![JUnit](https://img.shields.io/badge/tested%20with-JUnit6-25A162)
[![JSpecify](https://img.shields.io/badge/JSpecify-1.0.0%20%40NullMarked-25A162)](https://jspecify.dev)
Expand Down
1 change: 1 addition & 0 deletions docs/history/llama-cpp-breaking-changes.md
Original file line number Diff line number Diff line change
Expand Up @@ -515,3 +515,4 @@ Used during `llama.cpp` version bumps: when upgrading, scan this file from the r
| b10092–b10107 | project source change (`train_engine.cpp`) + upstream verification (sandbox) | `train_engine.cpp` updated for the `use_mmap`→`load_mode` API break (see the row above); `LLAMA_LOAD_MODE_NONE` resolves via the already-included `llama.h`. All **six** patches (`0001`–`0003`, `0006`–`0008`) re-verified against a clean b10107 checkout (ggml/llama.cpp commit `c0bc8591`): applied in filename order via `git apply`, all clean — no patch-target file changed in the range (`0006`'s standalone `git apply --check` fails only because its context is the post-`0001` tree, as documented). The OuteTTS generator anchors hold (`tools/tts/tts.cpp` unchanged). Per-platform build + `ctest` confirmation by the CI pipeline. |
| b10107–b10154 | `common/{common.h,chat.h,chat.cpp,arg.cpp,arg.h}` + `tools/server/server.cpp` (**patch-target restructure — MCP-server support**) + new `common/{subproc.*,trie.*}` + `ggml/**` backends + `tools/ui/**` | **Two local patches refreshed; no project-source change (47 commits, ~100 files, ~5k line changes).** Upstream added **MCP-server support** ([#26170](https://github.com/ggml-org/llama.cpp/pull/26170) et al.): `common/arg.{cpp,h}` gain `common_print_available_devices()` + `--mcp-servers-config`/`--mcp-servers-json`; `common/{common.h,chat.h,chat.cpp}` change `thinking_end_tag` → `thinking_end_tags` (vector) and add `mcp_servers_*` fields (reasoning-budget internals — inside upstream-compiled `llama-common`/chat TUs, not referenced by project source). In `tools/server/server.cpp`, `llama_server()` gains a `server_mcp mcp_mgr` lifecycle (`mcp_mgr.start()`, **2-arg** `tools.setup(server_tools, mcp_mgr)` — was 1-arg — `mcp_mgr.shutdown()` in the cleanup lambdas) + a "MCP servers (experimental)" warn branch, and the resumable-streaming routes were **renamed** `/v1/stream/:conv_id` → `/v1/stream` and **moved** into the middle of the route/CORS/tools block. **Patch `0001` refreshed:** one `common/arg.h` hunk drifted (b10154 inserted `common_print_available_devices()` between `common_params_parse` and `common_params_to_map`); the other 36 files (incl. `common/arg.cpp`, the ~34 `common_params_parse_main` call sites, `tests/test-arg-parser.cpp`, `tools/server/server.cpp`, `tools/tts/tts.cpp`) applied unchanged. **Patch `0007` reworked** (the MCP-server restructure broke its contiguous extraction): the shared helper `llama_server_register_common_routes()` was **narrowed to the pure core route table** (`health` … `slots`) — `llama_server()` now keeps the streaming/GCP/CORS-proxy/MCP/tools blocks inline, **byte-identical to upstream b10154** (only the route-table block is factored out) — and `llama_server_attach()` registers the route table + non-router streaming on the new `/v1/stream` paths + GCP compat + 403 stubs for the experimental `/cors-proxy` + `/tools` endpoints it does not wire. `0002`/`0003`/`0006`/`0008` applied unchanged. b10154 is the topmost release at bump time. |
| b10107–b10154 | patch refresh (`0001`,`0007`) + upstream verification (sandbox) | Both refreshed patches + `0002`/`0003`/`0006`/`0008` re-verified against a clean b10154 checkout (ggml/llama.cpp commit `0e4a03622`): applied in filename order via `git apply`, all clean, no rejects; the fail-loud `PATCH_COMMAND` (`apply-llama-patches.cmake`) and the OuteTTS generator (`tts.cpp @ b10154`, all anchors held) both passed on a fresh `cmake -B build -DBUILD_TESTING=ON`. **Full `jllama` shared-library build (compile + link) verified locally** (`libjllama.so`, `-O3`) — the reworked `server.cpp` (helper + `llama_server_attach`) and all project TUs compile and **link** against b10154 (no undefined references), confirming the MCP-server-era API surface. No project source needed changing (`train_engine.cpp`'s prior `load_mode` fix from b10092–b10107 still holds). Per-platform build + `ctest` confirmation by the CI pipeline. |
| b10154–b10173 | `common/{common.h,common.cpp,chat.h,speculative.cpp}` (**`common_memory` refactor**) + `tools/server/server-context.cpp` + `src/**` model TUs + `tools/ui/**` | **No patch change, no project-source change (85 files, ~5k line changes; ~393 KiB full diff — the chunking runbook flags it, but the reviewed non-WebUI surface is all additive/internal).** The one notable API change: the free functions `common_context_seq_rm`/`seq_add`/`seq_cp` (`common/common.h`/`common.cpp`) were made **`static`** and wrapped in a new **`struct common_memory`** (`init` + `seq_rm`/`seq_add`/`seq_cp` methods) that `tools/server/server-context.cpp` now calls via a per-slot `common_memory mem` member — **entirely upstream-internal**; no project C++ (`jllama.cpp`, `json_helpers.hpp`, `jni_helpers.hpp`, tests) references the removed free functions, so nothing to change. Everything else is additive: new `common_chat_format` enum value (`COMMON_CHAT_FORMAT_PEG_MINIMAX_M3`), new speculative enum (`COMMON_SPECULATIVE_TYPE_DRAFT_DSPARK`) + `common/speculative.cpp` DSpark support, `common/chat.cpp` PEG-parser additions, new `src/models/{dflash,eagle3}.cpp` model TUs, and trivial `server-task.{h,cpp}`/`server-schema.cpp` changes (param rename `ctx_main`→`ctx_tgt`, an added `SRV_TRC` trace, a blank line). New added files (`models/templates/MiniMax-M3.jinja`, `tools/mtmd/models/{mimo-audio,parakeet}.cpp`) are upstream-glob-managed (data/mtmd), not in our manually-listed `target_sources`; no server source we list was renamed or removed. All **7** patches (`0001`–`0003`, `0006`–`0009`) apply unchanged (only `0002`/`0003` touch `server-context.cpp`, and both hunks — line ~1152 load-progress-callback, line ~3965 slot-similarity getters — are clear of the memory-refactor hunks). **Full local verification:** fresh `cmake -B build -DBUILD_TESTING=ON` (fail-loud patch apply clean, OuteTTS generator regenerated from `tts.cpp @ b10173` with all anchors holding, ggml commit `e9fa0781f`) + full `cmake --build` (`libjllama.so` + `jllama_test` compile and link, `-O3`, no undefined references) + `ctest` **485/485 passing**. b10173 is the topmost release at bump time. Per-platform build + `ctest` confirmation by the CI pipeline. |
4 changes: 2 additions & 2 deletions llama/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -173,7 +173,7 @@ set(LLAMA_BUILD_APP OFF CACHE BOOL "" FORCE)
FetchContent_Declare(
llama.cpp
GIT_REPOSITORY https://github.com/ggerganov/llama.cpp.git
GIT_TAG b10154
GIT_TAG b10173
PATCH_COMMAND ${CMAKE_COMMAND}
-DPATCH_DIR=${CMAKE_CURRENT_SOURCE_DIR}/patches
-DLLAMA_SRC=<SOURCE_DIR>
Expand All @@ -196,7 +196,7 @@ execute_process(
COMMAND ${CMAKE_COMMAND}
-DTTS_SRC=${llama.cpp_SOURCE_DIR}/tools/tts/tts.cpp
-DOUT_CPP=${JLLAMA_TTS_GEN_CPP}
-DLLAMA_TAG=b10154
-DLLAMA_TAG=b10173
-P ${CMAKE_CURRENT_SOURCE_DIR}/cmake/generate-tts-upstream.cmake
RESULT_VARIABLE JLLAMA_TTS_GEN_RESULT
)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,28 +10,28 @@
* library was compiled against, exposed as a compile-time constant so callers can render a badge or
* emit a startup log line without loading the native library.
*
* <p>{@link #LLAMA_CPP_VERSION} is a pure-Java string ({@code "b10154"}) that mirrors the
* <p>{@link #LLAMA_CPP_VERSION} is a pure-Java string ({@code "b10173"}) that mirrors the
* {@code GIT_TAG} in {@code llama/CMakeLists.txt}. It is available even when {@code libjllama} is
* absent (pure-Java checkout, before {@code System.load}), which is what makes it suitable for a
* lightweight version badge in Android or other UIs.</p>
*
* <p>For the <em>authoritative</em> value that is baked into the native binary — the build number
* plus the resolved upstream commit, e.g. {@code "b10154-<commit>"} — call
* plus the resolved upstream commit, e.g. {@code "b10173-<commit>"} — call
* {@link net.ladenthin.llama.LlamaModel#getLlamaCppBuildInfo()} instead; that reads llama.cpp's own
* {@code build-info} through JNI and therefore cannot drift from the compiled library (but requires
* the native library to be loaded).</p>
*/
public final class LlamaCppVersion {

/**
* The pinned llama.cpp release tag this library was built against, e.g. {@code "b10154"}.
* The pinned llama.cpp release tag this library was built against, e.g. {@code "b10173"}.
*
* <p>Kept in lockstep with {@code GIT_TAG} in {@code llama/CMakeLists.txt} — see the
* "Upgrading/Downgrading llama.cpp Version" checklist in {@code CLAUDE.md}. This is the
* compile-time pin; use {@link net.ladenthin.llama.LlamaModel#getLlamaCppBuildInfo()} for the
* value actually linked into the native binary.</p>
*/
public static final String LLAMA_CPP_VERSION = "b10154";
public static final String LLAMA_CPP_VERSION = "b10173";

// Constants holder — not instantiable.
private LlamaCppVersion() {}
Expand Down
Loading