Upgrade llama.cpp to b10423 and adapt to server_slot_stats API

CLAUDE.md

@@ -6,7 +6,7 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co
 
 Java bindings for [llama.cpp](https://github.com/ggerganov/llama.cpp) via JNI, providing a high-level API for LLM inference in Java. The Java layer communicates with a native C++ library through JNI.
 
-Current llama.cpp pinned version: **b10360**
+Current llama.cpp pinned version: **b10423**
 
 ## Upgrading CUDA Version
 
@@ -429,7 +429,7 @@ needs no extra step here, `build-webui` re-reads the tag and rebuilds the matchi
 ships no UI):
 ```bash
 # needs node/npm + network; embed.cpp is plain C++17 (no npm)
-git clone --depth 1 --branch b10360 https://github.com/ggml-org/llama.cpp /tmp/lc
+git clone --depth 1 --branch b10423 https://github.com/ggml-org/llama.cpp /tmp/lc
 ( cd /tmp/lc/tools/ui && npm ci && npm run build \
   && ( cd dist && find . -type f -not -path './_gzip/*' \
        | while read -r f; do mkdir -p "_gzip/$(dirname "$f")"; gzip -9 -c "$f" > "_gzip/$f"; done ) \
@@ -469,7 +469,7 @@ cache lives in **Depot Cache** over sccache's **WebDAV** backend:
 - `SCCACHE_WEBDAV_TOKEN: ${{ secrets.DEPOT_TOKEN }}` — a Depot **organization** token, stored
   as the repo secret **`DEPOT_TOKEN`**.
 
-Because `sccache` is **content-addressed** and llama.cpp is pinned (`GIT_TAG b10360`), the
+Because `sccache` is **content-addressed** and llama.cpp is pinned (`GIT_TAG b10423`), the
 ~280 upstream object files are byte-identical every run, so a warm cache recompiles only the
 *changed* files. Depot's cache is **shared across all branches** (unlike GitHub's
 per-branch `actions/cache`), so every branch builds incrementally; a `b<nnnn>` version bump
@@ -1286,17 +1286,17 @@ ctest --test-dir build --output-on-failure -R "ResultsToJson"
 | File | Tests | Scope |
 |------|-------|-------|
 | `src/test/cpp/test_utils.cpp` | 162 | Upstream helpers: `server_tokens`, `server_grammar_trigger`, `gen_tool_call_id`, `json_value`, `json_get_nested_values`, UTF-8 helpers, `format_response_rerank`, `format_embeddings_response_oaicompat`, `oaicompat_completion_params_parse`, `oaicompat_chat_params_parse`, `are_lora_equal`, `strip_flag_from_argv`, `token_piece_value`, `json_is_array_and_contains_numbers`, `format_oai_sse`, `format_oai_resp_sse`, `format_anthropic_sse`, `parse_lora_request` |
-| `src/test/cpp/test_server.cpp` | 201 | Upstream result types: `result_timings`, `task_params::to_json()` (incl. `dry_sequence_breakers`, `preserved_tokens`, `timings_per_token`), `completion_token_output`, `server_task_result_cmpl_partial` (non-oaicompat + `to_json_oaicompat` + logprobs + `to_json_oaicompat_chat` + `to_json_anthropic` + dispatcher), `server_task_result_cmpl_final` (non-oaicompat + `to_json_oaicompat` + `to_json_oaicompat_chat` + `to_json_oaicompat_chat_stream` + `to_json_anthropic` + `to_json_anthropic_stream` + tool_calls + dispatcher), `server_task_result_embd`, `server_task_result_rerank`, `server_task_result_metrics`, `server_task_result_slot_save_load`, `server_task_result_slot_erase`, `server_task_result_apply_lora`, `server_task_result_get_lora`, `server_task_result_error`, `format_error_response`, `server_task::need_sampling()`, `server_task::n_tokens()`, `server_schema::eval_llama_cmpl_schema()` (parsing pipeline + grammar routing + error paths + per-request `dry_*` and `sse_ping_interval` field round-trips incl. hard-limit + server-default inheritance), `response_fields` projection |
+| `src/test/cpp/test_server.cpp` | 204 | Upstream result types: `server_slot_stats` (the `timings` JSON payload; replaced `result_timings` in b10408), `task_params::to_json()` (incl. `dry_sequence_breakers`, `preserved_tokens`, `timings_per_token`), `completion_token_output`, `server_task_result_cmpl_partial` (non-oaicompat + `to_json_oaicompat` + logprobs + `to_json_oaicompat_chat` + `to_json_anthropic` + dispatcher), `server_task_result_cmpl_final` (non-oaicompat + `to_json_oaicompat` + `to_json_oaicompat_chat` + `to_json_oaicompat_chat_stream` + `to_json_anthropic` + `to_json_anthropic_stream` + tool_calls + dispatcher), `server_task_result_embd`, `server_task_result_rerank`, `server_task_result_metrics` (`to_json()` = the `/slots` array + `to_metrics()` = the `/metrics` Prometheus text), `server_task_result_slot_save_load`, `server_task_result_slot_erase`, `server_task_result_apply_lora`, `server_task_result_get_lora`, `server_task_result_error`, `format_error_response`, `server_task::need_sampling()`, `server_task::n_tokens()`, `server_schema::eval_llama_cmpl_schema()` (parsing pipeline + grammar routing + error paths + per-request `dry_*` and `sse_ping_interval` field round-trips incl. hard-limit + server-default inheritance), `response_fields` projection |
 | `src/test/cpp/test_json_helpers.cpp` | 50 | All functions in `json_helpers.hpp`: `get_result_error_message`, `results_to_json`, `rerank_results_to_json` (incl. missing/out-of-range `index` rejection), `parse_encoding_format`, `extract_embedding_prompt`, `is_infill_request`, `parse_slot_prompt_similarity`, `parse_positive_int_config`, `wrap_stream_chunk` |
 | `src/test/cpp/test_log_helpers.cpp` | 13 | All functions in `log_helpers.hpp`: `log_level_name`, `format_log_as_json` |
 | `src/test/cpp/test_jni_helpers.cpp` | 54 | All functions in `jni_helpers.hpp` using a zero-filled `JNINativeInterface_` mock (incl. the `utf8_to_jstring_impl` byte-array string path: emoji byte-preservation, truncated-UTF-8 replace-not-throw) |
 | `src/test/cpp/test_tts_wav.cpp` | 2 | The in-memory WAV writer `pcm_to_wav16_bytes` in `tts_wav.hpp` (WAV header/payload + little-endian clamping) — our own code, not upstream. The Qwen3-TTS pipeline it pairs with (`mtmd_helper::gen_audio`) is entirely upstream-owned (no project-side DSP to unit-test here) and covered end-to-end by the Java `TtsIntegrationTest`. |
 
-**Current total: 482 tests (all passing).**
+**Current total: 485 tests (all passing).**
 
 #### Upstream source location (in CMake build tree)
 
-llama.cpp is fetched via CMake FetchContent, pinned to `GIT_TAG b10360`.
+llama.cpp is fetched via CMake FetchContent, pinned to `GIT_TAG b10423`.
 
 **GoogleTest** is a separate `BUILD_TESTING`-only FetchContent (`GIT_TAG v1.17.0`), used solely
 by the `jllama_test` C++ unit-test binary — not by the shipped library, and not coupled to the

README.md

@@ -11,7 +11,7 @@
 **Build:**  
 ![Java 8+](https://img.shields.io/badge/Java-8%2B-informational)  
 ![Platform](https://img.shields.io/badge/Platform-Linux%20%7C%20macOS%20%7C%20Windows%20%7C%20Android-lightgrey)  
-[![llama.cpp b10360](https://img.shields.io/badge/llama.cpp-%23b10360-informational)](https://github.com/ggml-org/llama.cpp/releases/tag/b10360)  
+[![llama.cpp b10423](https://img.shields.io/badge/llama.cpp-%23b10423-informational)](https://github.com/ggml-org/llama.cpp/releases/tag/b10423)  
 [![JPMS](https://img.shields.io/badge/JPMS-modular%20JAR-25A162)](https://openjdk.org/projects/jigsaw/)  
 ![JUnit](https://img.shields.io/badge/tested%20with-JUnit6-25A162)  
 [![JSpecify](https://img.shields.io/badge/JSpecify-1.0.0%20%40NullMarked-25A162)](https://jspecify.dev)  

docs/history/llama-cpp-breaking-changes.md

@@ -599,3 +599,14 @@ Used during `llama.cpp` version bumps: when upgrading, scan this file from the r
 | b10355–b10359 | `include/llama.h` (**two comment lines removed only, `llama_sampler_copy` signature unchanged**), `ggml/src/ggml-opencl/**` (**flash-attn kernel work — compiled but no header API called directly**), `ggml/src/ggml-webgpu/**` (**unused backend**) | **No project-source change (final step of the b10333→b10359 walk).** Rest of the range is upstream CI workflow files, no C++ surface this project touches. |
 | b10355–b10359 | upstream verification (sandbox, final target) | All **6** patches re-verified against a clean b10359 checkout (ggml/llama.cpp commit `84f712946`): fail-loud `PATCH_COMMAND` clean. **Full local verification (mandatory for the final target):** fresh configure + full `cmake --build` (`libjllama.so` + `jllama_test` compile and link, `-O3`, no undefined references) + `ctest` **482/482 passing**. Closes the b10333→b10359 walk (5 steps: b10333→b10338→b10342→b10344→b10353→b10354→b10355→b10359), on branch `claude/java-llama-cpp-b10359`. |
 | b10359–b10360 | `common/peg-parser.cpp` (**suppress incomplete escape sequences in the JSON PEG grammar, #26780**) | **No project-source change (single-step, final target).** Internal bugfix; `common/peg-parser.h` itself has zero diff (also on the "safe to skip" list). All **6** patches re-verified against a clean b10360 checkout (ggml/llama.cpp commit `48d22e295`): fail-loud `PATCH_COMMAND` clean. **Full local verification (mandatory for the final target):** fresh configure + full `cmake --build` (`libjllama.so` + `jllama_test` compile and link, `-O3`, no undefined references) + `ctest` **482/482 passing**. Same branch as the b10333→b10359 walk (`claude/java-llama-cpp-b10359`). |
+| b10360–b10362 | `src/models/exaone4.cpp` (**SWA not enabled for EXAONE 4.5, #26848**), `tests/test-backend-sampler.cpp` + `ci/run.sh` (**upstream CI only**) | **No project-source change.** Model-internal fix; zero diff under `common/`, `include/`, `tools/mtmd/`, `tools/server/`, `CMakeLists.txt`. First step of the b10360→b10423 walk. |
+| b10362–b10369 | `tools/mtmd/{mtmd.h,mtmd-helper.h,mtmd-helper-gen.cpp,clip*,models/pockettts-*}` (**pocket-tts support, #26871**), `include/llama.h` + `common/common.h` (**new `LLAMA_LOAD_MODE_AUTO`, now the `common_params::load_mode` default, #26081**) | **Project-source change: `src/main/cpp/tts_engine.cpp`.** `mtmd_helper_gen_audio_step_gen()` gained a `bool * out_stop` output parameter and now reports end-of-speech itself (also signalled by a null `h_state_out`); `mtmd_helper_gen_audio_inp` gained a `seed` field. The C++ wrapper `mtmd_helper::gen_audio::step_gen()` defaults `out_stop` to `nullptr`, so the old 3-arg call still *compiled* — the break was **semantic**, not a compile error: upstream's own `tools/tts/tts.cpp` dropped its `llama_vocab_is_eog(vocab, sampled)` loop guard in the same commit because pocket-tts has no discrete backbone token to test. `tts_engine.cpp` now mirrors upstream: loop on `out_stop` / null `h_next` instead of the EOG token (for Qwen3-TTS this is equivalent — upstream sets `out_stop` for exactly the `codec_eos`/EOG tokens the old check tested, verified in `mtmd-helper-gen.cpp`), and forwards the caller-supplied seed to `inp.seed` as well as to the backbone `common_sampler`. `mtmd.h` additions (`MTMD_GEN_AUDIO_TYPE_POCKETTTS`, `mtmd_gen_inp_default()`, `feats`/`is_eos`/`temp` fields) are appended, not renumbered. |
+| b10369–b10375 | `common/chat.cpp` (**Qwen bare-function tool-call parsing, #26793**), `src/models/{dflash,nemotron-h}.cpp`, `tools/imatrix/imatrix.cpp`, conversion scripts | **No project-source change.** `common/chat.h` itself has zero diff; the parser change is internal to an upstream-compiled TU. |
+| b10375–b10380 | `common/speculative.h` (**removes `common_speculative_need_embd()` / `common_speculative_need_embd_nextn()`, #26904**), `common/chat.cpp` (**muse-glimmer tool calls after EOM, #26879**), `ggml/src/ggml-cuda/**` | **No project-source change.** Confirmed via grep that no project source calls any `common_speculative_*` function, so the two removed declarations are confined to upstream translation units (`server-context.cpp` and `examples/speculative-simple` are their only callers). |
+| b10380–b10398 | **`CMakeLists.txt` + `common/CMakeLists.txt` + `src/CMakeLists.txt` + `tools/mtmd/CMakeLists.txt` (semantic versioning, #26839 — `LLAMA_INSTALL_VERSION` replaced by `LLAMA_VERSION_BASE`/`LLAMA_VERSION_MAJOR`)**, `tools/server/{server-common.h,server-common.cpp,server-context.cpp}` (**slot save/restore with media, #26640 — `server_tokens` loses `has_media()`, gains `serialize()`/`deserialize()`**), `common/{arg.cpp,common.h}` (**system/user `config.ini`, #26118 — new `fs_get_config_directory()`**), `common/build-info.{h,cpp.in}` (**`llama_print_build_info()` takes a version string**), `include/llama.h` (**additive `llama_version()`**) | **Project-source change: `llama/CMakeLists.txt` (build-system break).** This project adds `tools/mtmd` via `add_subdirectory()` from *its own* directory scope, where llama.cpp's variables are not visible, and `tools/mtmd/CMakeLists.txt` passes them to `set_target_properties(mtmd PROPERTIES VERSION … SOVERSION …)` — an undefined variable makes that argument list odd-length and aborts the configure. The pre-existing `LLAMA_INSTALL_VERSION` fallback (added for exactly this reason) no longer covers it, so the guard now defines `LLAMA_VERSION_BASE`/`LLAMA_VERSION_MAJOR` instead. Both stay neutral placeholders: `BUILD_SHARED_LIBS` is OFF, so `mtmd` is a static library and VERSION/SOVERSION are no-ops. The other API changes need no project change — `jllama.cpp` calls only `llama_build_info()` (unchanged), not `llama_print_build_info()`, and no project source calls `server_tokens::has_media()`. |
+| b10398–b10405 | `ggml/src/ggml-cpu/**` (**arm unused-variable fix**), `ggml/src/ggml-hip/CMakeLists.txt` (**drops `-funsafe-math-optimizations`**), `tools/ui/**` (**WebUI refactors**) | **No project-source change.** The WebUI is auto-followed by CI's `build-webui` job from the pinned tag, so its refactors need no per-bump review. |
+| b10405–b10408 | **`tools/server/{server-task.h,server-task.cpp,server-common.h,server-common.cpp,server-context.cpp}` (metrics refactor + correctness fixes, #26920)** | **Project-source change: `src/test/cpp/test_server.cpp`.** `result_timings` was **removed** and replaced by `server_slot_stats` (raw counters + absolute microsecond timestamps, with every duration/rate derived in accessors); `server_task_result_cmpl_{partial,final}::timings` became `::stats`, and the `"timings"` JSON block is now gated on `stats.is_set()` (`t_start > 0`) instead of a non-negative `prompt_n`. The emitted JSON keys are unchanged, so the **client-facing contract and the Java layer are unaffected** — no `src/main/cpp` file references any of these types (verified via grep). `server_task_result_metrics` lost its flat counter fields to an embedded `server_metrics`, its `to_json()` now returns the `/slots` array verbatim, and the new `to_metrics()` renders the `/metrics` Prometheus exposition text. The C++ regression suite was rewritten to match: `ResultTimings` → `ServerSlotStats` (same JSON assertions, plus `is_set()` and unstarted-generation coverage), and the metrics suite now parses `to_metrics()` samples (including that the per-draft-position labeled series appears only when populated). Suite total 482 → **485**. |
+| b10408–b10418 | `common/speculative.{h,cpp}` + `common/arg.cpp` (**additive `common_speculative_types_from_gguf()` for draft-type auto-detection, #26814**), `tools/server/server-http.cpp` (**`index.html` served `no-cache`, hashed assets stay `immutable`, #27006**), `ggml/src/ggml-{sycl,metal,cpu}/**` | **No project-source change.** The speculative addition is a new free function, no existing signature touched; the `server-http.cpp` caching change is internal to upstream's static-asset routes (and inert here unless the WebUI assets are compiled in). |
+| b10418–b10419 | `ggml/src/ggml-openvino/**` (**Qwen3.5 support, memory optimization, new ops, #26952**) | **No project-source change.** A single commit entirely inside the OpenVINO backend; no shared header touched, so only the `openvino-{linux,windows}-x86-64` classifier's compiled sources change. |
+| b10419–b10423 | `common/{common.h,common.cpp}` + `tools/completion/completion.cpp` (**shared CPU-parameter handling, #27026 — new `common_threadpools` RAII helper, `ggml_threadpool_params_from_cpu_params()` moved next to it**), `tools/ui/**` | **No project-source change (final step).** `ggml_threadpool_params_from_cpu_params()` moved within `common.h` but kept its signature; confirmed via grep that no project source mentions `threadpool` at all. |
+| b10360–b10423 | upstream verification (sandbox, final target) | All **6** patches re-verified against a clean b10423 checkout: sequential `git apply` (filename order, mirroring the `PATCH_COMMAND` applier) succeeded at **every** intermediate tag of the walk, and the fail-loud `PATCH_COMMAND` ran clean on a fresh `cmake -B build` at the target — no patch needed refreshing across the whole range. **Full local verification (mandatory for the final target):** fresh configure + full `cmake --build` (`libjllama.so` + `jllama_test` compile and link, `-O3`, no undefined references) + `ctest` **485/485 passing**. Walk of 10 steps (b10360→b10362→b10369→b10375→b10380→b10398→b10405→b10408→b10418→b10419→b10423) on branch `claude/update-b10423-ciih0h`. Four steps exceeded the 100 KiB chunking threshold unavoidably — upstream has **no** intermediate tags there (b10363–b10368, b10381–b10397, b10406–b10407 and b10420–b10422 do not exist), so the tag list itself, not the threshold, bounded the step size. |

llama/CMakeLists.txt

@@ -173,7 +173,7 @@ set(LLAMA_BUILD_APP OFF CACHE BOOL "" FORCE)
 FetchContent_Declare(
 	llama.cpp
 	GIT_REPOSITORY https://github.com/ggerganov/llama.cpp.git
-	GIT_TAG        b10360
+	GIT_TAG        b10423
 	PATCH_COMMAND  ${CMAKE_COMMAND}
 		-DPATCH_DIR=${CMAKE_CURRENT_SOURCE_DIR}/patches
 		-DLLAMA_SRC=<SOURCE_DIR>
@@ -193,12 +193,20 @@ endif()
 # LLAMA_BUILD_TOOLS defaults to LLAMA_STANDALONE, which is OFF when llama.cpp
 # is consumed via FetchContent.  Build mtmd explicitly so the target exists.
 if(NOT TARGET mtmd)
-    # LLAMA_INSTALL_VERSION is set inside llama.cpp's directory scope and is not
-    # visible here.  tools/mtmd/CMakeLists.txt uses it in set_target_properties()
-    # as a VERSION value; if the variable is empty the token list after PROPERTIES
-    # becomes odd-length and CMake aborts with "incorrect number of arguments".
-    if(NOT DEFINED LLAMA_INSTALL_VERSION)
-        set(LLAMA_INSTALL_VERSION "0")
+    # LLAMA_VERSION_BASE / LLAMA_VERSION_MAJOR are set inside llama.cpp's directory
+    # scope and are not visible here.  tools/mtmd/CMakeLists.txt uses them in
+    # set_target_properties() as VERSION/SOVERSION values; if a variable is empty the
+    # token list after PROPERTIES becomes odd-length and CMake aborts with "incorrect
+    # number of arguments".  Both are cosmetic for this build (BUILD_SHARED_LIBS is
+    # OFF, so mtmd is a static library and VERSION/SOVERSION are no-ops), hence the
+    # neutral placeholder rather than a mirror of upstream's numbers.
+    # (b10398 replaced the older LLAMA_INSTALL_VERSION with these two — upstream
+    # #26839, "cmake: introduce semantic versioning".)
+    if(NOT DEFINED LLAMA_VERSION_BASE)
+        set(LLAMA_VERSION_BASE "0")
+    endif()
+    if(NOT DEFINED LLAMA_VERSION_MAJOR)
+        set(LLAMA_VERSION_MAJOR "0")
     endif()
     add_subdirectory(${llama.cpp_SOURCE_DIR}/tools/mtmd ${llama.cpp_BINARY_DIR}/tools/mtmd)
 endif()