Skip to content

fix(litellm): parse DeepSeek-V3 proprietary inline tool-call tokens#5654

Open
fuchun1010 wants to merge 1 commit intogoogle:mainfrom
fuchun1010:fix/deepseek-tool-call-parsing
Open

fix(litellm): parse DeepSeek-V3 proprietary inline tool-call tokens#5654
fuchun1010 wants to merge 1 commit intogoogle:mainfrom
fuchun1010:fix/deepseek-tool-call-parsing

Conversation

@fuchun1010
Copy link
Copy Markdown

Closes #5024

Problem

DeepSeek-V3 emits tool calls using proprietary special tokens embedded in the content field:

<|tool▁calls▁begin|><|tool▁call▁begin|>function<|tool▁sep|>analysis_input
```json
{"work_dir_name":"..."}
```<|tool▁call▁end|><|tool▁calls▁end|>

When LiteLLM does not translate these into structured tool_calls (intermittent), ADK's fallback JSON parser finds the JSON object but rejects it because the function name (analysis_input) is embedded in the tokens (<|tool▁sep|>analysis_input) rather than as a name key inside the JSON payload.

Result: tool call is silently dropped and the raw tokens appear as text content.

Solution

  • Added _parse_deepseek_tool_calls_from_text — detects the proprietary token format, extracts function name + arguments, and emits standard ChatCompletionMessageToolCall objects
  • Added _extract_json_from_deepseek_args helper — handles optional Markdown code fences (```json ```) around the arguments payload
  • Integrated into the existing _parse_tool_calls_from_text as the first-pass parser, with fallback to generic inline JSON parsing
  • Supports: single tool calls, multi-tool calls, code-fenced JSON, bare JSON, surrounding text, mixed formats

Testing Plan

Unit Tests: Added 8 new tests covering:

  • Single tool call with code-fenced JSON args
  • Multiple tool calls in a single wrapped block
  • Bare JSON args (no code fences)
  • Tool call embedded in surrounding text
  • Text without DeepSeek tokens (no false positives)
  • Empty/whitespace-only text
  • Integration test via _parse_tool_calls_from_text
  • Mixed formats (DeepSeek tokens + standard inline JSON)

Regression: Full test_litellm.py: 264 passed, 0 failed

Files Changed

File Changes
src/google/adk/models/lite_llm.py +147 lines (2 new functions + integration)
tests/unittests/models/test_litellm.py +124 lines (8 new test functions)

@google-cla
Copy link
Copy Markdown

google-cla Bot commented May 10, 2026

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

@fuchun1010 fuchun1010 force-pushed the fix/deepseek-tool-call-parsing branch from c319bae to e91b1f6 Compare May 10, 2026 15:15
DeepSeek-V3 emits tool calls using proprietary special tokens
(<|tool▁calls▁begin|>…<|tool▁call▁begin|>function<|tool▁sep|>NAME)
embedded in the content field.  When LiteLLM does not translate these
into structured tool_calls (intermittent), the existing fallback JSON
parser rejects the payload because the function name is stored inside
the tokens rather than as a 'name' key in the JSON object.

Add _parse_deepseek_tool_calls_from_text that detects the proprietary
token format, extracts the function name and arguments, and emits
standard ChatCompletionMessageToolCall objects.  Integrate it into the
existing _parse_tool_calls_from_text pipeline.

Also add _extract_json_from_deepseek_args helper to handle optional
Markdown code fences (json … ) that DeepSeek wraps around
the arguments payload.

Closes google#5024
@fuchun1010 fuchun1010 force-pushed the fix/deepseek-tool-call-parsing branch from e91b1f6 to 08e864e Compare May 10, 2026 15:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

LiteLLM + DeepSeek-V3 multi-tool calling fails: tool call parsing error

1 participant