fix(copilot): route Claude models through Copilot's native anthropic provider by larsmoan · Pull Request #495 · databricks/ucode

larsmoan · 2026-09-04T16:08:43Z

Fixes #494.

What this does

For Claude models, Copilot now talks to Databricks using Copilot's anthropic connection mode instead of openai mode. That's the mode that lets Copilot's own caching logic actually run. Codex (gpt-5) is unchanged, still on openai mode.

This only kicks in on Copilot CLI 1.0.81-6 or newer. Below that version, Copilot sends a parameter current Claude models reject, so every request would just fail — the code checks the installed Copilot version and keeps older installs on the old (working, uncached) path.

Proof

Tested against a real workspace across Copilot versions 1.0.79 through 1.0.83, and checked Databricks' own usage table (system.ai_gateway.usage), filtered to Copilot's actual requests (not this session's).

Before, on 1.0.79/1.0.80 (what most people are likely running): requests succeed, but never cache — matches the original bug report exactly.
After, on 1.0.81-6+, with this fix: first request per model writes the cache, every request after that reads it back (confirmed in the usage table, not just Copilot's own token counter).

Test plan

uv run pytest -q passes (one unrelated pre-existing failure, same on main)
uv run ruff check . passes
Verified live as above

…provider Copilot CLI always got COPILOT_PROVIDER_TYPE=openai, even for Claude models, pointing it at the OpenAI-compatible MLflow gateway -- a wire format with no field for an Anthropic cache_control breakpoint, so Copilot's own prompt-cache logic never ran and Claude traffic never got a cache hit. Claude models now get COPILOT_PROVIDER_TYPE=anthropic against the native /ai-gateway/anthropic path claude.py uses, with two things the anthropic provider needs that live testing surfaced: Authorization: Bearer instead of the x-api-key it sends by default (Databricks' gateway 401s on that), and a canonical model id (COPILOT_PROVIDER_MODEL_ID) kept separate from the actual wire id (COPILOT_PROVIDER_WIRE_MODEL) -- an unrecognized id makes Copilot fall back to defaults that include temperature, which current-gen Claude models reject. Codex (gpt-5) is unchanged, still on openai/MLflow. Verified against a live workspace: cache_read_input_tokens > 0 in system.ai_gateway.usage on repeated claude-sonnet-5/claude-opus-5 requests after this change, versus every request 400ing before it. Fixes databricks#494

Copilot

🟡 Changes recommended

Canonical Claude model id extraction should normalize casing to avoid producing mixed-case COPILOT_PROVIDER_MODEL_ID values that may still be treated as unknown by Copilot and trigger incorrect fallback request defaults.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

This PR updates ucode’s Copilot CLI integration so Claude-family models are routed through Copilot’s native anthropic provider (to enable Copilot’s own cache_control prompt-caching logic), while keeping Codex (gpt-5) on the openai provider against the MLflow gateway.

Changes:

Added build_copilot_base_urls() to provide per-provider Copilot gateway base URLs (native Anthropic vs MLflow).
Updated the Copilot agent env overlay to select provider type/base URL by model family and to split canonical Claude model id vs wire model id.
Expanded tests to cover Copilot base URL building and the new Copilot env overlay/write behavior.

File summaries

File	Description
`src/ucode/databricks.py`	Introduces `build_copilot_base_urls()` and updates shared base URL mapping to include Copilot’s per-provider URLs.
`src/ucode/agents/copilot.py`	Switches Copilot provider routing based on Claude vs non-Claude models and adds canonical/wire model split for Claude.
`tests/test_databricks.py`	Adds coverage for the new Copilot base URL builder and ensures `build_tool_base_url("copilot", …)` correctly raises.
`tests/test_agent_copilot.py`	Adds coverage for Claude vs non-Claude env overlay behavior and config writing when switching model families.

Review details

Files reviewed: 4/4 changed files
Comments generated: 2
Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

+def _canonical_claude_model_id(model: str) -> str:
+    # e.g. "system.ai.claude-sonnet-5" -> "claude-sonnet-5" — the well-known
+    # name Copilot needs to recognize the model (see render_env_overlay).
+    match = _CANONICAL_CLAUDE_MODEL_ID_RE.search(model)
+    return match.group(0) if match else model


+    def test_claude_model_matches_case_insensitively(self):
+        env = copilot.render_env_overlay(WS, "us.anthropic.Claude-Opus-4-8", "tok")
+        assert env["COPILOT_PROVIDER_TYPE"] == "anthropic"
+        assert env["COPILOT_PROVIDER_MODEL_ID"] == "Claude-Opus-4-8"


…rsion Live-tested across Copilot CLI 1.0.79 through 1.0.83: below 1.0.81-6, Copilot always sends `temperature` on the anthropic BYOK path regardless of the model id, and current-gen Claude models (Sonnet 5, Opus 5) reject it outright -- every request 400s. That version boundary matches exactly the number in this repo's own CI comment ("1.0.81-6 sends unsupported parameters through BYOK providers"), which pins Copilot to 1.0.80 in CI for a related but distinct regression on the *openai* dialect. Without this gate, routing Claude through the anthropic provider unconditionally would have been a regression for anyone on Copilot < 1.0.81-6 (currently working, if uncached, via openai -> would 400 on every request instead). _supports_anthropic_provider() checks the installed Copilot version (agent_version(), same pattern as codex.py's MINIMUM_CODEX_VERSION) and keeps Claude on the openai path below the minimum -- unchanged from today's behavior, not a regression. At or above it, Claude gets the anthropic path and caches, confirmed live on 1.0.81-6, 1.0.81-7, 1.0.81, 1.0.82, and 1.0.83.

Copilot

🔵 Needs a closer look

_canonical_claude_model_id() currently preserves Bedrock-style -vN suffixes (e.g., ...-v1:0), which can produce a non-canonical model id and re-trigger Copilot’s “unrecognized model” fallback behavior the change is trying to avoid.

Review details

Suppressed comments (1)

Previously missed (1) — in code that hasn't changed since the last review.

src/ucode/agents/copilot.py:146

_canonical_claude_model_id() will return Bedrock-style ids including the trailing "-v1" segment (e.g. "us.anthropic.claude-opus-4-8-v1:0" -> "claude-opus-4-8-v1"). That is unlikely to be a Copilot-recognized canonical model id and can re-trigger the fallback behavior you’re trying to avoid (sending default params like temperature).

Files reviewed: 4/4 changed files
Comments generated: 0 new
Review effort level: Lite

Addresses two issues from automated PR review on databricks#495: - Lowercase the extracted id. A mixed-case input (e.g. from a Bedrock-style slug) produced a mixed-case COPILOT_PROVIDER_MODEL_ID, which Copilot's catalog wouldn't recognize -- re-triggering the exact "unrecognized model" fallback (sending temperature) this split exists to avoid. - Strip a trailing Bedrock version suffix (-vN or -vN:M), the same pattern usage.py's normalize_price_key already strips for the same reason.

Copilot

🟢 Approval recommended

The changes are well-scoped, version-gated to avoid regressions on older Copilot CLIs, and backed by targeted unit tests covering the new routing and config behavior.

Review details

Files reviewed: 4/4 changed files
Comments generated: 0 new
Review effort level: Lite

larsmoan mentioned this pull request Sep 5, 2026

Copilot CLI gets zero prompt-cache hits on Claude models — ucode forces the openai provider type instead of Copilot's native anthropic one #494

Open

larsmoan force-pushed the fix/copilot-anthropic-prompt-caching branch 2 times, most recently from dec3191 to 5245f82 Compare September 5, 2026 17:06

larsmoan force-pushed the fix/copilot-anthropic-prompt-caching branch from 5245f82 to 0e88068 Compare September 5, 2026 17:06

larsmoan marked this pull request as ready for review September 5, 2026 17:11

Copilot AI lite review requested due to automatic review settings September 5, 2026 17:11

Copilot started reviewing on behalf of larsmoan September 5, 2026 17:12 View session

Copilot AI reviewed Sep 5, 2026

View reviewed changes

Copilot AI review requested due to automatic review settings September 5, 2026 17:24

Copilot started reviewing on behalf of larsmoan September 5, 2026 17:24 View session

Copilot AI reviewed Sep 5, 2026

View reviewed changes

Copilot AI review requested due to automatic review settings September 5, 2026 17:36

Copilot started reviewing on behalf of larsmoan September 5, 2026 17:36 View session

Copilot AI reviewed Sep 5, 2026

View reviewed changes

Navigation Menu

Uh oh!

fix(copilot): route Claude models through Copilot's native anthropic provider - #495

fix(copilot): route Claude models through Copilot's native anthropic provider#495
larsmoan wants to merge 3 commits into
databricks:mainfrom
larsmoan:fix/copilot-anthropic-prompt-caching

larsmoan commented Sep 4, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

larsmoan commented Sep 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What this does

Proof

Test plan

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🔵 Needs a closer look

Suppressed comments (1)

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟢 Approval recommended

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

larsmoan commented Sep 4, 2026 •

edited

Loading