fix(google_genai): add provider metadata, route tools to metadata, drop eager serialization by starfolkai[bot] · Pull Request #590 · braintrustdata/braintrust-sdk-python

starfolkai · 2026-07-17T17:40:16Z

Summary

Align the Google GenAI integration with .agents/skills/sdk-integrations/SKILL.md. Audit turned up four issues:

Missing metadata.provider on every span — spec requires every llm span to carry both metadata.model and metadata.provider. Added metadata.provider = \"google\" on generate_content, generate_content_stream, embed_content, generate_images, and every interactions.* span.
Tools in input.config instead of metadata.tools — spec says tool definitions live in metadata.tools. Route the serialized Google-native tool declarations onto metadata.tools for the models path, and route the interactions tools kwarg onto metadata.tools for the interactions path. New _config_for_input() strips tools from the config that flows into input via a shallow field iteration (no full model_dump).
Excess serialization — dropped bt_safe_deep_copy(config) in _serialize_input and _prepare_generate_images_traced_call (span logging already runs bt_safe_deep_copy at send time, so the eager copy was duplicate work — same pattern as fix(anthropic): drop redundant bt_safe_deep_copy passes in tracing #583 for anthropic and fix(claude_agent_sdk): add provider metadata, drop deep-copy serialization, allowlist hook fields #585 for claude_agent_sdk). Also dropped the eager grounding_metadata.model_dump(exclude_none=True) in _aggregate_generate_content_chunks, and the eager _materialize_interaction_value(...) on interaction tools.
Denylist → allowlist in _tool_span_input / _tool_span_output. Replaced key not in {\"id\", \"name\", \"type\", \"signature\", \"server_name\", ...} with explicit _TOOL_CALL_INPUT_FIELDS = (\"arguments\", \"code\", \"language\", \"url\", \"query\") and _TOOL_RESULT_OUTPUT_FIELDS = (\"result\", \"output\", \"outcome\", \"content\"). Any unrecognized field on a call/result payload is dropped rather than passed through.

Also trimmed narrative docstrings/comments that just described what the code does; kept the section dividers and the two "why" comments about interaction tool span lifetime.

Test plan

Extended existing VCR-backed tests to assert metadata.provider == \"google\", metadata.tools presence for both models.generate_content and interactions.create tool cases, and negative assertions that tools do not leak into input.config / input.
cd py && CI=1 BRAINTRUST_TEST_PACKAGE_VERSION=latest pytest src/braintrust/integrations/google_genai/test_google_genai.py — 45 passed
cd py && CI=1 BRAINTRUST_TEST_PACKAGE_VERSION=1.75.0 pytest src/braintrust/integrations/google_genai/test_google_genai.py — 45 passed
cd py && CI=1 BRAINTRUST_TEST_PACKAGE_VERSION=1.30.0 pytest src/braintrust/integrations/google_genai/test_google_genai.py — 38 passed, 7 pre-existing skips (interactions API unavailable on that version)
No cassettes re-recorded, no new mocks/fakes — used the existing VCR-backed suite.

🤖 Generated with Claude Code

Created by abhijeet

Slack thread

…op eager serialization Align the Google GenAI integration with `.agents/skills/sdk-integrations/SKILL.md`. - **Missing `metadata.provider` on every span** — spec requires every `llm` span to carry both `metadata.model` and `metadata.provider`. Added `metadata.provider = "google"` on `generate_content`, `generate_content_stream`, `embed_content`, `generate_images`, and every `interactions.*` span. - **Tools in `input.config` instead of `metadata.tools`** — spec says tool definitions live in `metadata.tools`. Route the serialized Google-native tool declarations onto `metadata.tools` for the models path, and route the interactions `tools` kwarg onto `metadata.tools` for the interactions path. `_config_for_input` strips `tools` from the config that flows into `input` via a shallow field iteration. - **Excess serialization** — dropped `bt_safe_deep_copy(config)` in `_serialize_input` and `_prepare_generate_images_traced_call` (span logging already runs `bt_safe_deep_copy` at send time, so the eager copy was duplicate work — same pattern as #583 for anthropic). Dropped the eager `grounding_metadata.model_dump(exclude_none=True)` in `_aggregate_generate_content_chunks`; dropped the eager `_materialize_interaction_value(kwargs.get("tools"))` for interaction tools. - **Denylist → allowlist** in `_tool_span_input` / `_tool_span_output`. New `_TOOL_CALL_INPUT_FIELDS = ("arguments","code","language","url", "query")` and `_TOOL_RESULT_OUTPUT_FIELDS = ("result","output", "outcome","content")` replace the `key not in {...}` filters. Any unrecognized field is dropped rather than passed through. - Trimmed narrative docstrings/comments that just described what the code does; kept the two "why" comments about interaction tool span lifetime and the section-header dividers. Test plan - Extended existing VCR-backed tests to assert `metadata.provider == "google"`, `metadata.tools` presence for both `models.generate_content` and `interactions.create` tool cases, and negative assertions that tools do not leak into `input.config` / `input`. - `cd py && CI=1 BRAINTRUST_TEST_PACKAGE_VERSION=latest pytest src/braintrust/integrations/google_genai/test_google_genai.py` — 45 passed - `cd py && CI=1 BRAINTRUST_TEST_PACKAGE_VERSION=1.75.0 pytest src/braintrust/integrations/google_genai/test_google_genai.py` — 45 passed - `cd py && CI=1 BRAINTRUST_TEST_PACKAGE_VERSION=1.30.0 pytest src/braintrust/integrations/google_genai/test_google_genai.py` — 38 passed, 7 pre-existing skips (interactions API unavailable) - No cassettes re-recorded, no new mocks/fakes — used the existing VCR-backed suite. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

## Summary Audit of `py/src/braintrust/integrations/huggingface_hub/` against `.agents/skills/sdk-integrations/SKILL.md`. Two related fixes: - **Drop eager `_try_to_dict` in stream tool-call merge.** `_merge_tool_call_delta` was running `_try_to_dict(entry)` (Pydantic `model_dump("python")`) on every streaming tool-call delta chunk just to read `index` / `id` / `type` / `function` fields. `SpanImpl.log_internal` already runs `bt_safe_deep_copy` at send time, so the eager conversion was a redundant recursive walk on the hot path. Same pattern as #586 (cohere) and #590 (google_genai). - **Remove the `_get_field(obj, key)` dict-or-attr helper.** HuggingFace's `BaseInferenceType` (parent of every response type and stream chunk on both matrix pins — `latest`=1.23.0 and floor=`0.32.0`) inherits from `dict`: ``` >>> from huggingface_hub.inference._generated.types.base import BaseInferenceType >>> BaseInferenceType.__mro__ (BaseInferenceType, dict, object) ``` So `.get()` works uniformly on every response, chunk, delta, and nested field — the dict-or-attr dispatch was dead cleverness. The two places that read from an `InferenceClient` instance now use `getattr(instance, key, None)` directly. Net diff: **60 insertions / 89 deletions**, one file. Also picked up two small invariants that were previously implicit via `_get_field`: - Tool-call-delta loop now checks `isinstance(entry, dict)` (tighter than `is None`) and drops the corresponding `isinstance(incoming_fn, dict)` re-check inside. - `_text_generation_extra_metadata` gets an explicit `if details is None: return {}` guard at the top. **Everything else was already SKILL-aligned** and needed no change: - `metadata.provider` always set (kwarg > instance > `"huggingface"` fallback), `metadata.model` set when known. - Tool definitions routed to `metadata.tools` via `_CHAT_METADATA_KEYS` allowlist, not into `input`. - All metadata extraction is allowlist-based — no denylists anywhere in the file. - `context.span_origin.instrumentation.name = "huggingface-hub-auto"` via local `start_span` helper. - Only spec-listed metric keys. - One accumulated span per stream, errors propagate, setup + `wrap_huggingface_hub` idempotent. ## Test plan - [x] `CI=1 BRAINTRUST_TEST_PACKAGE_VERSION=latest pytest src/braintrust/integrations/huggingface_hub/test_huggingface_hub.py` — 24/24 passed (huggingface-hub 1.23.0) - [x] `CI=1 BRAINTRUST_TEST_PACKAGE_VERSION=0.32.0 pytest src/braintrust/integrations/huggingface_hub/test_huggingface_hub.py` — 22 passed, 2 pre-existing skips for text_generation on that pin (`_skip_if_text_generation_unavailable`) - [x] **No cassettes re-recorded, no new mocks/fakes** — used the existing VCR-backed suite. 🤖 Generated with [Claude Code](https://claude.com/claude-code)  Created by abhijeet  [Slack thread](https://starfolkai.slack.com/archives/C0AQDETAVT3/p1784312976815939?thread_ts=1784312976.815939&cid=C0AQDETAVT3) Co-authored-by: Starfolk <noreply@starfolk.ai> Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>

Abhijeet Prasad (AbhiPrasad) approved these changes Jul 17, 2026

View reviewed changes

Abhijeet Prasad (AbhiPrasad) enabled auto-merge (squash) July 17, 2026 17:43

Abhijeet Prasad (AbhiPrasad) merged commit 496cbe4 into main Jul 17, 2026
82 checks passed

Abhijeet Prasad (AbhiPrasad) deleted the fix/google-genai-align-with-skill branch July 17, 2026 17:47

starfolkai Bot mentioned this pull request Jul 17, 2026

perf(huggingface_hub): optimize how dict attributes are set #591

Merged

3 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

fix(google_genai): add provider metadata, route tools to metadata, drop eager serialization#590

fix(google_genai): add provider metadata, route tools to metadata, drop eager serialization#590
Abhijeet Prasad (AbhiPrasad) merged 1 commit into
mainfrom
fix/google-genai-align-with-skill

starfolkai Bot commented Jul 17, 2026 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

starfolkai Bot commented Jul 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Test plan

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

starfolkai Bot commented Jul 17, 2026 •

edited

Loading