Skip to content

fix(server-utils): Record Vercel AI experimental_telemetry.metadata on gen_ai spans - #24721

Open
isaacs wants to merge 2 commits into
developfrom
isaacschlueter/js-3800-vercelaiintegration-experimental_telemetrymetadata-no-longer
Open

isaacs wants to merge 2 commits into
developfrom
isaacschlueter/js-3800-vercelaiintegration-experimental_telemetrymetadata-no-longer

Conversation

@isaacs

@isaacs isaacs commented Sep 24, 2026

Copy link
Copy Markdown
Member

v10 got vercel.ai.telemetry.metadata.<key> attributes from the AI SDK's own OTel spans. In v11, ai 4 to 6 go through the orchestrion subscriber, which never read metadata, so v11 dropped it.

The orchestrion subscriber now passes the metadata to the channel subscriber, which sets it as span attributes when each span starts. The invoke_agent, generate_content and embeddings spans get the attributes, and execute_tool spans don't. This matches the AI SDK, which leaves metadata off its ai.toolCall spans.

fixes JS-3800
fixes #24720
Re: #24706

@isaacs
isaacs requested review from a team as code owners September 24, 2026 22:11
@isaacs
isaacs requested review from JPeer264 and mydea and removed request for a team September 24, 2026 22:11
@linear-code

linear-code Bot commented Sep 24, 2026

Copy link
Copy Markdown

JS-3800

@github-actions

github-actions Bot commented Sep 24, 2026 •

Copy link
Copy Markdown
Contributor

size-limit report 📦

Path Size % Change Change
@sentry/browser 29.24 kB - -
@sentry/browser - with treeshaking flags 27.5 kB - -
@sentry/browser - with treeshaking flags tracing without tracing 27.4 kB - -
@sentry/browser (incl. Tracing) 51.15 kB - -
@sentry/browser (incl. Tracing + Span Streaming) 51.17 kB - -
@sentry/browser (incl. Tracing, Profiling) 54.18 kB - -
@sentry/browser (incl. Tracing, Replay) 90.76 kB - -
@sentry/browser (incl. Tracing, Replay) - with treeshaking flags 79.86 kB - -
@sentry/browser (incl. Tracing, Replay with Canvas) 95.46 kB - -
@sentry/browser (incl. Tracing, Replay, Feedback) 108.41 kB - -
@sentry/browser (incl. Feedback) 46.76 kB - -
@sentry/browser (incl. sendFeedback) 34.3 kB - -
@sentry/browser (incl. FeedbackAsync) 39.41 kB - -
@sentry/browser (incl. Metrics) 30.25 kB - -
@sentry/browser (incl. Logs) 30.51 kB - -
@sentry/browser (incl. Metrics & Logs) 31.18 kB - -
@sentry/react 31 kB - -
@sentry/react (incl. Tracing) 53.45 kB - -
@sentry/vue 36.74 kB - -
@sentry/vue (incl. Tracing) 53.7 kB - -
@sentry/svelte 29.26 kB - -
CDN Bundle 30.93 kB - -
CDN Bundle (incl. Tracing) 51.69 kB - -
CDN Bundle (incl. Logs, Metrics) 33.2 kB - -
CDN Bundle (incl. Tracing, Logs, Metrics) 53.66 kB - -
CDN Bundle (incl. Replay, Logs, Metrics) 73.92 kB - -
CDN Bundle (incl. Tracing, Replay) 89.28 kB - -
CDN Bundle (incl. Tracing, Replay, Logs, Metrics) 91.25 kB - -
CDN Bundle (incl. Tracing, Replay, Feedback) 95.45 kB - -
CDN Bundle (incl. Tracing, Replay, Feedback, Logs, Metrics) 97.42 kB - -
CDN Bundle - uncompressed 91.4 kB - -
CDN Bundle (incl. Tracing) - uncompressed 153.77 kB - -
CDN Bundle (incl. Logs, Metrics) - uncompressed 97.97 kB - -
CDN Bundle (incl. Tracing, Logs, Metrics) - uncompressed 159.73 kB - -
CDN Bundle (incl. Replay, Logs, Metrics) - uncompressed 227.54 kB - -
CDN Bundle (incl. Tracing, Replay) - uncompressed 273.5 kB - -
CDN Bundle (incl. Tracing, Replay, Logs, Metrics) - uncompressed 279.44 kB - -
CDN Bundle (incl. Tracing, Replay, Feedback) - uncompressed 287.2 kB - -
CDN Bundle (incl. Tracing, Replay, Feedback, Logs, Metrics) - uncompressed 293.13 kB - -
@sentry/nextjs (client) 55.77 kB - -
@sentry/sveltekit (client) 51.59 kB - -
@sentry/core/server 39.95 kB - -
@sentry/core/browser 13.63 kB - -
@sentry/node 136.7 kB +0.05% +66 B 🔺
@sentry/node/import (ESM hook with diagnostics-channel injection) 82.56 kB - -
@sentry/node - without tracing 90.58 kB +0.02% +13 B 🔺
@sentry/node - without channel injection 115.15 kB +0.06% +63 B 🔺
@sentry/aws-serverless 98.85 kB +0.01% +2 B 🔺
@sentry/cloudflare (withSentry) - minified 206.49 kB - -
@sentry/cloudflare (withSentry) 513.73 kB - -

View base workflow run

@JPeer264 JPeer264 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Implementation LGTM. However I'm not sure if we should send vercel.ai specific attributes. I added @andreiborza and @RulaKhaled who know more about status quo on these fields and if we should (re-)add them

@RulaKhaled RulaKhaled left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also lgtm. I’d bring this up in the agent tracing channel before merging, vercel.ai.telemetry.metadata.* attributes aren’t in @sentry/conventions, they probably can’t be convention constants but maybe we could register a prefix

const baseAttributes: Record<string, string | number | boolean> = {
const baseAttributes: SpanAttributes = {
[SEMANTIC_ATTRIBUTE_SENTRY_ORIGIN]: ORIGIN,
...telemetryMetadataAttributes(event.telemetryMetadata),

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if this should be gated on recordInputs or dataCollection.genAI

… on gen_ai spans

v10 got `vercel.ai.telemetry.metadata.<key>` attributes from the AI
SDK's own OTel spans. In v11, `ai` 4 to 6 go through the orchestrion
subscriber, which never read `metadata`, so v11 dropped it.

The orchestrion subscriber now passes the metadata to the channel
subscriber, which sets it as span attributes when each span starts.
The `invoke_agent`, `generate_content` and `embeddings` spans get the
attributes, and `execute_tool` spans don't. This matches the AI SDK,
which leaves metadata off its `ai.toolCall` spans.

fixes JS-3800
fixes #24720
Re: #24706

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@isaacs
isaacs force-pushed the isaacschlueter/js-3800-vercelaiintegration-experimental_telemetrymetadata-no-longer branch from e652f6b to aea4552 Compare September 25, 2026 17:53
@isaacs

isaacs commented Sep 25, 2026

Copy link
Copy Markdown
Member Author

I’d bring this up in the agent tracing channel before merging

Good call! Pinged them for input.

if (isObjectLike(metadata)) {
for (const [key, value] of Object.entries(metadata)) {
attributes[`${VERCEL_AI_TELEMETRY_METADATA_ATTRIBUTE_PREFIX}${key}`] = value as SpanAttributeValue;
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: The telemetryMetadataAttributes function improperly handles non-primitive metadata values, leading to data corruption when they are cast to SpanAttributeValue without validation.
Severity: MEDIUM

Suggested Fix

Before assigning the value as a span attribute, check if it is a primitive type. If it is not a primitive, attempt to serialize it using JSON.stringify within a try-catch block to handle potential serialization errors gracefully. This aligns with the existing SDK pattern for handling user-provided data.

Prompt for AI Agent
Review the code at the location below. A potential bug has been identified by an AI
agent. Verify if this is a real issue. If it is, propose a fix; if not, explain why it's
not valid.

Location:
packages/server-utils/src/integrations/vercel-ai/vercel-ai-dc-subscriber.ts#L481

Potential issue: The `telemetryMetadataAttributes` function casts values from
`experimental_telemetry.metadata` to `SpanAttributeValue` without proper type
validation. The `SpanAttributeValue` type is restricted to primitives and arrays of
primitives, but the input metadata can contain nested objects. When a span with a
non-primitive attribute is serialized for Sentry, the object is converted to the string
`"[object Object]"`, leading to corrupted or lost telemetry data. This behavior is
inconsistent with established SDK patterns that safely handle user-provided objects by
checking for primitives and using `JSON.stringify` for complex types.

Did we get this right? 👍 / 👎 to inform future reviews.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

vercelAIIntegration: experimental_telemetry.metadata no longer recorded on gen_ai spans (v11 regression, ai <= 6)

3 participants