fix(strands): allowlist metrics, add span_origin, fix agent metadata reads#612
Merged
Merged
Conversation
Audit against `.agents/skills/sdk-integrations/SKILL.md`:
- Missing `_INSTRUMENTATION = "strands-auto"` and local `start_span`
wrapper. Every other integration has it (adk, autogen, anthropic,
mistral, etc.); strands was the only integration whose spans lacked
`context.span_origin.instrumentation.name`. Added the module-level
wrapper and pass `internal={"instrumentation": _INSTRUMENTATION}`
explicitly through nested `parent.start_span(...)` calls, which don't
route through the module wrapper.
- Denylist metrics extraction on the model-invoke span:
`{k: v for k, v in metrics.items() if _is_supported_metric_value(v)}`
passed every numeric key strands emits (present + future) into the
span's top-level `metrics` bag. Replaced with an explicit allowlist
(`latencyMs`, `timeToFirstByteMs`) landing in
`metadata.strands_metrics` since neither key is spec-listed.
- `_agent_metrics_and_metadata` was reading `getattr(result, "metrics")`
as a dict — but `AgentResult.metrics` is an `EventLoopMetrics`
dataclass, so the `isinstance(metrics, dict)` guard silently discarded
everything. Renamed to `_agent_metadata_from_result` and read
`cycle_count`, `accumulated_usage`, `accumulated_metrics` off the
dataclass through a unified `_pick_numeric(source, keys)` allowlist
helper. Moved `cycles` from `metrics` to `metadata` (not a spec key).
- Positional-arg reads: `_start_agent_span_wrapper` and
`_start_model_invoke_span_wrapper` did `kwargs.get("tools")` /
`kwargs.get("system_prompt")` etc. Strands' signatures put these at
fixed positional slots — switched to `_arg(args, kwargs, N, name)`
so positional callers work.
- Simplifications: collapsed the parent-vs-module `if/else` in
`_start_span_for_otel` by picking the callable once; replaced the
`_bt_parent_otel_span` metadata back-channel with a direct
`parent_otel_span=` kwarg on `_start_span_for_otel`; dropped the
now-orphan `metrics=` parameter from `_end_span_for_otel`; trimmed
the excessive `wrap_strands_tracer` and `_InvalidOtelSpanKey`
docstrings.
No cassettes re-recorded — HTTP behavior is unchanged, only in-process
span shaping was touched. The existing VCR test now also asserts
`span_origin.instrumentation.name == "strands-auto"` on every emitted
span.
Test plan
- nox -s "test_strands(latest)" — 2/2 pass
- nox -s "test_strands(1.20.0)" — 2/2 pass
- ruff check + ruff format --check on touched files — clean
- nox -s pylint — clean
Known SKILL gaps NOT fixed here (each deserves its own discussion):
- `metadata.provider` on LLM spans: Strands' Tracer only receives
`model_id`, no model-class hook. Provider inference would require
brittle id-pattern matching.
- Tokens under `metadata.strands_usage` rather than `metrics.tokens`
et al: fixing correctly means deciding whether model-invoke should
stay `llm` at all — strands' `OpenAIModel`/`AnthropicModel` delegate
to the underlying provider SDK, which the SKILL's "framework
llm-like spans that delegate" rule says should NOT be typed `llm`.
- No `test_auto_strands.py` subprocess auto-instrument test.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Abhijeet Prasad (AbhiPrasad)
approved these changes
Jul 21, 2026
Abhijeet Prasad (AbhiPrasad)
enabled auto-merge (squash)
July 21, 2026 19:12
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Aligns the Strands integration with
.agents/skills/sdk-integrations/SKILL.md, following the pattern of #606 (openai), #604 (mistral), #603 (livekit), #594 (litellm), #583 (anthropic). Audit turned up several issues:Missing span_origin instrumentation name. Every other integration has
_INSTRUMENTATION = "<provider>-auto"+ a module-levelstart_spanwrapper; strands was the only one whose spans lackedcontext.span_origin.instrumentation.name. Added the standard wrapper and passinternal={"instrumentation": _INSTRUMENTATION}explicitly through the nestedparent.start_span(...)path, which doesn't route through the module wrapper.Denylist metrics extraction (real leak vector).
_end_model_invoke_span_wrapperwas doing{k: v for k, v in metrics.items() if _is_supported_metric_value(v)}— every numeric key strands emits (present + future) flowed into the span's top-levelmetricsbag. Replaced with an explicit allowlist (latencyMs,timeToFirstByteMs). Since neither key is spec-listed, they land undermetadata.strands_metricsrather than top-levelmetrics.Silently-broken agent metadata extraction.
_agent_metrics_and_metadatadidmetrics = getattr(result, "metrics"); if not isinstance(metrics, dict): return {}, {}— butAgentResult.metricsis anEventLoopMetricsdataclass, so the guard discarded everything. Renamed to_agent_metadata_from_resultand readcycle_count/accumulated_usage/accumulated_metricsoff the dataclass through a single_pick_numeric(source, keys)allowlist helper. Movedcyclesfrommetricstometadata(not a spec metric key).Positional-arg reads.
_start_agent_span_wrapper/_start_model_invoke_span_wrapperwerekwargs.get("tools")/kwargs.get("system_prompt")etc. Strands' signatures put those at fixed positional slots — switched to_arg(args, kwargs, N, name)so positional callers work.Simplifications (via
/simplify): collapsed twin allowlist helpers into_pick_numeric; collapsed the parent-vs-moduleif/elsein_start_span_for_otel; replaced the_bt_parent_otel_spanmetadata back-channel with a directparent_otel_span=kwarg; dropped the now-orphanmetrics=parameter from_end_span_for_otel; trimmed the verbosewrap_strands_tracerand_InvalidOtelSpanKeydocstrings.Known SKILL gaps NOT fixed here
Each deserves its own PR/discussion — flagging so they don't get lost:
metadata.provideron LLM spans. Strands'Traceronly receivesmodel_id, not the model class — provider inference would require brittle id-pattern matching.metadata.strands_usagerather thanmetrics.tokens/prompt_tokens/completion_tokens. Fixing correctly means deciding whether model-invoke should stay typedllmat all — Strands'OpenAIModel/AnthropicModeldelegate to the underlying provider SDK, which per the SKILL's "framework llm-like spans that delegate" rule should NOT be typedllm.test_auto_strands.pysubprocess auto-instrument test. Strands is wired intoauto.pybut lacks the subprocess parity test other integrations have.Test plan
test_strands_openai_agent_traces_native_otel_lifecycle) covers the behavior; added aspan_origin.instrumentation.name == "strands-auto"assertion on every emitted span.cd py && nox -s "test_strands(latest)"— 2/2 passcd py && nox -s "test_strands(1.20.0)"— 2/2 passruff check+ruff format --checkon touched files — cleannox -s pylint— clean🤖 Generated with Claude Code
Created by abhijeet
Slack thread