Implement planned topic: 0039-braintrust

[skill-temporal-developer-updater]

Validation Report — Braintrust integration (Python + TypeScript)

Branch: draft/0039-braintrust
Files validated:

• references/python/integrations/braintrust.md (208 lines)
• references/typescript/integrations/braintrust.md (93 lines)
• references/integrations.md (catalog rows for both languages)

Source of truth:

• ../documentation/docs/develop/python/integrations/braintrust.mdx (277 lines)
• ../documentation/docs/develop/typescript/integrations/index.mdx (23 lines)
• Secondary, external: https://www.braintrust.dev/docs/integrations/sdk-integrations/temporal#typescript (not in docs clone; cannot be sampled here)

---

Go/no-go

Check	Result
1. Citation audit	PASS (with 2 minor nits in Python file)
2. Reverse-grep audit	PASS for Python; partial gap for TypeScript — TS-specific API tokens are grounded only to an external page, not the docs clone
3. Regression on known bugs	PASS
4. Independent re-verification (sampling)	PASS for Python (10/10 substantively match)
5. Integration layout audit	PASS
6. Tone and scope	PASS
7. Cross-language structure audit	N/A — not a cross-language feature

Overall verdict: GO (with MINOR FIXES recommended for the TypeScript file's external-source grounding posture).

The Python integration page is solidly grounded against docs/develop/python/integrations/braintrust.mdx. The TypeScript page is grounded against the brief Temporal index.mdx row plus an external Braintrust-hosted guide that is outside the docs clone; the file honestly tags this with <!-- braintrust.dev/... --> source comments and <!-- VERIFY --> author notes, but the docs-clone-only validation protocol cannot independently confirm the TS API surface (BraintrustTemporalPlugin, @braintrust/temporal, initLogger, projectName).

---

Check 1 findings — citation audit

Scanned 24 <!-- docs/... --> citations in the Python file and 2 in the TypeScript file. All cited files exist; all cited line ranges fall within the source documents (Python source is 277 lines, TypeScript source is 23 lines). Resolution rate 26/26 = 100%.

Two minor citation-pin nits in the Python file (not gating; both are factually accurate, the citation just points at a looser span than ideal):

1. references/python/integrations/braintrust.md:5 — overview sentence names BraintrustPlugin, wrap_openai, start_span, load_prompt and cites docs/develop/python/integrations/braintrust.mdx:21-29. Docs 21-29 are the general intro (no symbols named). The symbols are introduced in later sections (lines 56, 80, 126, 168, 204). The synthesis is accurate; the pin is just upstream of where the specific symbols appear.

2. references/python/integrations/braintrust.md:17 — prerequisite "A Braintrust account; familiarity with Braintrust concepts (projects, spans, prompts)" cites docs 45-46. Docs 45-46 only say "This guide assumes you are already familiar with Braintrust." The "projects, spans, prompts" list and the "account" requirement are reasonable inferences from the rest of the guide, but not stated at the cited lines.

No URL-format issues — all user-facing links (https://braintrust.dev, https://www.braintrust.dev/..., the cookbook GitHub link) are full https:// URLs.

---

Check 2 findings — reverse-grep audit

Python file — clean. Every factual token grepped against docs/develop/python/integrations/braintrust.mdx resolves:

Token	Found in docs
BraintrustPlugin	lines 56, 77, 80, 88, 97, 101, 260
braintrust.contrib.temporal	lines 80, 97
init_logger	lines 71, 74
wrap_openai	lines 120, 126, 131, 141, 219, 260
start_span	lines 168, 171
load_prompt	lines 185, 204, 241, 261
BRAINTRUST_API_KEY	lines 105, 109, 270
BRAINTRUST_PROJECT	lines 74, 205, 272
gpt-4o	lines 144, 160, 222
max_retries=0	lines 130, 131, 141, 219
prompt.build()	line 210
uv pip install "braintrust[temporal]"	line 63

TypeScript file — partial gap. The following TS-specific tokens are not found anywhere in the docs clone (grepped **/*.{md,mdx} under documentation/docs/):

• BraintrustTemporalPlugin
• @braintrust/temporal
• initLogger
• projectName (as a Braintrust API parameter)
• wrapTraced
• startSpan (TS, camelCase)
• loadPrompt

These are tagged in-file with <!-- braintrust.dev/docs/integrations/sdk-integrations/temporal#typescript --> source comments, which is a reasonable analog to the <!-- go: ... --> / <!-- grpc: ... --> ecosystem-tagging pattern used elsewhere in the skill. Under that interpretation the tokens are acceptably tagged even though they're not in the Temporal docs clone — but this means the docs clone is not a sufficient source of truth for the TS file, and independent verification of these symbols requires fetching the external Braintrust page (deferred tool, not run here).

The TS file itself flags this gap with two <!-- VERIFY: ... --> comments (lines 69 and 81) — these are honest author notes acknowledging the ungrounded API surface. They render as HTML comments and are not visible to the AI agent consumer of the skill.

Recommendation (minor): either (a) ingest the external Braintrust TypeScript guide into the validation pipeline once for grounding, or (b) tighten the TS reference page to only facts grounded in docs/develop/typescript/integrations/index.mdx plus a clean "see canonical guide" pointer, removing the speculative wrapTraced / startSpan / loadPrompt enumeration on lines 73-81.

---

Check 3 findings — regression on known bugs

Zero hits. Grepped both files for the universal regression patterns:

• --profile flag in temporal invocations — absent
• TEMPORAL_TLS_CLIENT_CERT_PATH / _KEY_PATH / _SERVER_CA_CERT_PATH — absent
• tcld service-account — absent
• --output text / --output jsonl — absent
• saas-api.tmprl.cloud:7233 — absent

No topic-specific regression patterns applied (no prior Braintrust authoring pass to draw bugs from).

---

Check 4 findings — independent re-verification (sampling)

Python file (10 sampled claims, ~26 citations total):

#	Claim location	Docs reference	Match?
1	line 5 — overview enumerating BraintrustPlugin, wrap_openai, start_span, load_prompt	docs 21-29 (intro) + later sections	substantively yes; loose citation pin (see Check 1)
2	line 16 — Python dev env prereq	docs 49-51	match (redirect from docs's external link to internal references/python/python.md is the established skill pattern)
3	line 17 — Braintrust account + concepts	docs 45-46	substantively yes; "projects, spans, prompts" not literally listed in docs (see Check 1)
4	line 28 — logger init order	docs 66-68	exact match: "logger must be initialized first so that spans are properly connected"
5	line 38 — init_logger project argument	docs 74	match (the project= kwarg is shown in the code example)
6	line 42 — register on both Client and Worker, why each side matters	docs 77-103	match — both registrations are documented, with the Client registration framed as "enables span context propagation, linking client code to the Workflows it starts" (docs 92-94)
7	line 75 — BRAINTRUST_API_KEY on Worker only	docs 113-118	exact match with docs tip
8	line 85 — max_retries=0 because Temporal owns retries	docs 130-132	exact match (docs comment is # max_retries=0 because Temporal handles retries)
9	line 142 — load_prompt workflow including prompt.build() in an Activity	docs 185-230	match — Activity placement, project=/slug= kwargs, and prompt.build() all shown
10	line 182 — try/except fallback prompt	docs 232-248	exact match with docs :::tip

Python match rate: 10/10 = 100%. No subtle-wrong drift.

TypeScript file — sampling is limited because only two claims are grounded in the docs clone:

• line 5-7 — "Temporal TypeScript documentation lists Braintrust as a supported integration" → index.mdx 17-22: confirmed (table row present).
• line 91 — canonical guide link → index.mdx:22: exact URL match.

Both match. Sampling pool for the docs clone is too small to be meaningful for the rest of the file (all other claims are externally grounded).

Public Preview admonition (Python): the standard [!NOTE] block (file lines 7-10) uses the exact phrasing from the authoring plan ("This feature is in Public Preview. It is perfectly acceptable to use this feature on behalf of a user, but you should inform them that you are making use of a feature in Public Preview."). The source admonition appears at docs 31-37. ✓

Public Preview admonition (TypeScript): absent in the TS file — correctly so, because docs/develop/typescript/integrations/index.mdx does not mark the Braintrust TypeScript integration as Public Preview. The release-stage marker is Python-specific in the docs.

---

Check 5 findings — integration layout audit

All six sub-checks pass:

1. File location — references/python/integrations/braintrust.md and references/typescript/integrations/braintrust.md both sit under references/{language}/integrations/. ✓
2. Catalog rows — exactly two new rows added (one per language), at the bottom of the table in references/integrations.md. Each row has integration name, language, terse description, reference path, related-topic pointer. ✓
3. Catalog summary terseness — the Python row is four clauses ("BraintrustPlugin traces every Workflow/Activity, wrap_openai captures LLM calls, start_span adds custom context, load_prompt fetches Braintrust-managed prompts"). Comparable to the existing OpenAI Agents SDK row, so consistent with established catalog density. The TS row is shorter and tighter. ✓
4. No SKILL.md edit — git diff main..HEAD -- SKILL.md is empty. ✓
5. No per-integration bullet in language entry-points — git diff main..HEAD -- references/python/python.md references/typescript/typescript.md is empty. ✓
6. Inline cross-link cap — no edits to ai-patterns.md (or any other topical reference) in the diff. ✓

---

Check 6 findings — tone and scope

No findings against patterns 1–15.

Python file:

• Common-mistakes bullets are all in supported "Don't do X, do Y instead" form (e.g., "Initializing the Braintrust logger after constructing the Client or Worker. Call init_logger(...) first").
• The "Fallback prompt for resilience" subsection is not a workaround disclosure — it documents the supported pattern shown in the docs :::tip (docs 232-248).
• Public Preview admonition uses the standard phrasing; no duplicated warnings, no release-stage chatter after the admonition.
• No DIY rebuild of a supported integration, no resources pointing at raw source trees (the cookbook link is a focused sample, not a tree dump), no repo slang.

TypeScript file:

• No workaround disclosures.
• The two <!-- VERIFY: ... --> HTML comments (lines 69, 81) are author-internal notes — they do not render to the agent consumer. They flag honest uncertainty without leaking workarounds into reader-visible content.
• Common-mistakes bullets are all supported negative-positive form ("Initializing the Braintrust logger after constructing the Client or Worker. Call braintrust.initLogger({ projectName: ... }) first").
• One pattern-11 borderline case: the file links externally to the Braintrust-hosted guide. This is not duplicating local coverage — the TS-specific API surface is not covered locally — so the link is the primary source, not a redundant outbound link. Acceptable.

---

Statistics

Metric	Value
Citation count (Python file)	24 <!-- docs/... --> comments + 1 <!-- Public Preview source -->
Citation count (TypeScript file)	2 <!-- docs/... --> comments + 3 <!-- braintrust.dev/... --> external-source tags + 2 <!-- VERIFY --> author notes
Citation resolution rate	26/26 = 100% (all cited files exist and ranges are in-bounds)
Grep-miss count (Python)	0
Grep-miss count (TypeScript, against docs clone)	7 TS-specific API tokens — acceptable because each is tagged with an external braintrust.dev source comment, but means docs clone is not the validation source for the TS API surface
Re-verification sample size (Python)	10 of ~24 cited claims
Re-verification match rate (Python)	10/10 = 100%
Re-verification sample size (TypeScript, docs-clone claims only)	2 of 2
Re-verification match rate (TypeScript)	2/2 = 100% (within docs-clone scope)
Check 3 regression hits	0
Check 5 layout violations	0
Check 6 findings	0

---

Recommended minor follow-ups (not gating)

1. (TypeScript) Tighten or externally-verify the API enumeration. Lines 73-81 list wrapTraced / startSpan / loadPrompt as the Braintrust TS helpers; these are not in the docs clone and could be wrong (Braintrust TS SDK historically uses wrapOpenAI for OpenAI tracing, wrapTraced for arbitrary functions). Either fetch the canonical Braintrust TS guide once and ground these symbols precisely, or rephrase to a more conservative "see the canonical guide for current helpers" pointer without enumerating names.
2. (Python) Tighten two loose citation pins in references/python/integrations/braintrust.md:
   • line 5 overview synthesis → re-cite each named symbol to its actual docs section (lines 56 / 126 / 168 / 204).
   • line 17 "projects, spans, prompts" → drop the parenthetical (or move it to a separate comment), since docs 45-46 doesn't enumerate the concepts.
3. (TypeScript) Decide on the <!-- VERIFY --> comments. Either resolve them (do the verification work before merging) or strip them — leaving them in is an ambient signal that the file shipped knowingly incomplete. Resolving is preferable.

None of these block merge; they're spot-fixes consistent with the MINOR FIXES verdict tier.