[bot] LangSmith JS SDK (`langsmith`) not instrumented — add tracing and eval migration support

[AbhiPrasad]

Summary

The JavaScript langsmith npm package (latest 0.8.0) exposes LangSmith's tracing and evaluation APIs for TypeScript/JavaScript apps: traceable, RunTree, Client trace ingestion methods, evaluate(), comparative evaluation, and Jest/Vitest eval helpers. Braintrust has Python migration support for LangSmith already, but the JavaScript SDK has no equivalent instrumentation/wrapper today.

Users migrating JS/TS apps from LangSmith should be able to keep their LangSmith tracing/eval code and emit Braintrust traces/experiments, with the same tandem vs. standalone behavior as Python where possible.

Existing Braintrust support/reference

Python support exists and should be used as the semantic reference:

• Docs: https://www.braintrust.dev/docs/integrations/sdk-integrations/langsmith
• Implementation: ~/workspace/braintrust-sdk-python/py/src/braintrust/wrappers/langsmith_wrapper.py
• Tests: ~/workspace/braintrust-sdk-python/py/src/braintrust/wrappers/test_langsmith_wrapper.py

The Python wrapper currently patches:

Python LangSmith API	Braintrust behavior
langsmith.traceable	Applies Braintrust traced while optionally also running LangSmith tracing
langsmith.evaluate	Converts LangSmith-style eval args into Braintrust Eval
langsmith.aevaluate	Converts LangSmith-style async eval args into Braintrust EvalAsync
langsmith.Client.evaluate / aevaluate	Same eval conversion on client methods

Python also supports BRAINTRUST_STANDALONE=1 / standalone mode to skip LangSmith and only send to Braintrust.

What is missing in JavaScript

There is no JS equivalent of setup_langsmith() / setupLangSmith() and no auto-instrumentation for the langsmith package.

Current JS repo state:

• No langsmith-specific wrapper under js/src/wrappers/
• No langsmith plugin/channels under js/src/instrumentation/plugins/
• No langsmith config under js/src/auto-instrumentations/configs/
• Existing js/src/auto-instrumentations/configs/langchain.ts only hooks @langchain/core CallbackManager.configure / _configureSync; it does not instrument the standalone langsmith SDK APIs.
• langsmith appears in e2e dependency lockfiles because LangChain depends on it, but there is no Braintrust LangSmith migration/instrumentation surface.

Relevant LangSmith JS APIs to cover

Research source: https://github.com/langchain-ai/langsmith-sdk, js/ package (name: "langsmith", latest npm 0.8.0). Public entrypoints are exposed as root langsmith plus subpaths such as langsmith/traceable, langsmith/run_trees, langsmith/evaluation, langsmith/client, langsmith/jest, and langsmith/vitest.

Tracing

SDK surface	Notes / desired Braintrust behavior
traceable(fn, config) from langsmith/traceable	Primary non-LangChain tracing API. Preserve nested spans, name, run_type, tags, metadata, input/output processors where feasible. In tandem mode, still allow LangSmith's wrapper to run; in standalone mode, only create Braintrust spans.
RunTree from root / langsmith/run_trees	Manual tracing API (new RunTree, createChild, postRun, end, patchRun, events). Users who do manual RunTree tracing should get equivalent Braintrust span trees.
Client.createRun, Client.updateRun, Client.batchIngestRuns, Client.flush	Catch-all trace ingestion paths used by traceable, RunTree, LangChain tracer handoff, and LangSmith wrappers. Instrumenting these may be necessary to cover all trace sources, especially manual RunTree and LangChain-to-LangSmith traces.
langsmith/langchain helpers (getLangchainCallbacks, RunnableTraceable)	Ensure LangChain ↔ LangSmith handoff does not bypass Braintrust. Avoid duplicate spans with existing Braintrust LangChain callback instrumentation.

Important: account for auto-batched/background tracing and make setup/teardown idempotent. Avoid double logging when both Braintrust LangChain instrumentation and LangSmith Client/RunTree interception are active.

Evaluations

SDK surface	Notes / desired Braintrust behavior
evaluate(target, options) from langsmith/evaluation	Convert standard LangSmith evals to Braintrust Eval/JS equivalent. Support data as dataset name, examples array, async iterable; evaluators, summaryEvaluators, metadata/experiment naming, concurrency/repetitions where compatible.
evaluate([exp1, exp2], options)	Comparative evaluation path in current evaluate() overloads. Map to Braintrust comparative support if available, or document unsupported gaps.
evaluateComparative(experiments, options)	Deprecated upstream but still exported; should be covered or explicitly rejected with guidance.
RunEvaluator / runEvaluator / evaluator functions	Convert LangSmith evaluator signatures/results to Braintrust scorers/scores, similar to Python _make_braintrust_scorer.
langsmith/jest, langsmith/vitest, utils/jestlike helpers (test, describe, expect, logFeedback, logOutputs, wrapEvaluator)	These create LangSmith datasets/experiments/feedback through Client/RunTree. Decide whether first-class coverage is required or whether Client/RunTree/eval instrumentation is sufficient; add tests for the chosen contract.

Suggested implementation direction

Add a JS migration wrapper/instrumentation surface analogous to Python, e.g.:

• setupLangSmith({ apiKey, projectId, projectName, standalone })
• wrapTraceable() / wrapEvaluate() / wrapClient() helpers
• Optional env support mirroring Python (BRAINTRUST_STANDALONE=1) and LangSmith env fallbacks (LANGSMITH_PROJECT / LANGCHAIN_PROJECT where appropriate)

For auto-instrumentation, prefer the normal Orchestrion config + plugin/channel path. Potential target files from the published package include:

• traceable.js / traceable.cjs for traceable
• run_trees.js / run_trees.cjs for RunTree methods
• client.js / client.cjs for Client.createRun, updateRun, batchIngestRuns
• evaluation.js / evaluation.cjs for evaluate / evaluateComparative
• jest.js, vitest.js, and utils/jestlike.js if test-eval helpers need explicit coverage

Prefer shared conversion logic between manual wrappers and auto-instrumentation.

Acceptance criteria

• JS users can migrate LangSmith tracing code to Braintrust with minimal changes, matching the Python docs as closely as possible.
• Coverage includes both direct traceable() usage and manual RunTree/Client trace ingestion, or documents any deliberate unsupported API with rationale.
• Coverage includes LangSmith evals (evaluate, comparative evals where possible, evaluator/scorer result conversion) and test helper behavior as appropriate.
• Tandem mode sends to both LangSmith and Braintrust; standalone mode only sends to Braintrust.
• Idempotent setup; no accidental double instrumentation with existing Braintrust LangChain instrumentation.
• E2E coverage for at least:
  • traceable() nested spans
  • manual RunTree parent/child spans
  • evaluate() with examples + evaluator(s)
  • comparative or explicit unsupported test
  • one Jest/Vitest eval helper path or a documented decision not to support it initially

Upstream references

• JS API reference: https://reference.langchain.com/javascript/langsmith
• SDK repo: https://github.com/langchain-ai/langsmith-sdk/tree/main/js
• npm package: https://www.npmjs.com/package/langsmith
• Braintrust Python LangSmith docs: https://www.braintrust.dev/docs/integrations/sdk-integrations/langsmith