Skip to content

feat(sandbox): accept PromptInputPart[] messages in prompt dispatch#187

Merged
vutuanlinh2k2 merged 2 commits into
mainfrom
feat/578-multimodal-prompt-parts
Jul 15, 2026
Merged

feat(sandbox): accept PromptInputPart[] messages in prompt dispatch#187
vutuanlinh2k2 merged 2 commits into
mainfrom
feat/578-multimodal-prompt-parts

Conversation

@vutuanlinh2k2

Copy link
Copy Markdown
Contributor

Substrate for gtm-agent#578 (native multimodal prompt dispatch): consumers can now dispatch structured PromptInputPart[] — text plus image/file parts — through the sandbox prompt seam instead of being forced through the flattened-string path.

  • streamSandboxPrompt, runSandboxPrompt, and driveSandboxTurn accept message: string | PromptInputPart[]. String messages take the exact flattenHistory path as before (byte-identical, pinned by the existing seam tests); array messages route through the new mergeHistoryIntoParts.
  • mergeHistoryIntoParts(parts, history?) folds the conversation transcript into the first text part (${transcript}\n\nUser: ${text} — same format as flattenHistory, both now built by one shared transcript helper so they can't drift). Non-text parts pass through untouched, order preserved. No text part → throws: a parts array with nothing to anchor history onto is a caller bug, not something to paper over.
  • PromptInputPart is re-exported for consumers. The sandbox SDK does not name-export this type from any public entry point (verified on 0.9.7 and 0.10.5), so it is derived structurally from SandboxInstance['streamPrompt']'s parameter — it stays in lockstep with whatever the SDK actually accepts instead of being a hand-copied union that can drift.
  • @tangle-network/sandbox devDep aligned ^0.9.7 → ^0.10.5 to match what consumers (gtm-agent) resolve, so the widened signatures typecheck against the same SDK surface downstream.

Tests: new seam coverage for array forwarding + history folding into parts[0], mergeHistoryIntoParts passthrough/order/throw cases; existing string-path tests unchanged and green (121 passed; the 3 writeProfileFilesToBox/ensureWorkspaceSandbox timeout flakes fail identically on a clean tree). tsc --noEmit clean; tsup build + dist .d.ts verified to carry the new exports.

Consumer: gtm-agent PR tangle-network/gtm-agent#588 (draft) builds against this branch via local link and needs the release + a dep bump once this merges.

streamSandboxPrompt, runSandboxPrompt, and driveSandboxTurn take
string | PromptInputPart[]; array messages fold conversation history into
the leading text part via mergeHistoryIntoParts (throws when no text part
is present). String messages keep the flattenHistory transcript unchanged.
PromptInputPart is derived from the sandbox SDK's streamPrompt signature
and re-exported for consumers.
@vutuanlinh2k2
vutuanlinh2k2 merged commit f6c3dc9 into main Jul 15, 2026
1 check passed
@vutuanlinh2k2
vutuanlinh2k2 deleted the feat/578-multimodal-prompt-parts branch July 15, 2026 06:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant