Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 6 additions & 4 deletions app/cli/internal/trace/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,14 +47,16 @@ What the user sees in the agent.

| Feature | What it means | Claude Code | Cursor | OpenCode 1.x | OpenCode 2 |
|---------|---------------|-------------|--------|--------------|------------|
| Welcome message | At session start, the user sees a message that tells that Chainloop records the session, and in which project. | Yes | No | No | No |
| Session link after push | After a `git push` that the agent runs, the user sees a link to the session in Chainloop. | Yes | No | No | No |
| Welcome message | At session start, the user sees a message that tells that Chainloop records the session, and in which project. | Yes | No | Unknown | Yes |
| Session link after push | After a `git push` that the agent runs, the user sees a link to the session in Chainloop, and the model is told to repeat it in its reply. | Yes | No | Unknown | Yes |

How OpenCode shows them: the hook writes a JSON response to stdout, and the plugin reads it. OpenCode 1.x shows the welcome message and the link as a TUI toast, and adds the link to the output of the shell command for the model. An OpenCode 2 plugin has no toast. The welcome message is the description of the synthetic message that carries the spec capture instruction, which the TUI shows in the transcript. The link is added to the result of the shell command, which the TUI shows in the command block and the model reads.

Why not Yes:

- **Welcome message, Cursor and OpenCode:** the provider gives the session-start context to the model only. It has no channel that shows a message to the user at session start, so the welcome message is dropped.
- **Welcome message, Cursor:** the provider gives the session-start context to the model only. It has no channel that shows a message to the user at session start, so the welcome message is dropped.
- **Welcome message and session link after push, OpenCode 1.x:** implemented with the TUI toast of the 1.x SDK, but not verified in a live 1.x session.
- **Session link after push, Cursor:** Cursor has no hook after a shell command, so there is no point at which to show the link.
- **Session link after push, OpenCode:** the plugin hook fires after a shell command, but the response that shows a message to the user is not verified yet. The link stays pending until it expires.

## Specs and skills

Expand Down
7 changes: 5 additions & 2 deletions app/cli/internal/trace/claude/announce_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -83,6 +83,9 @@ func TestAnnounceSessionStart(t *testing.T) {
const (
banner = "Chainloop Trace is recording this session."
instruction = "Write the specification this session is working from into /repo/.chainloop/specs/abc-123 now."
// Claude Code prints a systemMessage flush against the transcript, so
// the provider sets the banner apart with blank lines.
framedBanner = "\n\n" + banner + "\n"
)

testCases := []struct {
Expand All @@ -95,7 +98,7 @@ func TestAnnounceSessionStart(t *testing.T) {
{
name: "both channels in one document",
msg: trace.SessionStartMessage{Banner: banner, Instruction: instruction},
wantBanner: banner,
wantBanner: framedBanner,
wantInstruction: instruction,
wantEmitted: true,
},
Expand All @@ -104,7 +107,7 @@ func TestAnnounceSessionStart(t *testing.T) {
// still gets the banner, the model is told nothing.
name: "a banner with nothing to instruct",
msg: trace.SessionStartMessage{Banner: banner},
wantBanner: banner,
wantBanner: framedBanner,
wantEmitted: true,
},
{
Expand Down
13 changes: 11 additions & 2 deletions app/cli/internal/trace/claude/provider.go
Original file line number Diff line number Diff line change
Expand Up @@ -201,11 +201,20 @@ func (p *Provider) SupportsSessionStartInstruction() bool {
// systemMessage must stay top-level; nested inside hookSpecificOutput it is
// silently ignored. The event name is the one that fired, not the one
// AnnounceToUser hardcodes for its own, different hook.
//
// The blank lines around the banner are this client's framing: Claude Code
// prints a systemMessage flush against the transcript, so without them the
// banner reads as part of whatever came before.
func (p *Provider) AnnounceSessionStart(msg trace.SessionStartMessage) error {
if msg.Empty() {
return nil
}

banner := msg.Banner
if banner != "" {
banner = "\n\n" + banner + "\n"
}

type hookSpecificOutput struct {
HookEventName string `json:"hookEventName"`
AdditionalContext string `json:"additionalContext,omitempty"`
Expand All @@ -215,7 +224,7 @@ func (p *Provider) AnnounceSessionStart(msg trace.SessionStartMessage) error {
SystemMessage string `json:"systemMessage,omitempty"`
HookSpecificOutput hookSpecificOutput `json:"hookSpecificOutput"`
}{
SystemMessage: msg.Banner,
SystemMessage: banner,
HookSpecificOutput: hookSpecificOutput{
HookEventName: eventSessionStart,
AdditionalContext: msg.Instruction,
Expand Down Expand Up @@ -283,7 +292,7 @@ func (p *Provider) AnnounceToUser(msg string) error {
SystemMessage: msg,
HookSpecificOutput: hookSpecificOutput{
HookEventName: eventPostToolUse,
AdditionalContext: "Tell the user the following, including any link verbatim: " + msg,
AdditionalContext: trace.RelayToModelInstruction + msg,
},
}

Expand Down
66 changes: 52 additions & 14 deletions app/cli/internal/trace/opencode/announce_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -26,8 +26,9 @@ import (
"github.com/stretchr/testify/require"
)

// TestAnnounceSessionStart pins the session-start response: the Chainloop plugin reads instruction and posts it to the session as a context-only message.
// There is no channel to the user, so the banner is never emitted.
// TestAnnounceSessionStart pins the session-start response: the Chainloop
// plugin posts instruction to the session as a context-only message and shows
// banner to the user, each OpenCode major on its own channel.
func TestAnnounceSessionStart(t *testing.T) {
const (
banner = "Chainloop Trace is recording this session."
Expand All @@ -38,22 +39,22 @@ func TestAnnounceSessionStart(t *testing.T) {
name string
msg trace.SessionStartMessage
// want is the emitted document, or nil for no output at all.
want map[string]any
want *hookResponse
}{
{
name: "the instruction is emitted and the banner is not",
name: "the instruction and the banner share one document",
msg: trace.SessionStartMessage{Banner: banner, Instruction: instruction},
want: map[string]any{"instruction": instruction},
want: &hookResponse{Instruction: instruction, Banner: banner},
},
{
name: "an instruction on its own",
msg: trace.SessionStartMessage{Instruction: instruction},
want: map[string]any{"instruction": instruction},
want: &hookResponse{Instruction: instruction},
},
{
// A banner has nowhere to go, so the hook stays a no-op.
name: "a banner alone emits nothing",
name: "a banner on its own",
msg: trace.SessionStartMessage{Banner: banner},
want: &hookResponse{Banner: banner},
},
{
name: "nothing to say emits nothing",
Expand All @@ -72,9 +73,9 @@ func TestAnnounceSessionStart(t *testing.T) {
return
}

var got map[string]any
var got hookResponse
require.NoError(t, json.Unmarshal([]byte(out), &got))
assert.Equal(t, tc.want, got)
assert.Equal(t, *tc.want, got)
})
}
}
Expand All @@ -88,9 +89,9 @@ func TestAnnouncePromptSubmit(t *testing.T) {
testCases := []struct {
name string
reminder string
want map[string]any
want *hookResponse
}{
{name: "a reminder is emitted", reminder: reminder, want: map[string]any{"instruction": reminder}},
{name: "a reminder is emitted", reminder: reminder, want: &hookResponse{Instruction: reminder}},
{name: "nothing to say emits nothing", reminder: ""},
}

Expand All @@ -105,9 +106,46 @@ func TestAnnouncePromptSubmit(t *testing.T) {
return
}

var got map[string]any
var got hookResponse
require.NoError(t, json.Unmarshal([]byte(out), &got))
assert.Equal(t, tc.want, got)
assert.Equal(t, *tc.want, got)
})
}
}

// TestAnnounceToUser pins the response after a shell command: the plugin
// shows message to the user and adds relayToModel to the command result, so
// the model repeats the message in its reply.
func TestAnnounceToUser(t *testing.T) {
const msg = "Coding Session Available at https://app.chainloop.dev/u/acme/sessions/abc-123"

testCases := []struct {
name string
msg string
want *hookResponse
}{
{
name: "a message goes out on both channels",
msg: msg,
want: &hookResponse{Message: msg, RelayToModel: trace.RelayToModelInstruction + msg},
},
{name: "nothing to say emits nothing", msg: ""},
}

for _, tc := range testCases {
t.Run(tc.name, func(t *testing.T) {
out := captureStdout(t, func() {
require.NoError(t, New().AnnounceToUser(tc.msg))
})

if tc.want == nil {
assert.Empty(t, out)
return
}

var got hookResponse
require.NoError(t, json.Unmarshal([]byte(out), &got))
assert.Equal(t, *tc.want, got)
})
}
}
Expand Down
101 changes: 73 additions & 28 deletions app/cli/internal/trace/opencode/hooks.go
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,9 @@ var commandTools = []string{"bash", "shell"}
// pluginTemplate is the TypeScript plugin written to .opencode/plugins/chainloop-trace.ts.
// It fires chainloop trace hook subcommands on session lifecycle and tool events,
// and posts to the session the instruction the session-start hook returns and
// the reminder the user-prompt-submit hook returns at each user message.
// the reminder the user-prompt-submit hook returns at each user message. It
// shows the user the banner the session-start hook returns and the session
// link the hook after a shell command returns.
// The {{SessionEndBlock}} placeholder is replaced with the session.deleted handler
// for full install, or removed entirely for trace-run install.
//
Expand Down Expand Up @@ -136,41 +138,57 @@ function fire(directory: string, event: string, payload: Record<string, any>): P
})
}

// instructionFrom fires a hook that can answer with an instruction for the
// model, and returns the instruction it wrote to stdout, if Chainloop has one.
async function instructionFrom(directory: string, event: string, sessionID: string, hookEventName: string): Promise<string> {
const out = await fire(directory, event, { session_id: sessionID, hook_event_name: hookEventName })
if (!out.trim()) return ""
// HookResponse is what a chainloop hook writes to stdout when it has something
// to deliver. It mirrors the Go hookResponse type: the two are one contract
// and change together. A hook with nothing to say writes nothing.
type HookResponse = {
// instruction is for the model, posted to the session as context.
instruction?: string
// banner greets the user when the session starts.
banner?: string
// message is shown to the user after a shell command.
message?: string
// relayToModel is added to the shell command result, so the model repeats
// the message in its reply.
relayToModel?: string
}

// responseFrom fires a hook and returns the response it wrote to stdout.
async function responseFrom(directory: string, event: string, payload: Record<string, any>): Promise<HookResponse> {
const out = await fire(directory, event, payload)
if (!out.trim()) return {}
try {
return JSON.parse(out).instruction ?? ""
return JSON.parse(out) ?? {}
} catch (err) {
console.error("chainloop-trace: could not read the " + event + " response: " + err)
return ""
return {}
}
}

// Post adds an instruction to the session as a context-only message, which
// the model reads without replying to it. Each OpenCode major has its own API
// for it.
type Post = (sessionID: string, instruction: string) => Promise<void>
// the model reads without replying to it, and shows the user the description,
// if any. Each OpenCode major has its own API for it.
type Post = (sessionID: string, instruction: string, description?: string) => Promise<void>

// postInstruction posts the instruction and waits until it is stored, so a
// turn sent right away still finds it. A failed post costs the instruction
// only, so it is logged and never fails the caller.
async function postInstruction(post: Post, sessionID: string, instruction: string) {
async function postInstruction(post: Post, sessionID: string, instruction: string, description?: string) {
try {
await post(sessionID, instruction)
await post(sessionID, instruction, description)
} catch (err) {
console.error("chainloop-trace: could not post the session instruction: " + err)
}
}

async function sessionCreated(directory: string, sessionID: string, parentID: string | undefined, post: Post) {
const instruction = await instructionFrom(directory, "session-start", sessionID, "session.created")
const res = await responseFrom(directory, "session-start", { session_id: sessionID, hook_event_name: "session.created" })
// A child session belongs to a subagent, whose parent already has the
// instruction.
if (!instruction || parentID) return
await postInstruction(post, sessionID, instruction)
// instruction and the banner.
if (parentID || (!res.instruction && !res.banner)) return
// The banner is the description the user sees. It never goes to the model,
// so a session with no instruction posts nothing to the model.
await postInstruction(post, sessionID, res.instruction ?? "", res.banner)
}

// childSessions holds the OpenCode 1.x sessions of subagents, whose parent
Expand All @@ -182,8 +200,8 @@ const childSessions = new Set<string>()
// answer with a short reminder to capture a new or changed spec, which is
// posted ahead of the turn.
async function promptSubmitted(directory: string, sessionID: string, post: Post) {
const reminder = await instructionFrom(directory, "user-prompt-submit", sessionID, "chat.message")
if (reminder) await postInstruction(post, sessionID, reminder)
const res = await responseFrom(directory, "user-prompt-submit", { session_id: sessionID, hook_event_name: "chat.message" })
if (res.instruction) await postInstruction(post, sessionID, res.instruction)
}

async function sessionEvent(directory: string, type: string, sessionID: string, parentID: string | undefined, post: Post) {
Expand All @@ -193,7 +211,9 @@ async function sessionEvent(directory: string, type: string, sessionID: string,
{{SessionEndBlock}}
}

async function toolEvent(directory: string, hook: string, hookEventName: string, sessionID: string, tool: string, callID: string, args: any, skillDir = "") {
// toolEvent fires the hook of a tool call and returns its response. Only the
// hook after a shell command can have one: the session link left by a push.
async function toolEvent(directory: string, hook: string, hookEventName: string, sessionID: string, tool: string, callID: string, args: any, skillDir = ""): Promise<HookResponse | undefined> {
const payload = { session_id: sessionID, hook_event_name: hookEventName, tool_name: tool }
if (tool === skillTool) {
// The skill is loaded after the call, and the result names its folder.
Expand All @@ -204,8 +224,7 @@ async function toolEvent(directory: string, hook: string, hookEventName: string,
if (commandTools.includes(tool)) {
// The call ID pairs this hook with the other hook of the same call, so
// overlapping commands keep their own snapshots.
await fire(directory, hook, { ...payload, tool_use_id: callID })
return
return responseFrom(directory, hook, { ...payload, tool_use_id: callID })
}
if (!fileWritingTools.includes(tool)) return
for (const fp of filePathsFromArgs(args)) {
Expand All @@ -215,8 +234,19 @@ async function toolEvent(directory: string, hook: string, hookEventName: string,

// server is the OpenCode 1.x entry point.
async function server({ directory, client }: any) {
// noReply stores the message without asking the model for an answer.
const post: Post = async (sessionID, instruction) => {
// toast shows a message in the TUI. It is not awaited, so a slow TUI never
// holds back the session or a tool result, and a run without a TUI only
// logs the failure.
const toast = (message: string) => {
Promise.resolve()
.then(() => client.tui.showToast({ body: { message, variant: "info" } }))
.catch((err: unknown) => console.error("chainloop-trace: could not show the message: " + err))
}
// noReply stores the message without asking the model for an answer. The
// description is shown as a toast.
const post: Post = async (sessionID, instruction, description) => {
if (description) toast(description)
if (!instruction) return
await client.session.prompt({
path: { id: sessionID },
body: { noReply: true, parts: [{ type: "text", text: instruction, synthetic: true }] },
Expand Down Expand Up @@ -255,7 +285,14 @@ async function server({ directory, client }: any) {
await toolEvent(directory, "pre-tool-use", "tool.execute.before", input.sessionID, input.tool, input.callID, output.args)
},
"tool.execute.after": async (input: any, output: any) => {
await toolEvent(directory, "post-tool-use", "tool.execute.after", input.sessionID, input.tool, input.callID, input.args, input.tool === skillTool ? skillDirFrom(output) : "")
const res = await toolEvent(directory, "post-tool-use", "tool.execute.after", input.sessionID, input.tool, input.callID, input.args, input.tool === skillTool ? skillDirFrom(output) : "")
// The toast reaches the user now. The tool output reaches the model,
// whose reply stays on screen after the toast is gone. An aborted tool
// can have no output to add to.
if (res?.message) toast(res.message)
if (res?.relayToModel && typeof output?.output === "string") {
output.output = output.output + "\n\n" + res.relayToModel
}
},
}
}
Expand All @@ -269,12 +306,20 @@ async function setup(ctx: any) {
await toolEvent(directory, "pre-tool-use", "tool.execute.before", event.sessionID, event.tool, event.id, event.input)
})
await ctx.tool.hook("execute.after", async (event: any) => {
await toolEvent(directory, "post-tool-use", "tool.execute.after", event.sessionID, event.tool, event.id, event.input, event.tool === skillTool ? skillDirFrom(event.output, event.result, event) : "")
const res = await toolEvent(directory, "post-tool-use", "tool.execute.after", event.sessionID, event.tool, event.id, event.input, event.tool === skillTool ? skillDirFrom(event.output, event.result, event) : "")
// The model reads the content parts of the result, and the TUI shows them
// in the command block, so one part reaches both the user and the model.
// A failed command has no result to add to.
if (res?.relayToModel && Array.isArray(event.result?.content)) {
event.result.content.push({ type: "text", text: res.relayToModel })
}
})

// resume: false stores the message without asking the model for an answer.
const post: Post = async (sessionID, instruction) => {
await ctx.session.synthetic({ sessionID, text: instruction, resume: false })
// The TUI shows a synthetic message by its description only, and the model
// reads its text, which is empty when there is only a banner.
const post: Post = async (sessionID, instruction, description) => {
await ctx.session.synthetic({ sessionID, text: instruction, description, resume: false })
}

// A child session belongs to a subagent, whose parent session already gets
Expand Down
Loading
Loading