From 0901286bf179a53b8d8cf950875cbd02fc3c0812 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Mat=C3=ADas=20Insaurralde?= Date: Thu, 8 Oct 2026 21:58:44 -0300 Subject: [PATCH 1/6] spec: issue-3582 Trace evidence export to disk MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Design spec for a mode that runs the trace push-time assembly and writes the AI coding session evidence set to a local directory, with no control plane and no attestation. Refs #3582 Signed-off-by: Matías Insaurralde --- .../specs/issue-3582-trace-evidence-export.md | 122 ++++++++++++++++++ 1 file changed, 122 insertions(+) create mode 100644 docs/specs/issue-3582-trace-evidence-export.md diff --git a/docs/specs/issue-3582-trace-evidence-export.md b/docs/specs/issue-3582-trace-evidence-export.md new file mode 100644 index 000000000..a89860a8d --- /dev/null +++ b/docs/specs/issue-3582-trace-evidence-export.md @@ -0,0 +1,122 @@ +--- +owner: matiasinsaurralde +ticket: https://github.com/chainloop-dev/chainloop/issues/3582 +prd: +--- + + + +# Spec issue-3582: Trace evidence export to disk + +## Summary +Chainloop Trace records an AI coding session. On a git push, it assembles the session evidence and sends it to the control plane as a signed attestation. The assembly does real work. It reads the spec sources at push time, redacts secrets, and replaces copies with pointers. It stores each spec, skill, and image as its own material. Today that assembly runs only inside a push, so it needs a control plane, credentials, and a network. This spec adds a mode that runs the same assembly and writes the result to a local directory, with no control plane and no attestation. It gives developers and tests the exact evidence a push would produce, on disk, for inspection and comparison. + +## Problem +- The evidence a push produces is more than the raw transcript. The push reads the spec sources, redacts secrets, and adds the pointers, the pasted images, and the skills. +- This assembly runs only inside a push. So a developer needs a running control plane, credentials, and a network to see its output. +- A test cannot get the pushed evidence without a live backend. So the assembly is hard to test. +- No command writes the session evidence to disk. A developer cannot see what a session would send before they send it. +- The push also signs and stores the attestation. A developer who only wants to read the evidence pays for the whole backend path. + +## Goals and Non-Goals +- Goal: a mode runs the full push-time assembly and writes the result to a local directory, with no control plane. +- Goal: the written evidence equals what a push would upload for the same session. The fields, the redaction, the pointers, and the spec, skill, and image materials match. +- Goal: the output is laid out so a reader and a test find each material by its digest. +- Goal: a developer reads the evidence of a session with no backend and no credentials. +- Non-goal: a signed attestation. The mode writes the evidence set, not a signed statement or a signed bundle. A control-plane-free signed attestation needs a contract and a signing key, and it is a separate, larger effort. +- Non-goal: policy evaluation and contract-required materials. Those need a contract from the control plane. +- Non-goal: a change to the push path, or to what a push uploads. +- Non-goal: the agent or the model offline. The agent still calls its model over the network during the session. This mode is about the Chainloop backend, not the model. + +## Requirements + +### R-001: Write the evidence to a local directory +The mode MUST run the push-time assembly and write the result to a directory the user names. It MUST make no call to the control plane and emit no attestation. +- Done when: the mode runs with no network and no credentials, and it writes the directory. + +### R-002: The same evidence as a push +The written evidence record MUST equal the one a push would upload for the same session. The fields, the redaction, the pointers, and the spec, skill, and image materials MUST match. The one difference is the signature and the attestation wrapper, which the mode does not produce. +- Done when: a session that is exported and the same session that is pushed give the same evidence record and the same material digests. + +### R-003: Materials addressable by digest +The mode MUST write each material under its content digest. A reader and a test MUST find each material from the reference in the evidence record. + +### R-004: Redaction on by default +The mode MUST redact secrets in the evidence and in the materials, as a push does. An opt-out is for a trusted local run only, and the output can then hold secrets. +- Done when: a secret in a spec source does not appear in the exported files. + +### R-005: No control plane and no credentials +The mode MUST NOT need a control plane, an organization, a token, or a network. + +### R-006: A manifest +The mode MUST write an index. The index MUST list each material by name, kind, and digest. A reader MUST see the whole set of materials from the index, with no need to parse each file. The evidence record holds the warnings of the run, not the index. + +### R-007: Export from a recorded session +The mode MUST build the export from the session state that the hooks recorded. It MUST NOT depend on one way of running the agent. + +## Constraints +- The repository is public. The mode and its code hold no internal context. +- The agent calls its model over the network during the session. The mode changes the Chainloop backend path only. +- The export is not signed, so it is not proof of anything. It is for inspection and for tests. +- The mode reuses the push assembly. It does not fork that logic. +- An export can hold the content of the session, including local files the session read. A developer keeps the output directory as they keep any working copy of the session. + +## Proposal +The developer runs the mode and names an output directory. The mode records the session as the push path does, or it reads a session the hooks already recorded. It then runs the same assembly a push runs. It parses the session. It reads each spec source at that time. It redacts secrets. It replaces each copy in the transcript with a pointer. It gathers the pasted images and the skills. A push would store each material in the backend and send a signed attestation. Instead, the mode writes each material to the output directory under its digest, writes the session evidence record, and writes a manifest. + +The push path already separates the assembly from the step that stores one material. That store step is a single seam. The mode gives the seam a store that writes to disk and returns the digest, in place of the store that uploads to the backend. Every assembly step runs unchanged: the spec read, the redaction, the pointers, the images, the skills, and the evidence record. So the export cannot drift from a push. It is the same assembly with a different sink. + +The output directory holds three things. It holds the session evidence record, which carries the warnings of the run. It holds a folder of materials, each named by its digest. It holds a manifest that lists the materials. The layout mirrors the content-addressed store the backend uses. So a material is found by its digest, and two runs of the same session are easy to compare. + +The mode writes the evidence set, not a signed attestation. A signed attestation with no control plane needs a contract and a signing key. That is a separate, larger effort, and this spec puts it out of scope. The export is for a developer who reads the evidence, and for a test that checks the assembly. + +The mode is not named "offline". The agent still calls its model over the network during the session, so "offline" would claim something false. The name says what the mode does: it writes the evidence to disk. The verb is "export". The first surface is a flag on the single-session run that names the directory to write to. A standalone "export" command can come later, for a session the hooks already recorded. + +```mermaid +flowchart TD + Agent[Coding agent session] --> Hooks[Trace hooks record the session] + Hooks --> Assembly[Push-time assembly: parse, read specs, redact, add pointers, images, skills] + Assembly --> Sink{Material sink} + Sink -->|push| Backend[Control plane and CAS: stored, signed attestation] + Sink -->|export| Disk[Local directory: evidence record, materials by digest, manifest] +``` + +## Decision Record + +| ID | Decision | Choice | Why (and what we rejected) | Source | +|----|----------|--------|----------------------------|--------| +| D-001 | How the mode produces the evidence | Reuse the push assembly through its store seam, with a store that writes to disk | The export is then the same assembly as a push, so it cannot drift. Rejected: a new path that parses and assembles on its own (it forks the logic, and the two drift). | drafting | +| D-002 | What the mode writes | The evidence set: the evidence record and its materials. Not a signed attestation | A developer and a test need the evidence, not a signature. Rejected: a signed bundle with no control plane (it needs a contract and a signing key, which is a larger, separate effort). | drafting | +| D-003 | Output layout | Content-addressed: each material under its digest, plus a manifest | A reader maps a reference in the evidence to its material, and two runs are easy to compare. Rejected: a flat dump with ad-hoc names (hard to map, hard to compare). | drafting | +| D-004 | Secrets in the output | Redaction on by default, with an opt-out for a trusted local run | The output equals a push by default, so it holds no secret a push would drop. The opt-out serves a developer who debugs the redaction. Rejected: no redaction (the output holds secrets). Also rejected: redaction with no opt-out (a developer cannot see the raw text). | drafting | +| D-005 | Name of the mode | Name it for what it does, write the evidence to disk, for example "export" | The agent and the model are still online during the session, so "offline" claims something false. Rejected: "offline" (false claim). Also rejected: "dry-run" (it already means render without store, and it still calls the control plane). | owner request | +| D-006 | The surface and the verb | The verb is "export". The first surface is a flag on the single-session run that names the output directory. A standalone "export" command comes later | One command then gives the whole path: run the agent and write the evidence together. A standalone command serves a session the hooks already recorded, and it can wait. Rejected: a standalone command first (it does not cover the common case of running the agent and reading the evidence together). | owner request | +| D-007 | What the export carries | Everything a push uploads, including the raw transcript material | The export is faithful only when it holds what a push holds. Rejected: the evidence record and the spec, skill, and image materials only (it drops the transcript, so the export does not match a push). | owner request | +| D-008 | Determinism for a compare | The mode writes the faithful output. A shared helper normalizes the volatile fields for a compare | The export stays faithful to a push, with no special mode that hides a difference. One helper keeps each test from normalizing on its own. Rejected: a normalized form that the mode itself writes. Such a mode can drift from a push, and it can mask a real change. | owner request | + +## Open Questions +None. + +## Milestones +1. **Core export.** The mode writes the evidence set of a recorded session to a directory, with the disk store and no control plane. +2. **Export from the run.** The single-session run takes an export flag that names a directory, so one command runs the agent and writes the evidence. +3. **Layout and manifest.** The content-addressed layout and the manifest have a documented structure, so a test reads the output without guessing. + +## Risks + +| Risk | Mitigation | +|------|------------| +| The export drifts from a real push. | The export reuses the push assembly through its store seam, so the only difference is the sink. A test compares an export with a push for the same session (R-002). | +| The output holds secrets. | Redaction is on by default, as in a push. The opt-out is for a trusted local run, and it is documented as such (D-004). | +| A reader treats the export as a signed attestation. | The export is not signed, and the manifest says so. The mode is for inspection and for tests. | +| Timestamps and volatile paths make a test flaky. | The test normalizes the volatile fields before a compare (D-008). | From d6bf25435fcfae53dcf75680a76e3002ea62f497 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Mat=C3=ADas=20Insaurralde?= Date: Sun, 11 Oct 2026 19:16:54 -0300 Subject: [PATCH 2/6] spec: issue-3582 add local mode for trace evidence export MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Revise the design so trace init --local is the primary enabling surface, which records a reserved local organization and a repository project so the setup contacts no control plane. trace run --export then wraps the agent and writes the evidence set to disk when the session ends, with no push. Refs #3582 Chainloop-Trace-Sessions: d139ac66-b95f-4920-ac38-12997996037d Signed-off-by: Matías Insaurralde --- .../specs/issue-3582-trace-evidence-export.md | 43 ++++++++++++++----- 1 file changed, 32 insertions(+), 11 deletions(-) diff --git a/docs/specs/issue-3582-trace-evidence-export.md b/docs/specs/issue-3582-trace-evidence-export.md index a89860a8d..ce9ebdf8f 100644 --- a/docs/specs/issue-3582-trace-evidence-export.md +++ b/docs/specs/issue-3582-trace-evidence-export.md @@ -19,20 +19,22 @@ When your code goes against this spec, say so in your PR. Do not edit the spec t # Spec issue-3582: Trace evidence export to disk ## Summary -Chainloop Trace records an AI coding session. On a git push, it assembles the session evidence and sends it to the control plane as a signed attestation. The assembly does real work. It reads the spec sources at push time, redacts secrets, and replaces copies with pointers. It stores each spec, skill, and image as its own material. Today that assembly runs only inside a push, so it needs a control plane, credentials, and a network. This spec adds a mode that runs the same assembly and writes the result to a local directory, with no control plane and no attestation. It gives developers and tests the exact evidence a push would produce, on disk, for inspection and comparison. +Chainloop Trace records an AI coding session. On a git push, it assembles the session evidence and sends it to the control plane as a signed attestation. The assembly does real work. It reads the spec sources at push time, redacts secrets, and replaces copies with pointers. It stores each spec, skill, and image as its own material. Today that assembly runs only inside a push. The setup needs the control plane too. So a developer needs an account, credentials, and a network before they see any evidence. This spec adds a local mode. `trace init --local` sets up the repository with a reserved local organization and a project derived from the repository, so the setup contacts no control plane. `trace run --export` then wraps the agent. When the session ends, it runs the same assembly a push runs and writes the evidence set to a local directory. There is no attestation and no push. It gives developers and tests the exact evidence a push would produce, on disk, with no backend at all. ## Problem - The evidence a push produces is more than the raw transcript. The push reads the spec sources, redacts secrets, and adds the pointers, the pasted images, and the skills. - This assembly runs only inside a push. So a developer needs a running control plane, credentials, and a network to see its output. +- The setup also needs the control plane. The init step creates the workflow and needs a login, so there is no way to use Trace without a backend at all. - A test cannot get the pushed evidence without a live backend. So the assembly is hard to test. - No command writes the session evidence to disk. A developer cannot see what a session would send before they send it. - The push also signs and stores the attestation. A developer who only wants to read the evidence pays for the whole backend path. ## Goals and Non-Goals -- Goal: a mode runs the full push-time assembly and writes the result to a local directory, with no control plane. +- Goal: `trace init --local` sets up the repository for local mode and contacts no control plane. +- Goal: `trace run --export` runs the full push-time assembly and writes the result to a local directory when the session ends. It needs no control plane and does no push. - Goal: the written evidence equals what a push would upload for the same session. The fields, the redaction, the pointers, and the spec, skill, and image materials match. - Goal: the output is laid out so a reader and a test find each material by its digest. -- Goal: a developer reads the evidence of a session with no backend and no credentials. +- Goal: a developer sets up and reads the evidence of a session with no account, no credentials, and no network. - Non-goal: a signed attestation. The mode writes the evidence set, not a signed statement or a signed bundle. A control-plane-free signed attestation needs a contract and a signing key, and it is a separate, larger effort. - Non-goal: policy evaluation and contract-required materials. Those need a contract from the control plane. - Non-goal: a change to the push path, or to what a push uploads. @@ -56,7 +58,7 @@ The mode MUST redact secrets in the evidence and in the materials, as a push doe - Done when: a secret in a spec source does not appear in the exported files. ### R-005: No control plane and no credentials -The mode MUST NOT need a control plane, an organization, a token, or a network. +The mode MUST NOT need a control plane, a real organization, a token, or a network. A reserved local organization stands in for the control-plane organization. ### R-006: A manifest The mode MUST write an index. The index MUST list each material by name, kind, and digest. A reader MUST see the whole set of materials from the index, with no need to parse each file. The evidence record holds the warnings of the run, not the index. @@ -64,15 +66,26 @@ The mode MUST write an index. The index MUST list each material by name, kind, a ### R-007: Export from a recorded session The mode MUST build the export from the session state that the hooks recorded. It MUST NOT depend on one way of running the agent. +### R-008: Local mode set up with no control plane +`trace init --local` MUST set up the repository for local mode without contacting the control plane. It MUST record a reserved local organization and a project derived from the repository. The run MUST use these in place of a control-plane workflow and contract. +- Done when: `trace init --local` runs with no network and no credentials, and a later export needs no control-plane lookup. + +### R-009: Export when the session ends +`trace run --export` MUST wrap the agent and, when the session ends, write the evidence set to the named directory. It MUST NOT push an attestation. +- Done when: the agent runs, the session ends, and the directory holds the evidence with no push. + ## Constraints - The repository is public. The mode and its code hold no internal context. - The agent calls its model over the network during the session. The mode changes the Chainloop backend path only. - The export is not signed, so it is not proof of anything. It is for inspection and for tests. - The mode reuses the push assembly. It does not fork that logic. +- The reserved local organization is a local placeholder. It is never sent to a control plane, and it is not a real Chainloop organization. - An export can hold the content of the session, including local files the session read. A developer keeps the output directory as they keep any working copy of the session. ## Proposal -The developer runs the mode and names an output directory. The mode records the session as the push path does, or it reads a session the hooks already recorded. It then runs the same assembly a push runs. It parses the session. It reads each spec source at that time. It redacts secrets. It replaces each copy in the transcript with a pointer. It gathers the pasted images and the skills. A push would store each material in the backend and send a signed attestation. Instead, the mode writes each material to the output directory under its digest, writes the session evidence record, and writes a manifest. +The developer sets up the repository once with `trace init --local`. In local mode, init records a reserved local organization and a project derived from the repository. It creates no workflow and contacts no control plane. Today init needs a login and creates the workflow the attestations target. Local mode drops that step, so a developer with no account can set up Trace. The reserved organization and the project stand in for the control-plane workflow and contract. + +The developer then runs the agent through `trace run --export` with an output directory. The run wraps the agent as it does today. When the session ends, instead of signing and pushing an attestation, the run executes the same assembly a push runs. It parses the session. It reads each spec source at that time. It redacts secrets. It replaces each copy in the transcript with a pointer. It gathers the pasted images and the skills. A push would store each material in the backend and send a signed attestation. Instead, the run writes each material to the output directory under its digest, writes the session evidence record, and writes a manifest. The push path already separates the assembly from the step that stores one material. That store step is a single seam. The mode gives the seam a store that writes to disk and returns the digest, in place of the store that uploads to the backend. Every assembly step runs unchanged: the spec read, the redaction, the pointers, the images, the skills, and the evidence record. So the export cannot drift from a push. It is the same assembly with a different sink. @@ -80,12 +93,14 @@ The output directory holds three things. It holds the session evidence record, w The mode writes the evidence set, not a signed attestation. A signed attestation with no control plane needs a contract and a signing key. That is a separate, larger effort, and this spec puts it out of scope. The export is for a developer who reads the evidence, and for a test that checks the assembly. -The mode is not named "offline". The agent still calls its model over the network during the session, so "offline" would claim something false. The name says what the mode does: it writes the evidence to disk. The verb is "export". The first surface is a flag on the single-session run that names the directory to write to. A standalone "export" command can come later, for a session the hooks already recorded. +The mode is not named "offline". The agent still calls its model over the network during the session, so "offline" would claim something false. Two names say what the parts do. `--local` names the init mode: no control plane, evidence on local disk. `--export` names the run output: write the evidence to a directory. A standalone "export" command, for a session the hooks already recorded, can come later. ```mermaid flowchart TD - Agent[Coding agent session] --> Hooks[Trace hooks record the session] - Hooks --> Assembly[Push-time assembly: parse, read specs, redact, add pointers, images, skills] + Init[trace init --local: reserved local org, repository project, no control plane] --> Run[trace run --export: wraps the agent] + Run --> Session[Coding agent session, hooks record it] + Session --> End{Session ends} + End --> Assembly[Same push-time assembly: parse, read specs, redact, pointers, images, skills] Assembly --> Sink{Material sink} Sink -->|push| Backend[Control plane and CAS: stored, signed attestation] Sink -->|export| Disk[Local directory: evidence record, materials by digest, manifest] @@ -103,14 +118,19 @@ flowchart TD | D-006 | The surface and the verb | The verb is "export". The first surface is a flag on the single-session run that names the output directory. A standalone "export" command comes later | One command then gives the whole path: run the agent and write the evidence together. A standalone command serves a session the hooks already recorded, and it can wait. Rejected: a standalone command first (it does not cover the common case of running the agent and reading the evidence together). | owner request | | D-007 | What the export carries | Everything a push uploads, including the raw transcript material | The export is faithful only when it holds what a push holds. Rejected: the evidence record and the spec, skill, and image materials only (it drops the transcript, so the export does not match a push). | owner request | | D-008 | Determinism for a compare | The mode writes the faithful output. A shared helper normalizes the volatile fields for a compare | The export stays faithful to a push, with no special mode that hides a difference. One helper keeps each test from normalizing on its own. Rejected: a normalized form that the mode itself writes. Such a mode can drift from a push, and it can mask a real change. | owner request | +| D-009 | Primary enabling surface | `trace init --local` sets up local mode and contacts no control plane. It is the primary surface, ahead of the run flag | Today init needs a login and creates a workflow on the control plane, so a developer cannot use Trace without a backend. A local init drops that block and makes a fully backend-free path possible. This revises the ordering in D-006, which put the run flag first. Rejected: a run-only flag with no local init (init still needs the control plane, so the path is not backend-free). | owner request | +| D-010 | How local mode avoids the control plane | A reserved local organization and a project derived from the repository, in place of the control-plane workflow and contract | The run assembles with these local values and needs no control-plane lookup. Rejected: reusing a real organization and project from config (it mixes real and placeholder values, and a real organization implies a control plane). | owner request | +| D-011 | When the run writes the evidence | On session end, from `trace run --export`, with no push | The run already assembles at session end today, so the export reuses that point. One command runs the agent and writes the evidence together. Rejected: a git-push-triggered export (it adds a branch to the push hook and needs a git push, which the session-end run does not). | owner request | +| D-012 | The two names | `--local` for the init mode, `--export` for the run output | Each name says what its part does, and neither claims the model is offline. This refines D-005, which rejected "offline". Rejected: one name for both (the init mode and the run output are different actions). | owner request | ## Open Questions None. ## Milestones -1. **Core export.** The mode writes the evidence set of a recorded session to a directory, with the disk store and no control plane. -2. **Export from the run.** The single-session run takes an export flag that names a directory, so one command runs the agent and writes the evidence. -3. **Layout and manifest.** The content-addressed layout and the manifest have a documented structure, so a test reads the output without guessing. +1. **Local init.** `trace init --local` sets up local mode with a reserved local organization and a repository project, and contacts no control plane. +2. **Disk sink and core export.** The assembly writes the evidence set of a session to a directory through the disk store, with no control plane. +3. **Export from the run.** `trace run --export` wraps the agent and writes the evidence when the session ends, with no push. +4. **Layout and manifest.** The content-addressed layout and the manifest have a documented structure, so a test reads the output without guessing. ## Risks @@ -119,4 +139,5 @@ None. | The export drifts from a real push. | The export reuses the push assembly through its store seam, so the only difference is the sink. A test compares an export with a push for the same session (R-002). | | The output holds secrets. | Redaction is on by default, as in a push. The opt-out is for a trusted local run, and it is documented as such (D-004). | | A reader treats the export as a signed attestation. | The export is not signed, and the manifest says so. The mode is for inspection and for tests. | +| A reader treats the reserved local organization as a real Chainloop organization. | The reserved organization is a documented local placeholder. It is never sent to a control plane, and the export is not signed (D-010). | | Timestamps and volatile paths make a test flaky. | The test normalizes the volatile fields before a compare (D-008). | From 487ed5773470565fe348c8a7433458065bc6ba7a Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Mat=C3=ADas=20Insaurralde?= Date: Tue, 13 Oct 2026 00:50:24 -0300 Subject: [PATCH 3/6] spec: issue-3582 record local-mode implementation decisions MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Record the decisions the implementation settled. Local mode is marked per-clone in the trace state rather than in the shared repository config. The pre-push hook sends no attestation in local mode. A standalone trace export command writes the evidence of the sessions the hooks already recorded. Refs #3582 Chainloop-Trace-Sessions: d139ac66-b95f-4920-ac38-12997996037d Signed-off-by: Matías Insaurralde --- .../specs/issue-3582-trace-evidence-export.md | 30 ++++++++++++------- 1 file changed, 20 insertions(+), 10 deletions(-) diff --git a/docs/specs/issue-3582-trace-evidence-export.md b/docs/specs/issue-3582-trace-evidence-export.md index ce9ebdf8f..1cb677904 100644 --- a/docs/specs/issue-3582-trace-evidence-export.md +++ b/docs/specs/issue-3582-trace-evidence-export.md @@ -19,7 +19,7 @@ When your code goes against this spec, say so in your PR. Do not edit the spec t # Spec issue-3582: Trace evidence export to disk ## Summary -Chainloop Trace records an AI coding session. On a git push, it assembles the session evidence and sends it to the control plane as a signed attestation. The assembly does real work. It reads the spec sources at push time, redacts secrets, and replaces copies with pointers. It stores each spec, skill, and image as its own material. Today that assembly runs only inside a push. The setup needs the control plane too. So a developer needs an account, credentials, and a network before they see any evidence. This spec adds a local mode. `trace init --local` sets up the repository with a reserved local organization and a project derived from the repository, so the setup contacts no control plane. `trace run --export` then wraps the agent. When the session ends, it runs the same assembly a push runs and writes the evidence set to a local directory. There is no attestation and no push. It gives developers and tests the exact evidence a push would produce, on disk, with no backend at all. +Chainloop Trace records an AI coding session. On a git push, it assembles the session evidence and sends it to the control plane as a signed attestation. The assembly does real work. It reads the spec sources at push time, redacts secrets, and replaces copies with pointers. It stores each spec, skill, and image as its own material. Today that assembly runs only inside a push. The setup needs the control plane too. So a developer needs an account, credentials, and a network before they see any evidence. This spec adds a local mode. `trace init --local` sets the repository up with no control plane: no login, no organization, no workflow. In that mode the push hook sends no attestation, and the evidence is written to disk instead. `trace run --export` wraps the agent and writes the evidence when the session ends, and `trace export` writes the evidence of the sessions the hooks already recorded. Both run the same assembly a push runs. There is no attestation and no push. It gives developers and tests the exact evidence a push would produce, on disk, with no backend at all. ## Problem - The evidence a push produces is more than the raw transcript. The push reads the spec sources, redacts secrets, and adds the pointers, the pasted images, and the skills. @@ -58,7 +58,7 @@ The mode MUST redact secrets in the evidence and in the materials, as a push doe - Done when: a secret in a spec source does not appear in the exported files. ### R-005: No control plane and no credentials -The mode MUST NOT need a control plane, a real organization, a token, or a network. A reserved local organization stands in for the control-plane organization. +The mode MUST NOT need a control plane, an organization, a token, or a network. ### R-006: A manifest The mode MUST write an index. The index MUST list each material by name, kind, and digest. A reader MUST see the whole set of materials from the index, with no need to parse each file. The evidence record holds the warnings of the run, not the index. @@ -67,23 +67,27 @@ The mode MUST write an index. The index MUST list each material by name, kind, a The mode MUST build the export from the session state that the hooks recorded. It MUST NOT depend on one way of running the agent. ### R-008: Local mode set up with no control plane -`trace init --local` MUST set up the repository for local mode without contacting the control plane. It MUST record a reserved local organization and a project derived from the repository. The run MUST use these in place of a control-plane workflow and contract. -- Done when: `trace init --local` runs with no network and no credentials, and a later export needs no control-plane lookup. +`trace init --local` MUST set up the repository for local mode without contacting the control plane: no login, no organization, no workflow. In local mode the push hook MUST send no attestation. The evidence goes to disk instead. Local mode MUST be per-clone, so it does not change whether another clone of the same repository attests. +- Done when: `trace init --local` runs with no network and no credentials, and a push in that clone sends no attestation. ### R-009: Export when the session ends `trace run --export` MUST wrap the agent and, when the session ends, write the evidence set to the named directory. It MUST NOT push an attestation. - Done when: the agent runs, the session ends, and the directory holds the evidence with no push. +### R-010: Export a session the hooks already recorded +The CLI MUST provide a command that writes the evidence of the already-recorded sessions to a directory, without wrapping a run. A developer who set the repository up with `trace init --local` MUST be able to export the sessions the hooks recorded. +- Done when: after a recorded session, the command writes the evidence to the directory, with no push. + ## Constraints - The repository is public. The mode and its code hold no internal context. - The agent calls its model over the network during the session. The mode changes the Chainloop backend path only. - The export is not signed, so it is not proof of anything. It is for inspection and for tests. - The mode reuses the push assembly. It does not fork that logic. -- The reserved local organization is a local placeholder. It is never sent to a control plane, and it is not a real Chainloop organization. +- Local mode is per-clone. It is recorded in the local trace state, not in the shared repository config, so it never changes attestation for another clone. - An export can hold the content of the session, including local files the session read. A developer keeps the output directory as they keep any working copy of the session. ## Proposal -The developer sets up the repository once with `trace init --local`. In local mode, init records a reserved local organization and a project derived from the repository. It creates no workflow and contacts no control plane. Today init needs a login and creates the workflow the attestations target. Local mode drops that step, so a developer with no account can set up Trace. The reserved organization and the project stand in for the control-plane workflow and contract. +The developer sets up the repository once with `trace init --local`. Init installs the same hooks but creates no workflow and contacts no control plane. Today init needs a login and creates the workflow the attestations target. Local mode drops that step, so a developer with no account can set up Trace. The mode is recorded in the local trace state, not in the shared repository config. So it is per-clone, and never turns attestation off for a teammate. In local mode the push hook sends no attestation, because there is no control plane to send it to. The evidence is written to disk on demand instead. The developer then runs the agent through `trace run --export` with an output directory. The run wraps the agent as it does today. When the session ends, instead of signing and pushing an attestation, the run executes the same assembly a push runs. It parses the session. It reads each spec source at that time. It redacts secrets. It replaces each copy in the transcript with a pointer. It gathers the pasted images and the skills. A push would store each material in the backend and send a signed attestation. Instead, the run writes each material to the output directory under its digest, writes the session evidence record, and writes a manifest. @@ -93,14 +97,16 @@ The output directory holds three things. It holds the session evidence record, w The mode writes the evidence set, not a signed attestation. A signed attestation with no control plane needs a contract and a signing key. That is a separate, larger effort, and this spec puts it out of scope. The export is for a developer who reads the evidence, and for a test that checks the assembly. -The mode is not named "offline". The agent still calls its model over the network during the session, so "offline" would claim something false. Two names say what the parts do. `--local` names the init mode: no control plane, evidence on local disk. `--export` names the run output: write the evidence to a directory. A standalone "export" command, for a session the hooks already recorded, can come later. +The mode is not named "offline". The agent still calls its model over the network during the session, so "offline" would claim something false. Two names say what the parts do. `--local` names the init mode: no control plane, evidence on local disk. `--export` names the run output: write the evidence to a directory. A standalone `trace export` command writes the evidence of the sessions the hooks already recorded. A developer who set up local mode then exports without wrapping a run. ```mermaid flowchart TD - Init[trace init --local: reserved local org, repository project, no control plane] --> Run[trace run --export: wraps the agent] + Init[trace init --local: local mode, no control plane] --> Run[trace run --export: wraps the agent] + Init --> Export[trace export: already-recorded sessions] Run --> Session[Coding agent session, hooks record it] Session --> End{Session ends} End --> Assembly[Same push-time assembly: parse, read specs, redact, pointers, images, skills] + Export --> Assembly Assembly --> Sink{Material sink} Sink -->|push| Backend[Control plane and CAS: stored, signed attestation] Sink -->|export| Disk[Local directory: evidence record, materials by digest, manifest] @@ -122,15 +128,19 @@ flowchart TD | D-010 | How local mode avoids the control plane | A reserved local organization and a project derived from the repository, in place of the control-plane workflow and contract | The run assembles with these local values and needs no control-plane lookup. Rejected: reusing a real organization and project from config (it mixes real and placeholder values, and a real organization implies a control plane). | owner request | | D-011 | When the run writes the evidence | On session end, from `trace run --export`, with no push | The run already assembles at session end today, so the export reuses that point. One command runs the agent and writes the evidence together. Rejected: a git-push-triggered export (it adds a branch to the push hook and needs a git push, which the session-end run does not). | owner request | | D-012 | The two names | `--local` for the init mode, `--export` for the run output | Each name says what its part does, and neither claims the model is offline. This refines D-005, which rejected "offline". Rejected: one name for both (the init mode and the run output are different actions). | owner request | +| D-013 | Where local mode is recorded | A per-clone marker in the local trace state, nothing in the shared repository config | The repository config is committed and shared. A local flag there would turn attestation off for every teammate who clones the repository and runs init. The export assembles each material with no organization or project, so none needs recording. This supersedes D-010, which put a reserved organization and project in the config. | implementation | +| D-014 | What the push hook does in local mode | It sends no attestation and returns cleanly | Local mode has no control plane to attest to. The git-push-triggered export was already rejected (D-011), so the hook does nothing on push. The evidence is exported on demand with `trace export`. | implementation | +| D-015 | When the standalone export command ships | Now, as `trace export`, together with local mode | A clone set up with `trace init --local` records sessions through the installed hooks, with no wrapped run to export them. Without the command there is no way to export them. This brings forward the standalone command that D-006 deferred. | implementation | ## Open Questions None. ## Milestones -1. **Local init.** `trace init --local` sets up local mode with a reserved local organization and a repository project, and contacts no control plane. +1. **Local init.** `trace init --local` sets up local mode without a control plane, marked per-clone, and the push hook sends no attestation in that mode. 2. **Disk sink and core export.** The assembly writes the evidence set of a session to a directory through the disk store, with no control plane. 3. **Export from the run.** `trace run --export` wraps the agent and writes the evidence when the session ends, with no push. 4. **Layout and manifest.** The content-addressed layout and the manifest have a documented structure, so a test reads the output without guessing. +5. **Standalone export.** `trace export` writes the evidence of the already-recorded sessions to a directory. ## Risks @@ -139,5 +149,5 @@ None. | The export drifts from a real push. | The export reuses the push assembly through its store seam, so the only difference is the sink. A test compares an export with a push for the same session (R-002). | | The output holds secrets. | Redaction is on by default, as in a push. The opt-out is for a trusted local run, and it is documented as such (D-004). | | A reader treats the export as a signed attestation. | The export is not signed, and the manifest says so. The mode is for inspection and for tests. | -| A reader treats the reserved local organization as a real Chainloop organization. | The reserved organization is a documented local placeholder. It is never sent to a control plane, and the export is not signed (D-010). | +| A developer's local mode turns attestation off for the whole team. | Local mode is per-clone. It is recorded in the local trace state, not in the shared repository config, so it never changes attestation for another clone (D-013). | | Timestamps and volatile paths make a test flaky. | The test normalizes the volatile fields before a compare (D-008). | From 10efd4d263c37207dddc12045629f1e5ea2796b8 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Mat=C3=ADas=20Insaurralde?= Date: Tue, 13 Oct 2026 00:59:49 -0300 Subject: [PATCH 4/6] spec: issue-3582 address review on scope, manifest, and wording MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Scope R-001 and R-005 to the export phase, so a run that wraps the agent does not fall foul of the no-network requirement. Require the index to declare the export is unsigned and not an attestation. Replace the ambiguous "the mode" in the prose with "the export". Refs #3582 Chainloop-Trace-Sessions: d139ac66-b95f-4920-ac38-12997996037d Signed-off-by: Matías Insaurralde --- .../specs/issue-3582-trace-evidence-export.md | 38 +++++++++---------- 1 file changed, 19 insertions(+), 19 deletions(-) diff --git a/docs/specs/issue-3582-trace-evidence-export.md b/docs/specs/issue-3582-trace-evidence-export.md index 1cb677904..b7cf56f32 100644 --- a/docs/specs/issue-3582-trace-evidence-export.md +++ b/docs/specs/issue-3582-trace-evidence-export.md @@ -19,7 +19,7 @@ When your code goes against this spec, say so in your PR. Do not edit the spec t # Spec issue-3582: Trace evidence export to disk ## Summary -Chainloop Trace records an AI coding session. On a git push, it assembles the session evidence and sends it to the control plane as a signed attestation. The assembly does real work. It reads the spec sources at push time, redacts secrets, and replaces copies with pointers. It stores each spec, skill, and image as its own material. Today that assembly runs only inside a push. The setup needs the control plane too. So a developer needs an account, credentials, and a network before they see any evidence. This spec adds a local mode. `trace init --local` sets the repository up with no control plane: no login, no organization, no workflow. In that mode the push hook sends no attestation, and the evidence is written to disk instead. `trace run --export` wraps the agent and writes the evidence when the session ends, and `trace export` writes the evidence of the sessions the hooks already recorded. Both run the same assembly a push runs. There is no attestation and no push. It gives developers and tests the exact evidence a push would produce, on disk, with no backend at all. +Chainloop Trace records an AI coding session. On a git push, it assembles the session evidence and sends it to the control plane as a signed attestation. The assembly does real work. It reads the spec sources at push time, redacts secrets, and replaces copies with pointers. It stores each spec, skill, and image as its own material. Today that assembly runs only inside a push. The setup needs the control plane too. So a developer needs an account, credentials, and a network before they see any evidence. This spec adds a local mode. `trace init --local` sets the repository up with no control plane: no login, no organization, no workflow. In local mode the push hook sends no attestation, and the evidence is written to disk instead. `trace run --export` wraps the agent and writes the evidence when the session ends, and `trace export` writes the evidence of the sessions the hooks already recorded. Both run the same assembly a push runs. There is no attestation and no push. It gives developers and tests the exact evidence a push would produce, on disk, with no backend at all. ## Problem - The evidence a push produces is more than the raw transcript. The push reads the spec sources, redacts secrets, and adds the pointers, the pasted images, and the skills. @@ -35,36 +35,36 @@ Chainloop Trace records an AI coding session. On a git push, it assembles the se - Goal: the written evidence equals what a push would upload for the same session. The fields, the redaction, the pointers, and the spec, skill, and image materials match. - Goal: the output is laid out so a reader and a test find each material by its digest. - Goal: a developer sets up and reads the evidence of a session with no account, no credentials, and no network. -- Non-goal: a signed attestation. The mode writes the evidence set, not a signed statement or a signed bundle. A control-plane-free signed attestation needs a contract and a signing key, and it is a separate, larger effort. +- Non-goal: a signed attestation. The export writes the evidence set, not a signed statement or a signed bundle. A control-plane-free signed attestation needs a contract and a signing key, and it is a separate, larger effort. - Non-goal: policy evaluation and contract-required materials. Those need a contract from the control plane. - Non-goal: a change to the push path, or to what a push uploads. -- Non-goal: the agent or the model offline. The agent still calls its model over the network during the session. This mode is about the Chainloop backend, not the model. +- Non-goal: the agent or the model offline. The agent still calls its model over the network during the session. This feature is about the Chainloop backend, not the model. ## Requirements ### R-001: Write the evidence to a local directory -The mode MUST run the push-time assembly and write the result to a directory the user names. It MUST make no call to the control plane and emit no attestation. -- Done when: the mode runs with no network and no credentials, and it writes the directory. +The export MUST run the push-time assembly and write the result to a directory the user names. It MUST make no call to the control plane and emit no attestation. +- Done when: the export reaches no control plane, needs no credentials, and writes the directory. ### R-002: The same evidence as a push -The written evidence record MUST equal the one a push would upload for the same session. The fields, the redaction, the pointers, and the spec, skill, and image materials MUST match. The one difference is the signature and the attestation wrapper, which the mode does not produce. +The written evidence record MUST equal the one a push would upload for the same session. The fields, the redaction, the pointers, and the spec, skill, and image materials MUST match. The one difference is the signature and the attestation wrapper, which the export does not produce. - Done when: a session that is exported and the same session that is pushed give the same evidence record and the same material digests. ### R-003: Materials addressable by digest -The mode MUST write each material under its content digest. A reader and a test MUST find each material from the reference in the evidence record. +The export MUST write each material under its content digest. A reader and a test MUST find each material from the reference in the evidence record. ### R-004: Redaction on by default -The mode MUST redact secrets in the evidence and in the materials, as a push does. An opt-out is for a trusted local run only, and the output can then hold secrets. +The export MUST redact secrets in the evidence and in the materials, as a push does. An opt-out is for a trusted local run only, and the output can then hold secrets. - Done when: a secret in a spec source does not appear in the exported files. ### R-005: No control plane and no credentials -The mode MUST NOT need a control plane, an organization, a token, or a network. +The export MUST NOT need a control plane, an organization, a token, or a network. The agent run that records the session still uses the network for its model. ### R-006: A manifest -The mode MUST write an index. The index MUST list each material by name, kind, and digest. A reader MUST see the whole set of materials from the index, with no need to parse each file. The evidence record holds the warnings of the run, not the index. +The export MUST write an index. The index MUST state that the export is unsigned and is not an attestation. It MUST list each material by name, kind, and digest. A reader MUST see the whole set of materials from the index, with no need to parse each file. The evidence record holds the warnings of the run, not the index. ### R-007: Export from a recorded session -The mode MUST build the export from the session state that the hooks recorded. It MUST NOT depend on one way of running the agent. +The export MUST come from the session state that the hooks recorded. It MUST NOT depend on one way of running the agent. ### R-008: Local mode set up with no control plane `trace init --local` MUST set up the repository for local mode without contacting the control plane: no login, no organization, no workflow. In local mode the push hook MUST send no attestation. The evidence goes to disk instead. Local mode MUST be per-clone, so it does not change whether another clone of the same repository attests. @@ -79,25 +79,25 @@ The CLI MUST provide a command that writes the evidence of the already-recorded - Done when: after a recorded session, the command writes the evidence to the directory, with no push. ## Constraints -- The repository is public. The mode and its code hold no internal context. -- The agent calls its model over the network during the session. The mode changes the Chainloop backend path only. +- The repository is public. The export and its code hold no internal context. +- The agent calls its model over the network during the session. The export changes the Chainloop backend path only. - The export is not signed, so it is not proof of anything. It is for inspection and for tests. -- The mode reuses the push assembly. It does not fork that logic. +- The export reuses the push assembly. It does not fork that logic. - Local mode is per-clone. It is recorded in the local trace state, not in the shared repository config, so it never changes attestation for another clone. - An export can hold the content of the session, including local files the session read. A developer keeps the output directory as they keep any working copy of the session. ## Proposal -The developer sets up the repository once with `trace init --local`. Init installs the same hooks but creates no workflow and contacts no control plane. Today init needs a login and creates the workflow the attestations target. Local mode drops that step, so a developer with no account can set up Trace. The mode is recorded in the local trace state, not in the shared repository config. So it is per-clone, and never turns attestation off for a teammate. In local mode the push hook sends no attestation, because there is no control plane to send it to. The evidence is written to disk on demand instead. +The developer sets up the repository once with `trace init --local`. Init installs the same hooks but creates no workflow and contacts no control plane. Today init needs a login and creates the workflow the attestations target. Local mode drops that step, so a developer with no account can set up Trace. Local mode is recorded in the local trace state, not in the shared repository config. So it is per-clone, and never turns attestation off for a teammate. In local mode the push hook sends no attestation, because there is no control plane to send it to. The evidence is written to disk on demand instead. The developer then runs the agent through `trace run --export` with an output directory. The run wraps the agent as it does today. When the session ends, instead of signing and pushing an attestation, the run executes the same assembly a push runs. It parses the session. It reads each spec source at that time. It redacts secrets. It replaces each copy in the transcript with a pointer. It gathers the pasted images and the skills. A push would store each material in the backend and send a signed attestation. Instead, the run writes each material to the output directory under its digest, writes the session evidence record, and writes a manifest. -The push path already separates the assembly from the step that stores one material. That store step is a single seam. The mode gives the seam a store that writes to disk and returns the digest, in place of the store that uploads to the backend. Every assembly step runs unchanged: the spec read, the redaction, the pointers, the images, the skills, and the evidence record. So the export cannot drift from a push. It is the same assembly with a different sink. +The push path already separates the assembly from the step that stores one material. That store step is a single seam. The export gives the seam a store that writes to disk and returns the digest, in place of the store that uploads to the backend. Every assembly step runs unchanged: the spec read, the redaction, the pointers, the images, the skills, and the evidence record. So the export cannot drift from a push. It is the same assembly with a different sink. The output directory holds three things. It holds the session evidence record, which carries the warnings of the run. It holds a folder of materials, each named by its digest. It holds a manifest that lists the materials. The layout mirrors the content-addressed store the backend uses. So a material is found by its digest, and two runs of the same session are easy to compare. -The mode writes the evidence set, not a signed attestation. A signed attestation with no control plane needs a contract and a signing key. That is a separate, larger effort, and this spec puts it out of scope. The export is for a developer who reads the evidence, and for a test that checks the assembly. +The export writes the evidence set, not a signed attestation. A signed attestation with no control plane needs a contract and a signing key. That is a separate, larger effort, and this spec puts it out of scope. The export is for a developer who reads the evidence, and for a test that checks the assembly. -The mode is not named "offline". The agent still calls its model over the network during the session, so "offline" would claim something false. Two names say what the parts do. `--local` names the init mode: no control plane, evidence on local disk. `--export` names the run output: write the evidence to a directory. A standalone `trace export` command writes the evidence of the sessions the hooks already recorded. A developer who set up local mode then exports without wrapping a run. +The export is not named "offline". The agent still calls its model over the network during the session, so "offline" would claim something false. Two names say what the parts do. `--local` names the init mode: no control plane, evidence on local disk. `--export` names the run output: write the evidence to a directory. A standalone `trace export` command writes the evidence of the sessions the hooks already recorded. A developer who set up local mode then exports without wrapping a run. ```mermaid flowchart TD @@ -148,6 +148,6 @@ None. |------|------------| | The export drifts from a real push. | The export reuses the push assembly through its store seam, so the only difference is the sink. A test compares an export with a push for the same session (R-002). | | The output holds secrets. | Redaction is on by default, as in a push. The opt-out is for a trusted local run, and it is documented as such (D-004). | -| A reader treats the export as a signed attestation. | The export is not signed, and the manifest says so. The mode is for inspection and for tests. | +| A reader treats the export as a signed attestation. | The export is not signed, and the index says so (R-006). The export is for inspection and for tests. | | A developer's local mode turns attestation off for the whole team. | Local mode is per-clone. It is recorded in the local trace state, not in the shared repository config, so it never changes attestation for another clone (D-013). | | Timestamps and volatile paths make a test flaky. | The test normalizes the volatile fields before a compare (D-008). | From 7e73d63270fff8ff58b0519eaf83907c7c31316b Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Mat=C3=ADas=20Insaurralde?= Date: Tue, 13 Oct 2026 01:11:22 -0300 Subject: [PATCH 5/6] spec: issue-3582 clarify the push hook does not write the export MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit R-008, the summary, and the proposal read as though the push wrote the evidence to disk. State that the hook only skips attestation and writes nothing, and that the separate trace export command writes the evidence on demand. Refs #3582 Chainloop-Trace-Sessions: d139ac66-b95f-4920-ac38-12997996037d Signed-off-by: Matías Insaurralde --- docs/specs/issue-3582-trace-evidence-export.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/specs/issue-3582-trace-evidence-export.md b/docs/specs/issue-3582-trace-evidence-export.md index b7cf56f32..93478caae 100644 --- a/docs/specs/issue-3582-trace-evidence-export.md +++ b/docs/specs/issue-3582-trace-evidence-export.md @@ -19,7 +19,7 @@ When your code goes against this spec, say so in your PR. Do not edit the spec t # Spec issue-3582: Trace evidence export to disk ## Summary -Chainloop Trace records an AI coding session. On a git push, it assembles the session evidence and sends it to the control plane as a signed attestation. The assembly does real work. It reads the spec sources at push time, redacts secrets, and replaces copies with pointers. It stores each spec, skill, and image as its own material. Today that assembly runs only inside a push. The setup needs the control plane too. So a developer needs an account, credentials, and a network before they see any evidence. This spec adds a local mode. `trace init --local` sets the repository up with no control plane: no login, no organization, no workflow. In local mode the push hook sends no attestation, and the evidence is written to disk instead. `trace run --export` wraps the agent and writes the evidence when the session ends, and `trace export` writes the evidence of the sessions the hooks already recorded. Both run the same assembly a push runs. There is no attestation and no push. It gives developers and tests the exact evidence a push would produce, on disk, with no backend at all. +Chainloop Trace records an AI coding session. On a git push, it assembles the session evidence and sends it to the control plane as a signed attestation. The assembly does real work. It reads the spec sources at push time, redacts secrets, and replaces copies with pointers. It stores each spec, skill, and image as its own material. Today that assembly runs only inside a push. The setup needs the control plane too. So a developer needs an account, credentials, and a network before they see any evidence. This spec adds a local mode. `trace init --local` sets the repository up with no control plane: no login, no organization, no workflow. In local mode the push hook sends no attestation and writes nothing itself. `trace run --export` wraps the agent and writes the evidence when the session ends, and `trace export` writes the evidence of the sessions the hooks already recorded. Both run the same assembly a push runs. There is no attestation and no push. It gives developers and tests the exact evidence a push would produce, on disk, with no backend at all. ## Problem - The evidence a push produces is more than the raw transcript. The push reads the spec sources, redacts secrets, and adds the pointers, the pasted images, and the skills. @@ -67,7 +67,7 @@ The export MUST write an index. The index MUST state that the export is unsigned The export MUST come from the session state that the hooks recorded. It MUST NOT depend on one way of running the agent. ### R-008: Local mode set up with no control plane -`trace init --local` MUST set up the repository for local mode without contacting the control plane: no login, no organization, no workflow. In local mode the push hook MUST send no attestation. The evidence goes to disk instead. Local mode MUST be per-clone, so it does not change whether another clone of the same repository attests. +`trace init --local` MUST set up the repository for local mode without contacting the control plane: no login, no organization, no workflow. In local mode the push hook MUST send no attestation, and it writes nothing itself. A separate command, `trace export`, writes the evidence to disk on demand (R-010). Local mode MUST be per-clone, so it does not change whether another clone of the same repository attests. - Done when: `trace init --local` runs with no network and no credentials, and a push in that clone sends no attestation. ### R-009: Export when the session ends @@ -87,7 +87,7 @@ The CLI MUST provide a command that writes the evidence of the already-recorded - An export can hold the content of the session, including local files the session read. A developer keeps the output directory as they keep any working copy of the session. ## Proposal -The developer sets up the repository once with `trace init --local`. Init installs the same hooks but creates no workflow and contacts no control plane. Today init needs a login and creates the workflow the attestations target. Local mode drops that step, so a developer with no account can set up Trace. Local mode is recorded in the local trace state, not in the shared repository config. So it is per-clone, and never turns attestation off for a teammate. In local mode the push hook sends no attestation, because there is no control plane to send it to. The evidence is written to disk on demand instead. +The developer sets up the repository once with `trace init --local`. Init installs the same hooks but creates no workflow and contacts no control plane. Today init needs a login and creates the workflow the attestations target. Local mode drops that step, so a developer with no account can set up Trace. Local mode is recorded in the local trace state, not in the shared repository config. So it is per-clone, and never turns attestation off for a teammate. In local mode the push hook sends no attestation, because there is no control plane to send it to. It writes nothing itself. The export commands write the evidence to disk on demand. The developer then runs the agent through `trace run --export` with an output directory. The run wraps the agent as it does today. When the session ends, instead of signing and pushing an attestation, the run executes the same assembly a push runs. It parses the session. It reads each spec source at that time. It redacts secrets. It replaces each copy in the transcript with a pointer. It gathers the pasted images and the skills. A push would store each material in the backend and send a signed attestation. Instead, the run writes each material to the output directory under its digest, writes the session evidence record, and writes a manifest. From 707fa39fe27af4a8e86acc2d6f12c5d28d594108 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Mat=C3=ADas=20Insaurralde?= Date: Tue, 13 Oct 2026 19:17:11 -0300 Subject: [PATCH 6/6] spec: issue-3582 reframe to local trace mode MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Refocus the spec on recording AI coding sessions without a control plane, per review. A repository stores sessions on disk, configured in .chainloop.yml with traceStorage (CONTROL_PLANE default or LOCAL), an optional tracePath, and traceRequired (replacing requireTrace). The pre-push hook stores the session locally through the same assembly a push runs. Exporting an already-recorded session on demand is deferred to a future spec. Refs #3582 Chainloop-Trace-Sessions: d139ac66-b95f-4920-ac38-12997996037d Signed-off-by: Matías Insaurralde --- .../specs/issue-3582-trace-evidence-export.md | 138 +++++++++--------- 1 file changed, 72 insertions(+), 66 deletions(-) diff --git a/docs/specs/issue-3582-trace-evidence-export.md b/docs/specs/issue-3582-trace-evidence-export.md index 93478caae..9a20fbaf1 100644 --- a/docs/specs/issue-3582-trace-evidence-export.md +++ b/docs/specs/issue-3582-trace-evidence-export.md @@ -16,100 +16,102 @@ This spec records a design that humans reviewed and approved. Do not change it a When your code goes against this spec, say so in your PR. Do not edit the spec to match the code. --> -# Spec issue-3582: Trace evidence export to disk +# Spec issue-3582: Trace local mode ## Summary -Chainloop Trace records an AI coding session. On a git push, it assembles the session evidence and sends it to the control plane as a signed attestation. The assembly does real work. It reads the spec sources at push time, redacts secrets, and replaces copies with pointers. It stores each spec, skill, and image as its own material. Today that assembly runs only inside a push. The setup needs the control plane too. So a developer needs an account, credentials, and a network before they see any evidence. This spec adds a local mode. `trace init --local` sets the repository up with no control plane: no login, no organization, no workflow. In local mode the push hook sends no attestation and writes nothing itself. `trace run --export` wraps the agent and writes the evidence when the session ends, and `trace export` writes the evidence of the sessions the hooks already recorded. Both run the same assembly a push runs. There is no attestation and no push. It gives developers and tests the exact evidence a push would produce, on disk, with no backend at all. +Chainloop Trace records an AI coding session. Today it needs a control plane. The setup creates a workflow and needs a login, and a git push attests the session to the backend. So a developer cannot use Trace without an account, credentials, and a network. This spec adds a local mode. A repository records its sessions and stores them on disk, with no control plane. The mode is set in `.chainloop.yml` with `traceStorage: LOCAL` and an optional `tracePath`. The normal flow works. A developer runs the harness, writes code, and pushes. The pre-push hook runs the same assembly a push runs and stores the session under the path, and tells the user where it went. `traceRequired` replaces `requireTrace`. This spec is about recording a session without a control plane. Exporting an already-recorded session on demand is a separate feature, left to a future spec. ## Problem -- The evidence a push produces is more than the raw transcript. The push reads the spec sources, redacts secrets, and adds the pointers, the pasted images, and the skills. -- This assembly runs only inside a push. So a developer needs a running control plane, credentials, and a network to see its output. -- The setup also needs the control plane. The init step creates the workflow and needs a login, so there is no way to use Trace without a backend at all. -- A test cannot get the pushed evidence without a live backend. So the assembly is hard to test. -- No command writes the session evidence to disk. A developer cannot see what a session would send before they send it. -- The push also signs and stores the attestation. A developer who only wants to read the evidence pays for the whole backend path. +- Trace needs a control plane. The init step creates a workflow and needs a login, so there is no way to set Trace up without a backend. +- A push attests the session to the control plane. So recording a session needs an account, credentials, and a network. +- Some teams want to record AI coding sessions with no backend at all. An evaluation, an air-gapped machine, a privacy need, or a trial before adopting the backend all want local recording. +- The evidence a push produces is more than the raw transcript. It reads the spec sources, redacts secrets, and adds the pointers, the pasted images, and the skills. A local mode has to do the same work, so the stored session matches a push. ## Goals and Non-Goals -- Goal: `trace init --local` sets up the repository for local mode and contacts no control plane. -- Goal: `trace run --export` runs the full push-time assembly and writes the result to a local directory when the session ends. It needs no control plane and does no push. -- Goal: the written evidence equals what a push would upload for the same session. The fields, the redaction, the pointers, and the spec, skill, and image materials match. -- Goal: the output is laid out so a reader and a test find each material by its digest. -- Goal: a developer sets up and reads the evidence of a session with no account, no credentials, and no network. -- Non-goal: a signed attestation. The export writes the evidence set, not a signed statement or a signed bundle. A control-plane-free signed attestation needs a contract and a signing key, and it is a separate, larger effort. +- Goal: a repository records and stores its AI coding sessions on disk, with no control plane, no login, and no workflow. +- Goal: `.chainloop.yml` sets the storage mode: the control plane (the default) or local disk, with an optional path. +- Goal: the normal flow works. The developer runs the harness and pushes, and the session is stored locally. +- Goal: the user is told where the session is stored. +- Goal: the stored evidence equals what a push would upload for the same session, apart from the signature. +- Non-goal: exporting an already-recorded session to a directory on demand. That is a separate feature for a future spec. +- Non-goal: a signed attestation. Local mode stores the evidence set, not a signed statement. A control-plane-free signed attestation needs a contract and a signing key, and it is a separate, larger effort. - Non-goal: policy evaluation and contract-required materials. Those need a contract from the control plane. -- Non-goal: a change to the push path, or to what a push uploads. - Non-goal: the agent or the model offline. The agent still calls its model over the network during the session. This feature is about the Chainloop backend, not the model. ## Requirements -### R-001: Write the evidence to a local directory -The export MUST run the push-time assembly and write the result to a directory the user names. It MUST make no call to the control plane and emit no attestation. -- Done when: the export reaches no control plane, needs no credentials, and writes the directory. +### R-001: Store the session evidence on disk +In local mode, Trace MUST run the push-time assembly and store the result on disk under the configured path. It MUST make no call to the control plane and emit no attestation. +- Done when: a push in a local-mode repository reaches no control plane, needs no credentials, and stores the session on disk. ### R-002: The same evidence as a push -The written evidence record MUST equal the one a push would upload for the same session. The fields, the redaction, the pointers, and the spec, skill, and image materials MUST match. The one difference is the signature and the attestation wrapper, which the export does not produce. -- Done when: a session that is exported and the same session that is pushed give the same evidence record and the same material digests. +The stored evidence record MUST equal the one a push would upload for the same session. The fields, the redaction, the pointers, and the spec, skill, and image materials MUST match. The one difference is the signature and the attestation wrapper, which local mode does not produce. +- Done when: a session stored locally and the same session pushed give the same evidence record and the same material digests. ### R-003: Materials addressable by digest -The export MUST write each material under its content digest. A reader and a test MUST find each material from the reference in the evidence record. +The stored session MUST write each material under its content digest. A reader and a test MUST find each material from the reference in the evidence record. ### R-004: Redaction on by default -The export MUST redact secrets in the evidence and in the materials, as a push does. An opt-out is for a trusted local run only, and the output can then hold secrets. -- Done when: a secret in a spec source does not appear in the exported files. +Local mode MUST redact secrets in the evidence and in the materials, as a push does. An opt-out is for a trusted local run only, and the output can then hold secrets. +- Done when: a secret in a spec source does not appear in the stored files. ### R-005: No control plane and no credentials -The export MUST NOT need a control plane, an organization, a token, or a network. The agent run that records the session still uses the network for its model. +Local mode MUST NOT need a control plane, an organization, a token, or a network. The agent run that records the session still uses the network for its model. ### R-006: A manifest -The export MUST write an index. The index MUST state that the export is unsigned and is not an attestation. It MUST list each material by name, kind, and digest. A reader MUST see the whole set of materials from the index, with no need to parse each file. The evidence record holds the warnings of the run, not the index. +Local mode MUST write an index next to the stored session. The index MUST state that the stored evidence is unsigned and is not an attestation. It MUST list each material by name, kind, and digest. A reader MUST see the whole set of materials from the index, with no need to parse each file. The evidence record holds the warnings of the run, not the index. -### R-007: Export from a recorded session -The export MUST come from the session state that the hooks recorded. It MUST NOT depend on one way of running the agent. +### R-007: Store from a recorded session +The stored evidence MUST come from the session state that the hooks recorded. It MUST NOT depend on one way of running the agent. ### R-008: Local mode set up with no control plane -`trace init --local` MUST set up the repository for local mode without contacting the control plane: no login, no organization, no workflow. In local mode the push hook MUST send no attestation, and it writes nothing itself. A separate command, `trace export`, writes the evidence to disk on demand (R-010). Local mode MUST be per-clone, so it does not change whether another clone of the same repository attests. -- Done when: `trace init --local` runs with no network and no credentials, and a push in that clone sends no attestation. +`trace init --local` MUST set the repository up for local mode without contacting the control plane: no login, no organization, no workflow. It MUST record the local storage mode in `.chainloop.yml` (R-011), and MAY record a path. +- Done when: `trace init --local` runs with no network and no credentials, and later pushes store the session on disk. -### R-009: Export when the session ends -`trace run --export` MUST wrap the agent and, when the session ends, write the evidence set to the named directory. It MUST NOT push an attestation. -- Done when: the agent runs, the session ends, and the directory holds the evidence with no push. +### R-009: Store the session on push +In local mode the pre-push hook MUST store the session on disk instead of pushing an attestation. +- Done when: a `git push` in a local-mode repository stores the session under the configured path and sends no attestation. -### R-010: Export a session the hooks already recorded -The CLI MUST provide a command that writes the evidence of the already-recorded sessions to a directory, without wrapping a run. A developer who set the repository up with `trace init --local` MUST be able to export the sessions the hooks recorded. -- Done when: after a recorded session, the command writes the evidence to the directory, with no push. +### ~~R-010: Export a session the hooks already recorded~~ +~~The CLI MUST provide a command that writes the evidence of the already-recorded sessions to a directory, without wrapping a run.~~ Dropped. Exporting an already-recorded session on demand moved to a future spec (D-020). + +### R-011: Configure trace storage +`.chainloop.yml` MUST carry the trace storage mode in `traceStorage`, one of `CONTROL_PLANE` (the default) or `LOCAL`. In `LOCAL`, Trace stores sessions on disk and contacts no control plane. An optional `tracePath` MUST name where local sessions are stored, with a default when it is absent. `traceRequired` MUST replace `requireTrace` with the same meaning: block a push when a required trace is missing. The old `requireTrace` and `localTrace` fields are deprecated, and a later change drops them. +- Done when: a repository with `traceStorage: LOCAL` stores its sessions on disk, and `traceRequired` blocks a push with no recorded session. + +### R-012: Say where the session is stored +In local mode, Trace MUST tell the user where the session is stored. It MUST say so when the harness starts and when a push stores the session. +- Done when: a user who runs the harness and pushes sees the path the session was written to. ## Constraints -- The repository is public. The export and its code hold no internal context. -- The agent calls its model over the network during the session. The export changes the Chainloop backend path only. -- The export is not signed, so it is not proof of anything. It is for inspection and for tests. -- The export reuses the push assembly. It does not fork that logic. -- Local mode is per-clone. It is recorded in the local trace state, not in the shared repository config, so it never changes attestation for another clone. -- An export can hold the content of the session, including local files the session read. A developer keeps the output directory as they keep any working copy of the session. +- The repository is public. The code holds no internal context. +- The agent calls its model over the network during the session. Local mode changes the Chainloop backend path only. +- The stored evidence is not signed, so it is not proof of anything. +- Local mode reuses the push assembly. It does not fork that logic. +- `traceStorage` is a committed repository policy, so a local-trace repository is the same for every clone of it. +- A stored session can hold the content of the session, including local files the session read. A developer keeps the storage directory as they keep any working copy of the session. ## Proposal -The developer sets up the repository once with `trace init --local`. Init installs the same hooks but creates no workflow and contacts no control plane. Today init needs a login and creates the workflow the attestations target. Local mode drops that step, so a developer with no account can set up Trace. Local mode is recorded in the local trace state, not in the shared repository config. So it is per-clone, and never turns attestation off for a teammate. In local mode the push hook sends no attestation, because there is no control plane to send it to. It writes nothing itself. The export commands write the evidence to disk on demand. +The developer sets the repository up once with `trace init --local`. Init installs the hooks but creates no workflow and contacts no control plane. It writes the storage mode to `.chainloop.yml`: `traceStorage: LOCAL`, with an optional `tracePath` for where the sessions go. Today init needs a login and creates the workflow the attestations target. Local mode drops that step, so a developer with no account can set Trace up. The storage mode is a committed repository policy, visible to every clone. So a local-trace repository is a deliberate, readable choice, not a hidden per-developer flag. -The developer then runs the agent through `trace run --export` with an output directory. The run wraps the agent as it does today. When the session ends, instead of signing and pushing an attestation, the run executes the same assembly a push runs. It parses the session. It reads each spec source at that time. It redacts secrets. It replaces each copy in the transcript with a pointer. It gathers the pasted images and the skills. A push would store each material in the backend and send a signed attestation. Instead, the run writes each material to the output directory under its digest, writes the session evidence record, and writes a manifest. +The developer then runs the harness as usual. When the harness starts, Trace tells the user the session is recorded and where it will be stored. The hooks record the session as they do today. On `git push`, the pre-push hook runs the same assembly a push runs. It parses the session. It reads each spec source, redacts secrets, and replaces each copy in the transcript with a pointer. It gathers the pasted images and the skills. A push would store each material in the backend and send a signed attestation. In local mode the hook instead writes each material under `tracePath`, by its digest, with the session evidence record and a manifest. It then tells the user where it stored the session. There is no attestation and no control plane. -The push path already separates the assembly from the step that stores one material. That store step is a single seam. The export gives the seam a store that writes to disk and returns the digest, in place of the store that uploads to the backend. Every assembly step runs unchanged: the spec read, the redaction, the pointers, the images, the skills, and the evidence record. So the export cannot drift from a push. It is the same assembly with a different sink. +The push path already separates the assembly from the step that stores one material. That store step is a single seam. Local mode gives the seam a store that writes to disk and returns the digest, in place of the store that uploads to the backend. Every assembly step runs unchanged. So the stored evidence cannot drift from a push. It is the same assembly with a different sink. -The output directory holds three things. It holds the session evidence record, which carries the warnings of the run. It holds a folder of materials, each named by its digest. It holds a manifest that lists the materials. The layout mirrors the content-addressed store the backend uses. So a material is found by its digest, and two runs of the same session are easy to compare. +Under `tracePath`, each session holds three things. It holds the session evidence record, which carries the warnings of the run. It holds a folder of materials, each named by its digest. It holds a manifest that lists the materials and states the evidence is unsigned and not an attestation. The layout mirrors the content-addressed store the backend uses. So a material is found by its digest, and two runs of the same session are easy to compare. -The export writes the evidence set, not a signed attestation. A signed attestation with no control plane needs a contract and a signing key. That is a separate, larger effort, and this spec puts it out of scope. The export is for a developer who reads the evidence, and for a test that checks the assembly. +Local mode stores the evidence set, not a signed attestation. A signed attestation with no control plane needs a contract and a signing key. That is a separate, larger effort, out of scope here. Exporting an already-recorded session to a directory on demand is also out of scope. It is a separate feature with its own surface, left to a future spec. This spec is about recording a session without a control plane. -The export is not named "offline". The agent still calls its model over the network during the session, so "offline" would claim something false. Two names say what the parts do. `--local` names the init mode: no control plane, evidence on local disk. `--export` names the run output: write the evidence to a directory. A standalone `trace export` command writes the evidence of the sessions the hooks already recorded. A developer who set up local mode then exports without wrapping a run. +Local mode is not named "offline". The agent still calls its model over the network during the session, so "offline" would claim something false. The name says where the evidence goes: local, on disk. ```mermaid flowchart TD - Init[trace init --local: local mode, no control plane] --> Run[trace run --export: wraps the agent] - Init --> Export[trace export: already-recorded sessions] - Run --> Session[Coding agent session, hooks record it] - Session --> End{Session ends} - End --> Assembly[Same push-time assembly: parse, read specs, redact, pointers, images, skills] - Export --> Assembly - Assembly --> Sink{Material sink} - Sink -->|push| Backend[Control plane and CAS: stored, signed attestation] - Sink -->|export| Disk[Local directory: evidence record, materials by digest, manifest] + Init[trace init --local: writes traceStorage LOCAL and tracePath, no control plane] --> Harness[Run the harness: hooks record the session] + Harness --> Push{git push: pre-push hook} + Push --> Assembly[Same push-time assembly: parse, read specs, redact, pointers, images, skills] + Assembly --> Sink{Material sink by traceStorage} + Sink -->|CONTROL_PLANE| Backend[Control plane and CAS: stored, signed attestation] + Sink -->|LOCAL| Disk[tracePath: evidence record, materials by digest, manifest] ``` ## Decision Record @@ -131,23 +133,27 @@ flowchart TD | D-013 | Where local mode is recorded | A per-clone marker in the local trace state, nothing in the shared repository config | The repository config is committed and shared. A local flag there would turn attestation off for every teammate who clones the repository and runs init. The export assembles each material with no organization or project, so none needs recording. This supersedes D-010, which put a reserved organization and project in the config. | implementation | | D-014 | What the push hook does in local mode | It sends no attestation and returns cleanly | Local mode has no control plane to attest to. The git-push-triggered export was already rejected (D-011), so the hook does nothing on push. The evidence is exported on demand with `trace export`. | implementation | | D-015 | When the standalone export command ships | Now, as `trace export`, together with local mode | A clone set up with `trace init --local` records sessions through the installed hooks, with no wrapped run to export them. Without the command there is no way to export them. This brings forward the standalone command that D-006 deferred. | implementation | +| D-016 | What the spec is about | Recording and storing a session without a control plane, configured in `.chainloop.yml`. Not exporting an already-recorded session | A reviewer noted the value is removing the control-plane dependency, not inspecting an export. This supersedes the export framing of D-005 and D-006. Rejected: keeping the "export" framing (it reads as a dev and test tool and hides the real use case). | PR review | +| D-017 | The surface that stores locally | The pre-push hook stores the session on disk when the storage mode is local | The normal flow already runs at push time, so storing there needs no new wrapper. This supersedes D-011, which stored on session end from `trace run --export`, and the standalone command of D-015. `trace run` stays a one-off wrapper for a single run. Rejected: a `trace run --export` flag as the surface (it is a single-run command, not the normal push-time path). | PR review | +| D-018 | How local mode is configured | `.chainloop.yml` carries `traceStorage` (CONTROL_PLANE default, or LOCAL), an optional `tracePath`, and `traceRequired`. The old `requireTrace` and `localTrace` are deprecated | A committed storage mode is an explicit, visible repository policy, which is what a local-trace repository wants. This supersedes D-013, whose per-clone marker avoided an accidental shared flag. An explicit `traceStorage` is intentional, not accidental, so a shared field is right. The old boolean fields are replaced, and a later change drops them. Rejected: a per-clone-only local flag (a local-trace repository is a team choice, not a per-developer one). | owner request | +| D-019 | Where local sessions are stored | Under `tracePath`, content-addressed with a manifest, with a default path when `tracePath` is absent | A reader finds each material by its digest, and the layout matches the content-addressed store the backend uses. Rejected: a path with no default (every local repository would have to set one). | owner request | +| D-020 | Exporting an already-recorded session | Out of scope. A future spec | The value here is recording without a control plane. A command that exports an existing session on demand is a separate feature with its own surface. This reverses D-015. Rejected: folding it into this spec (it widens the scope and confuses the use case). | PR review | ## Open Questions -None. +- [ ] **Encrypt the stored sessions at rest?** Proposed: no by default. The stored session holds what a working copy of the session already holds, and redaction drops secrets by default. An opt-in encryption could come later. Left for reviewers. ## Milestones -1. **Local init.** `trace init --local` sets up local mode without a control plane, marked per-clone, and the push hook sends no attestation in that mode. -2. **Disk sink and core export.** The assembly writes the evidence set of a session to a directory through the disk store, with no control plane. -3. **Export from the run.** `trace run --export` wraps the agent and writes the evidence when the session ends, with no push. -4. **Layout and manifest.** The content-addressed layout and the manifest have a documented structure, so a test reads the output without guessing. -5. **Standalone export.** `trace export` writes the evidence of the already-recorded sessions to a directory. +1. **Config and local init.** `.chainloop.yml` carries `traceStorage`, `tracePath`, and `traceRequired`, and `trace init --local` writes the local storage mode with no control plane. +2. **Store on push.** The pre-push hook stores the session on disk in local mode, through the same assembly a push runs, with no attestation. +3. **Layout and manifest.** The content-addressed layout and the manifest have a documented structure, so a reader and a test find each material. +4. **Tell the user.** The harness start and the push say where the session is stored. ## Risks | Risk | Mitigation | |------|------------| -| The export drifts from a real push. | The export reuses the push assembly through its store seam, so the only difference is the sink. A test compares an export with a push for the same session (R-002). | -| The output holds secrets. | Redaction is on by default, as in a push. The opt-out is for a trusted local run, and it is documented as such (D-004). | -| A reader treats the export as a signed attestation. | The export is not signed, and the index says so (R-006). The export is for inspection and for tests. | -| A developer's local mode turns attestation off for the whole team. | Local mode is per-clone. It is recorded in the local trace state, not in the shared repository config, so it never changes attestation for another clone (D-013). | +| The stored evidence drifts from a real push. | Local mode reuses the push assembly through its store seam, so the only difference is the sink. A test compares the stored output with a push for the same session (R-002). | +| The stored output holds secrets. | Redaction is on by default, as in a push. The opt-out is for a trusted local run, and it is documented as such (D-004). | +| A reader treats the stored evidence as a signed attestation. | The stored evidence is not signed, and the index says so (R-006). | +| A committed storage mode surprises a teammate. | `traceStorage` is explicit and visible in `.chainloop.yml`, so a local-trace repository is a deliberate, readable choice that a teammate sees on clone (D-018). | | Timestamps and volatile paths make a test flaky. | The test normalizes the volatile fields before a compare (D-008). |