feat(xdev): the loop can write and verify — run_tests/write_file as real xdev turns - #33
Merged
Merged
Conversation
…eal xdev turns
Asked to "fix the off-by-one in parseConfig and run the tests", the loop
rotated four tools and changed nothing: `write_file` returned
`written: false` with a message saying `stub`, and `run_tests` returned
`passed: 0, output: ""`. Two of the four v1 tools were fiction, and the
approval gate in front of them was theatre.
## internal/xdev — the executor client
Speaks xdev's `rpc` JSONL protocol: `{"type":"ready"}` first (validated
against protocol version 1, so a future v2 fails loudly instead of
mis-parsing), then id-tagged prompts answered by a stream of events and one
terminal response.
Four properties are load-bearing, and each has a test against a real child
process rather than a mock:
- **the ready frame is a hard gate** — a child that dies without one
fails Start instead of blocking a run forever;
- **events are read until the response** — a client that waits for the
response without draining events deadlocks behind a full pipe, which
xdev's own Serve comment names;
- **one turn at a time** — xdev refuses a second concurrent prompt, so
serialising here turns a server-side error into a client-side wait;
- **the step's context kills the child** — a turn that outlives its
budget must not leave a half-read stream and a sandbox still mutating
a workspace.
## The tools
`run_tests` and `write_file` each run as one xdev turn. That is §4.1's
contract in code: agentloop owns the loop and the bound, xdev owns the turn
— the file mutation, the test run, the model call inside it.
Three outcomes are made distinguishable, because "no sandbox" must never
read as "the tests passed":
| Situation | Result |
|---|---|
| no executor configured | Success, `ran`/`written` false, message says so |
| the turn failed | Success=false with the child's reason — an observation the loop keeps |
| the turn ran | Success, `ran`/`written` true, the turn's text in `output` |
`write_file` with no path fails closed and says why.
## The seam that was missing on the loop side
`nextToolDefault` passed only `{step, goal}`, so every write was rejected
for a missing path before the sandbox saw it. It now gives each tool the
arguments that make it a real call — the readers get the goal as query text,
`run_tests` gets the workspace, `write_file` gets a target. A real planner
should name the file; this exists so the write path is exercisable end to
end until it does, and the comment says so.
## Wiring
AGENTLOOP_XDEV_BIN default "xdev"
AGENTLOOP_XDEV_DIR workspace turns run in (default: a fresh temp dir,
because a sandbox pointed at the server's cwd could
edit agentloop itself)
AGENTLOOP_XDEV_OFF disable
One child is shared across runs on purpose (a per-run child would pay
startup per step and lose the session between them) and it is still one turn
at a time. A child that will not start is not fatal: the registry is built
without a sandbox and the two tools report no executor — the same shape as
LeanKG being down.
The M6 eval factory keeps **no** sandbox and no knowledge client: the deploy
gate stays deterministic and offline, and still passes 4/4.
## Verified live
Against a sandbox speaking the protocol:
step 2 run_tests -> ran: true, output: "ran `go test ./...`: 12 passed, 0 failed"
step 3 write_file -> written: true, path: agentloop-<run>-step-3.txt
With `AGENTLOOP_XDEV_OFF=1` the same run reports `ran: false` / `written:
false` and names the missing executor. Against the real xdev binary the
client reaches the ready frame and Start then fails on that machine's
malformed models.yml — which is the correct shape: Start fails, and the
server logs a warning and runs without a sandbox rather than dying.
## Checks
`make check` green: gofmt, vet, 14/14 packages (7 new tests in
internal/xdev, 3 new in internal/tools), lint 0 issues, PRD OK, selftest
12/12.
Docs: PRD §4 marks three tools built and one stub; §13 records this; USAGE's
config table gains the three variables and §9 splits what reads/writes today
from what still does not.
linhdmn
added a commit
that referenced
this pull request
Sep 21, 2026
…inters (#38) The index said "the three remaining stub tools, the model-driven planner and tier-combo names are the next milestone". Three of those four are now landed (#33 execution, #34 the reasoner, #35 tier routing), so the note was stale and the real next step was unnamed. Rewritten as a handoff, with each item verified against the code rather than recalled: 1. **The loop cannot read a file** — four tools, none of which returns file *content*. `query` returns graph elements with a 400-char excerpt and a rung; it is not a reader. So the reasoner locates parseConfig, learns it is in config.go:41, and cannot look at it — which is exactly why the live demo wrote `func parseConfig() {}`. `write_file`'s description already promises a "read twin" that does not exist. 2. **The run has no answer** — `goal_met` stores its synthesis in `PartialSynthesis`, a field named for the bound case, and the model's rationale sits in a step's `why`. 3. **Success criteria are prose** — `ps.Success` appears exactly once in the codebase: as prompt text in reason.go. Nothing evaluates it. 4. `web_search` is the last stub (or should be deleted). 5. The two gaps #37 recorded: reply-out unscreened, ~740ms/call unmetered. 6. Planning is still a rule table (no longer blocking — the chooser carries). Plus the working notes a new session loses time rediscovering: the worktree rule, `make check` as the gate, the shell quirks (nohup for backgrounded servers, pkill matching its own command line, ports 8080/9699 taken), and the one that matters most — every defect this repo shipped lately was a check that could not fail (#25, #29, #32, #37, and #8's checklist ticked on the author's behalf). Verify by breaking the thing, not by watching it pass.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The question this answers
"What can this actually do in my repo?" I asked it to "fix the off-by-one in parseConfig and run the tests" and it rotated four tools and changed nothing —
write_filereturnedwritten: false,run_testsreturnedpassed: 0, output: "". Two of the four v1 tools were fiction, and the approval gate in front of them was theatre.Now it writes and it verifies.
internal/xdev— the executor clientSpeaks xdev's
rpcJSONL protocol: areadyframe first (validated against protocol version 1, so a future v2 fails loudly instead of mis-parsing), then id-tagged prompts answered by a stream of events and one terminal response.Four properties are load-bearing, and each has a test against a real child process rather than a mock:
Startinstead of blocking a run foreverServecomment names thisThe tools
run_testsandwrite_fileeach run as one xdev turn — which is PRD §4.1's contract in code: agentloop owns the loop and the bound, xdev owns the turn.Three outcomes are made distinguishable, because "no sandbox" must never read as "the tests passed":
ran/writtenfalse, message says soSuccess=falsewith the child's reason — an observation the loop keepsran/writtentrue, the turn's text inoutputwrite_filewith no path fails closed and says why.The seam that was missing on the loop side
nextToolDefaultpassed only{step, goal}, so every write was rejected for a missing path before the sandbox ever saw it. It now gives each tool the arguments that make it a real call: the readers get the goal as query text,run_testsgets the workspace,write_filegets a target path. A real planner should name that file — this exists so the write path is exercisable end to end until it does, and the comment says so.Wiring
One child is shared across runs on purpose — a per-run child would pay startup on every step and lose the session between them — and it is still one turn at a time. A child that will not start is not fatal: the registry is built without a sandbox and the two tools report no executor. Same shape as LeanKG being down.
The M6 eval factory keeps no sandbox and no knowledge client: the deploy gate stays deterministic and offline, and still passes 4/4.
Verified live
Against a sandbox speaking the protocol:
With
AGENTLOOP_XDEV_OFF=1the same run reportsran: false/written: falseand names the missing executor. Against the real xdev binary the client reaches the ready frame andStartthen fails on this machine's malformed~/.xdev/agent/models.yml("cannot unmarshal !!seq into string") — which is the correct shape:Startfails, the server logs a warning, and the run continues without a sandbox rather than dying.What this does not yet make it
It still cannot choose its own steps: the planner is deterministic and
nextToolDefaultrotates. What changed is that the rotation now does something real — it writes and it runs tests. A model-driven planner is the next piece, and until it lands the loop is a working write-test harness with a bound and a gate, not an agent that finds the bug itself.Checks
make checkgreen: gofmt clean, vet clean, 14/14 packages (7 new tests ininternal/xdev, 3 ininternal/tools), lint 0 issues, PRD OK, selftest 12/12.