Skip to content

Add /autobuilder-loop and /plan-deliverables skills#2276

Closed
sneakygriff wants to merge 2 commits into
garrytan:mainfrom
sneakygriff:skills/add-autobuilder-loop-and-plan-deliverables
Closed

Add /autobuilder-loop and /plan-deliverables skills#2276
sneakygriff wants to merge 2 commits into
garrytan:mainfrom
sneakygriff:skills/add-autobuilder-loop-and-plan-deliverables

Conversation

@sneakygriff

Copy link
Copy Markdown

Rebased onto latest main (a325940) — applies cleanly, no conflicts.

What this adds

Two skills that work as a unit, plus the supporting fixes they depend on.

/autobuilder-loop

Plan-driven build loop. Resolves an approved plan, freezes the milestone set, and drives it to completion with model-routed subagents (the orchestrator never reads source, diffs, or logs — subagents return compact envelopes). Every milestone must pass a mandatory gate before it counts as done:

  • /plan-eng-review + /review, each in a subagent that reads its own skill file and returns findings-only (fixes stay routed dispatches).
  • An adversarial cross-model forum — Codex gpt-5.6-sol at ultra effort and Grok grok-build, read-only. A [P1] from either model fails the gate. If both are unavailable it degrades to a single Claude adversarial subagent tagged [single-model] rather than silently passing.
  • Local Docker verification (a crashed app returning 500 is a FAIL, never a pass; NO_SERVER is a FAIL of "functional").

A bounded deferral valve allows batching at most one small, non-sensitive milestone's gate forward — never security/auth/migrations/public interfaces, and never the final milestone.

/plan-deliverables

Authors per-milestone acceptance criteria — each measurable and paired with the check that validates it (the deliverable) — into the design doc.

This closes a real gap in the pipeline: /autobuilder-loop re-reads per-milestone {next_milestone, acceptance_criteria} from the plan on every iteration, but nothing upstream authored that field. /office-hours writes a single prose ## Success Criteria blurb, /plan-eng-review audits proposed test coverage, and /spec's Definition of Done is issue-level (epics only).

Runs after /office-hours, before the plan reviews. Per-piece loop:

draft criteria → default-on cross-model gap-check (Codex + Grok, honoring the existing codex_reviews / grok_reviews kill-switches) → probe the user for judgment calls → pair each criterion with a repo-native validating check (or an explicit manual-verification when no automated check can honestly prove it) → write the block.

Idempotent and resume-safe: re-running reads existing progress and processes only what's missing.

How the two connect

<!-- status: pending | built-gate-pending | complete --> milestone markers. /plan-deliverables writes pending; /autobuilder-loop step 1.0 selects the first pending milestone, and step e / the deferral valve write status back. That's an explicit handoff contract instead of inferring "next" from unchecked checkboxes at both ends.

Hardening

/plan-deliverables was put through an adversarial audit (Codex gpt-5.6-sol ultra + Grok grok-build + independent verification agents) before this PR. Fixed:

  • A gap-check block that never wrote its prompt file — both external models caught it independently; it would have sent an empty prompt to both CLIs on every milestone.
  • mktemp "…-XXXXXXXX.txt" — the .txt after the X-template means zero randomization on BSD/macOS (collisions between concurrent runs).
  • BRANCH=$(git … | tr … || echo no-branch) — without pipefail the pipeline exits 0 on tr, so the fallback never fires and BRANCH ends up empty.
  • Missing idempotency/resume guards; section insert landing below the doc's closing section; prompt-injection fencing for doc-derived text; probe model availability once and run the two CLIs in parallel instead of sequentially.

Three findings were verified as false-positive / convention-consistent and deliberately not "fixed" (e.g. the markdown format is the shape autobuilder-loop's LLM parse subagent was co-designed for).

Validation

  • Rebased onto a325940; bun run gen:skill-docs --host all → regenerates byte-identical output against current main's generator.
  • --dry-run freshness gate → 0 stale.
  • bun run skill:check → all hosts fresh; gstack-plan-deliverables OK across hosts.
  • ⚠️ skill:check reports ❌ claude/SKILL.md — generated file missing. This pre-exists on main (upstream claude/ contains only SKILL.md.tmpl) and is unrelated to this branch.

Notes for review

  • Commit 1 also carries the gen-skill-docs and test-suite fixes that /autobuilder-loop depends on to generate and validate.
  • One conflict resolved in your favor: test/user-slug-fallback.test.ts overlapped with the recent test-infra work. This branch had tightened the assertion to user_slug_at_local; I kept main's more permissive user_slug_at_(local|[a-f0-9]{8}|[a-f0-9]{16}) rather than change an unrelated test.
  • Happy to split this into two PRs, trim scope, or take the skills separately — whatever's easiest to review.

🤖 Generated with Claude Code

sneakygriff and others added 2 commits July 15, 2026 15:55
Plan-driven build loop: resolves an approved plan, freezes the milestone set, and
drives it to completion with model-routed subagents. Every milestone must pass a
mandatory gate — /plan-eng-review + /review + an adversarial cross-model forum
(Codex gpt-5.6-sol at ultra effort, Grok grok-build) — plus local Docker
verification before it counts as done. A [P1] from either external model fails the
gate; if both are unavailable the gate degrades to a single Claude adversarial
subagent rather than silently passing.

Includes the gen-skill-docs and test-suite fixes the skill depends on to generate
and validate.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Authors per-milestone acceptance criteria — each measurable and paired with the
check that validates it (the deliverable) — into the design doc, in a format
/autobuilder-loop parses. Closes a real gap: autobuilder-loop re-reads
per-milestone {next_milestone, acceptance_criteria} from the plan each iteration,
but nothing upstream authored that field.

Runs after /office-hours, before the plan reviews. Per-piece loop: draft criteria
-> default-on cross-model gap-check (Codex gpt-5.6-sol ultra + Grok grok-build,
honoring the existing codex_reviews/grok_reviews kill-switches) -> probe the user
for judgment calls -> pair each criterion with a repo-native validating check (or
an explicit manual-verification) -> write the block. Idempotent and resume-safe.

Wires the two skills together via <!-- status: pending|built-gate-pending|complete -->
milestone markers: autobuilder-loop step 1.0 selects the first pending milestone;
step e and the deferral valve write status back.

Registered in the router, coverage matrix, office-hours/plan-eng-review chaining,
README and CLAUDE.md.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@trunk-io

trunk-io Bot commented Jul 15, 2026

Copy link
Copy Markdown

Merging to main in this repository is managed by Trunk.

  • To merge this pull request, check the box to the left or comment /trunk merge below.

After your PR is submitted to the merge queue, this comment will be automatically updated with its status. If the PR fails, failure details will also be posted here

@time-attack

Copy link
Copy Markdown
Contributor

@sneakygriff can you send some examples don't with this skill

@sneakygriff

Copy link
Copy Markdown
Author

/plan-deliverables
DON'T run it on a vague idea. It operationalizes an existing plan — it doesn't create one. With no design doc it stops and points you at /office-hours rather than inventing milestones from an empty read. That's deliberate: invented milestones produce invented criteria.

DON'T write criteria you can't fail. "Works correctly", "handles edge cases", "tests pass", "no regressions" are banned outright. If you can't state the observable outcome ("orders older than 30 days return HTTP 410 for all 4 roles"), you don't have a criterion — you have a wish.

DON'T delete a requirement because no test can prove it. This was the sharpest audit catch: the original "one criterion, one automated test — else drop or re-scope" rule would silently cut legitimate security, UX, and visual requirements to satisfy its own rule. Pair them with manual-verification / reviewed-by: instead. Dropping a requirement is not the same as validating it.

DON'T invent test paths. Naming test/foo.py::test_bar in a Jest or XCTest repo produces a fabricated deliverable nobody can build. Grep a sibling test and match the repo's own convention.

DON'T batch all milestones in one pass. The per-piece loop (draft → gap-check → probe → pair → write) is what keeps criteria specific. Batching regresses to generic filler.

DON'T pad to a number. "2–10" was reworded precisely because it invited padding. One criterion is correct when one genuinely covers the piece.

DON'T feed confidential plans to external models unaware. The gap-check ships criteria and requirement excerpts to Codex and Grok on every piece. That's on by default — kill it with gstack-config set codex_reviews off / grok_reviews off.

/autobuilder-loop
DON'T read the codebase "real quick" yourself. Its own rationalization table names this first: raw file contents in the orchestrator poison every downstream routing and gate decision, and force early compaction. Dispatch a scoped-read subagent.

DON'T make the trivial two-line edit yourself. The exception has no boundary — take it once and you're the worker, and the edit skips both routing and the review path.

DON'T batch the eng-review to the end. "Tests are green, I'll gate once at the end" is exactly how defects compound across milestones. The deferral valve allows one small, non-sensitive milestone forward — never the final one, and it's marked "built — gate pending", not complete.

DON'T treat green unit tests as done. Unit tests don't prove the composed system runs. NO_SERVER is a fail, and so is a 4xx/5xx — a crashed app returning 500 isn't "up".

DON'T work the next milestone from memory. It re-parses from disk every iteration specifically to catch in-place edits to criteria. "I've internalized the plan" is how scope silently drifts.

DON'T let it run business decisions. It suggests /plan-ceo-review, /design-review — it never auto-invokes them, and never auto-invokes /autoplan. Those are yours.

DON'T hand-flip a marker to complete. plan-deliverables writes pending; autobuilder-loop advances it only after the gate runs clean. Editing it by hand marks a milestone done while skipping the gate that defines "done" — you'd be lying to the loop with its own contract.

DON'T point autobuilder-loop at a plan with no acceptance criteria. It reads acceptance_criteria; it never authors them. Without them it has no definition of done to gate against — which is the exact gap plan-deliverables exists to close.

DON'T run plan-deliverables after the build starts. Criteria written post-hoc get shaped to the code that already exists. The whole value is criteria before code.

@time-attack

Copy link
Copy Markdown
Contributor

@16francej — recommend closing this PR. Thanks for putting this together. We aren’t accepting new standalone skills right now, so we’re going to pass on this proposal. We can revisit the area later if it becomes part of GStack’s core workflow.

@sneakygriff
sneakygriff deleted the skills/add-autobuilder-loop-and-plan-deliverables branch July 17, 2026 13:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants