Skip to content

feat(runtime): execute and grade immutable agent candidates#507

Merged
drewstone merged 6 commits into
mainfrom
feat/candidate-execution-plan
Jul 11, 2026
Merged

feat(runtime): execute and grade immutable agent candidates#507
drewstone merged 6 commits into
mainfrom
feat/candidate-execution-plan

Conversation

@drewstone

Copy link
Copy Markdown
Contributor

Summary

  • add one atomic path from a verified candidate bundle to a durable V2 run receipt
  • execute detached task/profile/code bytes behind a one-shot cross-process claim with crash recovery
  • bind the exact verified patch, task tree, executable grader bytes, raw grader output, protected trace, model ledger, and isolated memory evidence
  • enforce separate frozen execution, cleanup, and result budgets with cancellation and lease fencing
  • reject secret-bearing archives/manifests before persistence and keep recovery trace access read-only
  • consume the published @tangle-network/agent-interface 0.22.0 receipt contract and profile materializer 0.3.0

Verification

  • pnpm lint — 426 files clean
  • pnpm run typecheck — runtime and examples clean
  • pnpm test — 146 files, 1,420 passed, 2 skipped
  • pnpm build — JavaScript and declarations built
  • pnpm verify:package — public exports load
  • pnpm docs:freshness — version, peer, symbol, and generated-catalog checks clean
  • hostile re-audit — 4/4 reproduced timing/persistence failures fixed; 0 late result writes

No benchmark model calls were made in this change.

drewstone added 4 commits July 9, 2026 18:56
Extend the existing TraceStore, durable content addressing, candidate interface, and shared profile materializer into one verify-prepare-finalize execution path.
@drewstone

Copy link
Copy Markdown
Contributor Author

@tangletools review the current head for merge.

tangletools
tangletools previously approved these changes Jul 11, 2026

@tangletools tangletools left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Auto-approved drewstone PR — eedc44c8

This PR was opened by the trusted drewstone account.
The full PR reviewer audit still runs separately and will publish findings if it detects issues.

tangletools · auto-approval · reason: drewstone_author · 2026-07-11T06:00:22Z

@tangletools tangletools left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Value Audit — sound

Verdict sound
Concerns 0 (none)
Heuristic 0.0s
Duplication 0.0s
Interrogation 291.5s (2 bridge agents)
Total 291.5s

💰 Value — sound

Adds a durable, cross-process, exactly-once pipeline that turns a verified immutable candidate bundle into a tamper-evident V2 run receipt — a genuinely new layer that consumes the published substrate contracts rather than reinventing the in-process kernels, and it is in the repo's grain.

  • What it does: A new src/candidate-execution/ subsystem (~27 files) with one atomic path: verifyAgentCandidateBundle (verify.ts:37, consumes agentCandidateBundleSchema from @tangle-network/agent-interface) → prepareAgentCandidateExecution (prepare.ts:95; materializes task/candidate/profile workspaces, delegates profile→harness to @tangle-network/agent-profile-materialize at prepare.ts:167-174, reserv
  • Goals it achieves: Exactly-once, crash-recoverable execution of sealed benchmark candidates across processes/machines without double-grading, secret leakage, or unproven/late result writes (separate frozen execution/cleanup/result budgets, lease fencing, two-phase terminal commit, zero-model-call retry eligibility — claim.ts:76-91,215-222). It binds the full evidence chain (patch, task tree, executable grader bytes,
  • Assessment: Good on its merits and in-grain. It is a real capability gap the existing kernels do not fill, and it follows the repo's hard rules: domain-clean via injected adapters (CLAUDE.md layering), profile realization delegated to substrate (materializeCandidateProfile/applyAgentCandidateWorkspacePlan, prepare.ts:25-28,167-174 — honoring the §1.5 'never write a profile→harness realizer' law), neutral
  • Better / existing approach: None — this is the right approach. I checked the obvious existing equivalents and none cover durable cross-process candidate-bundle execution + receipts: src/runtime/supervise SpawnJournal/budget.ts:80 and driver-executor.ts are an in-process, single-tree content-addressed event log for equal-k replay/resume of the recursive atom (no cross-process claim/lease/terminal-CAS, no protected gra
  • Model: opencode/kimi-for-coding/k2p7
  • Bridge attempts: 1

🎯 Usefulness — sound

Adds a coherent verify→prepare→claim→execute→finalize spine that turns a sealed candidate bundle into a tamper-evident V2 run receipt, consuming real published contracts with no existing equivalent in the codebase.

  • Integration: Fully reachable: exported via export * from './candidate-execution' at src/index.ts:35, and its symbols are already in the generated API catalog (docs/api/primitive-catalog.md:110-111,142) with the docs-freshness gate green. It consumes the published @tangle-network/agent-interface 0.22.0 contract — I confirmed `AgentCandidateRunReceiptV2 extends Omit<AgentCandidateRunReceiptV1, 'schemaVersion'>
  • Fit with existing patterns: Fits the codebase grain precisely. Respects the layering law (agent-runtime → agent-eval, never reverse): imports TraceStore/BenchmarkEvaluation/isLlmSpan/REDACTION_VERSION from agent-eval and contract types from agent-interface, never imports agent-knowledge. Does NOT compete with the improvement/ module's CandidateGenerator/improvementDriver (src/improvement/improvement-driver.ts:36) — that
  • Real-world viability: Rigorously handles the non-happy paths. Tests cover exactly-one-concurrent-invocation (tests/candidate-execution-execute.test.ts:710), hanging process stop leaving the claim for recovery (:993), unknown settlement blocking zero-spend guessing (:1061), zero-call retry only for pre-model failures (:1126), lease expiry, cross-process claim races, and replay terminal writes — 1,420 tests pass. The exe
  • Model: opencode/zai-coding-plan/glm-5.2
  • Bridge attempts: 1

No concerns — sound change, no better or existing approach found. ✅


What this audit checks

It judges the change on its merits — not whether it was tasked out in an issue. Unticketed, fast-moving work is fine; the question is whether the change is good and whether a better or existing approach should be used instead.

Pass What it asks
Heuristic Vague title? Whitespace-only or cruft-bearing diff? (content signals only)
Duplication Do added function/class names already exist elsewhere in the repo?
Value Audit What does it do? What goal does it achieve? Is it good? Better architecture or already-exists?
Usefulness Audit Does it integrate and fit? Will it hold up in real use and actually get used?

Findings are concerns, not blocks — the human reviewer decides what to do with them.

value-audit · 20260711T060700Z

tangletools
tangletools previously approved these changes Jul 11, 2026

@tangletools tangletools left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Auto-approved drewstone PR — a4fec22e

This PR was opened by the trusted drewstone account.
The full PR reviewer audit still runs separately and will publish findings if it detects issues.

tangletools · auto-approval · reason: drewstone_author · 2026-07-11T06:15:17Z

@tangletools tangletools left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Auto-approved drewstone PR — fa6db037

This PR was opened by the trusted drewstone account.
The full PR reviewer audit still runs separately and will publish findings if it detects issues.

tangletools · auto-approval · reason: drewstone_author · 2026-07-11T06:19:52Z

@drewstone drewstone merged commit 164da02 into main Jul 11, 2026
2 checks passed
@drewstone drewstone mentioned this pull request Jul 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants