feat(enhance): the degenerate exits get their own diagnostic channel — exit_kind + the incomplete_summary histogram (#615) - #636
Open
gadievron wants to merge 1 commit into
Conversation
…— exit_kind + the incomplete_summary histogram (#615) The four degenerate exits are conflated under one classification 'incomplete', but they differ in kind and remedy: three cheap model-behavior exits (end-turn-without-finish, finish truncated at max_tokens, no-tool-calls — confidence 0.3, remedied by retry/prompt work) vs ONE budget-exhaustion exit (MAX_ITERATIONS — confidence 0.2, remedied by raising the budget). A consumer reading only classification cannot split the 4-vs-41. NO fourth classification (that would break every =='incomplete' consumer): an exit_kind field on AgentResult (present-only in to_dict; empty = a completed analysis; the four sites stamp their kinds), and the incomplete_summary kind histogram beside the existing error_summary — EnhanceResult carries it, and BOTH step-report writers (the scanner and the standalone CLI) thread it, so the split is visible without stderr mining. Legacy unstamped rows fall back to 'unstamped' in the aggregate; the per-record to_dict stays present- only. The classification and its consumers are UNCHANGED (pinned). serves: I-enh, I-out. de-balances-checked: every ==incomplete consumer keeps its semantics (the classification-unchanged test); the completed analysis serializes byte-identically (present-only). Refs #615
gadievron
requested review from
dgeyshis,
shahar-davidson and
sounil
as code owners
September 14, 2026 22:56
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #615.
What was wrong
The agentic enhance loop's four degenerate exits are conflated under one
classification: "incomplete", but they differ in kind and remedy: three cheap model-behavior exits (end-turn-without-finish, finish truncated at max_tokens, no-tool-calls — confidence 0.3, remedied by retry/prompt work) vs one budget-exhaustion exit (MAX_ITERATIONS — confidence 0.2, remedied by raising the budget, not retrying). In the referenced run the split was 41 "Analysis incomplete" + 4 "Could not complete analysis within iteration limit" — a 4-unit class a consumer reading onlyclassificationcannot distinguish from the 41.The fix
No fourth classification (that would break every
== "incomplete"consumer —checkpoint.py,enhancer.py, the degenerate-exit tests). Instead:exit_kindonAgentResult— the four exit sites stamp their kinds (end_turn_without_finish/finish_truncated/no_tool_calls/max_iterations); a completed analysis carries the empty string and serializes without the key (present-only — the completed-record byte-identity).incomplete_summaryhistogram beside the existingerror_summary: the enhancer's counting loop aggregatesexit_kind(legacy unstamped rows fall back to"unstamped");EnhanceResultcarries the histogram (present-only into_dict); and both step-report writers — the scanner's pipeline summary and the standalone CLI's enhance summary — thread it, so the 4-vs-41 split is visible in the artifacts without stderr mining.Evidence
AgentResulthad no field; the four exits stamped nothing; no histogram anywhere.EnhanceResultcarry + the present-only negative, both summary writers' pins, and two real-exit receipts driving the actual agent loop through the existing degenerate-exit harness (the end-turn-without-finish and the R2-B finish-truncated stamps on realAgentResults, not hand-constructed ones); the full suite 4171 passed + 33 skipped; ruff clean; Semgrep 0.#635merge shape named (four adjacent-insert hunks; the post-merge invariantsum(incomplete_summary.values()) == incomplete_countrecorded for the reconciler).Notes for the reviewer
EnhanceResultedits will meet feat(analyze): the enhancement non-verdicts are visible, out of the prompt, counted, and migrated on re-enhancement (#611) #635'sincomplete_count/total_unitsedits on merge — keep both; the post-merge invariant to pin: the histogram's sum equalsincomplete_count.context_enhancer's internal_summary_incompletecount and the resume path are kind-blind by design (they count the classification, not the kind) — out of this issue's stated scope ("alongside error_summary").Related: #611 (the sentinel-handling round that surfaced this), #609 (the same stage's metadata-loss fix).