You left a big uncommitted diff and a test now fails. Somewhere in those 400 lines are the few that broke it.
git bisect can't help — it needs commits, and it can't see inside one, let alone into a
dirty working tree. diffbisect works below commit granularity: it delta-debugs the diff
between a base rev and your working tree, splits it into change regions, and tells you
which hunks to revert.
npx @precisionutilityguild/diffbisect -- npm test
That's it. No config, no daemon, no API key, no LLM, no Python. diffbisect only reads the
exit code of the command after --, so vitest, jest, node --test, pytest, and cargo test
all work unmodified — nothing about any runner is special-cased, and the demo below drives a
plain node script. (mocha is the one exception: it exits with the failure count and needs a
one-line wrapper — see the probe exit-code convention.)
Deterministic given a deterministic test command.
The input is your uncommitted diff — staged, unstaged, and untracked — not a commit range. Your tree and index are never touched.
A region is a contiguous run of changed lines — the unit diffbisect turns on and off, and what the output calls a hunk. Renames, mode changes, binary files, and untracked additions are each one atomic unit instead.
Every terminal block in this README is real captured output. The demo is a git repo with a
10-function module whose working tree carries nine edits: seven benign refactors and one
seeded two-region conjunctive breakage. primary() and backup() were each flipped to
false, and the invariant route() === "ok" only fails when both are present — either one
alone still resolves via the other path. That's the hard case, interference rather than a lone
bad line, and the default mode isolates it exactly.
$ diffbisect -- node check.mjs
diffbisect · "node check.mjs" · base HEAD
9 regions · 26 probes (59 logged incl. memoized)
regions
H1 math.js:1-4 [region]
H2 math.js:5-9 [region]
H3 math.js:10-14 [region]
H4 math.js:15-19 [region]
H5 math.js:20-24 [region]
H6 math.js:25-29 [region]
H7 math.js:30-34 [region]
H8 math.js:35-39 [region]
H9 math.js:40-45 [region]
culprits (revert these): H5 H8
H5 math.js:20-24
diff --git a/math.js b/math.js
index 88d9f97..cb59967 100644
--- a/math.js
+++ b/math.js
@@ -20,5 +20,5 @@
export function primary() {
- return true;
+ return false;
}
H8 math.js:35-39
diff --git a/math.js b/math.js
index 88d9f97..cb59967 100644
--- a/math.js
+++ b/math.js
@@ -35,5 +35,5 @@
export function backup() {
- return true;
+ return false;
}
causes
#1 H5 H8 (1-minimal witness)
complete: reverting the culprits from the full diff passes (confirmed by a passing probe).
complete: ... is not a claim — it is backed by an actual passing probe: diffbisect ran the
command on a worktree with H5 and H8 reverted and observed exit 0. Perform that revert
yourself and the tree passes:
$ # revert exactly H5 and H8 (primary/backup back to true), keep the seven benign edits
$ node check.mjs; echo "exit=$?"
all ok
exit=0
culprits= the revert set. Remove exactly these hunks from the full diff and your tree passes — and whencompleteistrue, that is not inferred, it was confirmed by an actual passing run. Revert these, rerun, done.causes= the independent breakages, each with awitnessMinimalflag. A 400-line diff can carry more than one unrelated regression; each is reported separately, andwitnessMinimalsays whether diffbisect proved that cause 1-minimal (guaranteed under--minimal; a fast pass may leave it unproven).cluster= residue diffbisect could not carve. When hunks interfere in a way the search cannot separate within the budget, they land here instead of being force-fit into a false "minimal" answer. Treat the cluster as your suspect list: revert it as a block, or re-run with a larger--max-probesor--minimalto push further.
| Flag | Default | Meaning |
|---|---|---|
--json |
off | Emit the machine-readable contract. |
--base <rev> |
HEAD |
Base rev to diff the working tree against. |
--target <rev> |
— | Bisect the committed range <base>..<rev> instead of the working tree. |
--max-probes <n> |
500 |
Probe budget. Exhaustion is a clean exit 3, not an error. |
--confirm <n> |
1 |
Confirmation reruns for the first FAIL and the revert-confirming probes (0 disables). A false culprit is the worst output this tool can produce. |
--minimal |
off | Classic delta debugging end-to-end: a 1-minimal witness per cause, guaranteed but slower. |
--fast |
off | A single fast pass only; may return a residue cluster (explicitly labelled). |
--jobs <n> |
1 |
Speculative parallel probes within a round. Wall-clock only: the consumed probe sequence, log, and budget stay byte-identical to --jobs 1. |
--keep-worktrees |
off | Leave probe worktrees on disk (debug). |
-- <cmd> |
required | Everything after -- is the test command; its exit code is the oracle. |
This is the trap that catches everyone. Each probe runs in a fresh git worktree. Worktrees
share the object database — your committed and diffed content — but not untracked files:
node_modules/, build output, and anything else .gitignored is simply absent. A bare
npm test in that empty worktree fails to even start, so the base-precondition probe FAILs
and diffbisect exits 2 with the base tree does not PASS, before any bisection begins.
The fix is the same wrapper git bisect run users already write: symlink the untracked
artifacts into each probe's worktree before the tests run.
diffbisect -- sh -c '[ -e node_modules ] || ln -sfn /path/to/repo/node_modules node_modules; exec npm test'
/path/to/repo is your real checkout (where node_modules lives); the [ -e ... ] || guard
makes the wrapper idempotent across probes. The same applies to any generated artifact the
tests need — a dist/, a build cache, a compiled binary: symlink each one the same way.
Verified against a repo whose test depends on an ignored vendor/ dir: bare, it exits 2
(the base tree does not PASS); with the symlink wrapper, it isolates the culprit and exits 0.
The command after -- is the oracle, using git bisect run's convention so existing bisect
scripts work unchanged:
| Command exit | Meaning to diffbisect |
|---|---|
0 |
PASS |
1–127 (except 125) |
FAIL |
125 |
UNRESOLVED (skip / untestable) |
≥128 |
signal death — abort the whole run |
A failed git apply (a subset that will not compose) is a free UNRESOLVED — zero test
invocations, detected before the command ever runs.
Caution — runners that exit with the failure count. Some runners (mocha, for one) exit
with the number of failing tests rather than a flat 1. With 128 or more failures that
crosses the ≥128 threshold and diffbisect reads it as a signal death and aborts. Wrap such a
runner to normalise the code to a flat 1 on any failure:
diffbisect -- sh -c 'npx mocha; [ $? -eq 0 ]'
([ $? -eq 0 ] re-exits 0 on success and 1 on any nonzero count.) vitest, jest,
node --test, pytest, and cargo test already exit 1 on failure and need no wrapper.
Distinct from the probe convention above — branch on these, not on output text:
| Code | Meaning |
|---|---|
0 |
Culprit set isolated, cluster empty. |
1 |
Runtime/internal failure: not a git repo, worktree setup died, or the command was killed by a signal. |
2 |
Usage/precondition error: bad flags, no command, or a precondition failed — the base tree does not PASS, or the full diff does not FAIL. The message names the measured outcome and exit code. |
3 |
Residue: the probe budget was exhausted or an interdependency cluster remains. The partial result is still printed. |
Exit 2 fires when there is nothing to bisect. If you have already reverted the culprits, the full diff now passes and diffbisect refuses — which is also the agent's confirmation that the fix landed:
$ # after reverting H5 and H8, the full diff no longer fails
$ diffbisect -- node check.mjs; echo "exit=$?"
diffbisect: precondition failed — the full diff does not FAIL (outcome PASS, exit 0).
There is no regression in the diff to isolate.
exit=2
Exit 3 fires on residue — including a budget you set too low. Forcing a tiny budget on the same clean run shows the honest degraded result: everything unresolved, labelled as such, never dressed up as an answer.
$ diffbisect --max-probes 5 -- node check.mjs; echo "exit=$?"
diffbisect · "node check.mjs" · base HEAD
9 regions · 5 probes · budget exhausted
regions
H1 math.js:1-4 [region]
H2 math.js:5-9 [region]
H3 math.js:10-14 [region]
H4 math.js:15-19 [region]
H5 math.js:20-24 [region]
H6 math.js:25-29 [region]
H7 math.js:30-34 [region]
H8 math.js:35-39 [region]
H9 math.js:40-45 [region]
undetermined interdependency cluster: H1 H2 H3 H4 H5 H6 H7 H8 H9
9 hunks the search could not carve into minimal witnesses.
incomplete: the probe budget was exhausted before the revert set was confirmed.
exit=3
Pass --json when an agent or another program consumes the result. Same run as above:
$ diffbisect --json -- node check.mjs
{
"version": 1,
"tool": "diffbisect",
"command": ["node", "check.mjs"],
"base": "HEAD",
"hunks": [
{
"id": "H1",
"file": "math.js",
"lines": "1-4",
"kind": "region",
"patch": "diff --git a/math.js b/math.js\nindex 88d9f97..cb59967 100644\n--- a/math.js\n+++ b/math.js\n@@ -1,4 +1,4 @@\n export function add(a, b) {\n-\treturn a + b;\n+\treturn b + a;\n }\n "
},
{ "...": "H2 through H9, each with id, file, lines, kind, patch" }
],
"probes": [
{
"seq": 1,
"hunkIds": ["H1", "H2", "H3", "H4", "H5", "H6", "H7", "H8", "H9"],
"outcome": "FAIL",
"unresolvedReason": null,
"exitCode": 1,
"memoized": false
},
{ "...": "seq 2 through 59, one row per probe, in (round, index) order; memoized re-logs included" }
],
"result": {
"culprits": ["H5", "H8"],
"causes": [
{ "hunks": ["H5", "H8"], "witnessMinimal": true }
],
"cluster": [],
"complete": true,
"probeCount": 26,
"budgetExhausted": false
}
}
(Only the load-bearing rows are shown; the real output prints all nine hunks and all 59 probe rows verbatim.) The full object, field by field:
| Field | Shape |
|---|---|
version |
1 |
tool |
"diffbisect" |
command |
The test command after --, as an array. |
base |
The resolved base rev (default "HEAD"). |
hunks |
Array of { id, file, lines, kind, patch }. |
hunks[].kind |
"region", "rename", "mode", "binary", or "untracked". |
hunks[].patch |
The unit's patch text (file header + body), newline-joined. |
probes |
The full probe log: array of { seq, hunkIds, outcome, unresolvedReason, exitCode, memoized }. |
probes[].outcome |
"PASS", "FAIL", or "UNRESOLVED". |
probes[].unresolvedReason |
For UNRESOLVED probes: "apply", "exit125", or "flake"; else null. |
probes[].exitCode |
The command's exit code, or null when it never ran (apply failure). |
probes[].memoized |
Whether this row reused a cached subset outcome instead of re-running. |
result.culprits |
The revert set: hunk IDs that, removed together, make the tree pass. |
result.causes |
Array of { hunks, witnessMinimal } — one per independent cause. |
result.causes[].witnessMinimal |
Whether that cause was reduced to a 1-minimal witness. |
result.cluster |
Hunk IDs left in an undetermined interdependency cluster (empty on a clean result). |
result.complete |
true iff reverting culprits was confirmed by an actual passing probe. |
result.probeCount |
Distinct test invocations (memoized re-logs excluded). |
result.budgetExhausted |
Whether the --max-probes budget ran out before completion. |
The probe log is the raw evidence: a consumer can re-derive the verdict or extend the search itself. Same input plus a deterministic test command gives byte-identical output.
- Not
git bisect. That bisects commits and cannot look inside one or into a dirty tree. diffbisect's input is the uncommitted diff, below commit granularity. - Not
git-bifurcate. That grew a--strategy hunkbut stays commit-anchored, and it is Python; its own README names the hunk-interference problem without a shipped mitigation. diffbisect takes the dirty working tree as first-class input and treats interference residue as a labelled result. - Not
lockbisect/depbisect. Those bisect lockfile entries or dependency versions across commits. diffbisect bisects the source hunks in your working tree. - Not CI coverage / patch-coverage tooling. Codecov patch coverage and
diff-test-coveragegate lines changed between commits; diffbisect isolates which of your uncommitted hunks broke a specific test.
- Interdependent hunks can exhaust the budget into a cluster. Non-monotone interference (hunk A only fails alongside hunk B) is the normal case for real diffs, not an edge case, and there is no algorithmic fix for it — violations degrade minimality, never correctness. When the search cannot carve the residue within the budget, it reports a cluster rather than a false minimal answer.
- Flaky tests degrade to UNRESOLVED. The
--confirmguard protects the search from an inconsistent probe (an inconsistent result becomes UNRESOLVED and the search continues); it is not a flake detector and does not pretend to be. A truly flaky suite will not isolate cleanly. - Binary, rename, and mode units are atomic. They are not split further; each is a single all-or-nothing unit, as are untracked new files.
- Windows is unverified. The worktree and
git applypaths are documented but not validated there. - Your tree and index are never touched. Every probe runs in a fresh
git worktreechecked out at the base with the candidate subset applied; the diff under diagnosis is never committed, stashed, or mutated.
Delta debugging: Zeller & Hildebrandt, Simplifying and Isolating Failure-Inducing Input, IEEE TSE 2002 (https://doi.org/10.1109/32.988498).
Four tools, one idea: answers grounded in what your tests actually execute — evidence an agent can't hallucinate. Pick by the question you're holding:
| Your question | Tool |
|---|---|
| "A test is failing — which line is the bug?" | culprits — spectrum fault localization (npm) |
| "One test is failing — which function returns the wrong value?" | apd — algorithmic debugging, LLM-as-oracle (npm) |
| "Where is feature X implemented?" | recon — feature location by coverage diff (npm) |
| "My uncommitted diff broke the tests — which hunks?" | diffbisect — delta debugging below commit granularity (npm) |
MIT © Precision Utility Guild