Skip to content

feat: name the classes a pre-label run will and will not ask for - #703

Merged
JArmandoAnaya merged 4 commits into
mainfrom
feat/prelabel-prompt-classes
Aug 19, 2026
Merged

feat: name the classes a pre-label run will and will not ask for#703
JArmandoAnaya merged 4 commits into
mainfrom
feat/prelabel-prompt-classes

Conversation

@JArmandoAnaya

Copy link
Copy Markdown
Contributor

A pre-labeling run's prompt is the batch's pinned schema narrowed to the classes a bare box
prediction can be written as. A class is dropped for either of two reasons — it does not admit
bbox, so a detection has no shape to land as, or it declares a required attribute, which a
model's answer carries no value for — and until now both exclusions were silent everywhere.
Somebody with a vehicle class requiring a color started a run, watched it complete, and found
no vehicles labeled, with nothing anywhere saying why. That is the gap this closes (#677).

prompt_plan in visionset/inference/prelabel.py now returns both halves of the split, and
detectable_classes reads its asked list rather than computing its own, so the prompt a run uses
and the plan a surface publishes cannot be two derivations. reasons is a list because both
reasons can hold against one class: a caller told only that a class admits no box would add one
and watch it stay absent from the next run's prompt.

GET /batches/{batch_id}/pre-label publishes the plan before a run starts. It guards on the
launch's own terms, so a batch nobody may pre-label and a schema with no askable class refuse
there with the codes the launch already answers, and it takes no connection, since the prompt is
a property of the pinned schema alone. The dialog reads it on open and shows the asked-for classes
with the left-out ones beside them, and shows them again under a settled run's summary, which is
where a run that labeled nothing is actually read. At a terminal, visionset batch pre-label
writes the same two lines to stderr before the first forward pass.

PreLabelExclusionReason is an open vocabulary. It travels as a list a client renders member by
member, so a release that finds a third way a class cannot hold a detection must not cost an older
client the whole plan. The browser drops a reason it cannot word and still names the class, which
is the half that must not be lost.

Found, not fixed

The MCP surface gets nothing here: pre_label_batch still says nothing about the prompt. That
work is on a follow-up branch stacked on this one.

Issue-number references exist in prose in docs/batches.md and docs/mcp.md that predate this
change. They are left alone rather than swept in.

Test plan

Every stage run on the rebased branch, each exiting 0:

uv run pytest tests/architecture tests/cli tests/examples tests/formats tests/inference
    1058 passed, 19 skipped
uv run pytest tests/jobs tests/kernel tests/mcp tests/packaging tests/scripts tests/test_versioning.py
    2104 passed, 5 skipped
uv run pytest tests/server
    791 passed
uv run ruff check .                  All checks passed!
uv run ruff format --check .         419 files already formatted
uv run mypy src/visionset/kernel     Success: no issues found in 73 source files
uv run lint-imports                  Contracts: 4 kept, 0 broken.
pnpm -r build / -r test / -r lint    1062 + 1155 passed, lint clean
bash scripts/check.sh browser        278 passed (41.8s); 1 passed (21.6s)

Nine mutations were run against the new tests to show they fail when the code is wrong: dropping
each exclusion reason, silencing the plan announcement, truncating the route's exclusion list,
hiding the dialog's excluded line, hiding it in done-mode, removing the unknown-reason filter, and
removing the launch guard. Each reddened exactly the tests that claim it.

Closes #677

detectable_classes computed the exclusions and discarded them, so both
reasons a class is absent from the prompt — no bbox among its geometries,
or a required attribute a bare prediction carries no value for — were
invisible to every surface. prompt_plan returns both halves, and
detectable_classes reads its asked list so the prompt and the published
plan cannot be two derivations.

pre_label gains on_plan, announced after every refusal and before the
first forward pass, which is what lets a surface name the prompt a run is
actually about to use.
A run's prompt is invisible in its outcome: a schema whose vehicle class
requires an attribute completes a run, labels no vehicles, and says
nothing about why. GET /batches/{id}/pre-label answers asked_classes and
excluded_classes, each exclusion carrying every reason that holds against
it, so a client can name the prompt before a run starts.

Guarded on the launch's own terms — a batch nobody may pre-label and a
schema with no askable class refuse here with the same codes — and free
of the connection, since the prompt is a property of the pinned schema
alone.

PreLabelExclusionReason is an open vocabulary: it travels as a list a
client renders member by member, so a third reason must not cost an older
client the whole plan.
… does not

Printed to stderr before the first forward pass, where it is still
actionable — a class the prompt omits labels nothing, and afterwards
there is only the silence to explain.
The dialog said a batch's schema becomes the prompt without saying which
classes survive the narrowing, so a run that legitimately labels nothing
read exactly like a run that should have labeled something. It now reads
the plan when it opens and shows both halves: the classes it asks for,
and beside them each class it does not with the reason — no box, or an
attribute a prediction cannot supply.

Shown again under a settled run's summary, which is where a run that
labeled nothing is actually read. The lists come off the wire rather than
being derived from the pinned schema here: the same narrowing decides
what the run really prompts with, and a browser-side copy is how a dialog
comes to name a class no run asks about. A schema with nothing askable
refuses the read, and the dialog renders that refusal and leaves Start
dead rather than waiting for the press to produce it.
@JArmandoAnaya
JArmandoAnaya merged commit 88947cc into main Aug 19, 2026
15 checks passed
@JArmandoAnaya
JArmandoAnaya deleted the feat/prelabel-prompt-classes branch August 19, 2026 07:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

The pre-label dialog does not say which classes it will leave out of the prompt

1 participant