Batch pre-labeling now writes model labels into a batch at review_pending — the spine shipped in
the pull request opening this surface (#673). A run over a few hundred frames produces a few
hundred assets awaiting review, and the review flow they land in was built for a person working
one frame at a time. This issue is the surface for reviewing them at volume.
Three sub-decisions travel here from the work that surfaced provenance and confidence in the
review loop (#417), which handed them to the batch design rather than answering them with a wire it
would have had to invent:
- The per-asset aggregate. Confidence is per annotation and the review flow orders assets. A
minimum, a mean, and a count below a threshold are three different queues, and picking one is a
product decision rather than a derivation.
- Order or narrow, and any threshold that choice implies, read against the score distribution
a real run produces rather than an assumed one. That data now exists to be gathered, which is
what the spine was for.
- Server-side counts.
GalleryScreen takes its segment counts from the batch's
ProgressCounts rather than from loaded pages, so a confidence narrow that honours that rule
needs its counts answered server-side too.
An operational note recorded on the batch pre-labeling issue (#425) constrains all three: a
confidence score means something different per capability, so they cannot be pooled. Point-prompted
segmentation scores mask quality against a human-placed prompt; text detection scores prompt
affinity. A single "below 60%" filter over a mixed batch selects almost every detection and almost
no segmentation, which is a fact about the two scales rather than about the labels. Triage
aggregates, thresholds and queues are therefore per-capability, and any surface showing the number
names what it measures.
Beyond triage, the volume problem itself: bulk accept and reject, and whatever ordering makes a
reviewer's pass over a pre-labeled batch cheaper than opening each frame.
The current wire, for reference: BatchAssetOut carries no annotation count and no confidence
aggregate, and the asset-list endpoints take limit and offset only.
Per-class thresholds belong here too. The spine ships one minimum-confidence control for the whole
run, which is the shape a per-class refinement would build on.
#425 keeps its remaining scope until this is answered — its five recorded responsibilities are
discharged by the spine, and these are what is left.
Batch pre-labeling now writes model labels into a batch at
review_pending— the spine shipped inthe pull request opening this surface (#673). A run over a few hundred frames produces a few
hundred assets awaiting review, and the review flow they land in was built for a person working
one frame at a time. This issue is the surface for reviewing them at volume.
Three sub-decisions travel here from the work that surfaced provenance and confidence in the
review loop (#417), which handed them to the batch design rather than answering them with a wire it
would have had to invent:
minimum, a mean, and a count below a threshold are three different queues, and picking one is a
product decision rather than a derivation.
a real run produces rather than an assumed one. That data now exists to be gathered, which is
what the spine was for.
GalleryScreentakes its segment counts from the batch'sProgressCountsrather than from loaded pages, so a confidence narrow that honours that ruleneeds its counts answered server-side too.
An operational note recorded on the batch pre-labeling issue (#425) constrains all three: a
confidence score means something different per capability, so they cannot be pooled. Point-prompted
segmentation scores mask quality against a human-placed prompt; text detection scores prompt
affinity. A single "below 60%" filter over a mixed batch selects almost every detection and almost
no segmentation, which is a fact about the two scales rather than about the labels. Triage
aggregates, thresholds and queues are therefore per-capability, and any surface showing the number
names what it measures.
Beyond triage, the volume problem itself: bulk accept and reject, and whatever ordering makes a
reviewer's pass over a pre-labeled batch cheaper than opening each frame.
The current wire, for reference:
BatchAssetOutcarries no annotation count and no confidenceaggregate, and the asset-list endpoints take
limitandoffsetonly.Per-class thresholds belong here too. The spine ships one minimum-confidence control for the whole
run, which is the shape a per-class refinement would build on.
#425 keeps its remaining scope until this is answered — its five recorded responsibilities are
discharged by the spine, and these are what is left.