2026-11-13

C12: stripes the model finds but we throw away

Answer: most of them are recoverable. The model does draw 32 of the 43 missing stripes. It just scores them below the 0.45 cut-off. Only 4 are truly absent. The old finding was that 19 of 26 missed bottoms were absent. Today's model is very different: 4 of 24 missed bottoms are absent.

Lowering the cut-off is not free, though. In this test, 0.15 gains +13 photos net. 0.05 loses 9.

What was run

One RunPod run on the SAILSCAN account, about 15 minutes, H100, terminated afterwards. Cost roughly $1.
Model: v39 (sha f8a8f024), the 332 holdout photos, confidence floor 0.001, every candidate line kept (2,946 lines).
Check that the run is sound: at the 0.45 cut-off it scores 264 of 327 photos perfect, the same as the earlier v39 reference run.
Everything is saved in R2 under scratch/t54-c12/ (raw candidates, results, scripts).

Where the failing photos come from (at the 0.45 cut-off)

66 of the 332 photos fail (63 of the 327 live ones).
38 of them miss at least one stripe: 43 stripes in total (24 bottom, 11 top, 8 middle).
The other 28 fail only because of an extra line the model should not have drawn.
So it is about 66 photos, not 15. The "15" in the earlier note counted something narrower.

The 43 missed stripes

Verdict	Stripes	Bottom	Top	Middle
The model drew it, scored below 0.45 (a lower cut-off would pick it up)	32	17	9	6
Drawn only roughly (3 to 6% off the true line)	6	3	2	1
Not drawn at all, even at 0.001	4	4	0	0
Drawn at 0.45 or higher but already claimed by the neighbouring stripe	1	0	0	1

How confident the model was in the 32 recoverable lines: median 0.27, range 0.002 to 0.44. Twelve sit between 0.25 and 0.35, six between 0.35 and 0.45, and eight are under 0.15.

Bottom stripes only: 17 of 24 are recoverable, 3 are rough, 4 are absent. The v29-era run had 7 of 26 found and 19 of 26 absent. That version of the model missed bottoms because it never drew them. Today's version draws them and then loses them at the cut-off.

What a lower cut-off does (327 live photos, normal 3% rule)

Cut-off	Perfect photos	Fixed	Broken	Net
0.45 (today)	264
0.35	265	5	4	+1
0.25	273	14	5	+9
0.15	277	21	8	+13
0.10	274	21	11	+10
0.05	255	21	30	-9
0.02	236	20	48	-28

"Broken" means the photo was perfect at 0.45 and gets an extra line at the lower cut-off.

What this does not tell us

This is the raw model only. Production also runs the SAM3 refine step and the rules engine, and I did not replay those. So I can say which cut-off dropped a line, but not which production filter would then drop or keep it. The prod chain on the 332 would be the next measurement.
Lines added at a lower cut-off are de-duplicated with a simple rule (skip a line within 1% of one already kept). Production's own duplicate handling is probably better, so the "broken" column is likely too high at 0.05 and below.
One cut-off applied to every stripe is blunt. Of the 40 extra lines at 0.45, the median confidence is 0.58, so extras and missing stripes overlap in score. A cut-off tied to position (bottom stripe gets a lower bar) or to the stripe count could do better than 0.15 everywhere. That is the part worth testing on the production chain.