█████ █████ █████ █████ █████ █████
█ █ █ █ █ █ █ █ █
█████ █████ █ █ █ █████ █
█ █ █ █ █ █ █ █ █
█████ █ █ █ █████ █ █ █████Each row is scored on its own metric; the track runs from genesis (○) to the current king (●), with the specialist SOTA as a dashed ◆ bar — shown only where the benchmark+metric match ours. Where a specialist reports a different metric or no standard bench exists, the row says so rather than draw a false bar. Positions self-normalized per metric; lengths are within-row progress, not cross-row comparisons.
| Primitive | Bench · metric | Genesis | King | Δ | SOTA | Gate | Why |
|---|---|---|---|---|---|---|---|
| counting | CountBenchQA · acc | 0.787 | 0.982 | +.195 | 0.936 | ♛ crowned | targeted_data — LCB +0.102 ≫ δ; the fleet’s biggest single jump |
| description | COCO-Cap · CIDEr | 0.40 | 1.42 | +1.02 | 1.45 | ♛ crowned | strong_sft crowned @ fleet δ0.02 (LCB +.029, holdout +.146) — tops generalist VLMs, nears BLIP-2 1.45; product stays rich |
| detection | CPPE-5 · F1@.5 | 0.564 | 0.770 | +.206 | — F1≠mAP | ♛ crowned | strong_sft re-crowned genesis→0.770 (LCB +.200); tops prior 0.743; no fitting generalist to set an F1@.5 bar |
| grounding | RefCOCO · Acc@.5 | 0.850 | 0.920 | +.070 | 0.900 | beats bar | same-harness re-baseline: 72B teacher scores 0.900 here (paper 0.927 doesn’t reproduce); our 8B 0.920 beats it +2.0pt. Coord-convention artifact caught (72B would false-score 0.0 naively) |
| reasoning | A-OKVQA · acc | 0.900 | 0.900 | — | 0.900 | ties bar | same-harness re-baseline: 72B teacher = 0.900, identical to our 8B (paper 0.928 doesn’t reproduce). Wall was a paper artifact |
| ocr | DocVQA · ANLS | — | 0.943 | — | 0.947 | near-tie | same-harness re-baseline: 72B teacher 0.947 vs our 8B 0.943 = 0.4pt (within noise); paper 0.964 doesn’t reproduce. Only primitive with real (tiny) headroom → distill target |
| behaviour | UCF101 · top-1 | 0.875 | 0.980 | +.105 | ~0.98 | ♛ crowned | built a Satori video expert: 0.875→0.9796 TTA (windowed-training fix, full split-1 test, no leakage); +10.5pt, a statistical TIE with the 0.98 bar (2 clips short) |
| spatial | VSR · acc | 0.840 | 0.840 | — | ~0.78 | beats bar | 0.840 at genesis already clears the best-VLM VSR line (~0.775) — born above bar, no climb needed |
| pointing | PixMo-Points · point-acc | 0.66 | 0.790 | +.12 | ~0.79 | bar met | dedicated aug build carried holdout 0.66→0.790 (public 0.795) — both meet the ~0.79 Molmo bar, split-consistent +.12; crown gate held on n=200 public underpower, not a null |
| segmentation | RefCOCOg · cIoU | 64.1 | 66.5 | +2.4 | ~78.7 | decoder built | trained text-conditioned SAM2 decoder: box→SAM2 64.1→66.5; frozen-encoder wall ~67. VLM-in-the-loop tested (SEG-token → SAM2, H100): frozen-encoder near-parity 0.61, but UNFREEZING regressed to 0.45 (overfit at RefCOCOg scale) — hypothesis disconfirmed, 66.5 holds. 78.7 needs far more mask data, not encoder unfreeze |