The Cheap Model Double-Checked Its Way to First Place
A cheap model just outscored a far more expensive one on a terminal-task benchmark. It did not get smarter. It got to check itself more. Instead of answering once and shipping, it solves the same task repeatedly, compares the attempts, grades them, and keeps only what survives. At roughly a eleventh of the cost per run, the same budget buys an order of magnitude more attempts, and the attempt count is what shows up as accuracy. So the interesting question stops being which model is smarter, and becomes what a single verification pass costs you.
Source: RT @jackyk02: Scaling self-verification with DeepSeek V4 Flash beats Claude Fable 5 on Terminal-Bench 2.1, while being 1 · twitter
▶ Animated edition — the characters speak for themselves (Korean audio)
What this means for ThakiCloud
The reason this result is good news is narrow: scaling verification only works while verification is cheap. When every pass is a metered call, eleven checks are eleven invoices, and the accuracy you bought arrives with a bill attached. Paxis fans work out across agents and makes them argue each other down, keeping only what survives, for the same reason. That loop is only free to widen when it runs on hardware you own. Metis pushing serving cost down and on-prem keeping the infrastructure under your own control meet exactly here. Renting the clever model is one option. Owning the yard where a cheap model can run all day is the one that compounds.
An auto-generated comic riffing on this week’s industry news.