Kimi K3 tops coding arenas while finance-agent evals stall below 60% — the eval gap explains why AI acquisitions fail post-close and how to price the risk.
July 2026 exposes a structural problem acquirers ignore: the same model generation that wins public coding leaderboards fails real enterprise workflows. Kimi K3 scores 1,679 on Arena Frontend Code while Vals AI's Finance Agent v2 shows frontier models peaking below 58% on multi-step analyst tasks — planning, retrieval, calculation, and synthesis on actual financial documents.
Post-close integration fails when buyers assumed benchmark leadership transfers to customer workflows. Three failure modes recur. Domain mismatch: coding-optimized models underperform on structured business data (the gap Prior Labs' TabPFN explicitly targets). Harness dependency: Arena scores reflect prompt engineering, tool access, and sandbox setup — assets that may not transfer with the model weights. Regression risk: model updates can erase leaderboard advantages within a quarter; earnouts tied to static benchmarks create adversarial incentives.
Pricing fix: discount projected AI revenue where claims rely on public benchmarks not reproduced in buyer-controlled evals; require holdbacks for targets whose moat is model performance without proprietary data or workflow lock-in.
Our Take
the eval gap is the AI analogue of revenue quality diligence — separate headline metrics from durable capability. Before signing, run 30 tasks from the target's top three customer segments on the acquirer's infrastructure. If scores diverge more than 15 points from vendor claims, the synergy case is unproven and the multiple should reflect software-plus-services, not frontier IP.