Frontier model leaderboards rotate weekly. M&A and corp-dev teams need task-level eval frameworks — not vendor benchmarks — before pricing AI assets or signing LOIs.
July 2026 delivered three benchmark shocks in one week: Kimi K3 topped Arena Frontend Code, Vals AI's Finance Agent v2 showed no frontier model above 58% on real analyst workflows, and Meta's Muse Spark 1.1 launched with undisclosed full eval harness details. For M&A teams, the lesson is uniform: public leaderboards measure vendor marketing, not acquirer value.
A credible AI tech-diligence eval stack has four layers. Task fidelity: test on documents, tables, and workflows that match the target's customer base — credit agreements for fintech, frontend builds for dev tools, ERP records for enterprise SaaS. Harness transparency: demand prompts, scoring rubrics, and repeated-run rules; Kimi K3's launch post documented settings while many labs still publish headline numbers only. Regression cadence: re-run evals when target models update — leaderboard positions churn with every release cycle. Human-in-the-loop cost: sub-70% automated scores mean expert review remains mandatory; model the fully loaded cost of deployment, not token price alone.
Representations and warranties should reference eval methodology: define which benchmarks were used in financial projections, cap liability where claims exceed independently verified scores, and require escrow for earnouts tied to model-performance milestones.
Our Take
stop treating AI diligence like software diligence with a demo layer. Build a 20–30 task eval suite before IOI, run it on the target's production model (not the founder's laptop), and attach results to the data room. The gap between Arena Frontend Code glory and Finance Agent v2 reality proves that capability is task-specific — and that is exactly what purchase-price adjustments should reflect.