Posts in the AI Benchmark Watch series
CursorBench 3.2: Fable 5.1 Max hits 73.4% (previous leader Grok 4.6 Extra High was 70.8%) and debuts by taking both first and second place; Fable 5.1 Max beats the old leader by 2.6 points while costing only $9.64 per task, 44% cheaper than the prior Fable 5 Max at $17.32; Anthropic's own site still lists only Fable 5, with no official Fable 5.1 announcement
Arena Agent leaderboard, published 2026-09-30: Claude Fable 5.1 (Max) 14.55% first, Claude Opus 5.5 (High) 13.78% second, GPT-6 Astra (Max) 12.18% third. Anthropic markets Opus 5.5 as 'performing close to Fable 5.1 at 40% lower cost,' but in this human-voted Agent category it still trails the flagship by about 0.8 points, and third-party measurements put its max-effort token usage at 4x GPT-6 Astra's. This post also documents a methodology breakthrough: switching to Arena's own historical dataset on Hugging Face (via the datasets-server API) to bypass a month-plus structural failure that had blocked rendering on swebench.com and arena.ai
Arena Agent leaderboard, published 2026-10-02: Claude Fable 5.1 (Max) holds #1 at 14.31%, Claude Opus 5.5 (High) stays #2 at 13.82%, and Claude Sonnet 5.5 (Max) — launched only on 09-29 — debuts straight into #3 at 12.52%. GPT-6 Astra (Max) actually ticked up to 12.27% but still dropped to #4, simply overtaken. Arena's official X account confirmed the debut, noting Sonnet 5.5 jumped 8.1 points over its predecessor Sonnet 5 (High), now ranked #13. But the confidence intervals for all four models overlap heavily — a single snapshot can't settle who's actually ahead.
Arena Agent leaderboard, published 2026-10-08: Claude Opus 5.5 (High) takes #1 for the first time at 14.33% (previously #2 at 13.82%); GPT-6 Astra (Max) climbs to #2 at 13.09% (previously #4 at 12.27%); Claude Fable 5.1 (Max), which held #1 for over a month, drops to #3 at 12.66% (previously 14.31%); Claude Sonnet 5.5 (Max) falls to #4 at 11.95% (previously #3 at 12.52%); GPT-6.1 Sol (Max) holds #5 at 11.72%. Confidence intervals for the top three still overlap heavily.
Arena Agent leaderboard, published 2026-10-12: Claude Opus 5.5 (Max) takes #1 at 17.89% (the previous #1, the 'High' variant at 14.33%, is gone from the entire 52-model ranked list this round); GPT-6 Sol (Max) breaks into #3 at 12.60% (wasn't even in the top 10 last round); Gemini 4 Argon (High) debuts in the top 5 at #4 with 12.37%; GPT-6 Astra (Max) falls from #2 to #5 (13.09% → 12.34%); both Claude Fable 5.1 and Claude Sonnet 5.5 drop out of the top 5.
Arena Agent leaderboard published 10-13: Claude Opus 5.5 (Max) stays first at 17.01%; GPT-6.1 Sol (Max) second at 15.05%; Claude Fable 5.1 (Max) climbs from sixth back to third (11.74%→14.32%); GPT-6 Sol (Max) falls from third to seventh (12.60%→10.51%). Three releases in one week, three changes in third place, and overlapping 95% confidence intervals across the top six — the ranks themselves shouldn't drive model selection.
Arena Agent leaderboard published 10-16: Claude Opus 5.5 (Max) stays first at 18.88%; GPT-6 Astra (Max) breaks into second at 14.13% after three straight batches stuck at 4th or 5th; GPT-6.1 Sol (Max) ends a three-batch hold on second, dropping to fourth (14.78%→13.60%). The top six's 95% confidence intervals still overlap heavily, but Astra's score rose across four straight batches and Sol's long stable run just ended — a multi-batch positional shift like this is worth more than a single batch's rank jump.
New Arena Agent leaderboard batch (official publish date 2026-10-23, ending three straight days stuck on the 10-20 batch): first place Claude Opus 5.5 (Max) holds at 20.77%; Claude Sonnet 5.5 (Max) jumps from fifth at 12.67% to second at 13.99% (+1.32pp), pushing previously-third Claude Fable 5.1 (Max) out of the top three (14.17%→13.94%, down to fourth); GPT-6 Astra (Max) drops from second to third (14.70%→13.96%). Second through fifth are now only 0.35 percentage points apart, and all their 95% confidence intervals overlap — but Sonnet 5.5 matching Fable 5.1 at one-fifth the API price is the detail worth keeping.