How AI models are evaluated, compared and ranked, and how to tell which numbers still mean something. Updated October 2026.
Most famous benchmarks are now saturated. ARC-AGI-2, FrontierMath Tier 4 and SWE-bench Verified joined MMLU, GPQA and HumanEval in 2026. Labs now report on harder, more agentic suites: Humanity's Last Exam, ARC-AGI-3, SWE-bench Pro, Terminal-Bench 4.0, OSWorld 2.x, METR time horizons and GDPval.
Scores are dated and attributed. "Lab-reported" means the model card or launch post; "as reported" means press or third-party coverage we have not independently verified. Other evals worth watching include MCP-Atlas, BrowseComp, AutomationBench, the Vals Index and GraphWalks (long context).
HLE · Center for AI Safety + Scale AI
Expert-written, closed-ended questions across dozens of fields, built to outlast saturation. Scores climbed from single digits at launch (early 2025) to around 60% by late 2026; Claude Opus 4.6 had about 53% with tools in early 2026. Aggregators variously put Claude Opus 5.5, Claude Fable 5.1 and GPT-6 Astra on top, within about a point of each other.
Claude Fable 5.1, no tools / with tools · Sep 2026 (Anthropic-reported)
ARC Prize · interactive reasoning environments
Launched Mar 25, 2026. Replaces static grid puzzles with interactive game environments that the agent must explore and figure out on the fly. At launch, humans solved 100% while the best frontier LLM scored under 1% and the best purpose-built agent 12.58% (preview). Aggregators report large gains since then, but those numbers are unverified, so check arcprize.org for verified results.
Best frontier LLM at launch (Gemini 3.1 Pro) · Mar 2026; humans 100%
Epoch AI · research-level mathematics
Unpublished, research-level problems written by mathematicians. In 2026 Epoch audited the set and released FrontierMath v2, which fixed 12 Tier 4 problems and removed 7. A new FrontierMath Erdős variant (Sep 2026) is built on open Erdős problems.
GPT-5.5 · Apr 2026 (OpenAI-reported)
Scale AI · harder software-engineering tasks
Longer, more professional repository tasks. Labs moved their headline coding numbers here after SWE-bench Verified saturated. Scores depend heavily on the split and harness, and public leaderboards disagree widely, so we don't list a single top score. A cleaned-up "SWE-Bench Pro Verified" variant appeared in Sep 2026.
Leaderboards conflict · Oct 2026
Stanford / Laude Institute · tbench.ai
Hard command-line tasks run end to end in a sandbox. Version 4.0 has 66 tasks, an 8-hour agent timeout, and continuous semantic versioning (it followed 2.0, 2.1 and 3.0). Vendor-reported scores: Claude Opus 5.5 66.4%, GPT-6 Astra about 58%, Claude Fable 5.1 55.8%.
Claude Opus 5.5 · Sep 2026 (Anthropic-reported)
Computer-use agents on real desktops
Agents operate real desktop applications end to end. Agents reportedly passed the original OSWorld's ~72% human baseline around late 2025, so labs moved to OSWorld 2.0 and 2.1. Versions are not comparable: on 2.0, OpenAI reports GPT-6 Sol at 64.4% and Anthropic reports Fable 5.1 at 41.7% (strict); on 2.1, Anthropic reports Opus 5.5 at 81.8%.
Claude Opus 5.5 · Sep 2026 (Anthropic-reported)
METR · Time Horizon 1.1 (Jan 29, 2026)
The length of task, measured in skilled-human time, that an agent completes with 50% reliability. TH1.1 uses 228 tasks, 31 of them 8 hours or longer. Claude Opus 4.6 reached about 12 hours; an early Claude Mythos checkpoint was reported at 16 hours or more. METR says its suite cannot reliably measure horizons above ~16 hours, and it has published no horizon for Opus 5.5 or GPT-6.
Claude Mythos Preview, early checkpoint · 2026 (as reported)
OpenAI · economically valuable tasks
Real deliverables from 44 occupations, graded by industry experts against work by human professionals. It is now cited alongside AutomationBench as a "real job" measure in 2026 model launches. Compare scores only within a single lab's report.
Cited in 2026 frontier-lab launches
Agentic software engineering
A newer agentic coding suite that labs began citing in 2026, after SWE-bench Verified saturated. GPT-6 Luna was reported at 66.6% (Sep 2026).
Gemini 4 Argon · Oct 2026 (Google model card)
Coding-agent tasks · v3.2.0
A versioned coding-agent benchmark that labs cite next to SWE-bench Pro and Terminal-Bench. Always check which version a score refers to.
Claude Fable 5.1 · Sep 2026 (Anthropic-reported)
Scientific workflows in the terminal
A science-focused companion to Terminal-Bench, introduced in 2026, with research-style tasks run end to end in a terminal.
Claude Fable 5.1 · Sep 2026 (as reported at launch)
Practical biology research tasks
A follow-up to LAB-Bench covering everyday biology research work. Biology evals now matter for safety as well as capability, and labs report them alongside their bio-risk thresholds.
Gemini 4 Argon · Oct 2026 (Google model card)
Software weakness detection
A security benchmark built around Common Weakness Enumeration vulnerability classes. Cyber capability is now the main factor deciding when and how frontier models are released.
Gemini 4 Argon · Oct 2026 (Google model card)
Epoch AI · composite index
Stitches many benchmarks onto one scale, so it keeps discriminating as individual tests saturate. Sonnet 5.5 and Fable 5.1 were reported just behind, at 165.
Claude Opus 5.5 & GPT-6 Astra · Oct 1, 2026 (as reported, citing Epoch)
Sierra · tool-agent-user simulation
Simulated multi-turn customer-service conversations with tools, policies and a database. τ²-bench is reported to be near saturation, and τ³-bench is its successor. Treat leaderboard claims with care.
Successor τ³-bench emerging · 2026
Abstract Reasoning Corpus 2
Static visual-reasoning puzzles. Gemini 3.1 Pro scored 77.1% (ARC Prize-verified) in Feb 2026, and aggregators now show late-2026 models at about 90–95%. Kaggle's efficiency-limited ARC Prize 2026 track is led by Tufa Labs at 88.06. ARC-AGI-3 replaces it as the live test.
Aggregator range · Oct 2026 (77.1% verified, Gemini 3.1 Pro, Feb)
Epoch AI · hardest tier
Tier 4 records went from 31% (GPT-5.2 Pro, Jan) to 38% (GPT-5.4, Mar; Epoch-run). Per aggregators, every Tier 4 problem has now been solved at least once, with late-2026 models near 100% on v2. Attention has moved to FrontierMath Erdős and open research problems.
Per aggregators · Oct 2026 (Epoch-run record 38%, GPT-5.4, Mar)
500 human-validated GitHub issues
Real Python repository issues. Models must produce a patch that passes the tests. Scores reached the mid-90s in 2026, and the remaining failures are often disputed tasks. Superseded by SWE-bench Pro, DeepSWE and Terminal-Bench.
Claude Mythos Preview 93.9% · Apr 2026 (as reported)
Graduate-level Google-proof Q&A
198 hard graduate-level physics, chemistry and biology questions. Several frontier models are above 90% (GPT-5.4 92.8%). It is still quoted in launches, but it no longer separates the frontier.
Gemini 3.1 Pro · Feb 2026 (as reported)
International Mathematical Olympiad
In 2025, OpenAI and DeepMind systems scored 35/42 (gold level). At IMO 2026 (July), several AI systems reached 42/42. Officially graded perfect scores were reported for RedNote's dots-note-3.0 and Huawei's Celia, and GPT-5.6 Pro and Claude Opus 5 reportedly also scored full marks. No AI system was an official medal-eligible entrant.
Several AI systems · Jul 2026 (as reported)
Massive Multitask Language Understanding
A 57-subject multiple-choice knowledge test. Saturated above 90% by 2024. It was replaced by MMLU-Pro and then by expert-level exams such as HLE.
Saturated since 2024
OpenAI HumanEval
164 short Python functions. Frontier models score above 95%. Kept as a historical reference; repository-scale and agentic suites replaced it.
Saturated since 2024
Human preference ratings complement standardized tests. The Elo-style system, borrowed from chess, ranks models by head-to-head comparisons. Rating gaps are meaningful, but absolute numbers are not.
Formerly LMArena and, before that, LMSYS Chatbot Arena. The project has rebranded as arena.ai, and lmarena.ai now redirects there. Users chat with two anonymous models side by side and vote for the better answer, and millions of votes are turned into ratings across text, vision, image, video and coding arenas. As of October 2026, third-party trackers disagree about which model leads the text arena, and we could not verify the official board, so we don't name a #1 here. Check arena.ai directly and look at the confidence intervals.
Ratings come from a Bradley-Terry model fitted to pairwise votes and are reported on an Elo-like scale with confidence intervals. Only the gap between two models matters. Models whose intervals overlap are statistically tied, and in 2026 the top of the board is often a cluster of near-ties. Style (length, formatting, tone) affects votes, so the arena offers style-controlled views.
Equal ratings mean a 50% expected win rate. The absolute number means nothing on its own, because it shifts as models are added.
About a 57% expected win rate. This is often inside the confidence interval, so treat it as a tie.
About a 64% expected win rate. Users usually notice the difference on everyday tasks.
About a 76% expected win rate. Users notice the difference on almost every prompt.
About a 91% expected win rate. For example, a current frontier model against a small or much older one.
Where to check current rankings. Primary sources and independent evaluators come first.
Crowdsourced human-preference ratings for text, vision, image, video and coding. Formerly LMArena / LMSYS Chatbot Arena.
Verified ARC-AGI-2 and ARC-AGI-3 results, plotted against cost per task.
Independently run evaluations, FrontierMath, and the Epoch Capabilities Index.
Task-length measurements of agent autonomy. METR notes the page is no longer actively updated (Sep 2026).
The official Terminal-Bench 4.0 leaderboard and task suite for command-line agents.
Expert-built evals, including Humanity's Last Exam and SWE-bench Pro.
Holistic Agent Leaderboard: standardized, cost-aware agent evaluations across many benchmarks.
Independent quality, speed and price comparisons. Useful for cost and latency trade-offs.
Every benchmark follows the same life cycle. It launches with low scores, models climb quickly, scores pass 90%, and the field moves to a harder successor. In 2026 the cycle sped up. Stanford's AI Index 2026 counts about half of 60 major benchmarks as saturated, and ARC-AGI-2 and FrontierMath Tier 4 went from headroom to ceiling within months.
Benchmarks have also moved from exams to work: multi-hour terminal tasks, desktop computer use, and task-length horizons. Some labs pointed models at open research problems instead. In September 2026, 28 Fields Medalists signed an open letter warning against treating open problems as benchmarks.
SWE-bench Verified led to Pro and then Pro Verified. Terminal-Bench went 2.0 → 3.0 → 4.0, and OSWorld went to 2.0 and 2.1. A score with no version and no date is close to meaningless.
Lab reports name the harness and settings they used. Aggregator sites often disagree by 5 points or more on the same model, so treat their tables as indicative and write "as reported by" when you cite them.
Effort level (low through xhigh/max), tool access and agent scaffolding all change scores. On HLE, the same model scores 60.9% without tools and 65.0% with them.
Above about 90%, the remaining errors are often label noise. METR warns that its suite cannot resolve time horizons beyond ~16 hours, so the measurement runs out before the models do.
METR reported that if GPT-5.6 Sol's cheating attempts had counted as successes, its time horizon would have exceeded 270 hours. Reward hacking and test-set contamination both inflate static benchmarks.
The top systems sit within single digits of each other (Stanford AI Index 2026), and rankings flip from one benchmark to the next. Pick the eval closest to your task, then test on your own data.
Our models are continuously evaluated against these benchmarks. Try FullAI and see the difference.
Start Building for Free