Log inSign up free

AI Model Benchmarks101

Every major closed- and open-weight foundation model, version by version, on the standard public benchmarks, with what each costs. Pick a benchmark, toggle models on and off, and build the comparison you need.
Ask the room
Model versions
101
28 current
Open-weight
37
64 closed
Labs
12
frontier + open labs
Last refreshed
Oct 10, 2026
updates every two weeks
Benchmark
View
Weights
Labs
Models 3
Terminal-Bench 2.1Agentic tasks in a live terminal (Vals AI run, Terminus-2 harness). Measures how well a model works a shell unattended. Lounge
0%25%50%75%100%GPT-6 Astra87.3%Gemini 3.1 Pro70.8%
No published Terminal-Bench score: GPT-5
Download PNGlightFree to use with credit · how to cite
lounge.ai/benchmarks
GPT-6 Astra87.3%
Gemini 3.1 Pro70.8%
Model$ in / outTerminal-Bench▼SWE-benchGPQAHLEARC-AGI-2AIMEτ²-benchMMLU-ProAA IndexArena Elo
Claude Opus 5.5closed$4 / $2087.6%——67.7%91.7%—————
GPT-6 Astraclosed$10 / $5087.3%—96%57.2%95%—————
GPT-5.6 SolclosedSuperseded$5 / $3085.8%—94.6%—92.5%—85.1%—58.9—
Claude Fable 5.1closed$10 / $5085%——60.9%90%—————
Claude Opus 5closedSuperseded$5 / $2584.6%96%—56.6%90.4%———60.71505
Claude Sonnet 5.5closed$2 / $1083.1%——56.9%——————
GPT-6 SolclosedSuperseded$2 / $1083.1%———89.6%—————
Gemini 3.8 Flashclosed$0.75 / $3.7581.3%———89.2%—————
Kimi K3open
2.8T / 104B active · Kimi K3 License
$3 / $1580.9%93.4%93.5%56%60.4%———57.11476
Claude Fable 5closedSuperseded$10 / $5080.5%95%—57.8%89.2%—98.5%—59.91494
GPT-5.6 LunaclosedSuperseded$0.2 / $1.279%—92.3%—59.5%———51.2—
Muse Spark 1.3closed$1.25 / $4.2579%—————————
Grok 4.6closedSuperseded$2 / $678.3%———67.1%—————
GPT-5.6 TerraclosedSuperseded$2 / $1277.5%—92.9%—83.9%—86.3%—55—
Gemini 3.7 FlashclosedSuperseded$0.75 / $3.7577.5%———84.6%————1491
GPT-5.5closedSuperseded$5 / $3076.4%—93.6%41.4%85%—98%—54.81481
Claude Sonnet 5closedSuperseded$2 / $1074.5%85.2%—57.4%————53.4—
DeepSeek V4.1 Flashopen
552B / 16B active · MIT
$0.14 / $0.2874.5%—90.9%36.8%——————
Gemini 3.5 FlashclosedSuperseded$1.5 / $974.2%—92.7%40.2%72.1%—95.3%—50.21484
Gemini 3.6 FlashclosedSuperseded$0.75 / $3.7573.8%———60.4%———50.11476
Grok 4.7closed$2 / $673.4%—————————
GPT-6 Lunaclosed$0.1 / $0.573%———59.3%—————
Claude Opus 4.8closedSuperseded$5 / $2571.9%88.6%93.6%57.9%72.1%—94.4%—55.71484
GLM-5.3open
753B · GLM-5.3 License
$1.4 / $4.471.5%————————1475
Gemini 3.1 Proclosed$2 / $1270.8%—94.3%44.4%77.1%—95.6%—46.51480
Muse Spark 1.2closedSuperseded$1.25 / $4.2569.7%————————1489
Muse Spark 1.1closedSuperseded$1.25 / $4.2569.3%——62.1%————50.61479
Claude Opus 4.7closedSuperseded$5 / $2568.5%87.6%94.2%46.9%75.8%—88.6%—53.51490
Grok 4.5closedSuperseded$2 / $667.8%———52.6%———53.8—
GLM-5.2openSuperseded
744B / 40B active · MIT
$1.4 / $4.467.8%—91.2%54.7%—99.2%99.1%—51.1—
Qwen3.8 Maxopen
2.4T / 95B active · Qwen3.8-Max License
$2 / $667.4%—92.6%43.6%—————1480
DeepSeek V4 FlashopenSuperseded
284B / 13B active · MIT
$0.14 / $0.2867%79%88.1%34.8%61.4%——86.2%40.3—
Kimi K2.7 Codeopen
1T / 32B active · Modified MIT
$0.6 / $2.567%—————90.1%—42—
GLM-5.3 Flashopen
320B / 18B active · MIT
—62.9%—————————
Qwen3.7 MaxclosedSuperseded$2.5 / $7.561%80.4%92.4%41.4%——94.7%89.6%461474
Qwen3.8 27Bopen
27B · Apache-2.0
—58.4%—89.2%30.8%——————
Claude Sonnet 4.6closedSuperseded$3 / $1557.3%79.6%—49%58.3%—79.5%79.2%35.9—
GLM-5.1openSuperseded
744B / 40B active · MIT
—56.9%—86.2%52.3%—95.3%97.7%—40.2—
DeepSeek V4 Proopen
1.6T / 49B active · MIT
$0.435 / $0.8754.7%80.6%90.1%42.7%61.3%—96.2%87.5%44.3—
Gemini 3 FlashclosedSuperseded$0.5 / $353.9%78%90.4%33.7%——43.3%—27.4—
Kimi K2.6openSuperseded
1T / 32B active · Modified MIT
$0.6 / $2.553.6%80.2%90.5%34.7%—96.4%95.9%—44.2—
MiniMax M3open
428B / 23B active · MiniMax Community
$0.3 / $1.253.6%80.5%————88.9%—44.4—
Qwen3.6 PlusclosedSuperseded—53.2%78.8%—28.8%—95.3%97.7%88.5%39.6—
Qwen3.7 PlusclosedSuperseded—52.8%77.7%90.3%34.7%——93%88.5%39—
Nemotron 3 Ultraopen
550B / 55B active · OpenMDW-1.1
—50.9%71.9%87%26.7%——83.3%86.8%37.8—
MiniMax M2.7openSuperseded
230B / 10B active · Modified MIT
—48.7%—87%——89.8%84.8%—38.1—
Grok 4.20closedSuperseded$1.25 / $2.544.2%76.7%88.5%—53.3%—————
Claude Haiku 4.5closedSuperseded$1 / $543.8%73.3%—10.2%——————
Grok 4.3closedSuperseded$1.25 / $2.541.9%——35%——97.7%—37.6—
Kimi K2.5openSuperseded
1T / 32B active · Modified MIT
$0.6 / $2.541.9%76.8%87.6%30.1%—95.8%95.9%87.1%35.4—
Mistral Medium 3.5open
128B · Modified MIT
—39%77.6%————94.2%—29.9—
Claude Haiku 5.5closed$0.1 / $0.5———45.9%——————
Claude Mythos 5.1closed$10 / $50——————————
Claude Mythos 5closedSuperseded$10 / $50—95.5%—64.5%——————
Claude Opus 4.6closedSuperseded$5 / $25—80.8%89.2%53%68.8%—92.1%82%43.71503
Claude Opus 4.5closedSuperseded$5 / $25—80.9%—30.8%37.6%95.1%89.5%89.5%40.8—
Claude Sonnet 4.5closedSuperseded$3 / $15—77.2%——13.6%—————
Claude Opus 4.1closedRetired$15 / $75—74.5%————71.4%—33.7—
Claude Opus 4closedRetired$15 / $75—72.5%————————
Claude Sonnet 4closedRetired$3 / $15—72.7%————52.3%—25.5—
Claude 3.7 SonnetclosedRetired$3 / $15—62.3%————————
Claude 3.5 Sonnet (Oct 2024)closedRetired$3 / $15—49%————————
GPT-6.1 Solclosed$2 / $10————94.2%—————
GPT-5.5 ProclosedSuperseded$30 / $180———43.1%84.2%—————
GPT-5.4closedSuperseded$2.5 / $15——92.8%39.8%73.3%—98.9%—51.4—
GPT-5.3 CodexclosedSuperseded$1.75 / $14—85%————86%—44.3—
GPT-5.2closedSuperseded$1.75 / $14—80%——52.9%—84.8%—42.2—
GPT-5.1closedSuperseded$1.25 / $10——————81.9%—36.9—
GPT-5closedSuperseded$1.25 / $10—74.9%85.7%———84.8%—34.7—
gpt-oss-120bopen
117B / 5.1B active · Apache-2.0
——62.4%————65.8%—23.8—
o3closedSuperseded$2 / $8—69.1%————80.7%—30.4—
GPT-4.1closedSuperseded$2 / $8—54.6%————47.1%—19.4—
o3-miniclosedRetired$1.1 / $4.4—49.3%————28.7%—19—
o1closedRetired$15 / $60——————62.6%—23.4—
GPT-4oclosedRetired$2.5 / $10——————25.1%—11.2—
Gemini 4 ArgonclosedPreview$2 / $10——————————
Gemma 4 31Bopen
31B · Apache-2.0
——52%—26.5%——59.9%85.2%29.4—
Gemini 3 ProclosedSuperseded$2 / $12—76.2%91.9%37.5%31.1%—87.1%—39.51480
Gemini 2.5 FlashclosedRetired$0.3 / $2.5——————14.9%—14.1—
Gemini 2.5 ProclosedRetired$1.25 / $10—63.8%—18.8%——54.1%—25.8—
Grok 4.1closedSuperseded$3 / $15—————————1483
Grok 4closedSuperseded$3 / $15——87.5%25.4%15.9%—74.9%—33.3—
Muse Glimmer 30Bopen
30B · Apache-2.0
——76%83.5%——94.7%————
Muse SparkclosedSuperseded——77.4%89.5%50.4%42.5%—91.5%—43.11474
Llama 4 MaverickopenSuperseded
400B / 17B active · Llama 4 Community
———————17.8%—14.3—
Llama 4 ScoutopenSuperseded
109B / 17B active · Llama 4 Community
———————15.5%—10—
DeepSeek V3.2openSuperseded
671B / 37B active · MIT
$0.28 / $0.42——————78.9%—24.7—
DeepSeek V3.1openSuperseded
671B / 37B active · MIT
$0.56 / $1.68—66%————34.8%—21.1—
DeepSeek R1openSuperseded
671B / 37B active · MIT
$0.55 / $2.19—49.2%————36.5%—20.1—
DeepSeek V3openRetired
671B / 37B active · MIT
$0.27 / $1.1—42%————22.8%75.9%14.2—
Kimi K2openSuperseded
1T / 32B active · Modified MIT
$0.6 / $2.5—65.8%————61.1%—19.4—
Qwen3.8 Flash-Nextopen
125B / 6B active · Qwen Community 1.0
———91.7%35.9%——————
Qwen3.6 27BopenSuperseded
27B · Apache-2.0
——77.2%—24%—94.1%94.2%86.2%37—
Qwen3.5 397B-A17BopenSuperseded
397B / 17B active · Apache-2.0
——76.2%—28.7%—93.3%83.9%87.8%33.7—
GLM-5openSuperseded
744B / 40B active · MIT
——77.8%86%50.4%—95.8%98.2%85.7%39.5—
GLM-4.7openSuperseded
355B / 32B active · MIT
——73.8%—24.8%——95.9%84.3%33.7—
GLM-4.6openSuperseded
355B / 32B active · MIT
———————76.9%—23—
Mistral Large 4openPreview———————————
Mistral Small 4open
119B / 6.5B active · Apache-2.0
———————41.2%—19.6—
Mistral Large 3openSuperseded
675B / 41B active · Apache-2.0
———————24.6%—15.9—
MiniMax M2.5openSuperseded
230B / 10B active · Modified MIT
———————95.3%—33.6—
101 of 101 model versions shown · hover a score for its configuration, click it for the source · prices are USD per 1M tokens via each lab's API, standard tier
For journalists and researchers

Use these charts in your story

Every chart and number on this page is free to cite, screenshot and republish, in print or online, as long as you credit Lounge.ai and link to lounge.ai/benchmarks. Scores carry their source link and run configuration; the data is refreshed every two weeks (last on Oct 10, 2026). Need a custom cut, a quote or the full dataset? Email hello@lounge.ai.

Credit line
Source: Lounge.ai AI Model Benchmarks (lounge.ai/benchmarks), accessed October 10, 2026. https://lounge.ai/benchmarks
Embed a chart (links back automatically)
<a href="https://lounge.ai/benchmarks"><img src="https://lounge.ai/api/benchmarks/chart?b=tb" alt="Terminal-Bench 2.1: top AI models, from the Lounge.ai AI Model Benchmarks" width="1200" height="675" style="max-width:100%;height:auto" /></a>
<p>Source: <a href="https://lounge.ai/benchmarks">Lounge.ai AI Model Benchmarks</a></p>
Download charts (PNG, 1200 × 675)
Terminal-Bench 2.1Top 10 · darklightBest per labOpen-weight
SWE-bench VerifiedTop 10 · darklightBest per labOpen-weight
GPQA DiamondTop 10 · darklightBest per labOpen-weight
Humanity's Last ExamTop 10 · darklightBest per labOpen-weight
ARC-AGI-2Top 10 · darklightBest per labOpen-weight
AIME 2026Top 10 · darklightBest per labOpen-weight
τ²-bench (Telecom)Top 10 · darklightBest per labOpen-weight
MMLU-ProTop 10 · darklightBest per labOpen-weight
Artificial Analysis Intelligence IndexTop 10 · darklightBest per labOpen-weight
LMArena TextTop 10 · darklightBest per labOpen-weight
Building a comparison above? “Download PNG” under the chart exports exactly the models you picked. Each image carries the Lounge mark and the source line.