AIEvals

Benchmarks

CritPt

Research-level physics reasoning, scored by Artificial Analysis.

Scientific reasoningReported independently by Artificial Analysis386 models run by Artificial AnalysisUpdated 8 October 2026Reported by Reflection AI

How it is run

AA runs all 70 test problems against the authors’ grading server without an agent harness. CritPt is under expert review; the authors may revise the dataset and scores. Highest published effort per release; the table retains its snapshot date.

What it cannot tell you

Effort settings and evaluation versions matter. These source-specific results are kept separate from Vals AI and vendor reports. A missing score stays blank.

Leader
GPT-5.6 Sol32.29% · Artificial Analysis results, grouped here by release
Top-ten spread
2.58%Leader minus tenth place, or last place in smaller fields.
Median
0.14%of the 386 rows published
No published result
4of 46 models on this site

Artificial Analysis results by release

Highest published effort per model release, grouped here from AA’s public results. Grouping ranks are derived here; source settings and model-publisher reports stay separate.

#ModelScoreEffort
1GPT-5.6 Sol32.29%max
2GPT-6.1 Sol31.71%max
2GPT-6 Astra31.71%max
2Claude Opus 5.531.71%max
5Claude Sonnet 5.531.43%max
6GPT-6 Sol30.86%max
7GPT-5.5 Pro30.57%xhigh
8GPT-5.6 Terra30.00%max
8GPT-5.4 Pro30.00%xhigh
10Claude Fable 5.129.71%max
11Claude Opus 529.14%max
12Claude Fable 528.57%max
13GPT-5.527.14%xhigh
13Gemini 4 Argon27.14%high
15MiMo-V2.6-Pro26.57%Not specified
16Gemini 3 Deep Think25.71%Not specified
17Muse Spark 1.324.86%max
18GPT-5.423.43%xhigh
18Kimi K323.43%max
20Step 5 Preview20.86%Not specified
20Claude Opus 4.820.86%max
20GLM-5.220.86%max
23GPT-5.6 Luna20.57%max
24Qwen3.8 Max20.00%Not specified
24Qwen3.8 2.4T A95B20.00%Not specified
26Grok 4.619.71%xhigh
27GPT-6 Luna19.43%max
28GLM-5.319.14%max
29Claude Haiku 5.518.86%max
30Gemini 3.8 Flash18.29%high
31DeepSeek V4 Pro 081318.00%max
32Muse Spark 1.217.71%xhigh
32Grok 4.717.71%xhigh
32Qwen3.8 Max (0902)17.71%Not specified
32Gemini 3.1 Pro Preview17.71%Not specified
36Claude Sonnet 516.86%max
36GPT-5.3 Codex16.86%xhigh
38DeepSeek V4 Flash 073116.57%max
39GLM 5.3 Flash15.43%max
39Grok 4.515.43%high
41Muse Spark 1.115.14%xhigh
42DeepSeek V4.1 Flash14.29%max
42Gemini 3.7 Flash14.29%high
44JT-4.1 Flash 236B A21B14.00%Not specified
45Qwen3.7 Max13.43%Not specified
46Gemini 3.5 Flash13.14%high
47DeepSeek V4 Pro 042412.86%max
48Claude Opus 4.612.57%max
49MiMo-V2.6-Flash12.00%Not specified
49Claude Opus 4.712.00%max
51GPT-5.211.55%xhigh
52Muse Spark11.33%Not specified
53Qwen3.8-Flash-Next11.14%Not specified
54DeepSeek V4 Flash Vision10.86%max
55Gemini 3.6 Flash10.57%high
56GPT-5.4 mini10.00%xhigh
56Kimi K2.7 Code10.00%Not specified
58Quasar 438B9.43%max
59GPT-5.4 nano9.25%xhigh
60Grok Build 0.1 06169.14%Not specified
60Qwen3.7 Plus9.14%Not specified
60Gemini 3 Pro Preview9.14%high
63A.X-K28.86%Not specified
64GPT-5.2 Codex8.67%xhigh
65Nex-N2-Pro8.57%Not specified
66Inkling Small8.29%Not specified
67Kimi K2.68.00%Not specified
67Grok 4.38.00%high
69DeepSeek V3.2 Speciale7.43%Not specified
70DeepSeek V4 Flash 04207.14%max
71Motif 36.57%Not specified
72Grok 4.20 03096.00%Not specified
72Motif 3 (Beta)6.00%Not specified
74GPT-5.1 Codex5.71%high
74GPT-55.71%high
74Solar Open2 250B5.71%Not specified
77Inkling5.43%xhigh
77Solar Pro 45.43%Not specified
77Qwen3.8 27B5.43%xhigh
80GPT-5 Codex5.14%high
81GPT-5.14.86%high
81Hy34.86%Not specified
83Apodex 1.14.57%Not specified
83Claude Opus 4.54.57%Not specified
83GLM-5.14.57%Not specified
83Hy3-preview4.57%Not specified
83K2 Horizon 375B A23B4.57%Not specified
88MiMo-V2.5-Pro4.00%Not specified
89MiMo-V2.53.71%Not specified
89MiniMax-M33.71%Not specified
89Ring-2.6-1T3.71%xhigh
89Qwen3.6 Max Preview3.71%Not specified
93Claude Sonnet 4.63.14%max
93Nemotron 3 Ultra 550B A55B3.14%Not specified
93Nemotron 3 Super 120B A12B3.14%Not specified
93Kimi K2.53.14%Not specified
97MiMo-V2-Flash (Feb 2026)2.86%Not specified
97Grok 4.1 Fast2.86%Not specified
97Qwen3.6 Plus2.86%Not specified
100Gemini 2.5 Pro2.57%Not specified
100Muse Glimmer2.57%high
100LongCat 2.02.57%Not specified
100Ling-3.0-flash-Fin2.57%Not specified
100Kimi K2 Thinking2.57%Not specified
100GPT-5.5 Instant (May 2026)2.57%Not specified
106Step 3.5 Flash2.47%Not specified
107Step 3.5 Flash 26032.29%Not specified
107K2 Horizon MoVA 36B A4B2.29%Not specified
107Step 3.7 Flash2.29%high
110DeepSeek V3.12.00%Not specified
110GLM-52.00%Not specified
110Ling-3.0-flash-VL2.00%Not specified
110Grok 42.00%Not specified
114DeepSeek V3.1 Terminus1.71%Not specified
114Qwen3 Max Thinking1.71%Not specified
114Ling 3.0 Flash1.71%Not specified
114GLM-4.71.71%Not specified
114Qwen3.5 397B A17B1.71%Not specified
119DeepSeek V3.2 Exp1.43%Not specified
119gpt-oss-20b1.43%high
119Solar Mini 41.43%Not specified
119Gemini 3 Flash Preview1.43%Not specified
119ERNIE 5.0 Thinking Preview1.43%Not specified
119DeepSeek R1 0528 (May '25)1.43%Not specified
119Gemma 4 31B1.43%Not specified
126MiniMax-M2.51.14%Not specified
126o31.14%Not specified
126Gemini 2.5 Flash1.14%Not specified
126Gemini 3.1 Flash-Lite1.14%high
126GLM-4.61.14%Not specified
126Claude 4.5 Sonnet1.14%Not specified
126MiMo-V2-Omni1.14%Not specified
126Apriel-v1.5-15B-Thinker1.14%Not specified
126gpt-oss-120b1.14%high
135K-EXAONE1.12%Not specified
136Qwen3.5 122B A10B0.86%Not specified
136Claude 3.7 Sonnet0.86%Not specified
136K-EXAONE 2.0 08030.86%Not specified
136MiMo-V2-Omni-03270.86%Not specified
136HyperNova 60B 26050.86%high
136Qwen3.6 27B0.86%Not specified
136Qwen3.5 35B A3B0.86%Not specified
136Qwen3 Max (Preview)0.86%Not specified
136DeepSeek V3.20.86%Not specified
136Trinity Large Thinking0.86%Not specified
136MiniMax-M20.86%Not specified
147Qwen3.5 27B0.85%Not specified
147Mercury 20.85%high
149DeepSeek R1 (Jan '25)0.57%Not specified
149Ling-1T0.57%Not specified
149Qwen3.5 9B0.57%Not specified
149GLM 5V Turbo0.57%Not specified
149MiniMax-M2.70.57%Not specified
149o4-mini0.57%high
149Nemotron Cascade 2 30B A3B0.57%Not specified
149Ring-1T0.57%Not specified
149Grok 3 mini Reasoning0.57%high
158Qwen3.5 Omni Plus0.56%Not specified
159o3-mini0.29%high
159Granite 4.2 30B0.29%Not specified
159K2 Horizon 7B0.29%Not specified
159Mistral Small 40.29%Not specified
159GLM-5-Turbo0.29%Not specified
159North Mini Code0.29%Not specified
159MiniMax-M2.10.29%Not specified
159GLM-4.7-Flash0.29%Not specified
159Ling-2.6-1T0.29%Not specified
159Command A+0.29%Not specified
159INTELLECT-30.29%Not specified
159Doubao Seed Code0.29%Not specified
159EXAONE 4.5 33B0.29%Not specified
159Gemma 4 E4B0.29%Not specified
159o10.29%Not specified
159MiMo-V2-Pro0.29%Not specified
159Tri-21B-Think0.29%Not specified
159Qwen3 30B A3B 2507 (Reasoning)0.29%Not specified
159Nova 2.0 Lite0.29%high
159MiniCPM5-2B0.29%Not specified
159Grok 4.20 0309 v20.29%Not specified
159Magistral Medium 10.29%Not specified
159Magistral Small 1.20.29%Not specified
159Sarvam 30B0.29%high
159Hermes 4 - Llama-3.1 405B0.29%Not specified
159Magistral Medium 1.20.29%Not specified
159Apriel-v1.6-15B-Thinker0.29%Not specified
159Claude 4 Sonnet0.29%Not specified
159Granite 4.2 8B0.29%Not specified
159Qwen3 32B0.29%Not specified
159G9v3-39A5B0.29%Not specified
159DiffusionGemma 26B A4B0.29%Not specified
159Falcon-H1R-7B0.29%Not specified
159Qwen3 VL 8B (Reasoning)0.29%Not specified
193Ring-flash-2.00.28%Not specified
194GLM-4.5V0.00%Not specified
194JT-35B-Flash0.00%Not specified
194Nova 2.0 Omni0.00%medium
194Gemma 4 12B0.00%Not specified
194Claude 3.5 Haiku0.00%Not specified
194Gemma 3 270M0.00%Not specified
194Mistral Medium 3.10.00%Not specified
194GPT-5 nano0.00%high
194Qwen3 Coder 480B A35B Instruct0.00%Not specified
194Gemini 3.5 Flash-Lite0.00%high
194Qwen3.5 2B0.00%Not specified
194Devstral 20.00%Not specified
194Qwen3 Omni 30B A3B Instruct0.00%Not specified
194Granite 4.2 3B0.00%Not specified
194Nova 2.0 Pro Preview0.00%medium
194Qwen3 0.6B0.00%Not specified
194LFM2 1.2B0.00%Not specified
194Magistral Small 10.00%Not specified
194Llama 3.3 Nemotron Super 49B v10.00%Not specified
194GLM-4.50.00%Not specified
194Gemini 2.5 Flash-Lite0.00%Not specified
194Granite 4.0 Micro0.00%Not specified
194Claude 3 Haiku0.00%Not specified
194Mistral 7B Instruct0.00%Not specified
194Gemma 3 4B Instruct0.00%Not specified
194Hermes 4 - Llama-3.1 70B0.00%Not specified
194NVIDIA Nemotron 3 Nano 30B A3B0.00%Not specified
194Grok 30.00%Not specified
194Mistral Small 3.10.00%Not specified
194Qwen3 Next 80B A3B Instruct0.00%Not specified
194Llama Nemotron Super 49B v1.50.00%Not specified
194GPT-5 mini0.00%high
194Nemotron 3.5 Lightning0.00%Not specified
194Qwen3 4B 2507 (Reasoning)0.00%Not specified
194Qwen3 1.7B0.00%Not specified
194Gemma 3 27B Instruct0.00%Not specified
194Granite 3.3 8B0.00%Not specified
194Grok 4 Fast0.00%Not specified
194Gemma 3 12B Instruct0.00%Not specified
194NVIDIA Nemotron Nano 9B V20.00%Not specified
194EXAONE 4.0 32B0.00%Not specified
194G9v3-3B0.00%Not specified
194Qwen3.5 4B0.00%Not specified
194K2 Horizon 3.7B0.00%Not specified
194Qwen3 235B A22B 2507 Instruct0.00%Not specified
194Olmo 3 7B Think0.00%Not specified
194HyperCLOVA X SEED Think (32B)0.00%Not specified
194Nova Micro0.00%Not specified
194Sarvam M0.00%high
194Olmo 3.1 32B Think0.00%Not specified
194GPT-4.10.00%Not specified
194Command A0.00%Not specified
194LFM2.5-VL-1.6B0.00%Not specified
194Qwen3 VL 8B Instruct0.00%Not specified
194JT-MINI0.00%Not specified
194Ling-mini-2.00.00%Not specified
194Qwen3 30B A3B0.00%Not specified
194Molmo2-8B0.00%Not specified
194Qwen3.5 0.8B0.00%Not specified
194GLM-4.6V0.00%Not specified
194MiniCPM-V 4.6 1.3B0.00%Not specified
194Qwen3 235B A22B0.00%Not specified
194Mistral Small 3.20.00%Not specified
194DeepSeek V3 (Dec '24)0.00%Not specified
194Kimi K20.00%Not specified
194Solar Pro 20.00%high
194Qwen3 14B0.00%Not specified
194Sarvam 105B0.00%high
194Llama 3.1 Instruct 405B0.00%Not specified
194Llama 3.1 Instruct 70B0.00%Not specified
194LFM2.5-1.2B-Instruct0.00%Not specified
194Mistral Large 30.00%Not specified
194Apertus 70B Instruct0.00%Not specified
194DeepSeek R1 Distill Llama 70B0.00%Not specified
194Mistral Large 2 (Nov '24)0.00%Not specified
194Qwen3 VL 30B A3B Instruct0.00%Not specified
194Nova Pro0.00%Not specified
194Granite 4.0 350M0.00%Not specified
194Solar Open 100B0.00%high
194Mistral Medium 30.00%Not specified
194LFM2.5-1.2B-Thinking0.00%Not specified
194K2 Think V20.00%high
194Nova Lite0.00%Not specified
194Seed-OSS-36B-Instruct0.00%Not specified
194GLM-4.5-Air0.00%Not specified
194Granite 4.0 H Small0.00%Not specified
194Llama 3.2 Instruct 11B (Vision)0.00%Not specified
194DeepSeek R1 0528 Qwen3 8B0.00%Not specified
194LongCat Flash Lite0.00%Not specified
194Tri-21B-think Preview0.00%Not specified
194Kimi K2 09050.00%Not specified
194Motif-2-12.7B-Reasoning0.00%Not specified
194Qwen3.6 35B A3B0.00%Not specified
194Ministral 3 3B0.00%Not specified
194LFM2 8B A1B0.00%Not specified
194Qwen3 VL 32B Instruct0.00%Not specified
194Gemma 4 26B A4B0.00%Not specified
194NVIDIA Nemotron Nano 12B v2 VL0.00%Not specified
194Qwen3.5 Omni Flash0.00%Not specified
194Ministral 3 14B0.00%Not specified
194Gemini 2.5 Flash-Lite Preview (Sep '25)0.00%Not specified
194GPT-4.1 nano0.00%Not specified
194Llama 3.1 Nemotron Instruct 70B0.00%Not specified
194Llama 3.1 Instruct 8B0.00%Not specified
194LFM2 2.6B0.00%Not specified
194Gemma 4 E2B0.00%Not specified
194DeepSeek V3 03240.00%Not specified
194Mercury 2.50.00%Not specified
194MiniCPM5-1B0.00%Not specified
194Jamba 1.7 Large0.00%Not specified
194Qwen3 Max0.00%Not specified
194Nanbeige4.1-3B0.00%Not specified
194Ling 3.0 Tiny0.00%Not specified
194Qwen3 30B A3B 2507 Instruct0.00%Not specified
194Celeris-10.00%Not specified
194Llama 3.2 Instruct 1B0.00%Not specified
194Tiny Aya Global0.00%Not specified
194Exaone 4.0 1.2B0.00%Not specified
194Llama 3 Instruct 8B0.00%Not specified
194MiMo-V2-Flash0.00%Not specified
194Granite 4.1 8B0.00%Not specified
194Gemini 2.5 Flash Preview (Sep '25)0.00%Not specified
194Ling-flash-2.00.00%Not specified
194Apertus 8B Instruct0.00%Not specified
194GPT-4o (Aug '24)0.00%Not specified
194Nemotron 3 Nano Omni 30B A3B Reasoning0.00%Not specified
194Olmo 3 7B Instruct0.00%Not specified
194Granite 4.1 3B0.00%Not specified
194Ling 2.6 Flash0.00%Not specified
194Llama 4 Maverick0.00%Not specified
194Olmo 3.1 32B Instruct0.00%Not specified
194Ministral 3 8B0.00%Not specified
194Llama 3.3 Instruct 70B0.00%Not specified
194MiniMax M1 80k0.00%Not specified
194Qwen3 8B0.00%Not specified
194K2 Horizon 0.9B0.00%Not specified
194Devstral Small (Jul '25)0.00%Not specified
194NVIDIA Nemotron 3 Nano 4B0.00%Not specified
194GPT-4.1 mini0.00%Not specified
194Phi-4 Mini Instruct0.00%Not specified
194Grok Code Fast 10.00%Not specified
194Granite 4.0 1B0.00%Not specified
194K2-V20.00%high
194Claude 4.5 Haiku0.00%Not specified
194Granite 4.0 H 350M0.00%Not specified
194Gemma 3n E4B Instruct0.00%Not specified
194Qwen3 VL 235B A22B (Reasoning)0.00%Not specified
194Gemma 3n E2B Instruct0.00%Not specified
194Llama 3 Instruct 70B0.00%Not specified
194Devstral Medium0.00%Not specified
194Step3 VL 10B0.00%Not specified
194Llama 4 Scout0.00%Not specified
194ERNIE 4.5 300B A47B0.00%Not specified
194Qwen3 4B 2507 Instruct0.00%Not specified
194Jamba 1.7 Mini0.00%Not specified
194Qwen2.5 Instruct 72B0.00%Not specified
194Solar Pro 30.00%high
194Qwen3 Omni 30B A3B (Reasoning)0.00%Not specified
194Gemini 2.0 Flash (Feb '25)0.00%Not specified
194GPT-4o (Nov '24)0.00%Not specified
194GPT-5.5 Instant (June 2026)0.00%Not specified
194Olmo 3 32B Think0.00%Not specified
194Qwen3 Next 80B A3B (Reasoning)0.00%Not specified
194Qwen3 Max Thinking (Preview)0.00%Not specified
194Devstral Small (May '25)0.00%Not specified
194Claude 4.1 Opus0.00%Not specified
194Qwen3 VL 30B A3B (Reasoning)0.00%Not specified
194Devstral Small 20.00%Not specified
194Llama 3.1 Nemotron Ultra 253B v10.00%Not specified
194Gemma 3 1B Instruct0.00%Not specified
194Qwen3 VL 32B (Reasoning)0.00%Not specified
194Granite 4.1 30B0.00%Not specified
194Qwen3 VL 235B A22B Instruct0.00%Not specified
194Qwen3 Coder 30B A3B Instruct0.00%Not specified
194Granite 4.0 H 1B0.00%Not specified
194Reka Flash 30.00%Not specified
194Mistral Medium 3.50.00%high
194Mi:dm K 2.5 Pro0.00%Not specified
194Mi:dm K 2.5 Pro Preview0.00%Not specified
194LFM2.5-2.6B0.00%Not specified
194LFM2.5-8B-A1B0.00%Not specified
194Nova Premier0.00%Not specified
194Cogito v2.10.00%Not specified
194Phi-40.00%Not specified
194Qwen3 VL 4B Instruct0.00%Not specified
194Qwen3 235B A22B 2507 (Reasoning)0.00%Not specified
194Jamba Reasoning 3B0.00%Not specified
194Qwen3 VL 4B (Reasoning)0.00%Not specified
194KAT-Coder-Pro V10.00%Not specified
194LFM2 24B A2B0.00%Not specified
194KAT Coder Pro V20.00%Not specified
194Qwen3 Coder Next0.00%Not specified
194GPT-5.1 Codex mini0.00%high

No published result tracked

4 of the 46 models on this site have no published number on this benchmark, so their cells in the home table stay blank rather than reading as zero.

Read this with

Benchmarks in the same category, which usually disagree with this one on purpose.

As reported by each publisher

The same benchmark, measured by the labs themselves. Compare down a column, never across two publishers.

Reflection AI

Introducing Beam: model capability comparisons · 5 October 2026

RankModelScore
5Beam16.3%
7Inkling5.4%
8Nemotron 3 Ultra3.1%
2GLM 5.220.9%
4GLM 5.319.1%
1Kimi K323.4%
3Qwen3.8 Max20.0%
6DeepSeek V4.1 Flash14.3%

Ranks compare reported scores in the same published row; ties share a rank.

Reflection AI’s four capability tables, read 6 October 2026. All 21 rows and eight model columns are transcribed; NR means not reported and remains blank. The source’s Qwen 3.8 Max column uses this catalogue’s Qwen3.8 Max spelling. Beam results are publisher reports; Reflection AI cites Artificial Analysis and Datacurve for comparison-model evaluations. The announcement does not specify a common harness, effort or trial count for every row. Ranks are derived within each published row from reported values only, with ties sharing a rank; they do not establish an independent or overall model ranking. AutomationBench uses the public split; AA-Omniscience is an index on the public split only, calculated by Reflection AI. BrowseComp and DeepSearchQA use context management. AA-LCR has no stated version. These settings remain separate from other source fields.

reflection.ai/blog/introducing-beam

Sources

Every page used for this benchmark, read on 17 October 2026.