AIEvals

Models

GPT-5.6 Sol

OpenAI’s preceding Sol release, retained for comparison with GPT-6 and GPT-6.1.

OpenAIReleased 9 July 2026Context 1M tokensOutput 128k tokensPrivate weights$4.00 in / $20.00 out per Mtok

Family GPT-5.677 benchmarks with a published result2 independent first places among the 46 models this site listsRead 17 October 2026

Where it stands

Rank compares tracked models with a score. Published shows the source’s rank and field size, when available.

BenchmarkScoreRankPublishedLeaderBehind
Vals IndexVals AI58.01%10of 3310of 41Gemini 4 Argon10.89 pts
Artificial Analysis Intelligence IndexArtificial Analysis4711of 42no list readClaude Opus 5.511 pts
Terminal-Bench 4.0Artificial Analysis39.90%11of 40no list readClaude Sonnet 5.523.74 pts
SWE-bench (Verified)Vals AI96.20%3of 233of 88Claude Opus 50.80 pts
LiveCodeBenchVals AI82.60%17of 2152of 143Claude Fable 5.17.92 pts
GPQA DiamondVals AI95.20%2of 232of 138Gemini 3.1 Pro0.25 pts
MMLU ProVals AI89.10%11of 2416of 138Claude Fable 5.13.28 pts
MMMU ProVals AI88.84%6of 206of 93Claude Fable 5.11.80 pts
Vibe Code Bench v1.1Vals AI80.50%19of 3820of 106Claude Sonnet 5.511.89 pts
Terminal-Bench Science 0.1Artificial Analysis22.38%8of 24no list readGPT-6 Astra40.95 pts
ProgramBenchVals AI1.50%9of 299of 54Claude Opus 5.517.00 pts
τ³-BankingArtificial Analysis44.33%11of 30no list readQwen3.8 Max7.01 pts
ITBench-AAArtificial Analysis56.21%1of 18no list readGPT-5.6 Sol—
IFBenchArtificial Analysis72.65%5of 10no list readGrok 4 20 030910.28 pts
AA-AnalystAgentArtificial Analysis47.50%7of 18no list readGemini 3.7 Flash12.50 pts
MLCR-AAArtificial Analysis26.11%17of 37no list readClaude Sonnet 5.548.89 pts
ARC-AGI-3ARC Prize · Standard7.78%4of 11no list readGPT-6 Astra54.93 pts
SWE-Bench Pro V2 FullScale AI / SEAL95.50%6of 10no list readClaude Opus 53.90 pts
SWE-Bench Pro V2 HARDScale AI / SEAL82.40%8of 10no list readClaude Opus 515.60 pts
Vals RSI IndexVals AI23.88%10of 2010of 23Claude Opus 5.513.43 pts
Vibe Code Bench 1–100Vals AI20.01%7of 207of 20Claude Opus 5.510.35 pts
CUA-benchVals AI8.33%5of 85of 8GPT-6 Astra10.84 pts
τ²-Bench TelecomArtificial Analysis85.09%10of 10no list readJt 35b Flash14.03 pts
Harvey LAB-AAArtificial Analysis87.18%16of 20no list readMuse Spark 1.3 Max8.30 pts
EnterpriseOps-Gym-AAArtificial Analysis42.91%13of 22no list readClaude Fable 58.21 pts
AutomationBench-AAArtificial Analysis60.08%14of 40no list readGemini 4 Argon17.43 pts
SciCodeArtificial Analysis57.06%16of 40no list readClaude Opus 5.59.84 pts
GDP.pdfArtificial Analysis27.20%3of 40no list readGPT-6.1 Sol3.80 pts
CritPtArtificial Analysis32.29%1of 42no list readGPT-5.6 Sol—
AA-OmniscienceArtificial Analysis21.9720of 42no list readClaude Opus 5.524.45
AA-LCR v1.1Artificial Analysis84.00%7of 42no list readKimi K34.67 pts
Terminal-Bench 2.1Vals AI85.77%3of 353of 75Claude Opus 5.51.87 pts
ProofBench v1.1Vals AI83.00%10of 3412of 46Claude Sonnet 5.517.00 pts
LegalBenchVals AI86.97%7of 278of 149Claude Fable 51.59 pts
Harvey's Legal Agent BenchmarkVals AI2.50%31of 3740of 73Muse Spark 1.222.92 pts
SkillsBenchVals AI54.10%12of 2015of 35DeepSeek V4.1 Flash15.70 pts
MysteryMechanismVals AI33.33%10of 2110of 21GPT-6 Astra19.82 pts
Vals Multimodal IndexVals AI72.64%4of 174of 33Claude Fable 51.51 pts
BioMysteryBenchVals AI71.11%10of 1910of 21Claude Sonnet 5.510.00 pts
Code MigrationVals AI52.92%10of 3710of 71Claude Sonnet 5.516.91 pts
CyberBench v1.1Vals AI76.31%3of 313of 43GPT-6 Sol1.67 pts
EMBVals AI72.34%7of 377of 69Claude Fable 5.14.33 pts
Finance Agent (v2)Vals AI53.76%21of 3726of 73Gemini 4 Argon11.64 pts
IOIVals AI91.17%5of 315of 38Gemini 4 Argon8.83 pts
Legal Research BenchVals AI48.08%8of 378of 72Muse Spark 1.3 Max7.21 pts
MedCodeVals AI43.97%24of 3541of 104Claude Opus 519.60 pts
MedScribeVals AI85.23%23of 3629of 106Claude Opus 5.56.20 pts
Public Benefits Bench v1.1Vals AI66.51%16of 2916of 45Claude Opus 510.42 pts
SAGEVals AI52.56%5of 326of 90Claude Opus 4.73.54 pts
SRE BenchVals AI30.53%5of 155of 16GPT-6 Astra26.34 pts
Tax Agent BenchVals AI67.95%11of 3713of 64Claude Fable 5.19.69 pts
DeepSWE v1.1Datacurve72.67%4of 204of 28GPT-6 Astra1.45 pts
GDPval-AA v2.1Artificial Analysis1,587.920of 42no list readClaude Opus 5.5258.27
AA-Briefcase v1.1Artificial Analysis1,487.4219of 41no list readClaude Opus 5.5334.43
Humanity's Last ExamArtificial Analysis49.49%9of 42no list readClaude Opus 5.511.86 pts

Nearby on Vals Index

The two models above and below on the Vals Index (Vals AI), on every column where this model has a published number.

ModelVals Index%Artificial Analysis Intelligence IndexindexTerminal-Bench 4.0%SWE-bench%LiveCodeBench%GPQA Diamond%MMLU Pro%MMMU Pro%Vibe Code Bench v1.1%Terminal-Bench Science 0.1%ProgramBench%
GPT-6.1 SolOpenAI61.15%5256.06%–––––88.93%58.10%–
Muse Spark 1.3 MaxMeta58.16%4833.33%–––––85.86%10.95%–
GPT-5.6 SolOpenAI58.01%4739.90%96.20%82.60%95.20%89.10%88.84%80.50%22.38%1.50%
GPT-6 SolOpenAI57.54%4843.94%–––––87.82%30.00%2.00%
MiMo V2.6 ProXiaomi55.20%4634.85%–––––85.22%5.71%0.50%
Bold marks a figure better than this model’s on the same column, whichever model it belongs to. Ranks cover the models shown from the stated source.

Cost and speed, as published

Additional measurements from Artificial Analysis’ own row. Read 30 September 2026.

As publishedCost per taskOutput speed
GPT-5.6 Sol (max)$1.9980.5 tok/s

Results by benchmark

Independent and vendor results retain their own harnesses and effort settings.

BenchmarkIndependent runAs reported by a publisher
τ³-BankingTool use44.33%Artificial Analysisnot published
ITBench-AAIT operations56.21%Artificial Analysisnot published
IFBenchInstruction following72.65%Artificial Analysisnot published
AA-AnalystAgentData analysis47.50%Artificial Analysisnot published
MLCR-AAMedical reasoning26.11%Artificial Analysisnot published
ARC-AGI-3Interactive reasoning7.78%ARC Prize · Standardnot published
SWE-Bench Pro V2 FullRepository coding95.50%Scale AI / SEALnot published
SWE-Bench Pro V2 HARDRepository coding82.40%Scale AI / SEALnot published
Vals RSI IndexAutonomous research23.88%Vals AInot published
Vibe Code Bench 1–100Application development20.01%Vals AInot published
CUA-benchComputer use8.33%Vals AInot published
τ²-Bench TelecomTool use85.09%Artificial Analysisnot published
Harvey LAB-AALegal reasoning87.18%Artificial Analysisnot published
EnterpriseOps-Gym-AAEnterprise workflows42.91%Artificial Analysisnot published
Apex Bench v1.1Reasoningno published result
54.3%Apex Bench v1.1DeepSeek · 17 October 2026Rank 5 of 6 in this publisher table
AutomationBench-AAWorkflow automation60.08%Artificial Analysisnot published
SciCodeScientific coding57.06%Artificial Analysisnot published
GDP.pdfDocument reasoning27.20%Artificial Analysisnot published
CritPtScientific reasoning32.29%Artificial Analysisnot published
AA-OmniscienceFactual knowledge21.97Artificial Analysisnot published
AA-LCR v1.1Long-context reasoning84.00%Artificial Analysisnot published
Terminal-Bench 4.0Agentic coding39.90%Artificial Analysis
37.88%Vals AI
39.9%Terminal-Bench 4.0DeepSeek · 17 October 2026Rank 3 of 6 in this publisher table
39.9%Terminal-Bench 4.0 (Pass@1)DeepSeek · 10 September 2026Rank 2 of 7 in this publisher table
37.3%Terminal-Bench 4.0OpenAI · 3 September 2026Rank 5 of 6 in this publisher table
37.3%Terminal-Bench 4.0Anthropic · 22 September 2026Rank 5 of 5 in this publisher table
Terminal-Bench 2.1Agentic coding85.77%Vals AI
88.01%Artificial Analysis
88.8%Terminal Bench 2.1Z.ai · 14 August 2026Rank 1 of 8 in this publisher table
88.8%Terminal-Bench 2.1 (Pass@1)DeepSeek · 10 September 2026Rank 3 of 7 in this publisher table
SWE-bench (Verified)Software engineering96.20%Vals AInot published
LiveCodeBenchCode generation82.60%Vals AInot published
GPQA DiamondScience reasoning95.20%Vals AI
94.14%Artificial Analysis
94.1%GPQA Diamond (Pass@1)DeepSeek · 10 September 2026Rank 1 of 7 in this publisher table
94.6%GPQA DiamondOpenAI · 3 September 2026Rank 3 of 6 in this publisher table
MMLU ProKnowledge89.10%Vals AInot published
MMMU ProMultimodal reasoning88.84%Vals AI
83.41%Artificial Analysis
not published
Vibe Code Bench v1.1Product coding80.50%Vals AInot published
Terminal-Bench Science 0.1Agentic science22.38%Artificial Analysis
20.00%Vals AI
22.4%Terminal-Bench Science 0.1OpenAI · 3 September 2026Rank 4 of 5 in this publisher table
22.4%Terminal-Bench-Science 0.1Anthropic · 22 September 2026Rank 5 of 5 in this publisher table
ProgramBenchAgentic coding1.50%Vals AI
23.0%ProgramBenchDeepSeek · 17 October 2026Rank 3 of 6 in this publisher table
23.0%ProgramBench (Almost Solved)Z.ai · 14 August 2026Rank 2 of 7 in this publisher table
23.0%ProgramBench (Almost@1)DeepSeek · 10 September 2026Rank 2 of 6 in this publisher table
ProofBench v1.1Mathematics83.00%Vals AInot published
LegalBenchProfessional knowledge86.97%Vals AInot published
Harvey's Legal Agent BenchmarkProfessional knowledge2.50%Vals AInot published
SkillsBenchAgentic tooling54.10%Vals AInot published
MysteryMechanismScientific reasoning33.33%Vals AInot published
Vals Multimodal IndexComposite72.64%Vals AInot published
Vals IndexComposite58.01%Vals AInot published
BioMysteryBenchAgentic science71.11%Vals AInot published
Code MigrationAgentic coding52.92%Vals AInot published
CyberBench v1.1Security76.31%Vals AInot published
EMBFinancial modelling72.34%Vals AInot published
Finance Agent (v2)Financial analysis53.76%Vals AInot published
IOICompetitive programming91.17%Vals AInot published
Legal Research BenchProfessional knowledge48.08%Vals AInot published
MedCodeHealthcare administration43.97%Vals AInot published
MedScribeHealthcare administration85.23%Vals AInot published
Public Benefits Bench v1.1Public services66.51%Vals AInot published
SAGEEducation52.56%Vals AInot published
SRE BenchSecurity30.53%Vals AInot published
Tax Agent BenchProfessional knowledge67.95%Vals AInot published
OSWorld 2.0Computer useno published result
65.7%OSWorld 2.0 (offline set, partial reward)OpenAI · 3 September 2026Rank 3 of 3 in this publisher table
AutomationBenchBusiness workflowsno published result
45.8%AutomationBench (v1.0.6)DeepSeek · 17 October 2026Rank 6 of 6 in this publisher table
45.8%AutomationBench (v1.0.6)Z.ai · 14 August 2026Rank 4 of 8 in this publisher table
45.8%AutomationBench (Pass@1)DeepSeek · 10 September 2026Rank 5 of 7 in this publisher table
18.1%AutomationBench 1.0.6OpenAI · 3 September 2026Rank 4 of 5 in this publisher table
28.8%AutomationBenchAnthropic · 22 September 2026Rank 4 of 5 in this publisher table
CursorBench 4.0Agentic codingno published result
41.7%CursorBench 4.0Anthropic · 22 September 2026Rank 4 of 4 in this publisher table
DeepSWE v1.1Software engineering72.67%Datacurve
73.0%DeepSWE v1.1DeepSeek · 17 October 2026Rank 4 of 6 in this publisher table
72.7%DeepSWE (v1.1)Z.ai · 14 August 2026Rank 1 of 8 in this publisher table
73.0%DeepSWE v1.1 (Resolved)DeepSeek · 10 September 2026Rank 3 of 7 in this publisher table
72.7%DeepSWE v1.1OpenAI · 3 September 2026Rank 4 of 6 in this publisher table
FrontierCode v1.1Agentic codingno published result
47.5FrontierCode 1.1 MainOpenAI · 3 September 2026Rank 5 of 6 in this publisher table
47.5%FrontierCode v1.1 (Main)Anthropic · 22 September 2026Rank 5 of 5 in this publisher table
GDPval-AA v2.1Knowledge work1,587.9Artificial Analysis
1,588GDPval-AA v2.1Anthropic · 22 September 2026Rank 4 of 5 in this publisher table
AA-Briefcase v1.1Knowledge work1,487.42Artificial Analysisnot published
ChartographyVisual chart recognitionno published result
79.9%Chartography (w/tools)DeepSeek · 17 October 2026Rank 3 of 5 in this publisher table
79.9%Chartography w/ tools (Pass@1)DeepSeek · 10 September 2026Rank 2 of 4 in this publisher table
Humanity's Last ExamMultidisciplinary reasoning49.49%Artificial Analysis
44.5%HLE (w/o tools)DeepSeek · 17 October 2026Rank 2 of 5 in this publisher table
64.5%HLE w/ ToolsZ.ai · 14 August 2026Rank 1 of 8 in this publisher table
44.5%HLE (Pass@1)DeepSeek · 10 September 2026Rank 2 of 7 in this publisher table
ARC-AGI-2Abstract reasoningno published result
92.5%ARC-AGI-2OpenAI · 3 September 2026Rank 2 of 5 in this publisher table
FrontierMath Tier 4 (v2)Mathematicsno published result
83.0%FrontierMath Tier 4 (v2)OpenAI · 3 September 2026Rank 4 of 5 in this publisher table
Agents' Last ExamComputer useno published result
26.7%Agents' Last ExamDeepSeek · 17 October 2026Rank 6 of 6 in this publisher table
28.6%Agents' Last Exam (ALE-CLI)Z.ai · 14 August 2026Rank 1 of 8 in this publisher table
26.7%Agents' Last Exam (Pass@1)DeepSeek · 10 September 2026Rank 5 of 7 in this publisher table
53.6%Agents' Last ExamOpenAI · 3 September 2026Rank 3 of 4 in this publisher table
Artificial Analysis Intelligence IndexComposite47Artificial Analysisnot published
Terminal-Bench 3.0Agentic codingno published result
34.6%Terminal Bench 3.0Z.ai · 14 August 2026Rank 1 of 6 in this publisher table
34.4%Terminal-Bench 3.0 (Pass@1)DeepSeek · 10 September 2026Rank 2 of 7 in this publisher table
NL2Repo-BenchAgentic codingno published result
56.8%NL2Repo-BenchDeepSeek · 17 October 2026Rank 6 of 6 in this publisher table
56.8%NL2Repo-Bench (Score)DeepSeek · 10 September 2026Rank 6 of 7 in this publisher table
CyberGymSecurityno published result
83.6%CyberGymZ.ai · 14 August 2026Rank 3 of 8 in this publisher table
84.5%CyberGym (Pass@1)DeepSeek · 10 September 2026Rank 2 of 6 in this publisher table
SEC-bench ProSecurityno published result
74.3%SEC-Bench Pro (Pass@1)DeepSeek · 10 September 2026Rank 1 of 4 in this publisher table
ExploitGymSecurityno published result
216ExploitGym (2h)Z.ai · 14 August 2026Rank 1 of 7 in this publisher table
293ExploitGym (6h)Z.ai · 14 August 2026Rank 1 of 7 in this publisher table
33.7%ExploitGym (Pass@1)DeepSeek · 10 September 2026Rank 1 of 6 in this publisher table
BabyVisionMultimodalno published result
88.9%BabyVision (w/tools)DeepSeek · 17 October 2026Rank 4 of 5 in this publisher table
88.9%BabyVision w/tools (Pass@1)DeepSeek · 10 September 2026Rank 3 of 4 in this publisher table
ZeroBench (main set)Multimodalno published result
53.0%ZeroBench-main (w/tools)DeepSeek · 17 October 2026Rank 1 of 5 in this publisher table
53.0%ZeroBench-main w/tools (Pass@5)DeepSeek · 10 September 2026Rank 1 of 4 in this publisher table
FrontierSWEAgentic researchno published result
32.2%FrontierSWE v2Anthropic · 7 October 2026Rank 3 of 3 in this publisher table
SWE-MarathonAgentic codingno published result
42.5%SWE-Marathon (v1.1)Z.ai · 14 August 2026Rank 3 of 6 in this publisher table
PostTrainBenchAI researchno published result
36.2%PostTrainBenchZ.ai · 14 August 2026Rank 3 of 6 in this publisher table
ExploitBenchSecurityno published result
76.5%ExploitBenchZ.ai · 14 August 2026Rank 2 of 7 in this publisher table
Toolathlon VerifiedAgentic toolingno published result
74.9%Toolathlon VerifiedZ.ai · 14 August 2026Rank 3 of 8 in this publisher table
GDPval-AA v2Knowledge workno published result
1,730GDPval-AA v2Z.ai · 14 August 2026Rank 4 of 8 in this publisher table
Independent column: the suite named under each figure, one harness per suite. Publisher column: each lab’s own table, reproduced on the benchmark’s page. Publisher ranks compare reported scores in the same table row only; ties share a rank and unreported cells are excluded.

Sources

Read on 17 October 2026.