AIEvals

Models

Qwen3.8 Max

Alibaba’s August Qwen3.8 Max release, kept separate from the September 0902 revision.

AlibabaReleased 3 August 2026Context 1M tokensOutput 128k tokensOpen weights$2.00 in / $6.00 out per Mtok

Family Qwen 3.857 benchmarks with a published result1 independent first place among the 46 models this site listsRead 17 October 2026

Where it stands

Rank compares tracked models with a score. Published shows the source’s rank and field size, when available.

BenchmarkScoreRankPublishedLeaderBehind
Vals IndexVals AI48.27%28of 3329of 41Gemini 4 Argon20.63 pts
Artificial Analysis Intelligence IndexArtificial Analysis4025of 42no list readClaude Opus 5.518 pts
Terminal-Bench 4.0Artificial Analysis18.69%25of 40no list readClaude Sonnet 5.544.95 pts
SWE-bench (Verified)Vals AI85.60%14of 2317of 88Claude Opus 511.40 pts
LiveCodeBenchVals AI87.85%8of 2110of 143Claude Fable 5.12.67 pts
GPQA DiamondVals AI93.69%6of 236of 138Gemini 3.1 Pro1.76 pts
MMLU ProVals AI88.60%12of 2418of 138Claude Fable 5.13.78 pts
MMMU ProVals AI88.04%10of 2012of 93Claude Fable 5.12.60 pts
Vibe Code Bench v1.1Vals AI64.70%32of 3839of 106Claude Sonnet 5.527.69 pts
ProgramBenchVals AI0.00%19of 2923of 54Claude Opus 5.518.50 pts
τ³-BankingArtificial Analysis51.34%1of 30no list readQwen3.8 Max—
MLCR-AAArtificial Analysis19.44%21of 37no list readClaude Sonnet 5.555.56 pts
Vals RSI IndexVals AI21.76%12of 2012of 23Claude Opus 5.515.55 pts
Vibe Code Bench 1–100Vals AI12.83%17of 2017of 20Claude Opus 5.517.53 pts
AutomationBench-AAArtificial Analysis49.23%31of 40no list readGemini 4 Argon28.28 pts
SciCodeArtificial Analysis53.24%30of 40no list readClaude Opus 5.513.66 pts
GDP.pdfArtificial Analysis20.20%22of 40no list readGPT-6.1 Sol10.80 pts
CritPtArtificial Analysis20.00%20of 42no list readGPT-5.6 Sol12.29 pts
AA-OmniscienceArtificial Analysis3.432of 42no list readClaude Opus 5.543.02
AA-LCR v1.1Artificial Analysis78.33%37of 42no list readKimi K310.34 pts
Terminal-Bench 2.1Vals AI67.42%29of 3535of 75Claude Opus 5.520.22 pts
ProofBench v1.1Vals AI58.00%19of 3421of 46Claude Sonnet 5.542.00 pts
LegalBenchVals AI83.61%23of 2749of 149Claude Fable 54.95 pts
Harvey's Legal Agent BenchmarkVals AI10.42%11of 3714of 73Muse Spark 1.215.00 pts
SkillsBenchVals AI42.01%18of 2028of 35DeepSeek V4.1 Flash27.79 pts
MysteryMechanismVals AI23.87%14of 2114of 21GPT-6 Astra29.28 pts
Vals Multimodal IndexVals AI65.39%11of 1712of 33Claude Fable 58.76 pts
Code MigrationVals AI23.96%31of 3743of 71Claude Sonnet 5.545.87 pts
CyberBench v1.1Vals AI28.57%31of 3143of 43GPT-6 Sol49.41 pts
EMBVals AI60.07%26of 3732of 69Claude Fable 5.116.60 pts
Finance Agent (v2)Vals AI50.59%29of 3736of 73Gemini 4 Argon14.81 pts
IOIVals AI68.89%11of 3111of 38Gemini 4 Argon31.11 pts
Legal Research BenchVals AI47.60%11of 3711of 72Muse Spark 1.3 Max7.69 pts
MedCodeVals AI40.67%34of 3559of 104Claude Opus 522.90 pts
MedScribeVals AI84.95%24of 3631of 106Claude Opus 5.56.48 pts
Public Benefits Bench v1.1Vals AI67.12%14of 2914of 45Claude Opus 59.81 pts
SAGEVals AI51.25%9of 3214of 90Claude Opus 4.74.85 pts
Tax Agent BenchVals AI65.96%13of 3715of 64Claude Fable 5.111.68 pts
DeepSWE v1.1Datacurve57.46%16of 2016of 28GPT-6 Astra16.66 pts
GDPval-AA v2.1Artificial Analysis1,595.5318of 42no list readClaude Opus 5.5250.64
AA-Briefcase v1.1Artificial Analysis1,388.8125of 41no list readClaude Opus 5.5433.04
Humanity's Last ExamArtificial Analysis43.05%26of 42no list readClaude Opus 5.518.30 pts

Nearby on Vals Index

The two models above and below on the Vals Index (Vals AI), on every column where this model has a published number.

ModelVals Index%Artificial Analysis Intelligence IndexindexTerminal-Bench 4.0%SWE-bench%LiveCodeBench%GPQA Diamond%MMLU Pro%MMMU Pro%Vibe Code Bench v1.1%ProgramBench%
Hunyuan Hy4 PreviewTencent49.94%–––––––77.48%0.00%
Muse Spark 1.2Meta49.29%407.07%86.60%––88.28%86.13%79.10%–
Qwen3.8 MaxAlibaba48.27%4018.69%85.60%87.85%93.69%88.60%88.04%64.70%0.00%
DeepSeek V4 ProDeepSeek47.63%3614.14%96.40%87.53%92.42%86.97%–82.30%0.00%
Grok 4.5xAI44.72%3910.61%86.60%87.35%92.93%89.22%61.79%69.00%0.00%
Bold marks a figure better than this model’s on the same column, whichever model it belongs to. Ranks cover the models shown from the stated source.

Cost and speed, as published

Additional measurements from Artificial Analysis’ own row. Read 30 September 2026.

As publishedCost per taskOutput speed
Qwen3.8 Max$2.6738.6 tok/s

Results by benchmark

Independent and vendor results retain their own harnesses and effort settings.

BenchmarkIndependent runAs reported by a publisher
τ³-BankingTool use51.34%Artificial Analysis
55.2%tau3 bankingReflection AI · 5 October 2026Rank 1 of 6 in this publisher table
IFBenchInstruction followingno published result
82.8%IFBenchReflection AI · 5 October 2026Rank 1 of 5 in this publisher table
MLCR-AAMedical reasoning19.44%Artificial Analysisnot published
Vals RSI IndexAutonomous research21.76%Vals AInot published
Vibe Code Bench 1–100Application development12.83%Vals AInot published
SWE-Bench Pro v1Repository codingno published result
67.7%SWE Bench Pro v1Reflection AI · 5 October 2026Rank 1 of 5 in this publisher table
AutomationBench (public split)Business workflowsno published result
39.8%AutomationBench publicReflection AI · 5 October 2026Rank 4 of 6 in this publisher table
MCP AtlasAgentic toolingno published result
84.5%MCP AtlasReflection AI · 5 October 2026Rank 1 of 7 in this publisher table
AA-LCR (version not specified)Long-context reasoningno published result
80.3%AA-LCRReflection AI · 5 October 2026Rank 3 of 8 in this publisher table
LongBench v2Long-context reasoningno published result
66.3%LongBench v2Reflection AI · 5 October 2026Rank 1 of 4 in this publisher table
AutomationBench-AAWorkflow automation49.23%Artificial Analysisnot published
SciCodeScientific coding53.24%Artificial Analysis
52.1%SciCodeReflection AI · 5 October 2026Rank 3 of 7 in this publisher table
GDP.pdfDocument reasoning20.20%Artificial Analysisnot published
CritPtScientific reasoning20.00%Artificial Analysis
20.0%CriPT AAReflection AI · 5 October 2026Rank 3 of 8 in this publisher table
AA-OmniscienceFactual knowledge3.4Artificial Analysisnot published
AA-LCR v1.1Long-context reasoning78.33%Artificial Analysisnot published
Terminal-Bench 4.0Agentic coding18.69%Artificial Analysis
34.34%Vals AI
not published
Terminal-Bench 2.1Agentic coding67.42%Vals AI
81.27%Artificial Analysis
86.6%Terminal Bench v2.1Reflection AI · 5 October 2026Rank 4 of 8 in this publisher table
86.6%Terminal Bench 2.1Z.ai · 14 August 2026Rank 6 of 8 in this publisher table
SWE-bench (Verified)Software engineering85.60%Vals AInot published
LiveCodeBenchCode generation87.85%Vals AInot published
GPQA DiamondScience reasoning93.69%Vals AI
92.73%Artificial Analysis
92.6%GPQA DiamondReflection AI · 5 October 2026Rank 2 of 8 in this publisher table
MMLU ProKnowledge88.60%Vals AInot published
MMMU ProMultimodal reasoning88.04%Vals AI
82.31%Artificial Analysis
not published
Vibe Code Bench v1.1Product coding64.70%Vals AInot published
Terminal-Bench Science 0.1Agentic science
1.43%Vals AI
not published
ProgramBenchAgentic coding0.00%Vals AI
10.5%ProgramBench (Almost Solved)Z.ai · 14 August 2026Rank 6 of 7 in this publisher table
ProofBench v1.1Mathematics58.00%Vals AInot published
LegalBenchProfessional knowledge83.61%Vals AInot published
Harvey's Legal Agent BenchmarkProfessional knowledge10.42%Vals AInot published
SkillsBenchAgentic tooling42.01%Vals AInot published
MysteryMechanismScientific reasoning23.87%Vals AInot published
Vals Multimodal IndexComposite65.39%Vals AInot published
Vals IndexComposite48.27%Vals AInot published
Code MigrationAgentic coding23.96%Vals AInot published
CyberBench v1.1Security28.57%Vals AInot published
EMBFinancial modelling60.07%Vals AInot published
Finance Agent (v2)Financial analysis50.59%Vals AInot published
IOICompetitive programming68.89%Vals AInot published
Legal Research BenchProfessional knowledge47.60%Vals AInot published
MedCodeHealthcare administration40.67%Vals AInot published
MedScribeHealthcare administration84.95%Vals AInot published
Public Benefits Bench v1.1Public services67.12%Vals AInot published
SAGEEducation51.25%Vals AInot published
Tax Agent BenchProfessional knowledge65.96%Vals AInot published
AutomationBenchBusiness workflowsno published result
39.8%AutomationBench (v1.0.6)Z.ai · 14 August 2026Rank 7 of 8 in this publisher table
DeepSWE v1.1Software engineering57.46%Datacurve
51.0%DeepSWE v1.1Reflection AI · 5 October 2026Rank 4 of 6 in this publisher table
56.6%DeepSWE (v1.1)Z.ai · 14 August 2026Rank 7 of 8 in this publisher table
GDPval-AA v2.1Knowledge work1,595.53Artificial Analysisnot published
AA-Briefcase v1.1Knowledge work1,388.81Artificial Analysisnot published
Humanity's Last ExamMultidisciplinary reasoning43.05%Artificial Analysis
43.6%HLE no toolsReflection AI · 5 October 2026Rank 2 of 8 in this publisher table
56.2%HLE w/ ToolsZ.ai · 14 August 2026Rank 7 of 8 in this publisher table
Agents' Last ExamComputer useno published result
27.0%Agents' Last Exam (ALE-CLI)Z.ai · 14 August 2026Rank 4 of 8 in this publisher table
Artificial Analysis Intelligence IndexComposite40Artificial Analysisnot published
NL2Repo-BenchAgentic codingno published result
55.9%NL2RepoZ.ai · 14 August 2026Rank 5 of 6 in this publisher table
CyberGymSecurityno published result
78.5%CyberGymZ.ai · 14 August 2026Rank 6 of 8 in this publisher table
ExploitGymSecurityno published result
14ExploitGym (2h)Z.ai · 14 August 2026Rank 7 of 7 in this publisher table
26ExploitGym (6h)Z.ai · 14 August 2026Rank 7 of 7 in this publisher table
ExploitBenchSecurityno published result
28.8%ExploitBenchZ.ai · 14 August 2026Rank 6 of 7 in this publisher table
Toolathlon VerifiedAgentic toolingno published result
72.5%Toolathlon VerifiedZ.ai · 14 August 2026Rank 7 of 8 in this publisher table
GDPval-AA v2Knowledge workno published result
1,739GDPval-AA v2Z.ai · 14 August 2026Rank 3 of 8 in this publisher table
Independent column: the suite named under each figure, one harness per suite. Publisher column: each lab’s own table, reproduced on the benchmark’s page. Publisher ranks compare reported scores in the same table row only; ties share a rank and unreported cells are excluded.

Source notes

From the linked sources. Read 30 September 2026.

Weight classification. Vals AI labels this release open weights; Artificial Analysis labels it proprietary. The weight badge follows Vals AI’s model profile, and the conflicting classification is retained here.

Sources

Read on 17 October 2026.