Overview Compare models
Compare models Compare exact releases using results from the same evaluator.
Model 1 Kimi K3.1 Pro Kimi K3.1 Pro Highspeed Step 5 Preview GPT-6 Luna GPT-5.6 Terra GLM 5.3 Flash Qwen3.8 2.4T A95B Qwen3.8-Flash-Next MiMo-V2.6-Flash Qwen3.8 Max (0902) Claude Fable 5 Grok 4.6 Claude Opus 4.8 Claude Opus 4.7 Gemini 3.7 Flash Muse Spark 1.2 GPT-5.4 Grok 4.5 GPT-5.5 DeepSeek V4.1 Flash 1017 GLM-Legal Claude Haiku 5.5 Beam Gemini 4 Argon Claude Sonnet 5.5 Claude Opus 5.5 Claude Fable 5.1 Claude Opus 5 Claude Sonnet 5 GPT-6 Astra GPT-6 Sol GPT-6.1 Sol GPT-5.6 Sol GPT-5.6 Luna Gemini 3.8 Flash Gemini 3.1 Pro Grok 4.7 DeepSeek V4.1 Flash DeepSeek V4 Pro Kimi K3 Qwen3.8 Max GLM 5.3 Muse Spark 1.3 Max MiniMax M3 Hunyuan Hy4 Preview MiMo V2.6 Pro Inkling Mistral Medium 3.5 Model 2 Kimi K3.1 Pro Kimi K3.1 Pro Highspeed Step 5 Preview GPT-6 Luna GPT-5.6 Terra GLM 5.3 Flash Qwen3.8 2.4T A95B Qwen3.8-Flash-Next MiMo-V2.6-Flash Qwen3.8 Max (0902) Claude Fable 5 Grok 4.6 Claude Opus 4.8 Claude Opus 4.7 Gemini 3.7 Flash Muse Spark 1.2 GPT-5.4 Grok 4.5 GPT-5.5 DeepSeek V4.1 Flash 1017 GLM-Legal Claude Haiku 5.5 Beam Gemini 4 Argon Claude Sonnet 5.5 Claude Opus 5.5 Claude Fable 5.1 Claude Opus 5 Claude Sonnet 5 GPT-6 Astra GPT-6 Sol GPT-6.1 Sol GPT-5.6 Sol GPT-5.6 Luna Gemini 3.8 Flash Gemini 3.1 Pro Grok 4.7 DeepSeek V4.1 Flash DeepSeek V4 Pro Kimi K3 Qwen3.8 Max GLM 5.3 Muse Spark 1.3 Max MiniMax M3 Hunyuan Hy4 Preview MiMo V2.6 Pro Inkling Mistral Medium 3.5
Selected benchmark Compare the two releases within one evaluator and metric.
Benchmark BenchCAD w/o tools BenchCAD w/ tools ProgramBench (binary threshold 95%) PostTrain Bench v1.2 Kimi Code Bench 2.0 (Internal) Kimi Code Bench 3.0 Long-horizon (Internal) HLE-Full w/ tools CharXiv (RQ) w/ python MMMU-Pro w/ python GDP.pdf w/ python ChartoGraphy w/ python BabyVision w/ python ZeroBench_main w/ python (pass@5) Agents' Last Exam (ALL) Agents' Last Exam (CLI) APEX-Agents OfficeQA Pro v2 OfficeQA Pro (full image corpus) Frontier Finance HANDBOOK.md OSWorld V2.1 Business Operation (Internal) DECK-Arena (Internal, Elo-score) Design Bench (Internal, Elo-score) Kimi Hallu Bench Hard w/ tools (Internal, lower is better) HLE-Diamond CharXiv (RQ) ZeroBench_main w/o tools (avg@5) PerceptionBench Kimi Hallu Bench w/o tools (Internal, lower is better) xdev-bench-hard (Internal) HLE-Diamond (text-only, without tools) ZeroBench_main w/o tools (pass@5) Internal Financial Modeling Internal Financial Research xdev-bench-hard (average success rate) Terminal-Bench 4.0 cost per task SWE-Bench Pro (version unspecified) SWE-bench Multimodal FrontierCode v1.1 (Extended) ProgramBench (hidden test pass rate) DRACO (Anthropic harness) WANDR (frozen index) BenchCAD Vision2Code (no tools) BenchCAD Vision2Code (with tools) OSWorld 2.1 (offline strict pass) OfficeQA OfficeQA Pro HealthBench (length-adjusted) HealthBench (raw) HealthBench Professional (length-adjusted) HealthBench Professional (raw) PhysicianBench Global MMLU MILU SpatialBench Verified SingleCellBench Morphology-to-molecule matching Medicinal chemistry (ADME) Protein design (sequence generation) Protein design (library ranking) Biomedical image analysis Protocols (troubleshooting) Protocols (understanding V2) APEX-Agents-AA τ³-Banking ITBench-AA IFBench AIME 2025 AA-AnalystAgent MLCR-AA ARC-AGI-3 SWE-Bench Pro V2 Full SWE-Bench Pro V2 HARD Vals RSI Index Vibe Code Bench 1–100 CUA-bench τ²-Bench Telecom Harvey LAB-AA EnterpriseOps-Gym-AA Humanity's Last Exam (text-only, without tools) Apex Bench v1.1 PLawBench SWE-Bench Pro v1 SWE Atlas Codebase QnA SWE-bench Multilingual AIME 2026 AutomationBench (public split) MCP Atlas BrowseComp (context management) DeepSearchQA (context management) AA-LCR (version not specified) LongBench v2 AA-Omniscience (public split) LABBench 2 RiemannBench GraphWalks (up to 128k) GraphWalks (256k to 1M) LVBench CWE-bench v1 AutomationBench-AA SciCode GDP.pdf CritPt AA-Omniscience AA-LCR v1.1 Terminal-Bench 4.0 Terminal-Bench 2.1 SWE-bench (Verified) LiveCodeBench GPQA Diamond MMLU Pro MMMU Pro Vibe Code Bench v1.1 Terminal-Bench Science 0.1 ProgramBench ProofBench v1.1 LegalBench Harvey's Legal Agent Benchmark SkillsBench MysteryMechanism Vals Multimodal Index Vals Index BioMysteryBench Code Migration CyberBench v1.1 EMB Finance Agent (v2) IOI Legal Research Bench MedCode MedScribe Public Benefits Bench v1.1 SAGE SRE Bench Tax Agent Bench OSWorld 2.0 OSWorld 2.1 AutomationBench CursorBench 4.0 DeepSWE v1.1 FrontierCode v1.1 GDPval-AA v2.1 AA-Briefcase v1.1 Chartography Humanity's Last Exam ARC-AGI-2 FrontierMath Tier 4 (v2) Agents' Last Exam Artificial Analysis Intelligence Index Terminal-Bench 3.0 NL2Repo-Bench CyberGym SEC-bench Pro ExploitGym MathArena Apex Codeforces (rating) BabyVision ZeroBench (main set) FrontierSWE SWE-Marathon PostTrainBench ExploitBench Toolathlon Verified GDPval-AA v2 Comparison group aa · Terminal-Bench 4.0 · Artificial Analysis vals · Terminal-Bench 4.0 · Vals AI self · Kimi K3.1 Pro: complete release evaluation · Terminal-Bench 4.0 self · Terminal-Bench 4.0: complete score/cost chart · Terminal-Bench 4.0 (cost chart, low) self · Terminal-Bench 4.0: complete score/cost chart · Terminal-Bench 4.0 (cost chart, medium) self · Terminal-Bench 4.0: complete score/cost chart · Terminal-Bench 4.0 (cost chart, high) self · Terminal-Bench 4.0: complete score/cost chart · Terminal-Bench 4.0 (cost chart, xhigh) self · Terminal-Bench 4.0: complete score/cost chart · Terminal-Bench 4.0 (cost chart, max) self · Claude Haiku 5.5: launch evaluation table · Terminal-Bench 4.0 self · Claude Haiku 5.5: system card capability evaluations · Terminal-Bench 4.0 self · DeepSeek-V4.1-Flash-1017: complete release comparison · Terminal-Bench 4.0 self · Gemini 4 Argon: model evaluation · Terminal-Bench 4.0 self · DeepSeek-V4.1-Flash: comparison with closed and open-source models · Terminal-Bench 4.0 (Pass@1) self · GPT-6 Astra: evaluation table · Terminal-Bench 4.0 self · Claude Opus 5.5: evaluation table · Terminal-Bench 4.0 self · Claude Sonnet 5.5: evaluation table · Terminal-Bench 4.0
aa % · Higher is better 1 Kimi K3.1 Pro 67.17% 2 Claude Opus 5.5 59.60% Bars start at zero. 2 reported results for the selected models; ranks apply to this selection only; ties share a rank. Published effort settings may vary. Source ↗
Across the tests Each row uses one evaluator and scale. Percentage differences are in percentage points (pp).
Export comparison