AutomationBench-AA
Multi-step SaaS workflows, scored with partial credit by Artificial Analysis.
Workflow automationReported independently by Artificial Analysis140 models run by Artificial AnalysisUpdated 8 October 2026
How it is run
AA evaluates agent workflows using tools across business applications. The score is its partial-credit metric, not the full-pass metric from vendor AutomationBench 1.0.6 tables. Highest published effort per release; the table retains its snapshot date.
What it cannot tell you
Effort settings and evaluation versions matter. These source-specific results are kept separate from Vals AI and vendor reports. A missing score stays blank.
- Leader
- Gemini 4 Argon77.51% · Artificial Analysis results, grouped here by release
- Top-ten spread
- 15.31%Leader minus tenth place, or last place in smaller fields.
- Median
- 12.04%of the 140 rows published
- No published result
- 6of 46 models on this site
Artificial Analysis results by release
Highest published effort per model release, grouped here from AA’s public results. Grouping ranks are derived here; source settings and model-publisher reports stay separate.
| # | Model | Score | Effort |
|---|---|---|---|
| 1 | Gemini 4 Argon | 77.51% | high |
| 2 | Claude Sonnet 5.5 | 71.33% | max |
| 3 | Claude Opus 5.5 | 69.54% | max |
| 4 | DeepSeek V4.1 Flash | 68.89% | max |
| 5 | GPT-6 Astra | 68.49% | max |
| 6 | Grok 4.6 | 67.00% | xhigh |
| 7 | Grok 4.7 | 65.56% | xhigh |
| 8 | GPT-6.1 Sol | 64.87% | max |
| 9 | MiMo-V2.6-Flash | 64.05% | Not specified |
| 10 | GLM-5.3 | 62.20% | max |
| 11 | Gemini 3.7 Flash | 62.03% | high |
| 12 | GPT-6 Sol | 61.63% | max |
| 13 | GLM 5.3 Flash | 60.37% | max |
| 14 | GPT-5.6 Sol | 60.08% | max |
| 15 | Gemini 3.8 Flash | 59.93% | high |
| 16 | GPT-5.6 Terra | 59.65% | max |
| 17 | Claude Fable 5.1 | 59.38% | max |
| 18 | MiMo-V2.6-Pro | 58.62% | Not specified |
| 19 | Kimi K3 | 58.27% | max |
| 20 | Grok 4.5 | 57.93% | high |
| 21 | Muse Spark 1.3 | 57.85% | max |
| 22 | Qwen3.8 2.4T A95B | 57.25% | Not specified |
| 23 | DeepSeek V4 Pro 0813 | 56.71% | max |
| 24 | Claude Opus 5 | 56.57% | max |
| 25 | DeepSeek V4 Pro 0424 | 56.29% | max |
| 26 | Qwen3.8 Max (0902) | 56.19% | Not specified |
| 27 | Qwen3.8-Flash-Next | 55.91% | Not specified |
| 28 | Claude Fable 5 | 54.07% | max |
| 29 | DeepSeek V4 Flash 0731 | 53.97% | max |
| 30 | GPT-6 Luna | 53.17% | max |
| 31 | Gemini 3.6 Flash | 53.02% | high |
| 32 | Step 5 Preview | 50.96% | Not specified |
| 33 | GPT-5.6 Luna | 50.21% | max |
| 34 | Qwen3.8 Max | 49.23% | Not specified |
| 35 | Qwen3.8 27B | 48.24% | xhigh |
| 36 | DeepSeek V4 Flash Vision | 47.47% | max |
| 37 | GPT-5.5 | 47.30% | xhigh |
| 38 | JT-4.1 Flash 236B A21B | 46.47% | Not specified |
| 39 | Claude Opus 4.8 | 45.59% | max |
| 40 | Gemini 3.5 Flash | 42.08% | high |
| 41 | Muse Spark 1.2 | 40.61% | xhigh |
| 42 | Muse Spark 1.1 | 38.85% | xhigh |
| 43 | K2 Horizon 375B A23B | 37.21% | Not specified |
| 44 | Claude Sonnet 5 | 36.51% | max |
| 45 | GPT-5.5 Instant (June 2026) | 36.44% | Not specified |
| 46 | Gemini 3.1 Pro Preview | 35.43% | Not specified |
| 47 | Claude Haiku 5.5 | 35.41% | max |
| 48 | GLM-5.2 | 28.40% | max |
| 49 | Apodex 1.1 | 28.20% | Not specified |
| 50 | DeepSeek V4 Flash 0420 | 27.99% | max |
| 51 | GPT-5.4 mini | 26.20% | xhigh |
| 52 | K2 Horizon MoVA 36B A4B | 26.10% | Not specified |
| 53 | Gemini 3.5 Flash-Lite | 25.01% | high |
| 54 | Kimi K2.7 Code | 24.45% | Not specified |
| 55 | Quasar 438B | 23.86% | max |
| 56 | Qwen3.7 Max | 22.98% | Not specified |
| 57 | Solar Mini 4 | 22.25% | Not specified |
| 58 | MiniMax-M3 | 21.25% | Not specified |
| 59 | K2 Horizon 7B | 20.90% | Not specified |
| 60 | GLM-5.1 | 20.31% | Not specified |
| 61 | Claude Sonnet 4.6 | 20.10% | max |
| 62 | MiMo-V2.5 | 18.43% | Not specified |
| 63 | Hy3 | 18.41% | Not specified |
| 64 | Nex-N2-Pro | 17.63% | Not specified |
| 65 | Qwen3.7 Plus | 17.39% | Not specified |
| 66 | Ling-3.0-flash-VL | 15.68% | Not specified |
| 67 | K2 Horizon 3.7B | 14.70% | Not specified |
| 68 | Claude 4.5 Sonnet | 14.08% | Not specified |
| 69 | MiMo-V2.5-Pro | 13.70% | Not specified |
| 70 | Kimi K2.6 | 13.02% | Not specified |
| 71 | Qwen3.6 27B | 11.05% | Not specified |
| 72 | Solar Pro 4 | 9.08% | Not specified |
| 73 | Ling-3.0-flash-Fin | 7.39% | Not specified |
| 74 | Qwen3.5 397B A17B | 6.89% | Not specified |
| 75 | Muse Glimmer | 6.83% | high |
| 76 | Gemini 3.1 Flash-Lite | 6.80% | high |
| 77 | GPT-5 mini | 6.49% | high |
| 78 | Mistral Medium 3.5 | 6.30% | high |
| 79 | GPT-5.4 nano | 5.96% | xhigh |
| 80 | Qwen3.5 122B A10B | 5.48% | Not specified |
| 81 | Qwen3.6 35B A3B | 5.21% | Not specified |
| 82 | Inkling | 4.98% | xhigh |
| 83 | Gemma 4 31B | 4.93% | Not specified |
| 84 | Inkling Small | 4.81% | Not specified |
| 85 | LongCat 2.0 | 4.53% | Not specified |
| 86 | MiniMax-M2.7 | 4.44% | Not specified |
| 87 | Mistral Medium 3.1 | 4.15% | Not specified |
| 88 | Nemotron 3 Super 120B A12B | 3.78% | Not specified |
| 89 | Claude 4.5 Haiku | 3.25% | Not specified |
| 90 | Ling 3.0 Flash | 3.22% | Not specified |
| 91 | Devstral 2 | 3.11% | Not specified |
| 92 | Devstral Small 2 | 3.09% | Not specified |
| 93 | Nemotron 3 Ultra 550B A55B | 2.99% | Not specified |
| 94 | DeepSeek V3.1 Terminus | 2.71% | Not specified |
| 95 | A.X-K2 | 2.48% | Not specified |
| 96 | Mercury 2.5 | 2.35% | Not specified |
| 97 | Gemini 2.5 Pro | 2.17% | Not specified |
| 98 | Qwen3 235B A22B 2507 (Reasoning) | 2.15% | Not specified |
| 99 | Granite 4.2 8B | 2.06% | Not specified |
| 100 | Command A+ | 1.81% | Not specified |
| 101 | MiniCPM5-2B | 1.80% | Not specified |
| 102 | Mercury 2 | 1.68% | high |
| 103 | Granite 4.2 30B | 1.51% | Not specified |
| 104 | Mistral Large 3 | 1.50% | Not specified |
| 105 | North Mini Code | 1.49% | Not specified |
| 106 | Grok 4.3 | 1.38% | high |
| 107 | Trinity Large Thinking | 1.31% | Not specified |
| 108 | Mistral Small 3.1 | 1.20% | Not specified |
| 109 | Qwen3 Coder Next | 1.06% | Not specified |
| 110 | Qwen3.5 9B | 1.05% | Not specified |
| 111 | NVIDIA Nemotron 3 Nano 30B A3B | 0.97% | Not specified |
| 111 | Qwen3 32B | 0.97% | Not specified |
| 113 | Mistral Small 4 | 0.96% | Not specified |
| 114 | Nemotron 3.5 Lightning | 0.84% | Not specified |
| 115 | Qwen3 30B A3B 2507 (Reasoning) | 0.80% | Not specified |
| 116 | Ring-2.6-1T | 0.72% | xhigh |
| 117 | Ministral 3 3B | 0.67% | Not specified |
| 118 | DeepSeek V3 (Dec '24) | 0.58% | Not specified |
| 119 | Ministral 3 14B | 0.55% | Not specified |
| 120 | o3-mini | 0.53% | high |
| 121 | Ministral 3 8B | 0.52% | Not specified |
| 122 | Solar Pro 3 | 0.50% | high |
| 123 | Gemma 4 E4B | 0.46% | Not specified |
| 124 | NVIDIA Nemotron 3 Nano 4B | 0.45% | Not specified |
| 125 | Celeris-1 | 0.42% | Not specified |
| 126 | Granite 4.2 3B | 0.38% | Not specified |
| 127 | Qwen3 8B | 0.33% | Not specified |
| 128 | Mistral Small 3.2 | 0.29% | Not specified |
| 129 | Ling 3.0 Tiny | 0.28% | Not specified |
| 130 | DeepSeek V3 0324 | 0.27% | Not specified |
| 131 | DeepSeek R1 (Jan '25) | 0.24% | Not specified |
| 132 | Llama 4 Maverick | 0.21% | Not specified |
| 133 | K2 Horizon 0.9B | 0.20% | Not specified |
| 133 | gpt-oss-120b | 0.20% | high |
| 135 | Gemma 3 27B Instruct | 0.19% | Not specified |
| 135 | Gemma 3 12B Instruct | 0.19% | Not specified |
| 135 | gpt-oss-20b | 0.19% | high |
| 135 | Qwen3 14B | 0.19% | Not specified |
| 135 | Magistral Small 1.2 | 0.19% | Not specified |
| 135 | Llama 4 Scout | 0.19% | Not specified |
No published result tracked
6 of the 46 models on this site have no published number on this benchmark, so their cells in the home table stay blank rather than reading as zero.
Sources
Every page used for this benchmark, read on 17 October 2026.
- Benchmark site or the page that publishes itartificialanalysis.ai/evaluations/automationbench-aa
- Independent resultsartificialanalysis.ai/evaluations/automationbench-aa