APEX-Agents-AA
Long tasks spanning applications in professional-services work.
How it is run
AA runs 452 tasks using its Stirrup harness, with three attempts per task. The headline is pass@1.
What it cannot tell you
Agent tools and application environments materially affect success. This is AA’s implementation, not a merged vendor ranking.
- Leader
- Gemini 3.5 Flash47.05% · Artificial Analysis results, grouped here by release
- Top-ten spread
- 15.04%Leader minus tenth place, or last place in smaller fields.
- Median
- 26.18%of the 32 rows published
- No published result
- 39of 46 models on this site
Artificial Analysis results by release
Highest published effort per model release, grouped here from AA’s public results. Grouping ranks are derived here; source settings and model-publisher reports stay separate.
| # | Model | Score | Effort |
|---|---|---|---|
| 1 | Gemini 3.5 Flash | 47.05% | high |
| 2 | Qwen3.8 Max (0902) | 42.40% | Not specified |
| 3 | Kimi K3 | 41.30% | max |
| 4 | GPT-5.6 Terra | 38.94% | max |
| 5 | GPT-5.5 | 37.68% | xhigh |
| 6 | GPT-5.6 Luna | 35.84% | max |
| 7 | GLM-5.2 | 33.70% | max |
| 8 | GPT-5.4 | 33.26% | xhigh |
| 9 | Claude Opus 4.6 | 33.04% | max |
| 10 | Gemini 3.1 Pro Preview | 32.01% | Not specified |
| 11 | Apodex 1.1 | 31.19% | Not specified |
| 12 | Kimi K2.6 | 28.47% | Not specified |
| 13 | GPT-5.4 mini | 28.17% | xhigh |
| 14 | Claude Sonnet 4.6 | 28.02% | max |
| 15 | Gemini 3 Flash Preview | 27.73% | Not specified |
| 16 | Ling-3.0-flash-Fin | 27.43% | Not specified |
| 17 | GPT-5.4 nano | 24.93% | xhigh |
| 18 | DeepSeek V4 Pro 0424 | 24.26% | max |
| 19 | Qwen3.7 Plus | 22.42% | Not specified |
| 20 | Grok 4.3 | 17.04% | high |
| 21 | Qwen3.5 397B A17B | 15.34% | Not specified |
| 22 | Step 3.7 Flash | 14.82% | high |
| 23 | DeepSeek V3.2 | 14.53% | Not specified |
| 24 | GLM-5 | 14.45% | Not specified |
| 25 | Grok 4.20 0309 | 14.23% | Not specified |
| 26 | Gemini 3.1 Flash-Lite | 12.17% | high |
| 27 | Kimi K2.5 | 11.50% | Not specified |
| 28 | MiniMax-M2.7 | 10.62% | Not specified |
| 29 | gpt-oss-120b | 3.10% | high |
| 30 | MiMo-V2.5-Pro | 2.43% | Not specified |
| 31 | Nemotron 3 Super 120B A12B | 1.84% | Not specified |
| 32 | gpt-oss-20b | 0.74% | high |
No published result tracked
39 of the 46 models on this site have no published number on this benchmark, so their cells in the home table stay blank rather than reading as zero.
Read this with
Benchmarks in the same category, which usually disagree with this one on purpose.
Sources
Every page used for this benchmark, read on 17 October 2026.
- Benchmark site or the page that publishes itartificialanalysis.ai/evaluations/apex-agents-aa
- Independent resultsartificialanalysis.ai/evaluations/apex-agents-aa