AA-AnalystAgent
Quantitative analysis over source spreadsheets and documents.
Data analysisArtificial Analysis33 models run by Artificial AnalysisUpdated 30 September 2026
How it is run
80 questions across 14 domains, five attempts per question. Headline pass^5 requires all five attempts to be correct.
What it cannot tell you
A reliability metric, distinct from pass@1 or pass@5; the full question set is private.
- Leader
- Gemini 3.7 Flash60.00% · Artificial Analysis results, grouped here by release
- Top-ten spread
- 15.00%Leader minus tenth place, or last place in smaller fields.
- Median
- 27.50%of the 33 rows published
- No published result
- 28of 46 models on this site
Artificial Analysis results by release
Highest published effort per model release, grouped here from AA’s public results. Grouping ranks are derived here; source settings and model-publisher reports stay separate.
| # | Model | Score | Effort |
|---|---|---|---|
| 1 | Gemini 3.7 Flash | 60.00% | high |
| 2 | Claude Fable 5.1 | 57.50% | max |
| 3 | Claude Opus 5 | 53.75% | max |
| 4 | GPT-6 Astra | 51.25% | max |
| 5 | GPT-5.5 | 50.00% | xhigh |
| 6 | Claude Fable 5 | 48.75% | max |
| 7 | GPT-5.6 Sol | 47.50% | max |
| 8 | Claude Sonnet 5 | 46.25% | max |
| 9 | Gemini 3.5 Flash | 45.00% | high |
| 9 | Claude Opus 4.8 | 45.00% | max |
| 9 | Qwen3.8 Max (0902) | 45.00% | Not specified |
| 12 | Claude Opus 4.7 | 43.75% | max |
| 13 | Gemini 3.1 Pro Preview | 41.25% | Not specified |
| 13 | Grok 4.6 | 41.25% | high |
| 15 | Kimi K3 | 38.75% | max |
| 16 | Grok 4.5 | 35.00% | high |
| 17 | Inkling Small | 27.50% | Not specified |
| 18 | DeepSeek V4 Flash 0420 | 25.00% | max |
| 19 | Inkling | 23.75% | xhigh |
| 20 | MiMo-V2.5-Pro | 20.00% | Not specified |
| 20 | Claude Sonnet 4.6 | 20.00% | max |
| 22 | Qwen3.7 Max | 18.75% | Not specified |
| 22 | DeepSeek V4 Pro 0424 | 18.75% | max |
| 24 | Ling-3.0-flash-Fin | 16.25% | Not specified |
| 25 | Claude 4.5 Haiku | 15.00% | Not specified |
| 25 | GPT-5.4 mini | 15.00% | xhigh |
| 27 | Mistral Medium 3.5 | 12.50% | high |
| 28 | MiniMax-M2.7 | 11.25% | Not specified |
| 29 | MiniMax-M3 | 10.00% | Not specified |
| 30 | Gemini 3.1 Flash-Lite | 8.75% | high |
| 30 | Grok 4.3 | 8.75% | high |
| 32 | Nemotron 3 Ultra 550B A55B | 6.25% | Not specified |
| 33 | Mistral Small 4 | 1.25% | Not specified |
No published result tracked
28 of the 46 models on this site have no published number on this benchmark, so their cells in the home table stay blank rather than reading as zero.
Step 5 PreviewGPT-6 LunaGPT-5.6 TerraGLM 5.3 FlashQwen3.8 2.4T A95BQwen3.8-Flash-NextMiMo-V2.6-FlashMuse Spark 1.2GPT-5.4DeepSeek V4.1 Flash 1017GLM-LegalClaude Haiku 5.5BeamGemini 4 ArgonClaude Sonnet 5.5Claude Opus 5.5GPT-6 SolGPT-6.1 SolGPT-5.6 LunaGemini 3.8 FlashGrok 4.7DeepSeek V4.1 FlashDeepSeek V4 ProQwen3.8 MaxGLM 5.3Muse Spark 1.3 MaxHunyuan Hy4 PreviewMiMo V2.6 Pro
Sources
Every page used for this benchmark, read on 17 October 2026.
- Benchmark site or the page that publishes itartificialanalysis.ai/evaluations/aa-analyst-agent
- Independent resultsartificialanalysis.ai/evaluations/aa-analyst-agent