AIEvals

Benchmarks

GDP.pdf

Reasoning over long professional PDF documents, requiring all task criteria to pass.

Document reasoningReported independently by Artificial Analysis130 models run by Artificial AnalysisUpdated 8 October 2026

How it is run

AA implements Surge AI’s GDP.pdf benchmark and reports the all-pass score. Its runs and vendor figures retain separate settings. Highest published effort per release; the table retains its snapshot date.

What it cannot tell you

Effort settings and evaluation versions matter. These source-specific results are kept separate from Vals AI and vendor reports. A missing score stays blank.

Leader
GPT-6.1 Sol31.00% · Artificial Analysis results, grouped here by release
Top-ten spread
7.00%Leader minus tenth place, or last place in smaller fields.
Median
8.50%of the 130 rows published
No published result
6of 46 models on this site

Artificial Analysis results by release

Highest published effort per model release, grouped here from AA’s public results. Grouping ranks are derived here; source settings and model-publisher reports stay separate.

#ModelScoreEffort
1GPT-6.1 Sol31.00%max
1GPT-6 Astra31.00%max
3GPT-5.6 Sol27.20%max
4Muse Spark 1.326.60%max
5Claude Fable 5.126.20%max
5Claude Opus 5.526.20%max
7Claude Sonnet 5.525.80%max
8GPT-6 Sol25.20%max
9GPT-5.6 Luna24.00%max
9GPT-5.6 Terra24.00%max
9Claude Fable 524.00%max
12Gemini 3.7 Flash23.60%high
13GPT-6 Luna22.80%max
13Claude Opus 4.822.80%max
13Qwen3.8 Max (0902)22.80%Not specified
16Kimi K322.00%max
17Gemini 4 Argon21.80%high
18Claude Opus 521.60%max
19GPT-5.521.20%xhigh
20Gemini 3.8 Flash21.00%high
21Claude Haiku 5.520.80%max
22Qwen3.8 Max20.20%Not specified
22GPT-5.5 Instant (June 2026)20.20%Not specified
24Grok 4.720.00%xhigh
25Gemini 3.5 Flash19.80%high
26MiMo-V2.6-Pro19.20%Not specified
27Grok 4.518.80%high
28Gemini 3.1 Pro Preview17.80%Not specified
29Muse Spark 1.217.40%xhigh
29Gemini 3.6 Flash17.40%high
31Grok 4.617.20%xhigh
32Qwen3.8 27B16.60%xhigh
33Claude Sonnet 4.615.80%max
34Qwen3.8-Flash-Next15.60%Not specified
35GLM 5.3 Flash15.40%max
36Qwen3.8 2.4T A95B15.00%Not specified
37Step 5 Preview14.80%Not specified
38Muse Spark 1.114.40%xhigh
39Gemini 3.5 Flash-Lite13.60%high
40GPT-5.4 mini13.40%xhigh
40DeepSeek V4 Pro 042413.40%max
42Claude Sonnet 513.20%max
43Kimi K2.613.00%Not specified
44Inkling12.80%xhigh
44DeepSeek V4.1 Flash12.80%max
46Nex-N2-Pro12.60%Not specified
47Qwen3.5 397B A17B12.20%Not specified
47Qwen3.7 Plus12.20%Not specified
49DeepSeek V4 Flash Vision12.00%max
50DeepSeek V4 Pro 081311.40%max
51Apodex 1.111.20%Not specified
51Inkling Small11.20%Not specified
51Kimi K2.7 Code11.20%Not specified
51Hy311.20%Not specified
51GLM-5.311.20%max
56Qwen3.6 27B11.00%Not specified
56DeepSeek V4 Flash 073111.00%max
58DeepSeek V4 Flash 042010.80%max
59GLM-5.210.40%max
60Gemini 2.5 Pro10.20%Not specified
61Muse Glimmer10.00%high
62MiniMax-M39.80%Not specified
63MiMo-V2.6-Flash9.00%Not specified
64JT-4.1 Flash 236B A21B8.60%Not specified
64Qwen3.7 Max8.60%Not specified
66GPT-5 mini8.40%high
66Ling-3.0-flash-VL8.40%Not specified
66GLM-5.18.40%Not specified
69Gemini 3.1 Flash-Lite7.80%high
69GPT-5.4 nano7.80%xhigh
71K2 Horizon 375B A23B7.40%Not specified
72Qwen3.5 9B6.60%Not specified
72Qwen3.6 35B A3B6.60%Not specified
74LongCat 2.06.40%Not specified
74A.X-K26.40%Not specified
76MiniMax-M2.76.00%Not specified
76Gemma 4 31B6.00%Not specified
78Grok 4.35.80%high
79Solar Pro 45.60%Not specified
80DeepSeek V3.1 Terminus5.40%Not specified
80Ling-3.0-flash-Fin5.40%Not specified
80Ling 3.0 Flash5.40%Not specified
80Quasar 438B5.40%max
84Claude 4.5 Sonnet5.20%Not specified
84Motif 35.20%Not specified
86Nemotron 3 Ultra 550B A55B5.00%Not specified
86Solar Mini 45.00%Not specified
88Qwen3 235B A22B 2507 (Reasoning)4.80%Not specified
89K2 Horizon MoVA 36B A4B4.20%Not specified
90MiMo-V2.5-Pro4.00%Not specified
90gpt-oss-120b4.00%high
92Claude 4.5 Haiku3.80%Not specified
92Ring-2.6-1T3.80%xhigh
94Mercury 2.53.60%Not specified
95Qwen3.5 122B A10B3.40%Not specified
96K2 Horizon 7B3.20%Not specified
96K2 Horizon 3.7B3.20%Not specified
98Nemotron 3.5 Lightning3.00%Not specified
99Mistral Medium 3.52.80%high
100Nemotron 3 Super 120B A12B2.60%Not specified
100Qwen3 Coder Next2.60%Not specified
102Devstral 22.40%Not specified
102DeepSeek R1 (Jan '25)2.40%Not specified
104o3-mini2.00%high
104gpt-oss-20b2.00%high
106Granite 4.2 30B1.80%Not specified
106North Mini Code1.80%Not specified
108DeepSeek V3 03241.60%Not specified
108Solar Pro 31.60%high
110MiniCPM5-2B1.40%Not specified
111Mistral Medium 3.11.20%Not specified
111Mistral Large 31.20%Not specified
111Ministral 3 14B1.20%Not specified
111Celeris-11.20%Not specified
111Trinity Large Thinking1.20%Not specified
116Mistral Small 3.11.00%Not specified
116Gemma 3 12B Instruct1.00%Not specified
116Command A+1.00%Not specified
116NVIDIA Nemotron 3 Nano 30B A3B1.00%Not specified
116Devstral Small 21.00%Not specified
121DeepSeek V3 (Dec '24)0.80%Not specified
121Ling 3.0 Tiny0.80%Not specified
121Ministral 3 8B0.80%Not specified
121K2 Horizon 0.9B0.80%Not specified
125Granite 4.2 8B0.60%Not specified
126Qwen3 30B A3B 2507 (Reasoning)0.40%Not specified
127Granite 4.2 3B0.20%Not specified
127Gemma 3 27B Instruct0.20%Not specified
129Mistral Small 40.00%Not specified
129Ministral 3 3B0.00%Not specified
Source: artificialanalysis.ai/evaluations/gdp-pdf

No published result tracked

6 of the 46 models on this site have no published number on this benchmark, so their cells in the home table stay blank rather than reading as zero.

Claude Opus 4.7GPT-5.4DeepSeek V4.1 Flash 1017GLM-LegalBeamHunyuan Hy4 Preview

Read this with

Benchmarks in the same category, which usually disagree with this one on purpose.

  • OfficeQASame category. Answering grounded questions over a large corpus of historical United States Treasury Bulletin documents in an agentic workflow.
  • OfficeQA ProSame category. Grounded reasoning on the harder 133-question OfficeQA Pro subset of historical Treasury Bulletin documents.

Sources

Every page used for this benchmark, read on 17 October 2026.