GPQA Diamond

Ranks compare reported scores in the same published row; ties share a rank.

Reflection AI’s four capability tables, read 6 October 2026. All 21 rows and eight model columns are transcribed; NR means not reported and remains blank. The source’s Qwen 3.8 Max column uses this catalogue’s Qwen3.8 Max spelling. Beam results are publisher reports; Reflection AI cites Artificial Analysis and Datacurve for comparison-model evaluations. The announcement does not specify a common harness, effort or trial count for every row. Ranks are derived within each published row from reported values only, with ties sharing a rank; they do not establish an independent or overall model ranking. AutomationBench uses the public split; AA-Omniscience is an index on the public split only, calculated by Reflection AI. BrowseComp and DeepSearchQA use context management. AA-LCR has no stated version. These settings remain separate from other source fields.

reflection.ai/blog/introducing-beam

Ranks compare reported scores in the same published row; ties share a rank.

Table 3 of the V4.1 technical report, published with the model release. Every column is the vendor's own run at its maximum reasoning effort, each on that vendor's own harness, so this compares publication claims rather than one shared run. DeepSeek renamed nothing and disclosed no fallback rates; the report instead documents the opposite problem, agents searching for ways to game the harness, and says its visual-agent rows were run with a 512k-token context. The benchmark names are reproduced as DeepSeek wrote them — including rows this site keeps apart, such as HLE with and without tools — and the model columns are renamed to the names used on this site. DeepSeek's table is also republished, unchanged, on the model page at ollama.com; that copy prints 64.0 for the V4.1 row of NL2Repo-Bench where the report prints 65.4, and this site follows the report.

huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash/resolve/main/DeepSeek_V41_Tech_Report.pdf

Ranks compare reported scores in the same published row; ties share a rank.

OpenAI reports the maximum score at any effort level, in its own research environment or through its API. Claude and Gemini figures are taken by OpenAI from those vendors' published reports, so this table is a comparison between publication claims, not between runs.

openai.com/index/gpt-6-astra/