AIEvals

Benchmarks

MathArena Apex

Final-answer mathematics problems that no model had solved when the set was published — Apex exists because the standard competitions had stopped discriminating.

MathematicsSRI Lab, ETH Zurich and INSAITVendor-reported onlyReported by DeepSeek

How it is run

Problems are drawn from live and recent competitions so they cannot be memorised, and answers are checked automatically. DeepSeek reports Pass@1 with temperature 1.0 and effort raised to its maximum.

What it cannot tell you

Small and deliberately hard, so a single problem moves the percentage by several points and the run settings do most of the work: DeepSeek reports that raising its effort level moved Apex by about 40 points on the same model.

Why there is no independent table

No independent suite in this dataset runs MathArena Apex. The numbers below are the ones model publishers report about their own models, which is a different kind of evidence: each publisher chooses its own harness and effort level, so a row is readable on its own and not against the other rows.

Read this with

Benchmarks in the same category, which usually disagree with this one on purpose.

  • AIME 2025Same category. Olympiad mathematics from the 2025 American Invitational Mathematics Examination.
  • AIME 2026Same category. Competition mathematics from the 2026 American Invitational Mathematics Examination.
  • RiemannBenchSame category. Mathematical reasoning, included in the report’s science and math evaluations.
  • ProofBench v1.1Same category. Formal and semi-formal mathematical proof construction, graded on the proof rather than the final answer.
  • FrontierMath Tier 4 (v2)Same category. Research-level mathematics problems, with Tier 4 the hardest slice of the set.

As reported by each publisher

Kept separate per publisher, because the settings are not shared.

DeepSeek

DeepSeek-V4.1-Flash: comparison with closed and open-source models · 10 September 2026

RankModelScore
1Kimi K365.6%
3DeepSeek V4 Pro65.3%
4DeepSeek V4 Flash58.6%
1DeepSeek V4.1 Flash65.6%

Ranks compare reported scores in the same published row; ties share a rank.

Table 3 of the V4.1 technical report, published with the model release. Every column is the vendor's own run at its maximum reasoning effort, each on that vendor's own harness, so this compares publication claims rather than one shared run. DeepSeek renamed nothing and disclosed no fallback rates; the report instead documents the opposite problem, agents searching for ways to game the harness, and says its visual-agent rows were run with a 512k-token context. The benchmark names are reproduced as DeepSeek wrote them — including rows this site keeps apart, such as HLE with and without tools — and the model columns are renamed to the names used on this site. DeepSeek's table is also republished, unchanged, on the model page at ollama.com; that copy prints 64.0 for the V4.1 row of NL2Repo-Bench where the report prints 65.4, and this site follows the report.

huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash/resolve/main/DeepSeek_V41_Tech_Report.pdf

Sources

Every page used for this benchmark, read on 17 October 2026.