NL2Repo-Bench

Ranks compare reported scores in the same published row; ties share a rank.

The complete 11-row, six-model comparison from DeepSeek’s dated MIT model card, read 17 October 2026 at revision 1bae0e1088993b9b82de08f59de5be5a34e42170. All 63 published comparison cells and both additional parenthesized HLE scores are retained as 65 figures across 12 rows. The HLE text-only scores are split into their own row and benchmark page; GLM-5.3’s dagger-marked 42.0 is text-only and has no full-set score. Unreported values stay null and do not enter ranks. K3 and Opus 5 are displayed as Kimi K3 and Claude Opus 5. Terminal-Bench 4.0 uses Claude Code, DeepSWE v1.1 uses Mini-SWE-V2, and the release notes identify ZeroBench-main as Pass@5. Other effort levels, trial counts and judges are not fully specified. These are DeepSeek publication claims, separate from independent evaluations and earlier publisher snapshots.

huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash-1017

Ranks compare reported scores in the same published row; ties share a rank.

The comparison table Z.ai publishes with GLM-5.3, read on the model card itself. Z.ai states that GLM-5.3 uses the same base model as GLM-5.2 and that every gain comes from post-training, so the GLM-5.2 column is the same architecture one training round earlier. Column names were restated in the form this site uses, and Z.ai labels its own Fable column "Fable 5 (w/ fallback)", which is recorded here as Claude Fable 5 and is the only column whose runs used a fallback model. Rows whose scale is not a percentage keep their own unit: ExploitGym is reported as tasks completed with a 2-hour and a 6-hour budget and is split into two rows so both numbers stay readable, and GDPval-AA v2 is a score rather than a percentage. The table is the only source for several benchmarks on this site, and nothing in it is an independent run.

z.ai/blog/glm-5.3

Ranks compare reported scores in the same published row; ties share a rank.

NL2Repo-Bench (Score). The report prints 65.4 for the V4.1 row; the copy on ollama.com prints 64.0.

Table 3 of the V4.1 technical report, published with the model release. Every column is the vendor's own run at its maximum reasoning effort, each on that vendor's own harness, so this compares publication claims rather than one shared run. DeepSeek renamed nothing and disclosed no fallback rates; the report instead documents the opposite problem, agents searching for ways to game the harness, and says its visual-agent rows were run with a 512k-token context. The benchmark names are reproduced as DeepSeek wrote them — including rows this site keeps apart, such as HLE with and without tools — and the model columns are renamed to the names used on this site. DeepSeek's table is also republished, unchanged, on the model page at ollama.com; that copy prints 64.0 for the V4.1 row of NL2Repo-Bench where the report prints 65.4, and this site follows the report.

huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash/resolve/main/DeepSeek_V41_Tech_Report.pdf