AIEvals

Benchmarks

Terminal-Bench 3.0

Terminal work that is harder and more varied than 2.1 covers: the first release carries 74 tasks across 7 domains, and the task mix rolls forward instead of being frozen.

Agentic codingTerminal-Bench community and Harbor (open source)Vendor-reported onlyReported by Z.ai, DeepSeek

How it is run

A continuous benchmark with semantic versioning and result migrations, published by the team behind the earlier Terminal-Bench releases. At release the best models were at roughly 34%, and the 4.0 line continues the same idea at a harder level.

What it cannot tell you

Every number here is a publisher's own run under its own harness — the rows below are DeepSeek's, measured with its internal harness — and because the task set keeps moving, a figure belongs to the snapshot it was measured on rather than to the benchmark as a whole.

Why there is no independent table

No independent suite in this dataset runs Terminal-Bench 3.0. The numbers below are the ones model publishers report about their own models, which is a different kind of evidence: each publisher chooses its own harness and effort level, so a row is readable on its own and not against the other rows.

Read this with

Other versions of the same evaluation. Versions are not comparable: the task set changes between them.

  • Terminal-Bench 4.0Same family. Whether an agent can complete a long, realistic piece of work in a sandboxed terminal: a working service, a proof, a CAD model, a trained kernel, a forensic report.
  • Terminal-Bench 2.1Same family. The previous generation of the same idea: can an agent drive a shell to a finished result rather than produce a plausible answer.
  • FrontierCode v1.1 (Extended)Same category. Producing mergeable patches on the full 150-task FrontierCode v1.1 software-engineering evaluation.
  • ProgramBenchSame category. Whether an agent can produce a program that fully satisfies a specification, scored on complete resolution rather than partial credit.
  • Code MigrationSame category. Whether a model can reimplement a working program in another language and preserve its behaviour — the work of porting a tool to a new ecosystem or modernising legacy code, including COBOL to Java.
  • CursorBench 4.0Same category. Coding-agent work inside an IDE-driven workflow, measured on tasks the Cursor harness is built to run.

As reported by each publisher

Kept separate per publisher, because the settings are not shared.

Z.ai

GLM-5.3: Frontier Coding with Emergent Cyber Capabilities — evaluation table · 14 August 2026

RankModelScore
3GLM 5.328.3%
6GLM 5.24.6%
5Kimi K317.4%
4Claude Opus 4.821.1%
2Claude Fable 533.7%
1GPT-5.6 Sol34.6%

Ranks compare reported scores in the same published row; ties share a rank.

The comparison table Z.ai publishes with GLM-5.3, read on the model card itself. Z.ai states that GLM-5.3 uses the same base model as GLM-5.2 and that every gain comes from post-training, so the GLM-5.2 column is the same architecture one training round earlier. Column names were restated in the form this site uses, and Z.ai labels its own Fable column "Fable 5 (w/ fallback)", which is recorded here as Claude Fable 5 and is the only column whose runs used a fallback model. Rows whose scale is not a percentage keep their own unit: ExploitGym is reported as tasks completed with a 2-hour and a 6-hour budget and is split into two rows so both numbers stay readable, and GDPval-AA v2 is a score rather than a percentage. The table is the only source for several benchmarks on this site, and nothing in it is an independent run.

z.ai/blog/glm-5.3

DeepSeek

DeepSeek-V4.1-Flash: comparison with closed and open-source models · 10 September 2026

RankModelScore
1Claude Opus 543.3%
2GPT-5.6 Sol34.4%
5Kimi K317.7%
4GLM 5.328.3%
6DeepSeek V4 Pro11.8%
7DeepSeek V4 Flash7.6%
3DeepSeek V4.1 Flash30.0%

Ranks compare reported scores in the same published row; ties share a rank.

Table 3 of the V4.1 technical report, published with the model release. Every column is the vendor's own run at its maximum reasoning effort, each on that vendor's own harness, so this compares publication claims rather than one shared run. DeepSeek renamed nothing and disclosed no fallback rates; the report instead documents the opposite problem, agents searching for ways to game the harness, and says its visual-agent rows were run with a 512k-token context. The benchmark names are reproduced as DeepSeek wrote them — including rows this site keeps apart, such as HLE with and without tools — and the model columns are renamed to the names used on this site. DeepSeek's table is also republished, unchanged, on the model page at ollama.com; that copy prints 64.0 for the V4.1 row of NL2Repo-Bench where the report prints 65.4, and this site follows the report.

huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash/resolve/main/DeepSeek_V41_Tech_Report.pdf

Sources

Every page used for this benchmark, read on 17 October 2026.