AIEvals

Benchmarks

SWE-Marathon

Whether an agent can stay coherent across multi-hour software engineering work, including tasks that need a GPU rather than only a shell.

Agentic codingSWE-Marathon projectVendor-reported onlyReported by Z.ai

How it is run

Version 1.1 carries twenty revised tasks with tighter verification, execution with the internet closed and GPU post-training inside the task. The project states that a fresh leaderboard is still being prepared, so the only figures here are the ones a model publisher reports.

What it cannot tell you

One publisher, its own harness, no independent run to check against: Z.ai reports 42.5% for GLM-5.3, 19.4% for GLM-5.2 and 48.1% for Kimi K3, which is the opposite ordering from Terminal-Bench 3.0 in the same table and a good illustration of how task mix decides a ranking.

Why there is no independent table

No independent suite in this dataset runs SWE-Marathon. The numbers below are the ones model publishers report about their own models, which is a different kind of evidence: each publisher chooses its own harness and effort level, so a row is readable on its own and not against the other rows.

Read this with

Benchmarks in the same category, which usually disagree with this one on purpose.

  • FrontierCode v1.1 (Extended)Same category. Producing mergeable patches on the full 150-task FrontierCode v1.1 software-engineering evaluation.
  • Terminal-Bench 4.0Same category. Whether an agent can complete a long, realistic piece of work in a sandboxed terminal: a working service, a proof, a CAD model, a trained kernel, a forensic report.
  • Terminal-Bench 2.1Same category. The previous generation of the same idea: can an agent drive a shell to a finished result rather than produce a plausible answer.
  • ProgramBenchSame category. Whether an agent can produce a program that fully satisfies a specification, scored on complete resolution rather than partial credit.
  • Code MigrationSame category. Whether a model can reimplement a working program in another language and preserve its behaviour — the work of porting a tool to a new ecosystem or modernising legacy code, including COBOL to Java.
  • CursorBench 4.0Same category. Coding-agent work inside an IDE-driven workflow, measured on tasks the Cursor harness is built to run.

As reported by each publisher

Kept separate per publisher, because the settings are not shared.

Z.ai

GLM-5.3: Frontier Coding with Emergent Cyber Capabilities — evaluation table · 14 August 2026

RankModelScore
3GLM 5.342.5%
6GLM 5.219.4%
2Kimi K348.1%
1Claude Opus 4.848.8%
5Claude Fable 533.1%
3GPT-5.6 Sol42.5%

Ranks compare reported scores in the same published row; ties share a rank.

The comparison table Z.ai publishes with GLM-5.3, read on the model card itself. Z.ai states that GLM-5.3 uses the same base model as GLM-5.2 and that every gain comes from post-training, so the GLM-5.2 column is the same architecture one training round earlier. Column names were restated in the form this site uses, and Z.ai labels its own Fable column "Fable 5 (w/ fallback)", which is recorded here as Claude Fable 5 and is the only column whose runs used a fallback model. Rows whose scale is not a percentage keep their own unit: ExploitGym is reported as tasks completed with a 2-hour and a 6-hour budget and is split into two rows so both numbers stay readable, and GDPval-AA v2 is a score rather than a percentage. The table is the only source for several benchmarks on this site, and nothing in it is an independent run.

z.ai/blog/glm-5.3

Sources

Every page used for this benchmark, read on 17 October 2026.