AIEvals

Benchmarks

FrontierCode v1.1 (Extended)

Producing mergeable patches on the full 150-task FrontierCode v1.1 software-engineering evaluation.

Agentic codingReported by AnthropicVendor-reported onlyReported by Anthropic

How it is run

Section 8.3 quotes Cognition runs in Claude Code or Codex, averaged over five attempts per task. Extended contains all 150 tasks.

What it cannot tell you

Main is the 100 hardest tasks and is displayed separately. Max and xhigh settings retain separate rows rather than taking a best score.

Why there is no independent table

No independent suite in this dataset runs FrontierCode v1.1 (Extended). The numbers below are the ones model publishers report about their own models, which is a different kind of evidence: each publisher chooses its own harness and effort level, so a row is readable on its own and not against the other rows.

Read this with

Other versions of the same evaluation. Versions are not comparable: the task set changes between them.

  • FrontierCode v1.1Same family. Whether a coding agent produces changes that are ready to merge into a real codebase, rather than a patch that merely passes a test.
  • Terminal-Bench 4.0Same category. Whether an agent can complete a long, realistic piece of work in a sandboxed terminal: a working service, a proof, a CAD model, a trained kernel, a forensic report.
  • Terminal-Bench 2.1Same category. The previous generation of the same idea: can an agent drive a shell to a finished result rather than produce a plausible answer.
  • ProgramBenchSame category. Whether an agent can produce a program that fully satisfies a specification, scored on complete resolution rather than partial credit.
  • Code MigrationSame category. Whether a model can reimplement a working program in another language and preserve its behaviour — the work of porting a tool to a new ecosystem or modernising legacy code, including COBOL to Java.
  • CursorBench 4.0Same category. Coding-agent work inside an IDE-driven workflow, measured on tasks the Cursor harness is built to run.

As reported by each publisher

Kept separate per publisher, because the settings are not shared.

Anthropic

Claude Haiku 5.5: system card capability evaluations · 7 October 2026

RankRowModelScore
2FrontierCode v1.1 (Extended)Claude Haiku 5.558.4%
1FrontierCode v1.1 (Extended)Claude Sonnet 5.559.1%
1FrontierCode v1.1 (Extended, xhigh)Claude Sonnet 5.564.4%

Ranks compare reported scores in the same published row; ties share a rank.

FrontierCode v1.1 (Extended). §8.3: all 150 tasks, mean of five attempts at max effort; distinct from Main.

FrontierCode v1.1 (Extended, xhigh). §8.3 reports Sonnet’s best Extended score at xhigh; no exact Haiku xhigh value is stated.

Section 8, pages 111–139 of the Haiku 5.5 system card, read 8 October 2026. Values come from exact prose, tables and labelled figures. Unless specified, Haiku uses adaptive thinking at max effort and five trials with evaluation-specific context up to 1M. Each row retains its subset, metric, effort and source notes; no score is assigned to an unspecified SWE-Bench Pro version. Terminal runs use safeguards without fallback and no internet egress. ProgramBench uses filtered tasks and hidden-test pass rate without a time limit. OSWorld partial and strict results use the official 82-task offline subset. CAD uses 0–1 voxel IoU. Healthcare raw and length-adjusted scores stay separate. Life-science results disable Haiku biology safeguards. Independently run figures quoted by Anthropic remain publisher reports here. Missing comparison values are not inferred from rounded charts.

www.anthropic.com/claude-haiku-5-5-system-card

Sources

Every page used for this benchmark, read on 17 October 2026.