AIEvals

Benchmarks

ARC-AGI-2

Novel visual reasoning puzzles that are easy for people and hard for models that rely on memorised patterns.

Abstract reasoningARC Prize FoundationVendor-reported onlyReported by OpenAI

How it is run

OpenAI reports the ARC family — ARC-AGI-1, 2 and 3 — for Astra and its comparison models; Astra reports 95.0% on ARC-AGI-2 and 99.9% on the interactive ARC-AGI-3.

What it cannot tell you

ARC-AGI-3 is a different, interactive benchmark and a score there says nothing about ARC-AGI-2. OpenAI notes it ran ARC-AGI-3 with settings chosen for real-world behaviour rather than benchmark conditions.

Why there is no independent table

No independent suite in this dataset runs ARC-AGI-2. The numbers below are the ones model publishers report about their own models, which is a different kind of evidence: each publisher chooses its own harness and effort level, so a row is readable on its own and not against the other rows.

Read this with

Nothing else on this site shares this benchmark's family or category, so these are other evaluations the same publisher reports.

  • Terminal-Bench 4.0Whether an agent can complete a long, realistic piece of work in a sandboxed terminal: a working service, a proof, a CAD model, a trained kernel, a forensic report.
  • GPQA DiamondGraduate-level biology, chemistry and physics questions written to be unanswerable by web search alone.
  • Terminal-Bench Science 0.1Scientific research workflows driven through code and a terminal: analysing data, running simulations, fitting models.
  • OSWorld 2.0Whether an agent can carry out a long-horizon computer-use workflow on a real desktop: opening applications, moving files, filling in software.

As reported by each publisher

Kept separate per publisher, because the settings are not shared.

OpenAI

GPT-6 Astra: evaluation table · 3 September 2026

RankModelScore
1GPT-6 Astra95.0%
2GPT-5.6 Sol92.5%
4Claude Fable 5.190.0%
5Claude Fable 589.2%
3Claude Opus 590.4%

Ranks compare reported scores in the same published row; ties share a rank.

OpenAI reports the maximum score at any effort level, in its own research environment or through its API. Claude and Gemini figures are taken by OpenAI from those vendors' published reports, so this table is a comparison between publication claims, not between runs.

openai.com/index/gpt-6-astra/

Sources

Every page used for this benchmark, read on 17 October 2026.