AIEvals

Benchmarks

Global MMLU

Knowledge and reasoning accuracy across 42 languages, including both high-resource and low-resource languages.

Multilingual knowledgeReported by AnthropicVendor-reported onlyReported by Anthropic

How it is run

Section 8.12.1 averages accuracy across languages in a single trial, with max adaptive thinking, no tools, no custom system prompt, and safety classifiers.

What it cannot tell you

Blocked and unanswered questions are excluded rather than counted wrong. This multilingual aggregate is separate from MMLU Pro and language-specific scores.

Why there is no independent table

No independent suite in this dataset runs Global MMLU. The numbers below are the ones model publishers report about their own models, which is a different kind of evidence: each publisher chooses its own harness and effort level, so a row is readable on its own and not against the other rows.

Read this with

Benchmarks in the same category, which usually disagree with this one on purpose.

  • MILUSame category. Culturally grounded knowledge comprehension in ten Indic languages and English, reported as mean accuracy across eleven languages.

As reported by each publisher

Kept separate per publisher, because the settings are not shared.

Anthropic

Claude Haiku 5.5: system card capability evaluations · 7 October 2026

RankModelScore
4Claude Haiku 5.587.8%
3Claude Sonnet 5.592.1%
1Claude Opus 5.594.3%
2Claude Fable 5.194.0%

Ranks compare reported scores in the same published row; ties share a rank.

Global MMLU. §8.12.1: average across 42 languages, one trial, max, no tools/custom prompts. Blocked or unanswered questions excluded.

Section 8, pages 111–139 of the Haiku 5.5 system card, read 8 October 2026. Values come from exact prose, tables and labelled figures. Unless specified, Haiku uses adaptive thinking at max effort and five trials with evaluation-specific context up to 1M. Each row retains its subset, metric, effort and source notes; no score is assigned to an unspecified SWE-Bench Pro version. Terminal runs use safeguards without fallback and no internet egress. ProgramBench uses filtered tasks and hidden-test pass rate without a time limit. OSWorld partial and strict results use the official 82-task offline subset. CAD uses 0–1 voxel IoU. Healthcare raw and length-adjusted scores stay separate. Life-science results disable Haiku biology safeguards. Independently run figures quoted by Anthropic remain publisher reports here. Missing comparison values are not inferred from rounded charts.

www.anthropic.com/claude-haiku-5-5-system-card

Sources

Every page used for this benchmark, read on 17 October 2026.