AIEvals

Benchmarks

PhysicianBench

Successfully carrying out 100 physician tasks inside an electronic health record, measured by the share of attempts passed.

HealthcareReported by AnthropicVendor-reported onlyReported by Anthropic

How it is run

Section 8.11.2 reports five attempts per task at max effort, with Claude Opus 5 judging and safety classifiers enabled. GPT-6 Luna is run by Anthropic.

What it cannot tell you

This EHR workflow pass rate differs from HealthBench response rubrics. Lower effort runs took place three days after max runs, affecting timing comparisons.

Why there is no independent table

No independent suite in this dataset runs PhysicianBench. The numbers below are the ones model publishers report about their own models, which is a different kind of evidence: each publisher chooses its own harness and effort level, so a row is readable on its own and not against the other rows.

Read this with

Benchmarks in the same category, which usually disagree with this one on purpose.

  • HealthBench (length-adjusted)Same category. Quality of healthcare responses under a rubric score adjusted to account for response length rather than raw scoring alone.
  • HealthBench (raw)Same category. Raw healthcare response quality before the length adjustment used in Anthropic’s HealthBench reporting.
  • HealthBench Professional (length-adjusted)Same category. Professional healthcare response quality after Anthropic’s response-length adjustment, with deployed safety classifiers enabled.
  • HealthBench Professional (raw)Same category. Raw professional healthcare rubric scores before Anthropic’s response-length correction is applied.

As reported by each publisher

Kept separate per publisher, because the settings are not shared.

Anthropic

Claude Haiku 5.5: system card capability evaluations · 7 October 2026

RankModelScore
4Claude Haiku 5.543.0%
6Claude Haiku 4.51.4%
2Claude Sonnet 5.563.2%
1Claude Opus 5.568.4%
3Claude Fable 5.161.0%
5GPT-6 Luna33.6%

Ranks compare reported scores in the same published row; ties share a rank.

PhysicianBench. §8.11.2 / Figure 8.11.2.A: 100 EHR tasks, five attempts; Opus 5 judge, classifiers on. Haiku low/medium/high/xhigh 17.8/25.2/31.6/35.8%.

Section 8, pages 111–139 of the Haiku 5.5 system card, read 8 October 2026. Values come from exact prose, tables and labelled figures. Unless specified, Haiku uses adaptive thinking at max effort and five trials with evaluation-specific context up to 1M. Each row retains its subset, metric, effort and source notes; no score is assigned to an unspecified SWE-Bench Pro version. Terminal runs use safeguards without fallback and no internet egress. ProgramBench uses filtered tasks and hidden-test pass rate without a time limit. OSWorld partial and strict results use the official 82-task offline subset. CAD uses 0–1 voxel IoU. Healthcare raw and length-adjusted scores stay separate. Life-science results disable Haiku biology safeguards. Independently run figures quoted by Anthropic remain publisher reports here. Missing comparison values are not inferred from rounded charts.

www.anthropic.com/claude-haiku-5-5-system-card

Sources

Every page used for this benchmark, read on 17 October 2026.