AIEvals

Benchmarks

Humanity's Last Exam (text-only, without tools)

Expert-level reasoning on the text-only subset of Humanity’s Last Exam, with no image questions or tools.

Multidisciplinary reasoningCenter for AI Safety and Scale AI; subset reported by DeepSeekVendor-reported onlyReported by DeepSeek

How it is run

DeepSeek’s 17 October 2026 model card identifies text-only results with daggers: 44.0 for Flash 1017, 39.1 for the September Flash release and 42.0 for GLM-5.3. Only these explicitly marked subset results enter this table.

What it cannot tell you

The text-only subset differs from the full multimodal exam. It has its own ranks and cannot replace full-set HLE scores. Other comparison models have no published text-only result in this source; effort, trial count and judging are not specified.

Why there is no independent table

No independent suite in this dataset runs Humanity's Last Exam (text-only, without tools). The numbers below are the ones model publishers report about their own models, which is a different kind of evidence: each publisher chooses its own harness and effort level, so a row is readable on its own and not against the other rows.

Read this with

Other versions of the same evaluation. Versions are not comparable: the task set changes between them.

  • Humanity's Last ExamSame family. A 2,500-question exam written by domain experts across disciplines, designed to be at the frontier of human knowledge rather than a curriculum.

As reported by each publisher

Kept separate per publisher, because the settings are not shared.

DeepSeek

DeepSeek-V4.1-Flash-1017: complete release comparison · 17 October 2026

RankModelScore
1DeepSeek V4.1 Flash 101744.0%
3DeepSeek V4.1 Flash39.1%
2GLM 5.342.0%

Ranks compare reported scores in the same published row; ties share a rank.

Text-only HLE (w/o tools). The dagger-marked HLE text-only subset: 44.0 for 1017, 39.1 for September V4.1 Flash and 42.0 for GLM-5.3. No text-only score is published here for the other three models; ranks cover only these three reported scores.

The complete 11-row, six-model comparison from DeepSeek’s dated MIT model card, read 17 October 2026 at revision 1bae0e1088993b9b82de08f59de5be5a34e42170. All 63 published comparison cells and both additional parenthesized HLE scores are retained as 65 figures across 12 rows. The HLE text-only scores are split into their own row and benchmark page; GLM-5.3’s dagger-marked 42.0 is text-only and has no full-set score. Unreported values stay null and do not enter ranks. K3 and Opus 5 are displayed as Kimi K3 and Claude Opus 5. Terminal-Bench 4.0 uses Claude Code, DeepSWE v1.1 uses Mini-SWE-V2, and the release notes identify ZeroBench-main as Pass@5. Other effort levels, trial counts and judges are not fully specified. These are DeepSeek publication claims, separate from independent evaluations and earlier publisher snapshots.

huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash-1017

Sources

Every page used for this benchmark, read on 17 October 2026.