AIEvals

Benchmarks

BabyVision

Core visual ability rather than knowledge: 388 items in 22 subclasses that a person solves by looking, without needing language to reason about the answer.

MultimodalChen et al. (2026)Vendor-reported onlyReported by DeepSeek, DeepSeek

How it is run

Four categories of visual primitives, each with human baselines collected for the paper. DeepSeek reports a with-tools score against the published text-only baselines.

What it cannot tell you

The benchmark was built to show how far models fall short — the best model at publication scored 49.7 against an adult average of 94.1 — so a high number here is a claim about a setting that differs from the paper's leaderboard.

Why there is no independent table

No independent suite in this dataset runs BabyVision. The numbers below are the ones model publishers report about their own models, which is a different kind of evidence: each publisher chooses its own harness and effort level, so a row is readable on its own and not against the other rows.

Read this with

Benchmarks in the same category, which usually disagree with this one on purpose.

  • ZeroBench (main set)Same category. An "impossible" visual benchmark: questions written so that no current multimodal model is expected to answer them from the image alone.

As reported by each publisher

Kept separate per publisher, because the settings are not shared.

DeepSeek

DeepSeek-V4.1-Flash-1017: complete release comparison · 17 October 2026

RankModelScore
2DeepSeek V4.1 Flash 101792.5%
3DeepSeek V4.1 Flash89.6%
5Kimi K385.7%
4GPT-5.6 Sol88.9%
1Claude Opus 594.1%

Ranks compare reported scores in the same published row; ties share a rank.

BabyVision (w/tools). With tools. GLM-5.3 is not reported.

The complete 11-row, six-model comparison from DeepSeek’s dated MIT model card, read 17 October 2026 at revision 1bae0e1088993b9b82de08f59de5be5a34e42170. All 63 published comparison cells and both additional parenthesized HLE scores are retained as 65 figures across 12 rows. The HLE text-only scores are split into their own row and benchmark page; GLM-5.3’s dagger-marked 42.0 is text-only and has no full-set score. Unreported values stay null and do not enter ranks. K3 and Opus 5 are displayed as Kimi K3 and Claude Opus 5. Terminal-Bench 4.0 uses Claude Code, DeepSWE v1.1 uses Mini-SWE-V2, and the release notes identify ZeroBench-main as Pass@5. Other effort levels, trial counts and judges are not fully specified. These are DeepSeek publication claims, separate from independent evaluations and earlier publisher snapshots.

huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash-1017

DeepSeek

DeepSeek-V4.1-Flash: comparison with closed and open-source models · 10 September 2026

RankModelScore
1Claude Opus 594.1%
3GPT-5.6 Sol88.9%
4Kimi K385.7%
2DeepSeek V4.1 Flash89.6%

Ranks compare reported scores in the same published row; ties share a rank.

Table 3 of the V4.1 technical report, published with the model release. Every column is the vendor's own run at its maximum reasoning effort, each on that vendor's own harness, so this compares publication claims rather than one shared run. DeepSeek renamed nothing and disclosed no fallback rates; the report instead documents the opposite problem, agents searching for ways to game the harness, and says its visual-agent rows were run with a 512k-token context. The benchmark names are reproduced as DeepSeek wrote them — including rows this site keeps apart, such as HLE with and without tools — and the model columns are renamed to the names used on this site. DeepSeek's table is also republished, unchanged, on the model page at ollama.com; that copy prints 64.0 for the V4.1 row of NL2Repo-Bench where the report prints 65.4, and this site follows the report.

huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash/resolve/main/DeepSeek_V41_Tech_Report.pdf

Sources

Every page used for this benchmark, read on 17 October 2026.