AIEvals

Benchmarks

Vibe Code Bench v1.1

Whether a model can build a working product from a short prompt — the "vibe coding" workflow — rather than patch an existing repository.

Product codingVals AI106 models run by Vals AIUpdated 29 September 2026Reported by Google DeepMind

How it is run

Generated applications must build and run in an isolated container. Browser Use executes workflow tests through point-and-click interactions and checks the expected outcomes, recording screenshots and DOM snapshots.

What it cannot tell you

Passing these application workflows does not establish reliability on every product or deployment. The score depends on the requested features, environment and test coverage.

Leader
Claude Sonnet 5.592.39% · Vals AI's own list
Top-ten spread
6.22%Leader minus tenth place, or last place in smaller fields.
Median
79.80%of the 38 models on this site
No published result
8of 46 models on this site

Independent leaderboard · Vals AI

Top 106 of 106 runs by Vals AI, one harness.

#ModelScore
1Claude Sonnet 5.592.39%
2Gemini 4 Argon91.91%
3Claude Fable 590.35%
4Claude Opus 5.590.29%
5Claude Fable 5.190.26%
6GPT-6 Astra89.59%
7GPT-6.1 Sol88.93%
8Claude Opus 588.40%
9GPT-6 Sol87.82%
10Grok 4.786.17%
11Muse Spark 1.3 Max85.86%
12MiMo V2.6 Pro85.22%
13Kimi K384.96%
14DeepSeek V4.1 Flash84.74%
15Muse Spark 1 382.86%
16Claude Opus 4.882.72%
17DeepSeek V4 Pro82.30%
18GPT-6 Luna81.65%
19Claude Sonnet 581.33%
20GPT-5.6 Sol80.50%
21Muse Spark 1.279.10%
22MiMo-V2.6-Flash78.96%
23Gemini 3.8 Flash78.65%
24GLM 5.378.13%
25Claude Opus 4 8 Claude Code77.48%
25Hunyuan Hy4 Preview77.48%
27GPT-5.6 Luna77.06%
28Grok 4.676.24%
29Deepseek V4 Flash 073174.74%
30GPT-5.6 Terra74.59%
31Muse Spark 1 172.16%
32Claude Opus 4.771.00%
33Gemini 3.7 Flash70.39%
34GPT-5.569.85%
35Grok 4.569.00%
36Gpt 5.4 2026 03 0567.42%
37Gpt 5.5 Factory67.39%
38Qwen3.8 27b64.85%
39Qwen3.8 Max64.70%
40Gemini 3.6 Flash64.00%
41Glm 5.263.96%
42Gpt 5.3 Codex61.77%
43Gpt 5.5 Codex58.19%
44Claude Opus 4 657.57%
45Claude Sonnet 4 6 Claude Code55.77%
46Gpt 5.2 2025 12 1153.50%
46Claude Opus 4 6 Thinking53.50%
48Claude Sonnet 4 651.48%
49Deepseek V4 Pro49.93%
50Composer 2.549.61%
51Gemini 3.5 Flash48.68%
52GPT-5.448.47%
53Gpt 5.4 Mini 2026 03 1747.97%
54Qwen3.7 Max47.67%
55MiniMax M347.57%
56Kimi K2.7 Code47.21%
57Qwen3.7 Plus46.39%
58Mimo V2.542.17%
59Gpt 5.2 Codex37.91%
60Kimi K2.637.89%
61Gemini 3.5 Flash Lite37.16%
62Mimo V2.5 Pro34.11%
63Gemini 3.1 Pro32.03%
64Glm 5.131.46%
65GLM 5.3 Flash30.76%
66Gpt 5.4 Nano 2026 03 1726.10%
67Qwen3.6 Plus25.57%
68Gpt 5.1 2025 11 1324.61%
69Glm 5 Thinking23.36%
70Claude Sonnet 4 5 20250929 Thinking22.62%
71Gpt 5.1 Codex Max22.17%
72Claude Opus 4 5 20251101 Thinking20.63%
73Gemini 3 Flash Preview20.20%
74Gpt 5 2025 08 0720.09%
75Muse Spark19.67%
76Grok 4.319.40%
77Inkling19.21%
78Inkling Small19.05%
79Kimi K2.5 Thinking17.54%
80Qwen3.5 Plus Thinking15.74%
81MiniMax M2.514.85%
82Gemini 3 Pro Preview14.30%
83Gpt 5 Mini 2025 08 0714.17%
84Grok Build 0.113.35%
85Gpt 5.1 Codex13.12%
86Qwen3.6 27b11.94%
87MiniMax M2.711.93%
88Claude Haiku 4 5 20251001 Thinking11.39%
89Laguna M.111.04%
90Nemotron 3 Ultra 550b A55b7.64%
91Swe 1 6 Fast7.22%
92Laguna Xs.25.21%
93Deepseek V3p2 Thinking5.11%
94Grok 4.20 0309 Reasoning4.06%
95Mercury 2.53.68%
96Qwen3 Max3.51%
97Ling 3.0 Flash Af Rc33.40%
98Glm 4.63.09%
99Ling 3.0 Flash 26072.91%
100Mistral Medium 3.52.89%
101Grok 4 1 Fast Reasoning1.20%
102Gemini 2.5 Pro0.40%
103Gemini 3.1 Flash Lite Preview0.00%
103Nemotron Lightning 3p5 30b A3b0.00%
103Grok 4 Fast Reasoning0.00%
103Mistral Small 26030.00%

The same model, two sources

Independent and vendor figures for the same model, widest gap first. Run settings differ.

ModelIndependentAs publishedGapPublisher
Claude Fable 5.190.26%90.30%+0.04%Google DeepMind
Claude Opus 5.590.29%90.30%+0.01%Google DeepMind
Gemini 4 Argon91.91%91.90%-0.01%Google DeepMind
GPT-6 Astra89.59%89.60%+0.01%Google DeepMind
Gap is the publisher’s figure minus the independent one, in this benchmark’s own points. Both numbers stay visible: averaging them would invent a figure nobody published.

No published result tracked

8 of the 46 models on this site have no published number on this benchmark, so their cells in the home table stay blank rather than reading as zero.

Read this with

Other versions of the same evaluation. Versions are not comparable: the task set changes between them.

As reported by each publisher

The same benchmark, measured by the labs themselves. Compare down a column, never across two publishers.

Google DeepMind

Gemini 4 Argon: model evaluation · October 2026

RankModelScore
1Gemini 4 Argon91.9%
4GPT-6 Astra89.6%
2Claude Fable 5.190.3%
2Claude Opus 5.590.3%

Ranks compare reported scores in the same published row; ties share a rank.

Page 5 of Google DeepMind’s model evaluation, read 1 October 2026. Gemini runs use the API at the highest thinking setting, generally pass@1 without majority voting or parallel test-time compute; smaller benchmarks average multiple trials. Other models use their highest available published reasoning results. Google quotes Vals AI for knowledge-work and Vibe Code results, Zapier’s private-set AutomationBench leaderboard, Proximal for FrontierSWE v2, Surge AI for RiemannBench and no-tools Chartography, and the CWE-bench v1 leaderboard. Argon’s DeepSWE uses mini-swe; Terminal-Bench 4.0 is self-computed for Argon and quoted from the author board for other models. PostTrainBench v1.1 uses OpenCode with one H100 and a 10-hour budget for all models. Terminal-Bench Science uses a 6x verifier timeout for Argon. LABBench 2 uses terminal, bioinformatics tools, Python, R and internet access for all models. GraphWalks uses BFS F1 on two separate subsets: 650 items up to 128k tokens and 200 items from 256k to 1M. Agent’s Last Exam uses ALE-Claw, a 5-hour window and binary pass rate, with safety filters enabled for Argon. OSWorld 2.0 uses offline partial reward, the best of three single-attempt runs for Argon, safety filters, 1080p, 500 steps, Gemini CUA with parallel batch tools and compaction, screenshot-only observation and the 08.08 patch; Claude’s combined online/offline results are omitted. LVBench runs without tools, using 1 FPS for Gemini, 800 frames for Astra, 300 for Fable and 600 for Opus. Missing cells remain blank. These are the report’s published figures, preserved separately from this site’s independent source snapshots.

storage.googleapis.com/deepmind-media/gemini/gemini_4_argon_model_evaluation.pdf

Sources

Every page used for this benchmark, read on 17 October 2026.