Public Benefits Bench v1.1
Whether a general-purpose model can answer SNAP benefits questions correctly enough to act as a first point of contact for people seeking or keeping food assistance.
How it is run
459 SNAP scenarios covering seven phases of the benefits lifecycle, with expected responses and rubrics validated by SNAP policy experts. Vals ran each model in four conditions — the raw model, web search only, a multi-turn conversation only, and both together — and models added after the June 2026 study were evaluated only in the combined condition.
What it cannot tell you
The best model answers 76.9% of questions correctly, so nearly a quarter of answers would be wrong on a high-stakes subject. Vals reads the result as a grounding problem rather than a model-size problem: SNAP rules vary by state and county and are held by agencies, so retrieval over state material matters more here than general capability, and rows added after the study are single-condition rather than four-condition runs.
- Leader
- Claude Opus 576.93% · Vals AI's own list
- Top-ten spread
- 8.66%Leader minus tenth place, or last place in smaller fields.
- Median
- 62.45%of the 45 rows published
- No published result
- 17of 46 models on this site
Independent leaderboard · Vals AI
Top 45 of 45 runs by Vals AI, one harness.
| # | Model | Score |
|---|---|---|
| 1 | Claude Opus 5 | 76.93% |
| 2 | Claude Fable 5.1 | 74.90% |
| 3 | Claude Opus 5.5 | 70.64% |
| 4 | Claude Fable 5 | 70.43% |
| 5 | Gemini 4 Argon | 69.76% |
| 6 | MiMo V2.6 Pro | 68.94% |
| 7 | Hunyuan Hy4 Preview | 68.61% |
| 8 | GLM 5.3 | 68.54% |
| 9 | Muse Spark 1.2 | 68.47% |
| 10 | Kimi K3 | 68.27% |
| 11 | Claude Opus 4.8 | 68.13% |
| 12 | MiMo-V2.6-Flash | 67.59% |
| 13 | Claude Sonnet 5.5 | 67.19% |
| 14 | Qwen3.8 Max | 67.12% |
| 15 | Grok 4.6 | 66.85% |
| 16 | GPT-5.6 Sol | 66.51% |
| 17 | Claude Sonnet 5 | 66.03% |
| 18 | Grok 4.7 | 65.63% |
| 19 | Gemini 3.8 Flash | 65.29% |
| 20 | DeepSeek V4.1 Flash | 64.28% |
| 21 | MiniMax M3 | 64.14% |
| 22 | Deepseek V4 Pro | 62.92% |
| 23 | Claude Sonnet 4 6 | 62.45% |
| 24 | GPT-5.6 Terra | 62.38% |
| 25 | Glm 5.1 | 61.84% |
| 26 | Muse Spark 1 1 | 61.30% |
| 27 | GPT-5.6 Luna | 61.16% |
| 28 | GPT-5.5 | 60.89% |
| 29 | Gemini 3.5 Flash | 59.47% |
| 30 | Inkling Small | 59.41% |
| 31 | GPT-6.1 Sol | 59.34% |
| 32 | Inkling | 58.46% |
| 33 | GPT-6 Luna | 57.65% |
| 34 | Kimi K2.6 | 56.63% |
| 34 | GPT-6 Sol | 56.63% |
| 36 | Gemini 3.6 Flash | 56.56% |
| 37 | Claude Haiku 4 5 20251001 | 54.26% |
| 38 | Gemini 3.1 Pro | 53.79% |
| 39 | Gemini 3.5 Flash Lite | 51.83% |
| 40 | Grok 4.3 | 51.69% |
| 41 | Ling 3.0 Flash 2607 | 50.74% |
| 42 | Mercury 2.5 | 45.26% |
| 43 | Grok 4 1 Fast Reasoning | 44.79% |
| 44 | Laguna M.1 | 43.84% |
| 45 | Laguna Xs.2 | 41.41% |
No published result tracked
17 of the 46 models on this site have no published number on this benchmark, so their cells in the home table stay blank rather than reading as zero.
Sources
Every page used for this benchmark, read on 17 October 2026.
- Benchmark site or the page that publishes itwww.centerforcivicfutures.org/
- Independent resultswww.vals.ai/benchmarks/public-benefits-bench