MedCode
Whether a model can perform medical coding — assigning the diagnosis codes a hospital bills from — given realistic de-identified documentation rather than a clean textbook case.
How it is run
Built with Protege, the dataset carries up to 2,755 primary and secondary diagnosis codes per patient record, so a whole hospitalisation can be coded from admission through discharge, including sequencing, modifiers and payer rules.
What it cannot tell you
The leader reaches only 63.57%, and every model is weaker on mental-health and other complex categories than on physical conditions such as diabetes and hypertension. That gap to the rules billing depends on is the reliability question for any deployment, and Vals frames the field as deployed without much independent oversight.
- Leader
- Claude Opus 563.57% · Vals AI's own list
- Top-ten spread
- 10.35%Leader minus tenth place, or last place in smaller fields.
- Median
- 41.33%of the 104 rows published
- No published result
- 11of 46 models on this site
Independent leaderboard · Vals AI
Top 104 of 104 runs by Vals AI, one harness.
| # | Model | Score |
|---|---|---|
| 1 | Claude Opus 5 | 63.57% |
| 2 | Gemini 3.1 Pro | 59.06% |
| 3 | Gemini 4 Argon | 58.80% |
| 4 | Claude Fable 5 | 56.07% |
| 5 | Gemini 3 Flash Preview | 55.92% |
| 6 | Gemini 3.5 Flash | 55.83% |
| 7 | Claude Opus 4.7 | 54.86% |
| 8 | Claude Fable 5.1 | 53.51% |
| 9 | Gemini 3.7 Flash | 53.39% |
| 10 | Claude Opus 4.8 | 53.22% |
| 11 | Gemini 3.6 Flash | 53.15% |
| 12 | Claude Sonnet 5.5 | 52.92% |
| 13 | Gpt 5.1 2025 11 13 | 52.73% |
| 14 | Gemini 3 Pro Preview | 52.20% |
| 15 | Muse Spark | 51.31% |
| 16 | Gemini 2.5 Pro | 50.59% |
| 17 | Claude Opus 5.5 | 49.80% |
| 18 | Gpt 5.2 2025 12 11 | 49.75% |
| 19 | Gpt 5 2025 08 07 | 49.63% |
| 20 | Grok 4.7 | 49.55% |
| 21 | Muse Spark 1.2 | 49.35% |
| 22 | Claude Opus 4 5 20251101 Thinking | 49.16% |
| 23 | Claude Opus 4 6 Thinking | 49.13% |
| 24 | GPT-5.5 | 49.10% |
| 25 | Kimi K3 | 48.88% |
| 26 | GPT-6.1 Sol | 48.84% |
| 27 | GPT-6 Astra | 48.49% |
| 28 | Claude Opus 4 6 | 48.24% |
| 29 | Gemini 3.8 Flash | 48.13% |
| 30 | Gemini 3.1 Flash Lite Preview | 47.60% |
| 31 | Claude Sonnet 5 | 47.54% |
| 32 | O3 2025 04 16 | 47.29% |
| 33 | Claude Opus 4 1 20250805 Thinking | 47.23% |
| 34 | GPT-6 Sol | 47.07% |
| 35 | MiniMax M3 | 46.29% |
| 36 | Claude Opus 4 5 20251101 | 45.17% |
| 37 | MiMo V2.6 Pro | 44.97% |
| 38 | Grok 4.6 | 44.71% |
| 39 | GPT-6 Luna | 44.69% |
| 40 | Claude Sonnet 4 5 20250929 Thinking | 44.13% |
| 41 | GPT-5.6 Sol | 43.97% |
| 42 | Gemini 3.5 Flash Lite | 43.49% |
| 43 | GPT-5.6 Terra | 43.41% |
| 44 | Grok 4.5 | 43.29% |
| 45 | Hunyuan Hy4 Preview | 43.25% |
| 46 | Gpt 5 Mini 2025 08 07 | 43.05% |
| 47 | GLM 5.3 | 42.86% |
| 48 | DeepSeek V4 Pro | 42.47% |
| 49 | GPT-5.6 Luna | 42.39% |
| 50 | Glm 5.1 | 41.60% |
| 51 | Deepseek V4 Flash 0731 | 41.41% |
| 52 | Claude Opus 4 1 20250805 | 41.37% |
| 53 | Gpt 5.4 2026 03 05 | 41.29% |
| 54 | Inkling | 41.19% |
| 55 | DeepSeek V4.1 Flash | 41.17% |
| 56 | MiMo-V2.6-Flash | 41.06% |
| 57 | Gpt 5.4 Nano 2026 03 17 | 41.03% |
| 58 | Glm 5.2 | 40.77% |
| 59 | Qwen3.8 Max | 40.67% |
| 60 | Claude Sonnet 4 5 20250929 | 40.57% |
| 61 | Gemini 2.5 Flash Preview 09 2025 | 40.54% |
| 62 | Deepseek V4 Pro | 40.45% |
| 63 | Gemini 2.5 Flash Thinking | 40.36% |
| 64 | Gemini 2.5 Flash Preview 09 2025 Thinking | 40.33% |
| 65 | Kimi K2.6 | 40.14% |
| 66 | Kimi K2.5 Thinking | 39.32% |
| 67 | Qwen3.7 Max | 38.75% |
| 68 | Nemotron 3 Ultra 550b A55b | 38.62% |
| 69 | Gemini 2.5 Flash | 38.42% |
| 70 | Grok 4 0709 | 38.08% |
| 71 | Grok 4.3 | 38.07% |
| 72 | Inkling Small | 37.89% |
| 73 | Grok 4 Fast Reasoning | 37.38% |
| 74 | Qwen3.6 Plus | 36.89% |
| 75 | Llama4 Maverick Instruct Basic | 36.51% |
| 76 | Claude Sonnet 4 20250514 Thinking | 34.96% |
| 77 | MiniMax M2.7 | 34.44% |
| 78 | Gemini 2.5 Flash Lite Preview 09 2025 Thinking | 34.19% |
| 79 | MiniMax M2.1 | 34.08% |
| 80 | Claude Sonnet 4 20250514 | 33.94% |
| 81 | O4 Mini 2025 04 16 | 33.79% |
| 82 | Mistral Medium 3.5 | 33.75% |
| 83 | Qwen3.5 Flash | 33.00% |
| 84 | Glm 4.7 | 32.77% |
| 85 | Claude Haiku 4 5 20251001 Thinking | 32.68% |
| 86 | Mimo V2.5 Pro | 32.48% |
| 87 | Ling 3.0 Flash 2607 | 32.27% |
| 88 | Grok 4.20 0309 Reasoning | 32.16% |
| 89 | Mimo V2.5 | 31.89% |
| 90 | Qwen3 Vl Plus 2025 09 23 | 31.65% |
| 91 | Qwen3 Max 2026 01 23 | 31.37% |
| 92 | Mercury 2.5 | 31.33% |
| 93 | Gpt 5 Nano 2025 08 07 | 30.44% |
| 94 | Grok 4 Fast Non Reasoning | 30.04% |
| 95 | Ling 3.0 Flash Af Rc3 | 29.30% |
| 96 | Qwen3.8 27b | 28.70% |
| 97 | Grok 4 1 Fast Non Reasoning | 28.35% |
| 98 | Grok 4 1 Fast Reasoning | 28.08% |
| 99 | Gemini 2.5 Flash Lite | 27.11% |
| 100 | Gemini 2.5 Flash Lite Preview 09 2025 | 27.08% |
| 101 | Llama 4 Scout 17B 16E Instruct | 23.31% |
| 102 | Laguna M.1 | 23.11% |
| 103 | Laguna Xs.2 | 21.25% |
| 104 | Command A Plus 05 2026 | 19.72% |
No published result tracked
11 of the 46 models on this site have no published number on this benchmark, so their cells in the home table stay blank rather than reading as zero.
Read this with
Benchmarks in the same category, which usually disagree with this one on purpose.
Sources
Every page used for this benchmark, read on 17 October 2026.
- Benchmark site or the page that publishes itwww.withprotege.ai/
- Independent resultswww.vals.ai/benchmarks/medcode