BenchmarksKimi Hallu Bench Hard w/ tools (Internal, lower is better)
Kimi Hallu Bench Hard w/ tools (Internal, lower is better)
Claim-level hallucinations on hard practical prompts with tools; a lower rate is better.
Test definition
- Author / evaluator
- Reported by Moonshot AI
- Version / family
- Kimi Hallu Bench Hard w/ tools (Internal, lower is better) / Kimi Hallucination Benchmark
- Evidence available
- 8 results in 2 comparison groups
How the test is scored
Internal Hard hallucination stress test with tools. Claim-level hallucination rate; lower is better. Not the share of responses containing an error or an everyday-use error rate.
Comparison notes
Publisher-reported results, read 24 October 2026. Compare reported values only within the same source row; tool settings, subsets, versions, judges and metrics must not be combined.
A different task split, tool policy, judge or reasoning effort can change the result. Choose one comparison group below.
Results by evaluator
Select a publication and metric to compare the recorded field.
selfKimi K3.1 Pro: complete release evaluation · Kimi Hallu Bench Hard w/ tools (Internal, lower is better)
- 5.5%
- 8.3%
- 15.0%
- 22.6%
- 23.3%
- 23.3%
Bars start at zero. 6 reported results in this comparison group; ties share a rank. Source ↗
All observations
Every numeric result, with its source and reported run conditions.
8 results
| Model | Result (%) | Source | Run |
|---|---|---|---|
| Kimi K3.1 ProLower is better | 8.3 | self | DetailsInternal Hard hallucination stress test with tools. Claim-level hallucination rate; lower is better. Not the share of responses containing an error or an everyday-use error rate. Read 24 October 2026. All 47 source rows and six comparison columns are retained, with text-only HLE and both ZeroBench metrics split explicitly. Kimi uses max reasoning effort, temperature 1.0 and top-p 0.95 unless a row states otherwise; proprietary comparison columns are labelled max with fallback. Comparison figures include quoted external results and mixed harnesses. Missing cells remain null and do not enter ranks. Lower hallucination rates rank first; every other rank compares larger values within the same row. Terminal-Bench Science’s September DeepSeek score is attributed separately according to footnote 2. These publication claims do not populate independent boards. Highspeed has no independently identified results and inherits none of these scores.
|
| 8.3Kimi Hallu Bench Hard with tools (predecessor) | self | DetailsClaim-level hallucination rate, lower is better. Error-containing responses are 42.4% of the prior rate; that ratio is not an absolute error rate. Chart labels and explicit release prose, read 24 October 2026. These are internal publisher evaluations. The finance-chart Business Operation value differs from the full-table value and retains its own row. No Highspeed scores are inferred.
| |
| Claude Fable 5.1Lower is better | 23.3 | self | DetailsInternal Hard hallucination stress test with tools. Claim-level hallucination rate; lower is better. Not the share of responses containing an error or an everyday-use error rate. Read 24 October 2026. All 47 source rows and six comparison columns are retained, with text-only HLE and both ZeroBench metrics split explicitly. Kimi uses max reasoning effort, temperature 1.0 and top-p 0.95 unless a row states otherwise; proprietary comparison columns are labelled max with fallback. Comparison figures include quoted external results and mixed harnesses. Missing cells remain null and do not enter ranks. Lower hallucination rates rank first; every other rank compares larger values within the same row. Terminal-Bench Science’s September DeepSeek score is attributed separately according to footnote 2. These publication claims do not populate independent boards. Highspeed has no independently identified results and inherits none of these scores.
|
| GPT-6 AstraLower is better | 5.5 | self | DetailsInternal Hard hallucination stress test with tools. Claim-level hallucination rate; lower is better. Not the share of responses containing an error or an everyday-use error rate. Read 24 October 2026. All 47 source rows and six comparison columns are retained, with text-only HLE and both ZeroBench metrics split explicitly. Kimi uses max reasoning effort, temperature 1.0 and top-p 0.95 unless a row states otherwise; proprietary comparison columns are labelled max with fallback. Comparison figures include quoted external results and mixed harnesses. Missing cells remain null and do not enter ranks. Lower hallucination rates rank first; every other rank compares larger values within the same row. Terminal-Bench Science’s September DeepSeek score is attributed separately according to footnote 2. These publication claims do not populate independent boards. Highspeed has no independently identified results and inherits none of these scores.
|
| Claude Opus 5.5Lower is better | 15.0 | self | DetailsInternal Hard hallucination stress test with tools. Claim-level hallucination rate; lower is better. Not the share of responses containing an error or an everyday-use error rate. Read 24 October 2026. All 47 source rows and six comparison columns are retained, with text-only HLE and both ZeroBench metrics split explicitly. Kimi uses max reasoning effort, temperature 1.0 and top-p 0.95 unless a row states otherwise; proprietary comparison columns are labelled max with fallback. Comparison figures include quoted external results and mixed harnesses. Missing cells remain null and do not enter ranks. Lower hallucination rates rank first; every other rank compares larger values within the same row. Terminal-Bench Science’s September DeepSeek score is attributed separately according to footnote 2. These publication claims do not populate independent boards. Highspeed has no independently identified results and inherits none of these scores.
|
| DeepSeek V4.1 Flash 1017Lower is better | 23.3 | self | DetailsInternal Hard hallucination stress test with tools. Claim-level hallucination rate; lower is better. Not the share of responses containing an error or an everyday-use error rate. Read 24 October 2026. All 47 source rows and six comparison columns are retained, with text-only HLE and both ZeroBench metrics split explicitly. Kimi uses max reasoning effort, temperature 1.0 and top-p 0.95 unless a row states otherwise; proprietary comparison columns are labelled max with fallback. Comparison figures include quoted external results and mixed harnesses. Missing cells remain null and do not enter ranks. Lower hallucination rates rank first; every other rank compares larger values within the same row. Terminal-Bench Science’s September DeepSeek score is attributed separately according to footnote 2. These publication claims do not populate independent boards. Highspeed has no independently identified results and inherits none of these scores.
|
| GLM 5.3Lower is better | 22.6 | self | DetailsInternal Hard hallucination stress test with tools. Claim-level hallucination rate; lower is better. Not the share of responses containing an error or an everyday-use error rate. Read 24 October 2026. All 47 source rows and six comparison columns are retained, with text-only HLE and both ZeroBench metrics split explicitly. Kimi uses max reasoning effort, temperature 1.0 and top-p 0.95 unless a row states otherwise; proprietary comparison columns are labelled max with fallback. Comparison figures include quoted external results and mixed harnesses. Missing cells remain null and do not enter ranks. Lower hallucination rates rank first; every other rank compares larger values within the same row. Terminal-Bench Science’s September DeepSeek score is attributed separately according to footnote 2. These publication claims do not populate independent boards. Highspeed has no independently identified results and inherits none of these scores.
|
| Kimi K3Lower is better | 23.4Kimi Hallu Bench Hard with tools (predecessor) | self | DetailsClaim-level hallucination rate, lower is better. Error-containing responses are 42.4% of the prior rate; that ratio is not an absolute error rate. Chart labels and explicit release prose, read 24 October 2026. These are internal publisher evaluations. The finance-chart Business Operation value differs from the full-table value and retains its own row. No Highspeed scores are inferred.
|
Reproduction
Public execution instructions have not been verified in this catalogue. This status does not claim that the benchmark is closed or impossible to run.
Execution documentation ↗