AIEvalsReproduce

OverviewReproduce a result

Reproduce a result

Create a local evaluation plan with reference results, recipes, and required settings.

1. Choose the evaluation scope

1tests in this plan
0verified public harness
1need access or recipe work

The plan preserves every selected test, including blocked ones. Public code alone is not enough to reproduce an evaluator’s score.

2. Inspect recipes and missing settings

Kimi Hallu Bench Hard w/ tools (Internal, lower is better)Recipe not catalogued

Public execution instructions have not been verified in this catalogue. This status does not claim that the benchmark is closed or impossible to run.

  1. Open the benchmark’s author page.
  2. Confirm code and dataset availability, licence, task split and scoring.
  3. Record the exact harness commit, model, sampling and judge before adding a command.
Author / evaluator instructions ↗

1 source-specific reference results for this model; each remains in the downloaded plan.

Configuration still required

  • Immutable model or endpoint version
  • Dataset revision and split
  • Harness commit
  • Prompt and tool policy
  • Reasoning effort
  • Sampling and seed
  • Task limit and repetitions
  • Judge and aggregation
  • Provider spending limits

A fresh evaluation cannot reproduce the published score exactly until its model version, data, harness, prompts, tools, judge and sampling settings are matched. Unknown source settings remain unknown.

Understand this test →