Choose the task
A coding agent, a finance task and a reasoning exam answer different questions. Start with the capability you need.
OverviewHow to read the evidence
How to read the evidence
How sources, run conditions, and missing results affect a comparison.
Check the comparison
Match evaluator, dataset version, subset, tools, metric and reasoning effort. A shared benchmark name alone is not enough.
Inspect the evidence
Open the original source and the conditions on each result. Missing settings remain unknown, and missing results remain missing.
Reproduce responsibly
Use the author’s harness, pin the configuration and keep the run artifact. A new model run is new evidence, not automatically a reproduction.
Source labels
aa Artificial Analysis. vals Vals AI. Scores compare within each evaluator’s recorded field.
self A model lab’s release table or chart. It may include quoted results for other models; this label identifies the publication, not the runner of every quoted result. Open the chip for the original source.
AIEvals collects published evidence. It does not claim to have independently run the stored reference results. Local user-run reports remain separate.
What a rank means
Ranks are calculated within one comparison group. Ties share a rank, and missing scores are excluded. Error rates and cost prefer lower values; capability metrics usually prefer higher values.
Incomplete tracked-model composites are labelled as such. A catalogue rank is not a rank across every model in the evaluator’s full database.
Confidence intervals, task counts and effort are retained where reported. A small numerical gap does not establish a statistically meaningful difference.
Reading the charts
Score bars
One source group, one unit and a zero baseline. The chart shows the top recorded results, with the complete ledger below it.
Capability & cost
AA Index versus its measured cost per task. This includes workload usage; it is different from API price per million tokens. Hover for the source date.
Evidence coverage
Tests grouped by source chip, plus tests with no result, divided by the full catalogue. Each test counts once; evaluator results take precedence over lab release tables. More coverage is not a higher capability score.
What is needed for a reproducible run
Record the immutable model and endpoint, harness commit, dataset revision and split, prompts, tool policy, reasoning effort, token limits, sampling and seed, repetitions, judge and aggregation. Save raw outputs, per-task outcomes and logs. A public benchmark does not guarantee access to the evaluator’s exact run configuration.
Further reading
Explore Epoch’s benchmark explorer, Artificial Analysis’s capability and cost methodology, and Terminal-Bench’s author execution guide. Original score sources remain attached to each observation in this catalogue.