Benchmarks
A home for public, reproducible model evaluations.
Model evaluation results are planned alongside benchmark environment releases. No model scores have been reported yet.
| Model | CloseBench | SettleBench | Pass^8 | Updated |
|---|---|---|---|---|
| Model 1 · To be selected | — | — | — | — |
| Model 2 · To be selected | — | — | — | — |
| Model 3 · To be selected | — | — | — | — |
| Model 4 · To be selected | — | — | — | — |
| Model 5 · To be selected | — | — | — | — |
Last updated:Not yet published · Contamination check:Not yet assessed
Download the environment validation report →How we evaluate
Invariant-based grading
A task succeeds only when the final outcome and required process constraints pass. Releases will specify verifier versions, task splits, and failure reasons.
Reliability across repeated runs
Pass^k measures whether all k independent runs succeed; pass@k measures whether at least one succeeds. Results will include sampling settings and per-task records.
Contamination checks
Evaluation instances are separated from training data. Reports document checking methods and scope. Unreviewed items are never labeled as passed.