JustinTong commited on
Commit
d2df1d2
·
verified ·
1 Parent(s): 775335f

Align Terminal-Bench denominator to full 89-task set

Browse files
Files changed (1) hide show
  1. README.md +1 -1
README.md CHANGED
@@ -123,7 +123,7 @@ The benchmark results below were produced with this NVFP4 checkpoint on 8x NVIDI
123
  | Benchmark | Evaluation protocol | Score |
124
  |---|---|---:|
125
  | GSM8K | Full 1,319-example split, single-shot, sgl-eval | **97.42% (1,285/1,319)** |
126
- | Terminal-Bench 2.1 | terminus-2 harness, `temperature=1.0`, `top_p=0.95` | **87.5% (77/88)** |
127
  | DeepSWE | mini-swe-agent harness, `temperature=0.95`, `top_p=1.0` | **68.1% (77/113)** |
128
  | AIME 2026 | 30 problems x 16 rollouts, pass@1, sgl-eval | **94.17%** (majority@16 **100%**) |
129
 
 
123
  | Benchmark | Evaluation protocol | Score |
124
  |---|---|---:|
125
  | GSM8K | Full 1,319-example split, single-shot, sgl-eval | **97.42% (1,285/1,319)** |
126
+ | Terminal-Bench 2.1 | terminus-2 harness, `temperature=1.0`, `top_p=0.95` | **86.5% (77/89)** |
127
  | DeepSWE | mini-swe-agent harness, `temperature=0.95`, `top_p=1.0` | **68.1% (77/113)** |
128
  | AIME 2026 | 30 problems x 16 rollouts, pass@1, sgl-eval | **94.17%** (majority@16 **100%**) |
129