Align Terminal-Bench denominator to full 89-task set
Browse files
README.md
CHANGED
|
@@ -123,7 +123,7 @@ The benchmark results below were produced with this NVFP4 checkpoint on 8x NVIDI
|
|
| 123 |
| Benchmark | Evaluation protocol | Score |
|
| 124 |
|---|---|---:|
|
| 125 |
| GSM8K | Full 1,319-example split, single-shot, sgl-eval | **97.42% (1,285/1,319)** |
|
| 126 |
-
| Terminal-Bench 2.1 | terminus-2 harness, `temperature=1.0`, `top_p=0.95` | **
|
| 127 |
| DeepSWE | mini-swe-agent harness, `temperature=0.95`, `top_p=1.0` | **68.1% (77/113)** |
|
| 128 |
| AIME 2026 | 30 problems x 16 rollouts, pass@1, sgl-eval | **94.17%** (majority@16 **100%**) |
|
| 129 |
|
|
|
|
| 123 |
| Benchmark | Evaluation protocol | Score |
|
| 124 |
|---|---|---:|
|
| 125 |
| GSM8K | Full 1,319-example split, single-shot, sgl-eval | **97.42% (1,285/1,319)** |
|
| 126 |
+
| Terminal-Bench 2.1 | terminus-2 harness, `temperature=1.0`, `top_p=0.95` | **86.5% (77/89)** |
|
| 127 |
| DeepSWE | mini-swe-agent harness, `temperature=0.95`, `top_p=1.0` | **68.1% (77/113)** |
|
| 128 |
| AIME 2026 | 30 problems x 16 rollouts, pass@1, sgl-eval | **94.17%** (majority@16 **100%**) |
|
| 129 |
|