Affine-Specified Model
Affine Subnet Performance (Updated)
This model has been validated on the Affine subnet benchmark suite across interactive and long-horizon environments. In the latest internal subnet-aligned evaluation run, the model demonstrates strong consistency in tool use, planning, and multi-step recovery under real miner-serving constraints.
| Environment | Metric | Previous | Current | Delta |
|---|
| GAME | Episode Success (%) | 72.4 | 79.8 | +7.4 |
| LGC-V2 | Goal Completion (%) | 68.1 | 76.9 | +8.8 |
| LIVEWEB | Task Pass@1 (%) | 54.6 | 62.7 | +8.1 |
| PRINT | Constraint Accuracy (%) | 84.3 | 89.6 | +5.3 |
| SWE-SYNTH | Verified Patch Rate (%) | 39.2 | 47.8 | +8.6 |
Affine-specific notes
- Stability improves most in environments requiring repeated tool interactions and explicit rollback behavior.
- LIVEWEB and SWE-SYNTH gains are driven by better recovery from transient API/tool failures.
- PRINT remains the highest reliability environment, with low variance across task IDs.
Evaluation setup (Affine subnet aligned)
- Decode:
temperature=0.0 for deterministic scoring runs (with additional stress runs at 0.7)
- Sampling: fixed task IDs per model pair for fair comparison
- Execution mode: Affine local Docker runner (
af eval) with chute-backed OpenAI-compatible endpoints
- Retry policy: bounded retries for timeout/rate-limit events
- Reported values: multi-run aggregate from matched task pools
