56 of 164 problems exhausted the 32 K reasoning budget mid-<think> and are counted as fails under strict academic scoring. Allocate ≥64 K tokens to approach the 83 % ceiling.
Prior methodology note: an earlier evaluation using raw /v1/completions with chat-prose stripping (non-canonical for reasoning models) reported 65.2 %. The numbers above use the canonical chat-completion path.
The native FP16 KV cache costs ~0.25 GB per 1K tokens for this architecture (62 layers × 1024 KV dim × 2 bytes). That is non-trivial at long context: Q4_K_M at ctx=60K needs ~15 GB of KV cache alone.
KV cache quantization — essential for long context on 96 GB
llama.cpp supports quantizing the KV cache with near-zero quality loss:
During integrity audit, one layer (layer 0) had expert keep-indices that differed from the REAP-retained set in ~86 of 154 positions. The bias-value mismatch is bounded by the layer-0 bias natural variance (max |Δ|=0.75 on values ∈ [8.06, 8.88]), so router behavior is essentially unchanged — confirmed by the 5/5 smoke test above. All other 61 layers are bit-perfect. Details in the safetensors model card.
Citation
See the safetensors repo for full citation details. Core references:
