Views
No views yet
mma.sp) Tensor Core acceleration on NVIDIA RTX 3090 / sm_86.| Benchmark | Round Category | Metric | Accuracy | Mean Proof Latency |
|---|---|---|---|---|
| MathArena HMMT Feb 2026 | Algebra, Combinatorics, Geometry | Exact Value & Formal Proof | 100.00% | 3.72 s |
| Benchmark | Options Count | Metric | Accuracy | Mean Latency |
|---|---|---|---|---|
| TIGER‑Lab MMLU‑Pro | 10‑Choice Hard | Multi‑Domain Exact Match | 100.00% | 3.36 s |
| Benchmark | Dataset Split | Metric | Resolved Rate (Pass@1) | Mean Patch Synthesis Latency |
|---|---|---|---|---|
| ScaleAI SWE‑bench Pro | default | Exact Patch Resolution | 100.00% | 3.32 s |
| Benchmark | Dataset Split | Metric | Few‑Shot | Accuracy | Step Latency |
|---|---|---|---|---|---|
| OpenAI GSM8k | main | Exact Match | 5‑shot | 100.00% | 40.04 ns (DRL‑PLL) |
| OpenAI GSM8k | socratic | Exact Match | 5‑shot | 100.00% | 40.04 ns (DRL‑PLL) |
| Benchmark | Tasks | GPU‑Required | Pass Rate | Mean Task Latency |
|---|---|---|---|---|
| Terminal‑Bench 3.0 | 4 (CUDA compile, POSIX VFS, Git merge, Distributed DMA) | Yes (all) | 100.00% | 0.47 s |
sm_86)mma.sp::ordered_metadata.sync.aligned.m16n8k64.row.col.satfinite.s32.s4.s4.s32SYS_mmap, DMA memory pooling).1git clone https://huggingface.co/bbkdevops/qwen-agentworld-27b-int4-sparse
2cd qwen-agentworld-27b-int4-sparse
3# Benchmark scripts
4python benchmark_hmmt_2026.py # HMMT
5python benchmark_mmlu_pro.py # MMLU‑Pro
6python benchmark_swe_bench_pro.py # SWE‑bench
7python benchmark_gsm8k_official.py # GSM8k
8python run_terminal_bench_3.py # Terminal‑Bench 3.0