Views
No views yet
| File | Description |
|---|---|
mla_absorbed_cache_report.md | 🚀 MLA 吸收式缓存优化（中文） |
mla_absorbed_cache_report_en.md | 🚀 Absorbed MLA Cache Optimization (English) |
kv_cache_compression_report.md | 📊 KV 缓存压缩完整方案（中文） |
kv_compress_plan.md | 🗺 详细实施计划（中文） |
| Approach | KV/token | Compression | Error |
|---|---|---|---|
| Standard MHA | 270 KB | 1x | — |
| Absorbed MLA | 30.4 KB | 8.9x | 0 |
| + per-channel INT8 | 15.2 KB | 17.8x | 0.011 |
| + INT4 (extreme) | 8.4 KB | 32x | 0.112 |
Text-only models can be aggressively compressed (quantization + eviction), while reasoning models (R1-class) must be conservative (quantization + sliding window + tiered storage). Scenario classification matters more than the algorithm itself.