Views
No views yet
| Tensor Type | Quant | Reason |
|---|---|---|
| Attention Q/K/V/O | Q8_0 | Precision critical for reasoning & tool use |
| FFN Sparse Experts | Q4_K | 32 experts, only 6 active/token |
| FFN Shared Expert | Q6_K | Always active, don't crush quality |
| MoE Router Gate | Q6_K | Wrong routing = wrong expert = garbage |
| Token Embeddings | Q8_0 | Input quality matters |
| Output Projection | Q8_0 | Output quality matters |
| RMS Norms | F32 | Tiny, numerically sensitive |
| SSM (alpha,beta,dt,a,ba,out,conv1d) | F32 | Recurrence — must stay F32 |
llama-quantize --tensor-type (merged Q8 base)