Views
No views yet
| config | size | PPL | HumanEval | decode |
|---|---|---|---|---|
| Q4_K_M | 17.97 GB | 7.67 | 82.3% | slow |
| IQ4_XS + imatrix v5 | 15.08 GB | 7.71 | 84.8% | 56 tok/s |
| IQ4_XS + imatrix + MTP | 15.31 GB | 7.66 | 82.9% | 89-97 tok/s |
| IQ4_XS + imatrix, out=Q5_K | 14.94 GB | 7.68 | 83.5% | 56 tok/s |
| pure IQ4_XS + imatrix | 14.31 GB | 7.97 | 82.9% | 47 tok/s |