Views
No views yet
| Type | Bits | Quality | Description |
|---|---|---|---|
| IQ1 | 1-bit | very Low | Minimal footprint; worse than Q2/IQ2 |
| Q2/IQ2 | 2-bit | 🟥 Low | Minimal footprint; only for tests |
| Q3/IQ3 | 3-bit | 🟧 Low–Med | “Medium” variant |
| Q4/IQ4 | 4-bit | 🟩 Med–High | “Medium” — 4-bit |
| **Q5 ** | 5-bit | 🟩🟩 High | Excellent general-purpose quant |
| **Q6_K ** | 6-bit | 🟩🟩🟩 Very High | Almost FP16 quality, larger size |
| **Q8 ** | 8-bit | 🟩🟩🟩🟩 | Near-lossless baseline |