Link	Type	Size/GB	Notes
P1 P2 P3 P4 P5 P6 P7 P8	Q2_K	373.0
P1 P2 P3 P4 P5 P6 P7 P8 P9	Q3_K_S	442.2
P1 P2 P3 P4 P5 P6 P7 P8 P9 P10	Q3_K_M	489.2	lower quality
P1 P2 P3 P4 P5 P6 P7 P8 P9 P10 P11	Q3_K_L	530.6
P1 P2 P3 P4 P5 P6 P7 P8 P9 P10 P11 P12	IQ4_XS	551.6
P1 P2 P3 P4 P5 P6 P7 P8 P9 P10 P11 P12 P13	Q4_K_S	582.6	fast, recommended
P1 P2 P3 P4 P5 P6 P7 P8 P9 P10 P11 P12 P13	Q4_K_M	620.7	fast, recommended
P1 P2 P3 P4 P5 P6 P7 P8 P9 P10 P11 P12 P13 P14 P15	Q5_K_S	706.5
P1 P2 P3 P4 P5 P6 P7 P8 P9 P10 P11 P12 P13 P14 P15 P16 P17	Q5_K_M	728.3
P1 P2 P3 P4 P5 P6 P7 P8 P9 P10 P11 P12 P13 P14 P15 P16 P17 P18	Q6_K	842.7	very good quality
P1 P2 P3 P4 P5 P6 P7 P8 P9 P10 P11 P12 P13 P14 P15 P16 P17 P18 P19 P20 P21 P22 P23 P24 P25	Q8_0	1091.2	fast, best quality

Here is a handy graph by ikawrakow comparing some lower-quality quant types (lower is better):

FAQ / Model Request

See https://huggingface.co/mradermacher/model_requests for some answers to questions you might have and/or if you want some other model quantized.

Thanks

I thank my company, nethype GmbH, for letting me use its servers and providing upgrades to my workstation to enable this work in my free time.