Link	Type	Size/GB	Notes
P1 P2 P3 P4 P5 P6	Q2_K	267.5
P1 P2 P3 P4 P5 P6	IQ3_XS	289.4
P1 P2 P3 P4 P5 P6 P7	IQ3_S	303.1	beats Q3_K*
P1 P2 P3 P4 P5 P6 P7	Q3_K_S	303.1
P1 P2 P3 P4 P5 P6 P7	IQ3_M	326.4
P1 P2 P3 P4 P5 P6 P7 P8	Q3_K_M	353.4	lower quality
P1 P2 P3 P4 P5 P6 P7 P8	IQ4_XS	383.6
P1 P2 P3 P4 P5 P6 P7 P8 P9	Q3_K_L	393.6
P1 P2 P3 P4 P5 P6 P7 P8 P9	IQ4_NL	403.7	prefer IQ4_XS
P1 P2 P3 P4 P5 P6 P7 P8 P9	Q4_K_S	403.7	fast, recommended
P1 P2 P3 P4 P5 P6 P7 P8 P9	Q4_K_M	434.1	fast, recommended
P1 P2 P3 P4 P5 P6 P7 P8 P9 P10 P11	Q5_K_S	484.2
P1 P2 P3 P4 P5 P6 P7 P8 P9 P10 P11	Q5_K_M	503.6
P1 P2 P3 P4 P5 P6 P7 P8 P9 P10 P11 P12	Q6_K	577.4	very good quality
P1 P2 P3 P4 P5 P6 P7 P8 P9 P10 P11 P12 P13 P14 P15 P16 P17	Q8_0	747.5	fast, best quality
P1 P2 P3 P4 P5 P6 P7 P8 P9 P10 P11 P12 P13 P14 P15 P16 P17 P18 P19 P20 P21 P22 P23 P24 P25 P26 P27 P28 P29 P30 P31 P32	SOURCE	1405.9	source gguf, only provided when it was hard to come by

Here is a handy graph by ikawrakow comparing some lower-quality quant types (lower is better):

FAQ / Model Request

See https://huggingface.co/mradermacher/model_requests for some answers to questions you might have and/or if you want some other model quantized.

Thanks

I thank my company, nethype GmbH, for letting me use its servers and providing upgrades to my workstation to enable this work in my free time.