Qwen3 is the latest generation of large language models in Qwen series, offering a comprehensive suite of dense and mixture-of-experts (MoE) models. Built upon extensive training, Qwen3 delivers groundbreaking advancements in reasoning, instruction-following, agent capabilities, and multilingual support

Model Files

File Name	Size	Quantization	Format	Description
`Qwen3_0.6B.F32.gguf`	2.39 GB	FP32	GGUF	Full precision (float32) version
`Qwen3_0.6B.BF16.gguf`	1.2 GB	BF16	GGUF	BFloat16 precision version
`Qwen3_0.6B.F16.gguf`	1.2 GB	FP16	GGUF	Float16 precision version
`Qwen3_0.6B.Q3_K_M.gguf`	347 MB	Q3_K_M	GGUF	3-bit quantized (K M variant)
`Qwen3_0.6B.Q3_K_S.gguf`	323 MB	Q3_K_S	GGUF	3-bit quantized (K S variant)
`Qwen3_0.6B.Q4_K_M.gguf`	397 MB	Q4_K_M	GGUF	4-bit quantized (K M variant)
`Qwen3_0.6B.Q4_K_S.gguf`	383 MB	Q4_K_S	GGUF	4-bit quantized (K S variant)
`Qwen3_0.6B.Q5_K_M.gguf`	444 MB	Q5_K_M	GGUF	5-bit quantized (K M variant)
`Qwen3_0.6B.Q8_0.gguf`	639 MB	Q8_0	GGUF	8-bit quantized
`.gitattributes`	2.04 kB	—	—	Git LFS tracking file
`config.json`	31 B	—	—	Configuration placeholder
`README.md`	3.53 kB	—	—	Model documentation

Quants Usage

(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants)

Link	Type	Size/GB	Notes
GGUF	Q2_K	0.4
GGUF	Q3_K_S	0.5
GGUF	Q3_K_M	0.5	lower quality
GGUF	Q3_K_L	0.5
GGUF	IQ4_XS	0.6
GGUF	Q4_K_S	0.6	fast, recommended
GGUF	Q4_K_M	0.6	fast, recommended
GGUF	Q5_K_S	0.6
GGUF	Q5_K_M	0.7
GGUF	Q6_K	0.7	very good quality
GGUF	Q8_0	0.9	fast, best quality
GGUF	f16	1.6	16 bpw, overkill

Here is a handy graph by ikawrakow comparing some lower-quality quant types (lower is better):

prithivMLmods/Qwen3-0.6B-GGUF

Qwen3-0.6B-GGUF

Model Files

Quants Usage

prithivMLmods/
Qwen3-0.6B-GGUF