Quantization	Size	Description
Q4_K_M	~40% of original	Best balance of quality and size
Q5_K_M	~50% of original	Higher quality, larger size
Q8_0	~80% of original	Highest quality quantization

Quick Start

Using Ollama

bash

# Pull and run directly from HuggingFace
ollama pull hf.co/ermiaazarkhalili/Qwen3-0.6B-SFT-UltraChat-GGUF:Q4_K_M
ollama run hf.co/ermiaazarkhalili/Qwen3-0.6B-SFT-UltraChat-GGUF:Q4_K_M "Hello!"

Using llama.cpp

bash

# Download the GGUF file
huggingface-cli download ermiaazarkhalili/Qwen3-0.6B-SFT-UltraChat-GGUF \
    qwen3_0.6b_sft_ultrachat-q4_k_m.gguf --local-dir ./models

# Run inference
./llama-cli -m ./models/qwen3_0.6b_sft_ultrachat-q4_k_m.gguf \
    -p "What is the capital of France?" -n 128

Using LM Studio

Download the GGUF file from this repository
Import into LM Studio
Select the model and start chatting

Source Model

This is a quantized version of Qwen3-0.6B-SFT-UltraChat, which was fine-tuned on the HuggingFaceH4/ultrachat_200k dataset using SFT.

See the source model card for full training details and usage examples.

License

CC-BY-NC-4.0 (same as source model)

Acknowledgments

Quantization performed using llama.cpp
Source model by ermiaazarkhalili

Citation

If you use this model in your research or applications, please cite:

bibtex

@misc{azarkhalili2026_qwen3_0_6b_sft_ultrachat_gguf,
    author = {Azarkhalili, Behrooz},
    title = {Qwen3-0.6B-SFT-UltraChat-GGUF},
    year = {2026},
    publisher = {Hugging Face},
    url = {https://huggingface.co/ermiaazarkhalili/Qwen3-0.6B-SFT-UltraChat-GGUF}
}

To generate a citable DOI, click "Cite this model" on the model page.

Training Method

Trained using SFT (Supervised Fine-Tuning) on HuggingFaceH4/ultrachat_200k.

Base (merged) model: Qwen3-0.6B-SFT-UltraChat

Acknowledgments

Hugging Face TRL Team for the training library
llama.cpp for the GGUF quantization format
Compute Canada / DRAC for HPC resources
Base model developers for making their weights openly available