File	Size	Description
Q4_K_M	~986 MB	Best balance between quality and size
Q5_K_M	~1.13 GB	Better quality while remaining compact
Q6_K	~1.27 GB	High quality
Q8_0	~1.65 GB	Highest quality quantization

llama.cpp

bash

./llama-cli \
-m codemate-qwen-Q4_K_M.gguf \
-c 32768 \
-p "Explain Python decorators with an example."

Ollama

Create a Modelfile:

dockerfile

FROM ./codemate-qwen-Q8_0.gguf

PARAMETER num_ctx 32768
PARAMETER stop "<|im_end|>"

TEMPLATE \"\"\"
<|im_start|>system
{{ .System }}<|im_end|>
<|im_start|>user
{{ .Input }}<|im_end|>
<|im_start|>assistant
\"\"\"

Build:

bash

ollama create codemate -f Modelfile
ollama run codemate

Notes

This repository contains quantized versions only.

For the original merged model:

https://huggingface.co/micymike/CodeMate-Qwen-1.5B-32K-Distilled-on-Claude-Fable-5

Disclaimer

This project is an independent research effort and is not affiliated with or endorsed by Anthropic, Claude, Alibaba, or Qwen.

Author

Built by micymike 🇰🇪

micymike/CodeMate-Qwen-1.5B-32K-Distilled-on-Claude-Fable-5-gguf

CodeMate-Qwen-1.5B-32K-Distilled-on-Claude-Fable-5-GGUF

Available Quantizations

llama.cpp

Ollama

Notes

Disclaimer

Author

micymike/
CodeMate-Qwen-1.5B-32K-Distilled-on-Claude-Fable-5-gguf