Grug 27B quantized with
oMLX oQ5E (exponent-aware 5-bit) at
float16 precision. A token-efficient, reasoning-optimized variant of
ProCreations/grug-27b, built on the Qwen3.6-27B architecture.
Grug 27B is a LoRA fine-tuned reasoning model that adopts the "grug" thinking style — producing concise, token-efficient chain-of-thought reasoning. The model was fine-tuned with r=32 on all text-stack linear layers and the LoRA weights were merged back into the base bf16 weights. This quantized variant converts the model to float16 for ~20% faster prefill on Apple Silicon (M1/M2/M3/M4), then applies the oQ5E exponent-aware 5-bit quantization scheme from oMLX v0.5.7.
The oQ5E format uses exponent-aware quantization to preserve the dynamic range of larger-magnitude weights while compressing the bulk of the weight distribution. The fp16 base dtype was chosen over bf16 for ~20% faster prefill throughput on Apple Silicon GPUs.
1from mlx_lm import load, generate
2
3model, tokenizer = load("your-username/grug-27b-oQ5E-fp16")
4response = generate(
5 model, tokenizer,
6 prompt="Your prompt here",
7 max_tokens=1024,
8 temp=0.6,
9 top_p=0.95
10)
11print(response)
This quantized model card and accompanying files are released under the
Apache License 2.0, inherited from the original
ProCreations/grug-27b model. The base architecture is derived from
Qwen/Qwen3.6-27B.
See the original model card for full license terms:
https://huggingface.co/ProCreations/grug-27b1@misc{grug27b,
2 title={Grug 27B — Token-Efficient Reasoning with the "Grug" Thinking Style},
3 author={ProCreations},
4 year={2025},
5 url={https://huggingface.co/ProCreations/grug-27b}
6}
7
8@misc{qwen3,
9 title={Qwen3.6-27B Technical Report},
10 author={Qwen Team},
11 year={2025},
12 url={https://huggingface.co/Qwen/Qwen3.6-27B}
13}
