furina-qwen3.6-27b-merged-2.0-GGUF
This is a quantized GGUF version of the original model
hutaobentao/furina-qwen3.6-27b-merged-2.0.
The quantization was performed locally (compiled on Apple Silicon M3 Max) to provide the community with efficient, offline-ready files for local inference, as the original author stated they did not have the hardware resources to quantize it themselves.
Original Model Information & Author Notes
- Original Author: HuTaoDeDog / hutaobentao
- Base Model: Qwen/Qwen3.6-27B
- Model Characteristics: The original author injected a full prompt and dataset for roleplay purposes. Note from the author: “It might be slightly mechanical because it is a smaller 27B model.”
- Contact the Original Author: If you have questions about the base model, the author prefers contact via the Modelscope China Platform (modelscope.cn) under the username HuTaoDeDog, as they might not check Hugging Face regularly.
Provided Files
This repository contains the model in GGUF format, which is ideal for CPU inference and Apple Metal / unified memory setups.
Provided Files
This repository contains the model in GGUF format, which is ideal for CPU inference and Apple Metal / unified memory setups.
This repository contains multiple GGUF variants to suit different hardware capabilities:
furina-qwen3.6-27b-f16.gguf (Uncompressed / 16-bit): The full-precision model converted directly from the Safetensors without any quality loss. It requires substantial unified memory/RAM (around 54 GB).furina-qwen3.6-27b-Q8_0.gguf (8-bit): Excellent quality, virtually indistinguishable from F16, but significantly smaller. Recommended if you have enough memory for it.furina-qwen3.6-27b-Q6_K.gguf (6-bit): A great high-end compromise, retaining almost all nuances of the model while being memory efficient.furina-qwen3.6-27b-Q4_K_M.gguf (4-bit): Recommended for most users. The standard choice for local inference. It strikes the best balance between small file size, very fast text generation, and almost unnoticeable quality degradation.
How to run
You can use these GGUF files with any compatible local UI or CLI tool, such as:
Disclaimer: I am not the creator of the original model or the fine-tuning dataset. I only provide these GGUF files for the community to run the model locally.
