Property	Value
Source model	Carnice-9B-SFT-Fable5
Base model	kai-os/Carnice-9b
Training dataset	ermiaazarkhalili/Fable-5-Complete-2M-Clean
Architecture	`Qwen3_5ForCausalLM`
Parameters	8.95 B
Layers	32
Vocab size	248,320
License	APACHE-2.0
Developed by	Behrooz Azarkhalili

Available quantizations

File	Quant	Size	Notes
`carnice-9b-sft-fable5.q4_k_m.gguf`	Q4_K_M	5.24 GiB	Smallest here; the common choice for local inference on limited VRAM.
`carnice-9b-sft-fable5.q5_k_m.gguf`	Q5_K_M	6.02 GiB	Balanced. The usual default when Q4_K_M feels lossy.
`carnice-9b-sft-fable5.q8_0.gguf`	Q8_0	8.87 GiB	Largest, closest to the merged weights. Use when disk is not the constraint.

Sizes above are the actual file sizes in this repository, not estimates.

Quick start

Ollama

ollama pull hf.co/ermiaazarkhalili/Carnice-9B-SFT-Fable5-GGUF:Q4_K_M

llama.cpp

llama-cli -hf ermiaazarkhalili/Carnice-9B-SFT-Fable5-GGUF:Q4_K_M -p "Tell me a fable about a clever fox." -n 256

Download a single file

python

from huggingface_hub import hf_hub_download

path = hf_hub_download("ermiaazarkhalili/Carnice-9B-SFT-Fable5-GGUF", "carnice-9b-sft-fable5.q4_k_m.gguf")
print(path)

Limitations

Quantization is lossy: outputs will not match the source model exactly, and the effect grows as the quant level drops.
Primarily English training data.
Not safety-tuned. Use appropriate guardrails.
Research and prototyping, not production without further evaluation.

Citation

bibtex

@misc{azarkhalili2026_carnice_9b_sft_fable5_gguf,
    author = {Azarkhalili, Behrooz},
    title = {Carnice-9B-SFT-Fable5-GGUF},
    year = {2026},
    publisher = {Hugging Face},
    url = {https://huggingface.co/ermiaazarkhalili/Carnice-9B-SFT-Fable5-GGUF}
}

Acknowledgments

llama.cpp for the GGUF format and tooling
Unsloth and Hugging Face TRL
Compute Canada / DRAC for HPC resources