GGUF quantizations of https://huggingface.co/ornith-ai/Ornith-1.5-9B for use with llama.cpp, LM Studio, Ollama (GGUF import), KoboldCpp, and other GGUF-compatible runtimes.

About the Base Model

Ornith-1.5-9B is a 9B-parameter Qwen3.5-family model using a hybrid architecture consisting of:

32 transformer layers
Hybrid linear attention (Gated DeltaNet) and full attention
MRoPE positional encoding
Dense architecture (no routed experts)

These GGUF files are text-only conversions intended for inference in llama.cpp-compatible software.

Conversion Notes

Converted using upstream llama.cpp with native Qwen3.5 support.
The model was converted directly to Q8_0 GGUF and all other quants were generated from that Q8_0 source.
Importance matrix (imatrix) calibration was generated using WikiText-2 and applied to IQ quantizations.
The original Hugging Face configuration advertises an MTP/next-token prediction layer that is not present in the released weights. GGUF metadata was patched so the layer count matches the actual tensors contained in the model.

Available Quantizations

Standard Quants

Q2_K
Q3_K_S
Q3_K_M
Q3_K_L
Q4_0
Q4_1
Q4_K_S
Q4_K_M
Q5_K_S
Q5_K_M
Q6_K
Q8_0

IQ Quants

IQ2_M
IQ3_XXS
IQ3_XS
IQ3_M
IQ4_XS
IQ4_NL

Quant Recommendations

Quant	Recommended Use
IQ2_M / Q2_K	Maximum memory savings
Q3_K_M	Very low RAM systems
Q4_K_M	Best balance of quality and size
Q5_K_M	High-quality daily use
Q6_K	Near-original quality
Q8_0	Highest quality quant available in this repository

Usage with llama.cpp

bash

./llama-cli \
  -m Ornith-1.5-9B-Q4_K_M.gguf \
  -p "Explain quantum computing in simple terms."

NANI-Nithin/Ornith-1.5-9B-GGUF

Ornith-1.5-9B GGUF