Views
No views yet
| Quant | Size | Download | Notes |
|---|---|---|---|
| Q2_K | ~12 GB | Download | Smallest - noticeable quality loss |
| Q3_K_M | ~15 GB | Download | Good for low-VRAM setups |
| Q4_K_M | 21 GB | Download | Recommended - best size/quality balance |
| Q6_K | 29 GB | Download | Near-lossless |
| Q8_0 | 37 GB | Download | Essentially full precision |
1# Server - OpenAI-compatible API on :8080
2llama-server -m ornith-1.0-35b-uncensored-Q4_K_M.gguf -ngl 99 -c 2048 --jinja --port 8080
3
4# CLI
5llama-cli -m ornith-1.0-35b-uncensored-Q4_K_M.gguf -ngl 99 --jinja--jinja is required - without it the model uses a generic template and compliance degrades.Qwen3_5MoeForConditionalGeneration (256 routed experts + 1 shared expert per layer, 8 active per token).