Views
No views yet
qwen3_5_moe, Qwen3.5 MoE) for llama.cpp,
Ollama, LM Studio, etc. Text-only conversion.-0000N-of-0000M.gguf). Download all
shards of your chosen quant — llama.cpp/Ollama load them automatically from the
first shard.| Quant | Bits (approx) | Use |
|---|---|---|
| Q2_K | ~2.6 | smallest, lowest RAM, lossy |
| Q3_K_S / Q3_K_M / Q3_K_L | ~3 | small, budget RAM |
| Q4_K_S / Q4_K_M | ~4 | recommended balance |
| Q5_K_S / Q5_K_M | ~5 | higher quality |
| Q6_K | ~6 | near-lossless |
| Q8_0 | 8 | highest fidelity GGUF |
1# download all shards of one quant (e.g. Q4_K_M), then:
2./llama-cli -m CyberStrike-OffSec-35B-Q4_K_M-00001-of-0000N.gguf -p "What is SQL injection?"1# after downloading a quant's shards:
2printf 'FROM ./CyberStrike-OffSec-35B-Q4_K_M-00001-of-0000N.gguf\n' > Modelfile
3ollama create cyberstrike -f Modelfile
4ollama run cyberstrikeQwen/Qwen3.6-35B-A3B