Views
No views yet
Join the community: Discord
heterodoxin/qwen2.5-7b-instruct-apostate, an
Apostate weight-edit uncensored model.
Pick a single quant or the full-quality bf16.| file | quant | notes |
|---|---|---|
qwen2.5-7b-instruct-apostate-bf16.gguf | BF16 | full quality (default) |
qwen2.5-7b-instruct-apostate-Q2_K.gguf | Q2_K | smallest, lowest quality |
qwen2.5-7b-instruct-apostate-Q3_K_M.gguf | Q3_K_M | small |
qwen2.5-7b-instruct-apostate-Q4_K_M.gguf | Q4_K_M | recommended balance |
qwen2.5-7b-instruct-apostate-Q5_K_M.gguf | Q5_K_M | higher quality |
qwen2.5-7b-instruct-apostate-Q6_K.gguf | Q6_K | near-lossless |
qwen2.5-7b-instruct-apostate-Q8_0.gguf | Q8_0 | highest quality quant |
ollama run hf.co/heterodoxin/qwen2.5-7b-instruct-apostate-gguf:Q4_K_M # any quant tag above, or BF161huggingface-cli download heterodoxin/qwen2.5-7b-instruct-apostate-gguf qwen2.5-7b-instruct-apostate-Q4_K_M.gguf --local-dir .
2./llama-cli -m qwen2.5-7b-instruct-apostate-Q4_K_M.gguf -p "your prompt"Q4_K_M for any quant in the table, or qwen2.5-7b-instruct-apostate-bf16.gguf for full quality.