This repository contains multiple GGUF quantizations optimized for local inference with llama.cpp, koboldcpp, LM Studio, Ollama, and other GGUF-compatible runtimes.

Available Quantizations

Quant	Recommended Use
Q2_K	Ultra low RAM / mobile
Q3_K_M	Lightweight balanced inference
Q4_K_M	Recommended default
Q5_K_M	Higher quality
Q6_K	Maximum quality GGUF

Base Model

Base: Qwen/Qwen3-0.6B
Fine-tune: CoNDeNse-AI/GLM-5.1-Qwen3-0.6B-CoNDeNse

Features

Strong reasoning for model size
Compact and efficient
Optimized for local inference
GGUF support for CPU/GPU runtimes
Works well for:
- Coding
- Chat
- Lightweight reasoning
- Instruction following

Recommended Runtime

llama.cpp

bash

./llama-cli \\
  -m model.gguf \\
  -c 4096 \\
  -ngl 999

Suggested Settings

Setting	Value
Temperature	0.7
Top_p	0.9
Context Length	4096
Repeat Penalty	1.05

Compatibility

Tested with:

llama.cpp
LM Studio
Ollama
KoboldCpp
Jan

Hardware Recommendations

Quant	RAM Needed
Q2_K	~1 GB
Q3_K_M	~1.5 GB
Q4_K_M	~2 GB
Q5_K_M	~2.5 GB
Q6_K	~3+ GB

Usage Notes

Q4_K_M is recommended for most users as the best balance between quality and speed. Q6_K provides the highest quality but requires more memory.

Credits

Base model by Qwen team
Fine-tune by CoNDeNse-AI
GGUF conversion using Unsloth + llama.cpp ecosystem

Disclaimer

This model may generate incorrect, biased, or fabricated information. Use responsibly.

License

Please follow the original license and usage terms of the base model and fine-tuned model.

CoNDeNse-AI/GLM-5.1-Qwen3-0.6B-CoNDeNse-GGUF

GLM-5.1-Qwen3-0.6B-CoNDeNse-GGUF