You can deploy this model and start earning money today!

kusakana/
ReasoningQAT-Qwen3-4B-2bit

Name: ReasoningQAT-Qwen3-4B-2bit
Availability: InStock

Views

No views yet

ReasoningQAT-Qwen3-4B-2bit

This model is a 2-bit pseudo-quantized version of Qwen3-4B, trained with Quantization-Aware Training (QAT) for reasoning tasks.

Details

Base model: Qwen3-4B
Quantization: W2G128 (2-bit weights, group size 128)
Format: Pseudo-quantized (stored in FP16; weights lie on 2-bit quantization grids)
Method: ReasoningQAT — QAT combining knowledge distillation with teacher-confidence-weighted DFT loss, trained end-to-end on reasoning data

Citation

bibtex

@inproceedings{
  okoshi2026towards,
  title={Towards Quantization-Aware Training for Ultra-Low-Bit Reasoning {LLM}s},
  author={Yasuyuki Okoshi and Hikari Otsuka and Daichi Fujiki and Masato Motomura},
  booktitle={The Fourteenth International Conference on Learning Representations},
  year={2026},
  url={https://openreview.net/forum?id=Azsd2qyK6C}
}

kusakana/ReasoningQAT-Qwen3-4B-2bit

ReasoningQAT-Qwen3-4B-2bit

Details

Citation

kusakana/
ReasoningQAT-Qwen3-4B-2bit