This model is a fine-tuned version of Qwen2.5-0.5B-Instruct optimized for strict instruction-following and concise, human-aligned responses. It was trained using a two-stage alignment pipeline:

Supervised Fine-Tuning (SFT): Trained on the HuggingFaceH4/no_robots dataset to improve structural formatting and zero-shot task execution.

Direct Preference Optimization (DPO): Aligned using the argilla/dpo-mix-7k dataset to reduce conversational verbosity and prioritize helpful, direct answers.

The model utilizes PEFT/LoRA adapters and was trained efficiently within memory constraints using the Unsloth framework.

This qwen2 model was trained 2x faster with Unsloth and Huggingface's TRL library.