This model is a fine-tuned version of Qwen/Qwen3-4B-Instruct-2507 using Direct Preference Optimization (DPO) via the Unsloth library.
This repository contains the full-merged 16-bit weights. No adapter loading is required.
Training Configuration
Base model: Qwen/Qwen3-4B-Instruct-2507
Epochs: 1
Learning rate: 1e-06
Beta: 0.1
Max sequence length: 1024
Additional Notes
This DPO model was trained on top of a structured-output SFT LoRA model
(r=16, alpha=32, target_modules=[q_proj, k_proj, v_proj, o_proj]).
The SFT stage was used to improve JSON/YAML/XML formatting stability before applying preference optimization.
