Qwen1.5-0.5B-Chat DPO fine-tuned on the Orca Math dataset that consists of ~200K grade school math word problems and open-ended and multiple choice questions from different EPFL courses.
The model was developed during the course Modern Natural Language Processing (CS-552).
Its aim is to fine-tune the base model (Qwen/Qwen1.5-0.5B-Chat) to accurately
answer open-ended and multiple-choice questions from Orca Math dataset and various EPFL courses.
HuggingFace dataset : microsoft/orca-math-word-problems-200k
The EPFL dataset is not publicly available.
-
Training regime: The model is trained on EPFL dataset with cDPO with bf16 mixed precision, $\beta=0.2$, $lr=3 \times 10^{-6}$, and $label_smoothing=0.2$. It is then trained on Orca dataset but without label_smoothing and thus original DPO.
-
PEFT 0.10.0
