GPT-OSS-20B fine-tuned on adamo1139/HESOYAM_v0.4 dataset, 1 epoch, chatml format that erases reasoning.
1024 rank, 128 alpha QLoRA made with Unsloth.
It will be undergoing further preference alignment once some issues preventing me from doing it right now will be patched out.
Total batch size 16, learning rate 0.0002 with cosine schedule, with sample packing enabled. Training took about 8 hours on single 3090 Ti.
