This is the Final Submission Model for the competition. It is a merged 16-bit model, fine-tuned using Direct Preference Optimization (DPO).

Base Model: SFT-Tuned Qwen3-4B
Adapter Checkpoint: Step 50
Method: DPO with Unsloth

Usage

python

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model_id = "takenoko888/dpo-qwen-cot-merged-v2-final-50"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.float16,
    device_map="auto"
)