Note: You may encounter ImportError: Found an incompatible version of torchao if your torchao version is below 0.16.0. Run pip install --upgrade torchao to fix this.

Inference Code

python

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base_model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-0.6B-Base", torch_dtype="auto", device_map="auto")
tokenizer = AutoTokenizer.from_pretrained("Abdullah121212/qwen3-0.6b-sft-best")

model = PeftModel.from_pretrained(base_model, "Abdullah121212/qwen3-0.6b-sft-best")
model.eval()

prompt = "Explain the difference between supervised and unsupervised learning in simple terms."
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=256, do_sample=True, temperature=0.7)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Pipeline

Baseline (Qwen3-0.6B-Base)
  -> SFT (5 LoRA trials, this is trial 3 = best)
    -> DPO (see Abdullah121212/qwen3-0.6b-dpo-best)

Framework Versions

PEFT 0.19.1
Transformers 4.x
TRL 0.x
PyTorch 2.x