from peft import PeftModel
from transformers import AutoModelForCausalLM

base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-1.5B", torch_dtype="bfloat16")
model = PeftModel.from_pretrained(base, "sandywong/distractor-lora-clean-only-qwen25-1.5b")

# Or merge into base for inference:
model = model.merge_and_unload()

Part of

Base model: Qwen/Qwen2.5-1.5B
Method: LoRA rank-16 adapters
Training data: HotPotQA + MuSiQue contrastive pairs (19K train samples)
Training steps: 594 (1 epoch), effective batch size 32

Distractor Robustness Training for Multi-hop QA project.