from peft import PeftModel
from transformers import AutoModelForCausalLM

base = AutoModelForCausalLM.from_pretrained("meta-llama/Llama-3.2-1B", torch_dtype="bfloat16")
model = PeftModel.from_pretrained(base, "sandywong/distractor-lora-clean-only-llama1b")

# Or merge into base for inference:
model = model.merge_and_unload()

Part of

Base model: meta-llama/Llama-3.2-1B
Method: LoRA rank-16 adapters
Training data: HotPotQA + MuSiQue contrastive pairs (19K train samples)
Training steps: 594 (1 epoch), effective batch size 32

Distractor Robustness Training for Multi-hop QA project.