from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-7B", torch_dtype="bfloat16")
model = PeftModel.from_pretrained(base, "muzaffercky/Sorjin1-LoRA")

# Merge adapter into base weights
model = model.merge_and_unload()

Training Details

Base model: Qwen/Qwen2.5-7B
Dataset: muzaffercky/kurdish-kurmanji-theses
LoRA rank: 64 | Alpha: 128 | Dropout: 0.05
Target modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
Trainable parameters: 161,480,704 (2.08% of total)
Final train loss: 2.057 | Final eval loss: 1.8552

Model Family

Model	Description
Sorjîn1-LoRA (this model)	Raw LoRA adapter weights
Sorjîn1-7B	Continually pretrained base model
Sorjîn1-7B-Instruct	Merged with Qwen2.5-7B-Instruct