lab21-qwen2.5-3b-r16 – AI Model by hieuhieu3603 | AlphaNeural AI

You can deploy this model and start earning money today!

hieuhieu3603/
lab21-qwen2.5-3b-r16

like

0

unsloth/Qwen2.5-3B-bnb-4bit

Views

No views yet

Lab 21 — LoRA Fine-tuned Qwen2.5-3B (Vietnamese)

Author: Nguyễn Đoàn Hiếu (2A202600144) Course: AICB-P2T3 · VinUniversity Date: 2026-05-07

Model Description

LoRA adapter fine-tuned trên Qwen2.5-3B (4-bit quantized) với dataset Vietnamese Alpaca. Đây là bài Lab 21 về LoRA/QLoRA Fine-tuning.

Base model: unsloth/Qwen2.5-3B-bnb-4bit
Method: QLoRA 4-bit + Unsloth
LoRA rank: 16, alpha=32
Target modules: q_proj, v_proj
Dataset: 5CD-AI/Vietnamese-alpaca-gpt4-gg-translated (200 samples)
GPU: Tesla T4 (15.6 GB VRAM)
Training time: ~4 min
Peak VRAM: 6.62 GB

Training Hyperparameters

Parameter	Value
Epochs	3
Learning rate	2e-4
LR schedule	Cosine
Warmup ratio	0.10
Optimizer	adamw_8bit
Effective batch size	8 (1 × grad_accum=8)
Max seq length	1024

Evaluation Results

Rank	Trainable Params	Eval Loss	Perplexity
8	1,843,200	1.5577	4.75
16	3,686,400	1.5161	4.55
64	14,745,600	1.4768	4.38

How to Use

python

from peft import PeftModel
from unsloth import FastLanguageModel

# Load base + adapter
model, tokenizer = FastLanguageModel.from_pretrained(
    "unsloth/Qwen2.5-3B-bnb-4bit",
    max_seq_length=1024,
    load_in_4bit=True,
)
model = PeftModel.from_pretrained(model, "hieuhieu3603/lab21-qwen2.5-3b-r16")

# Generate
FastLanguageModel.for_inference(model)
prompt = "### Instruction:\nGiải thích machine learning cho người mới.\n\n### Response:\n"
inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
output = model.generate(**inputs, max_new_tokens=200, temperature=0.7)
print(tokenizer.decode(output[0], skip_special_tokens=True))

References

LoRA paper (Hu et al. 2021)
QLoRA paper (Dettmers et al. 2023)
Unsloth