LoRA adapter fine-tuned to turn a Vietnamese customer-support ticket into a strict 4-field JSON triage object: intent, urgency, product, sentiment.

Trained as coursework for AICB-P2T3 Day 21 — Fine-tuning & Safety (Track 3).

⚠️ Read this before using the adapter

This adapter fails its own regression gate and is published as a teaching artifact, not as a production model.

It improves the target task (0.765 → 0.970) but degrades unrelated general-knowledge ability by 0.302 (0.758 → 0.456), which is over 15× the lab's 0.020 tolerance. This is textbook catastrophic forgetting: 100% of the training data was ticket → JSON, with no general-domain replay mixed in.

Use it to reproduce the measurement, not to serve traffic. If you want to fix it, mix 1–5% general instruction data into the training set and re-run.

Results

Measured on a frozen 50-item target set and a 15-item general-knowledge regression set. All three rows use the same eval harness; (a) and (b) share the base weights and differ only in prompt.

Run	target	regression	format	latency (ms/sample)
(a) base + naive prompt	0.000	0.758	0.00	3332.1
(b) base + optimized prompt	0.765	0.758	1.00	1031.4
(c) this adapter	0.970	0.456	1.00	1520.8

Verdict: FAILED — target Δ +0.205, regression Δ −0.302 (tolerance 0.020).

Baseline (a) scores 0.000 on target because it scores 0.00 on format: it rarely emits parseable JSON at all, so no field can be scored. The failure is one of output format, not of classification.

Known failure mode

The adapter gets 44/50 target items fully correct. All 6 errors are the same error: urgency predicted as trung_binh when the gold label is thap, on tickets containing the phrase "Khi nào tiện" ("whenever it's convenient").

Low-urgency marker	eval items	correct	wrong	train examples
`Khi nào tiện`	6	0	6	30
`Không vội`	7	7	0	34
`Hỏi cho biết thôi`	5	5	0	22

This is not a data-coverage gap — Khi nào tiện appears 30 times in training, all labelled thap, more often than Hỏi cho biết thôi (22) which the adapter learns perfectly. The likely cause is a conflict with the base model's prior: the phrase opens with Khi nào ("when"), an interrogative about timing that reads as a delivery-chasing question. 30 optimizer steps are enough to teach rules that agree with the prior, but not enough to overwrite one that contradicts it.

Training


Base model	`unsloth/Qwen3.5-4B`
Method	LoRA (PEFT 0.20.0), fp16, not quantized
Placement	text-linear — 12 modules (`q,k,v,o,gate,up,down` + projections)
Rank / alpha	r=16 / α=32
Trainable params	32,464,896
Learning rate	1e-4 (10× the full-fine-tune scale)
Max steps	30 · effective batch 16 (1 × 16 grad-accum)
`max_length`	1024 (measured p95 was 98 — see note below)
Loss mask	`assistant-only`, `supervised_fraction = 0.4149`, verified by decoding the masked and supervised spans back to text
Data	250 synthetic Vietnamese support tickets, 225 train / 25 val, seed 42
Hardware	Colab Free Tesla T4 16 GB, peak 12.01 GB, 1037.5 s

max_length=1024 is the lab's hardware-tier constant, not a tuned value; the measured token-length p95 was 98 and the longest example was 101, so nothing was truncated.

Usage

python

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

BASE = "unsloth/Qwen3.5-4B"
ADAPTER = "minhlt12/qwen3.5-4b-lora-vi-ticket-triage"

tok = AutoTokenizer.from_pretrained(BASE)
model = AutoModelForCausalLM.from_pretrained(BASE, torch_dtype="float16", device_map="auto")
model = PeftModel.from_pretrained(model, ADAPTER)
model.eval()

SYSTEM = "Phân loại ticket sau."
ticket = "Shop ơi, mình đặt nồi chiên không dầu mã đơn DH249548. Thiếu phụ kiện. Khi nào tiện."

msgs = [{"role": "system", "content": SYSTEM}, {"role": "user", "content": ticket}]
prompt = tok.apply_chat_template(msgs, tokenize=False, add_generation_prompt=True)
out = model.generate(**tok(prompt, return_tensors="pt").to(model.device),
                     max_new_tokens=64, do_sample=False)
print(tok.decode(out[0], skip_special_tokens=True))
# -> {"intent": "san_pham_loi", "urgency": "trung_binh", ...}
#    note: gold urgency here is "thap" — this is the known failure mode above

Greedy decoding (do_sample=False) is what the reported numbers use.

Limitations

Fails its regression gate (see the warning at the top). Do not deploy.
Trained on synthetic tickets from a fixed template family. Real support text is messier, and these numbers will not transfer unchanged.
Vietnamese only. The label vocabulary is closed: intent ∈ {doi_tra, van_chuyen, hoan_tien, san_pham_loi, hoi_thong_tin}, urgency ∈ {cao, trung_binh, thap}, sentiment ∈ {tieu_cuc, trung_tinh, tich_cuc}.
30 training steps on 225 examples. This is a lab-scale run, not a converged one.
The eval set is small (50 target / 15 regression); treat differences under a few points as noise.

Framework versions

PEFT 0.20.0
Base model: unsloth/Qwen3.5-4B