A LoRA adapter that rewrites colloquial Vietnamese legal questions into formal statutory
queries, built as the input node of a RAG system over 242,063 Vietnamese statutory
articles. Published together with the retrieval-level evaluation used to assess it.
Scope note: on the held-out test set this adapter did not improve retrieval — passing
the user's question through unchanged scored higher. It is released for reproducibility of
the evaluation and for the diagnostic findings below, not as a production component.
Tiếng Việt: Adapter LoRA viết lại câu hỏi pháp luật thông thường thành truy vấn pháp
lý chuẩn, dùng làm node đầu vào cho hệ RAG trên 242.063 điều luật Việt Nam. Trên tập test
độc lập, adapter không cải thiện truy hồi so với dùng nguyên câu hỏi gốc, nên nó được công
khai để tái lập phép đo và để chia sẻ các phát hiện kỹ thuật bên dưới.
The problem it addresses
Vietnamese legal search has a wide vocabulary gap between how people ask and how statutes
are written:
user asks
statute says
nghỉ đẻ được mấy tháng? ("how many months of maternity leave?")
thời gian nghỉ việc hưởng chế độ thai sản
Closing that gap by rewriting the query is a natural design; this repo records what
happened when it was built and measured properly.
Evaluation
Held-out test: 122 questions, split by source document, so no document appears in both
train and test. Retrieval = hybrid dense + BM25 with RRF fusion.
mode
R@1
R@3
R@5
R@10
R@20
MRR
off — question unchanged
0.213
0.328
0.500
0.607
0.680
0.321
base — base model + prompt
0.205
0.352
0.434
0.516
0.590
0.306
adapter — this model
0.123
0.180
0.205
0.279
0.336
0.164
Prompting the base model is roughly neutral against the no-op baseline; the fine-tuned
adapter scores below both.
These absolute numbers come from a retrieval configuration with reranking and the
legal-hierarchy prior disabled, so they understate the host system — the same question
set scores R@1 0.376 through the full pipeline. The three rows above are measured under
identical conditions, so the comparison between them holds.
Diagnosis: the training target, not the training run
Training pairs were generated by distillation — a 35B model read each statutory article and
wrote both a colloquial question and a "proper" legal query for it. Because the teacher
could see the article, it wrote queries the way a librarian would: by naming the document.
Measured on the training set itself:
property of the target query
value
mean question length
27.0 words
mean target-query length
15.5 words
content words from the question retained
25%
targets beginning with a document-type name
67%
What the adapter therefore learned to produce:
input : Khi cửa khẩu bị tạm đóng vì an ninh hay dịch bệnh, những ai được quyền
ra quyết định và tối đa bao lâu?
output: Luật Sửa đổi, bổ sung một số điều của Luật Biên phòng về cửa khẩu
The retriever matches on article body text, not document titles. Targets that discard
75% of the content words — the exact signal dense retrieval uses — and substitute a
document-type name remove more information than they add.
A second training round addressed this directly: targets were rebuilt algorithmically from
the question's own content words plus tf-idf-selected legal terminology from the article,
raising content-word retention from 25% to 100% and eliminating document references
entirely. Retrieval quality fell further (R@1 0.072). That result points at the premise
rather than the data: the original question is already fluent Vietnamese, which is the
distribution the bi-encoder was trained on, so a rewrite that reduces fluency has little
room to help.
Findings worth reusing
Epoch-boundary memorisation is visible in the loss curve. Training loss dropped
abruptly from 1.607 to 0.831 at step 240. With 894 pairs at batch size 4, one epoch is
~224 steps — so step 224 is where epoch 2 begins and the model starts seeing the same data
again. The train/validation gap then widened to nearly 3× (0.544 vs 1.586) while validation
loss flattened. One epoch would have been sufficient for this set.
mlx-lm defaults to mask_prompt: false, computing loss over the prompt as well as the
completion. For a rewriting task the model should learn to write the query, not to ask
the question — this config sets mask_prompt: true.
Validation loss is not a proxy for retrieval quality. Loss fell from 4.567 to 1.586
across the run. Recall@k is the metric that decides whether a rewriting node earns its
place, and it moved in the opposite direction.
Training configuration
base
mlx-community/Qwen2.5-7B-Instruct-4bit (QLoRA — 4-bit base frozen)
method
LoRA, rank 16, scale 20.0, dropout 0.05, last 16 of 28 layers
optimiser
AdamW, lr 1e-4, cosine decay to 1e-5, warmup 30
steps
448 (≈2 epochs over 894 pairs, batch 4, max_seq_len 512)
mask_prompt
true
final
train loss 0.544 · val loss 1.586 (from 4.567)
hardware
Apple M3 Max 64 GB, ~35 tok/s, peak 10.1 GB
Usage
python
1from mlx_lm import load, generate
2model, tok = load("mlx-community/Qwen2.5-7B-Instruct-4bit",3 adapter_path="path/to/this/adapter")
If you are building this node
Build the retrieval-level evaluation before training. It is the only signal that
answers whether the node helps.
Do not let the teacher model see the article when generating rewriting targets, or it
will produce document references instead of content paraphrases.
Keep the question's content words. In the raw distillation output, 66% of targets
contained an actual document number; training on those teaches citation fabrication —
the most damaging failure mode for a legal tool.
Check whether the node is needed at all. With cross-encoder reranking and a
legal-hierarchy prior, the host system reaches R@1 0.870 on a hand-verified question set
with no rewriting. When retrieval is already strong, an intermediate transformation has
more information to lose than to add.
Data & attribution
Training pairs were distilled from Vietnamese statutory text sourced from
vbpl.vn — the legal document portal of the Ministry of Justice of
Viet Nam — via the CC BY 4.0 dataset
th1nhng0/vietnamese-legal-documents.
Original statutory text belongs to the Ministry of Justice of Viet Nam.
Adapter weights are released under Apache-2.0. Base model terms (Qwen2.5-7B-Instruct)
apply to any derived use.
Not legal advice
Neither this adapter nor the system it was built for constitutes legal advice. Statutory
lookup output must always be checked against the full text of the cited provision.
