Student model distilled from an XLM-RoBERTa-Large teacher for cross-lingual (EN → VI) fake-news / claim verification. Trained with Knowledge Distillation (soft + hard loss, α=0.7, temperature T≈1.094).

Backbone: nreimers/mMiniLMv2-L6-H384-distilled-from-XLMR-Large
Params: 107.1M
Labels: 0 = fake (tin giả), 1 = real (tin thật)

Results (best seed = 42)

Split	Score
VI test (F1-macro)	0.8262

Retention vs teacher: 93.3%

Usage

python

from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

REPO = "phuy1125/mminilm-l6-fakenews-kd"
tok = AutoTokenizer.from_pretrained(REPO, trust_remote_code=True)
model = AutoModelForSequenceClassification.from_pretrained(REPO, trust_remote_code=True).eval()

# Input format used in training: "claim | evidence" (drop the pipe if one side is empty)
text = "Một tuyên bố cần kiểm chứng | bằng chứng liên quan"
enc = tok(text, truncation=True, max_length=256, return_tensors="pt")
with torch.no_grad():
    logits = model(**enc).logits
pred = int(logits.argmax(-1))
print(model.config.id2label[pred])   # 'fake' or 'real'

Note: training used head-tail truncation (ids[:127] + ids[-127:]) at max_length=256. Plain truncation above is fine for most short inputs.

Citation

Distilled within an undergraduate thesis on cross-lingual fake-news detection (EN→VI). Teacher: XLM-RoBERTa-Large (vi_test F1 = 0.8858).