nrl-ai/vn-spell-correction-small — Vietnamese spell correction (BARTpho-syllable fine-tune)

Quick start

Evaluation — 8-split spell-correction grid

• light noise — ~5 % char-level edit distance, models a person typing Vietnamese on a keyboard with a few accent slips and the occasional fat-finger.
• heavy noise — ~15-20 % edit distance, models OCR output of a mid-quality scan with diacritic drops + char confusions.

How we compare

• light avg = mean word accuracy across the 4 light-noise splits.
• heavy avg = mean word accuracy across the 4 heavy-noise splits.

Training

• Base: vinai/bartpho-syllable-base (MIT license)
• Corpus: 545,000 (noisy, clean) pairs from nrl-ai/vn-spell-correction-train. Eval-leak guarded against nrl-ai/vn-spell-correction-eval and nrl-ai/vn-diacritic-eval.
• Validation: 5,000 held-out pairs.
• Epochs: 5
• Effective batch size: 32 (32 per device, grad-accum 1)
• Learning rate: 0.0005 with cosine schedule, 500 warmup steps
• Precision: bf16
• Sequence length: input 256, target 256
• Early stopping: patience=0 on eval_loss
• Training time: 96.6 min on a single NVIDIA RTX 3090 24 GB
• Seed: 42

Intended use

• Recommended: cleaning up noisy Vietnamese text — OCR output, user-generated text from non-VN-IME keyboards, form data with typos, social-media short-form. Strictly harder than diacritic restoration but covers it as a subset.
• Not recommended: text generation, classification, sentiment, NER, or any task the input distribution doesn't match.

Limitations

• In-distribution metric, real-world is harder — measured. Training and eval both use nom.text.noise. The synthetic 8-split numbers above measure how well we invert our noise generator. We also benchmark on a 200-sentence OOD eval whose noise comes from real Vietnamese error sources rather than our generator (7 slices, bootstrap 95 % CI):

  Slice	this model	Toshiiiii1 (public)	bmd1905 (public)	chamdentimem (public)
  forum_25	64.64 %	60.11 %	59.02 %	62.19 %
  mobile_25	95.29 %	96.95 %	88.09 %	86.15 %
  telex_real_25	16.45 %	18.54 %	11.58 %	17.14 %
  ocr_25	94.19 %	94.22 %	47.42 %	44.17 %
  legal_real_25	93.54 %	93.80 %	54.90 %	61.76 %
  news_real_25	91.34 %	94.07 %	30.62 %	34.81 %
  furniture_50	88.89 %	83.77 %	71.43 %	63.64 %
  Aggregate (n=200)	78.99 %	78.19 %	52.04 %	53.18 %

  The synthetic grid above measures how well we invert our own noise generator; the aggregate here (78.99 %) is what to plan around. The gap is the cost of a noise model that captures the surface of typos but not real Telex keystroke artefacts (dduwojc for được) or forum-style abbreviations (ko bt for không biết). Real Telex input is the weakest slice by a wide margin and is the primary target of the next training round.
• Heading and letterhead layout is a known blind spot. The training corpus is sentence-segmented, so document furniture (letterheads, all-caps titles, form labels, signature blocks) was filtered out during construction. That is the furniture_50 slice above, and it is the weakest non-Telex register. The model can leave a real-word tone error uncorrected where the same error is fixed in ordinary prose: Độc lập - Tự do - Hạnh phục is echoed back unchanged, while Tôi rất hạnh phục khi gặp lại bạn is corrected. Two conditions have to coincide, an adverse frequency prior (phục outnumbers phúc 3,810 to 1,311 in the corpus) and a layout the encoder has not seen corrected. Neither alone reproduces it.
  nom.text.heading ships a conservative recovery pass, enabled by default on HFDiacriticModel. When the first pass makes no edit and the input is heading-shaped, it retries on a lowercased copy and keeps only tone-level edits on purely alphabetic tokens. On furniture_50 that moves word accuracy 88.89 % to 91.11 % and sentence-exact 42.00 % to 56.00 %, while every other slice stays bit-identical:
  python

  1from nom.text.diacritic_models import HFDiacriticModel 2 3speller = HFDiacriticModel(model_id="nrl-ai/vn-spell-correction-small") 4speller("Độc lập - Tự do - Hạnh phục") 5# 'Độc lập - Tự do - Hạnh phúc'
• Heavy-noise corner cases. OCR outputs that drop entire words or add hallucinated text are out-of-scope; the noise generator we trained on caps edits per sentence (max 25 % edit ratio).
• Long sequences truncate at 256 sub-word tokens. Split paragraphs at sentence boundaries before calling.
• No grammar or stylistic correction. This model fixes character / syllable / diacritic errors but doesn't rewrite phrasing.
• Confidence intervals on small splits. business_55 (44/55 sents) and formal_72 (65/72 sents) have ±3-4 pp 95 % CI; the larger literary_800 split has ±1 pp. Treat single-pp differences with care.

License & attribution