Continually pretrained on the International Corpus of English (18 varieties, ~20M tokens) using GaLore for memory-efficient full-parameter optimisation. This is the shared foundation checkpoint both the implicit and explicit adaptation threads branch from.

Code, checkpoints, preference datasets, linguistic-analysis toolkit: https://github.com/surrey-nlp/diallm

Paper: https://arxiv.org/abs/2607.07669

Citation

Base model: Llama 3.1-8B
Pipeline stage: continual pretraining (CPT), precedes dialect-specific branching

bibtex

@article{painter2026diallm,
  title     = {DiaLLM: An Investigation into the Robustness-Generation Gap in English Dialect Adaptation},
  author    = {Painter, Jordan and Srirag, Dipankar and Kappiyath, Adarsh and Kanojia, Diptesh and Joshi, Aditya and Yin, Lu},
  year      = {2026},
  eprint    = {2607.07669},
  archivePrefix = {arXiv}
}

jordanpainter/diallm-llama-cpt

DiaLLM — Llama 3.1-8B — CPT

Citation

jordanpainter/
diallm-llama-cpt