Views
No views yet
llama-eus-8B-DIPLomA is a large language model derived from Llama-eus-8B that introduces instruction-following capabilities in Basque through the DIPLomA (Decoupled Instruction-Preserving Language Adaptation) method. It combines the Basque language knowledge of Llama-eus-8B, continually pretrained on Basque, with the instruction-tuned behavior of Meta-Llama-3.1-8B-Instruct using a lightweight delta-merging technique that transfers instruction alignment without costly retraining.| Attribute | Description |
|---|---|
| Model name | llama-eus-8B-DIPLomA |
| Base model | Llama-eus-8B |
| Instruction delta from | Meta-Llama-3.1-8B-Instruct and Meta-Llama-3.1-8B |
| Architecture | LLaMA 3.1 (8B parameters) |
| Tokenizer | LLaMA 3.1 tokenizer |
| Languages | Basque (eu), English (en) |
| Training method | DIPLomA (post-training delta merging) |
| License | Llama 3.1 Community License |
| Developed by | Orai NLP Technologies |
(Winstruct − Wbase) from the instructed model onto the Basque-adapted weights: W_DIPLOmA = W_LangAdapted + α · (Winstruct − Wbase)| Models | EU Instr. Follow. | EU Ling. Prof. | EU Safety | EN Inst. Follow. | EN Safety |
|---|---|---|---|---|---|
| Llama-eus-8B-DIPLomA | 69 | 79 | 55 | 76 | 94 |
| Llama-3.1-8B-Instruct | 15 | 37 | 7 | 81 | 94 |
1@inproceedings{sarasua-etal-2025-diploma,
2 title = "{DIPL}om{A}: Efficient Adaptation of Instructed {LLM}s to Low-Resource Languages via Post-Training Delta Merging",
3 author = "Sarasua, Ixak and
4 Corral, Ander and
5 Saralegi, Xabier",
6 editor = "Christodoulopoulos, Christos and
7 Chakraborty, Tanmoy and
8 Rose, Carolyn and
9 Peng, Violet",
10 booktitle = "Findings of the Association for Computational Linguistics: EMNLP 2025",
11 month = nov,
12 year = "2025",
13 address = "Suzhou, China",
14 publisher = "Association for Computational Linguistics",
15 url = "https://aclanthology.org/2025.findings-emnlp.1355/",
16 pages = "24898--24912",
17 ISBN = "979-8-89176-335-7",
18 abstract = "This paper investigates how open-weight instruction-tuned large language models (LLMs) can be efficiently adapted to low-resource languages without requiring costly large-scale post-training. We introduce DIPLomA (Decoupled Instruction-Preserving Language Adaptation), a lightweight delta-based transfer strategy that provides a practical and effective solution for this scenario. DIPLomA decouples language adaptation from post-training alignment by first continually pretraining a foundational LLM on a modest amount of monolingual target-language data while anchoring on English replay, and then injecting instruction-following capabilities via delta-based weight merging from the instructed counterpart of the base LLM. We evaluate DIPLomA on Basque and validate its generality on Welsh and Swahili, demonstrating consistent and substantial gains in instruction-following, linguistic proficiency, and safety. Compared to strong baselines, our method achieves average relative improvements of 50 points in Basque, 63 in Welsh, and 51 in Swahili, while preserving the original model{'}s multilingual performance. These results highlight DIPLomA as an effective, resource-efficient strategy for bringing high-quality instruction alignment to underrepresented languages at scale."
19}