You can deploy this model and start earning money today!

jurgiraud/
latxa-eseu-wmt26-augmented

Name: latxa-eseu-wmt26-augmented
Availability: InStock

Views

No views yet

latxa-eseu-wmt26-augmented

Spanish → Basque translation model for the automotive and energy domains, fine-tuned from HiTZ/Latxa-Llama-3.1-8B-Instruct.

This is the translation component of Agenteak, a multi-agent terminology-aware translation pipeline submitted to the WMT26 Terminology Translation Task (Track 1, es→eu). It can also be used on its own as a domain-adapted es→eu translator.

A single set of LoRA adapters covers both domains. The domain is selected at inference time through an Ámbito: line in the system prompt — this line is the conditioning signal, and omitting or changing it loses most of the benefit of domain fine-tuning.

Important: this model does not enforce terminology by itself. In Agenteak, terminology is selected by a separate agent, injected into the prompt, and verified downstream. Used standalone, this model gives you domain register and vocabulary, not guaranteed term usage.

Quick start

python

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

MODEL = "jurgiraud/latxa-eseu-wmt26-augmented"

tok = AutoTokenizer.from_pretrained(MODEL)
model = AutoModelForCausalLM.from_pretrained(
    MODEL, torch_dtype=torch.bfloat16, device_map="auto"
)

DOMAIN_PHRASE = {
    "automotive": "el ámbito de la automoción",
    "energy": "el ámbito de la energía",
}
DOMAIN_TAG = {"automotive": "automoción", "energy": "energía", "general": "general"}

def system_prompt(domain):
    if domain in DOMAIN_PHRASE:
        opening = ("Eres un traductor experto del castellano al euskera (euskara batua), "
                   f"especializado en {DOMAIN_PHRASE[domain]}.")
        registro = "y empleando la terminología asentada de ese ámbito. "
    else:
        opening = ("Eres un traductor experto del castellano al euskera (euskara batua), "
                   "capaz de trabajar con textos de cualquier ámbito.")
        registro = "y empleando un registro estándar. "
    return (
        opening + "\n"
        f"Ámbito: {DOMAIN_TAG[domain]}.\n"
        "Traduce con precisión y naturalidad, respetando la declinación y la sintaxis "
        "propias del euskera, " + registro +
        "Conserva exactamente todas las cifras, unidades y nombres propios tal como "
        "aparecen en el original, así como el formato del texto de partida (mayúsculas, "
        "puntuación, saltos de línea y etiquetas). "
        "No añadas explicaciones, comentarios ni el texto de partida. "
        "Responde únicamente con la traducción."
    )

def translate(text, domain="automotive", max_new_tokens=512):
    msgs = [{"role": "system", "content": system_prompt(domain)},
            {"role": "user", "content": text.strip()}]
    inputs = tok.apply_chat_template(
        msgs, add_generation_prompt=True, return_tensors="pt"
    ).to(model.device)
    out = model.generate(
        inputs,
        max_new_tokens=max_new_tokens,
        do_sample=False,
        eos_token_id=tok.eos_token_id,
        pad_token_id=tok.eos_token_id,
    )
    return tok.decode(out[0][inputs.shape[-1]:], skip_special_tokens=True).strip()

print(translate(
    "El motor de combustión interna alcanza su par máximo a 3.500 rpm.",
    domain="automotive",
))

Domain values

`domain`	`Ámbito:` value	Use for
`automotive`	`automoción`	vehicles, components, powertrains, transport technology
`energy`	`energía`	generation, renewables, fuels, electrical engineering
`general`	`general`	out-of-domain text; falls back to standard register

Training data

Mined bitext — comparable es/eu Wikipedia articles reached from domain root categories, aligned within article pairs with LaBSE (cosine ≥ 0.80) and filtered with Bicleaner AI (≥ 0.50). Unit: sentence pair.
Synthetic — three-sentence paragraphs generated by the base model with explicit bilingual terminology prompting, filtered with the same Bicleaner AI threshold. Unit: paragraph pair.
General — EusParallel filtered to 40–700 characters and eu/es character ratio 0.55–1.70, deduplicated, and sampled at 1.5× the in-domain count. This component is what stops the model overfitting to the narrow in-domain register.

Development sets are held out at the article level: 217 automotive, 252 energy, 300 general.

Training procedure

LoRA SFT with TRL on a single A100 40GB.


Precision	bf16, no quantisation, SDPA attention
LoRA	r=16, α=32, dropout=0.1, bias none
Target modules	`q_proj` `k_proj` `v_proj` `o_proj` `gate_proj` `up_proj` `down_proj`
Trainable params	41.9M (0.52% of 8.07B)
Max length	1024, packing disabled
Learning rate	1e-4, cosine, warmup ratio 0.03
Weight decay	0.01
Batch	4 per device × 4 accumulation (effective 16)
Loss	completion only (prompt tokens masked)
Early stopping	patience 4 (epoch ceiling 4)

Final training loss 0.447; dev loss 0.436 automotive, 0.431 energy, 0.455 general

Evaluation

Held-out in-domain development sets, greedy decoding.

		BLEU ↑	chrF2 ↑	TER ↓	COMET ↑
Automotive	Latxa-3.1-8B-Instruct (base)	20.36	56.82	82.05	80.73
	+ FT (mined + general)	25.76	62.27	69.44	84.21
	+ FT (mined + general + synthetic)	27.39	62.65	67.36	84.58
Energy	Latxa-3.1-8B-Instruct (base)	17.62	58.46	87.12	82.96
	+ FT (mined + general)	27.33	63.64	67.38	86.21
	+ FT (mined + general + synthetic)	28.01	64.50	65.79	86.48

Citation

Please also cite the base model:

bibtex

@inproceedings{sainz-etal-2025-instructing,
  title     = {Instructing Large Language Models for Low-Resource Languages: A Systematic Study for Basque},
  author    = {Sainz, Oscar and Perez, Naiara and Etxaniz, Julen and Fernandez de Landa, Joseba
               and Aldabe, Itziar and García-Ferrero, Iker and Zabala, Aimar and Azurmendi, Ekhi
               and Rigau, German and Agirre, Eneko and Artetxe, Mikel and Soroa, Aitor},
  booktitle = {Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing},
  year      = {2025},
  pages     = {29136--29160},
  url       = {https://aclanthology.org/2025.emnlp-main.1484/}
}

@inproceedings{etxaniz-etal-2024-latxa,
  title     = {Latxa: An Open Language Model and Evaluation Suite for Basque},
  author    = {Etxaniz, Julen and Sainz, Oscar and Perez, Naiara and Aldabe, Itziar and Rigau, German
               and Agirre, Eneko and Ormazabal, Aitor and Artetxe, Mikel and Soroa, Aitor},
  booktitle = {Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics},
  year      = {2024},
  pages     = {14952--14972},
  url       = {https://aclanthology.org/2024.acl-long.799/}
}

License

Derived from Latxa, which is itself derived from Llama-3.1, so the Llama 3.1 Community License applies and its use restrictions carry over.