AitanaTA-7b-Instruct is a translation LLM that has been instruction-tuned from SalamandraTA-7b-Instructed. This model results from instruction-tuning on parallel data from Salamandra-7b-base. AitanaTA-7b-Instruct has been specifically instruction-tuned for translation between Spanish and Valencian, with a focus on sentence-level translation.
This model follows the same instruction pattern as SalamandraTA-7b-Instructed. The only adaptation was focusing the instructions on translation between Valencian and Spanish, in both directions.
Training Details
Training Data
The data comes from L'Associació de Mitjans d'Informació i Comunicació (AMIC), from which we built a parallel sentence-level dataset. This dataset was specifically created to align Spanish and Valencian sentences, ensuring high-quality parallel examples for training and evaluation.
== Language pairs==
Valencian -> Spanish: 738777 (mean_src_len=165.6, mean_tgt_len=168.6)
Training Hyperparameters
Training regime:
epochs: 1
learning_rate: 1e-5
beta1: 0.9
beta2: 0.99
weight_decay: 0
global_batch_size: 64
micro_batch_size: 2
log_interval: 5
save_interval: 5
lr_warmup_steps: 100
max_seq_length: 2048
How to use
You can translate between Spanish to Valencian. The instruction-following model uses the commonly adopted ChatML template:
The easiest way to apply it is by using the tokenizer's built-in functions, as shown in the following snippet.
python
1from datetime import datetime
2from transformers import AutoTokenizer, AutoModelForCausalLM
3import transformers
4import torch
56model_id ="gplsi/Aitana-TA-7b-S"78source ='Spanish'9target ='Valencian'10sentence ="La inteligencia artificial está transformando el mundo."1112text =f"Translate the following text from {source} into {target}.\n{source}: {sentence} \n{target}:"1314tokenizer = AutoTokenizer.from_pretrained(model_id)1516model = AutoModelForCausalLM.from_pretrained(17 model_id,18 device_map="auto",19 dtype=torch.bfloat16
20)2122message =[{"role":"user","content": text }]2324prompt = tokenizer.apply_chat_template(25 message,26 tokenize=False,27 add_generation_prompt=True,28)2930inputs = tokenizer.encode(prompt, add_special_tokens=False, return_tensors="pt")31input_length = inputs.shape[1]32outputs = model.generate(input_ids=inputs.to(model.device),33 max_new_tokens=400,34 early_stopping=True,35 num_beams=5)3637print(tokenizer.decode(outputs[0, input_length:], skip_special_tokens=True))38# La intel·ligència artificial està transformant el món.
Using this template, each turn is preceded by a <|im_start|> delimiter and the role of the entity
(either user, for content supplied by the user, or assistant for LLM responses), and finished with the <|im_end|> token.
Evaluation
Testing Data
Our objective was to evaluate translation between Spanish and Valencian. To test the model, we used the Phrases task from IberoBench.
Metrics
For evaluation, we relied on a set of standard translation metrics: COMET, BLEU, TER, and ChrF.
Results on Phrases tasks
Task
Metric
Aitana-TA-7b-S
salamandraTA-7b-instruct
phrases_es-va
BLEU
65.14
61.92
phrases_va-es
BLEU
82.04
75.12
phrases_va-ca
BLEU
79.52
72.25
phrases_ca-va
BLEU
78.63
72.30
===============
=======
===================
phrases_es-va
ChrF
84.24
82.82
phrases_va-es
ChrF
91.63
87.72
phrases_va-ca
ChrF
91.02
85.09
phrases_ca-va
ChrF
90.88
87.56
===============
=======
===================
phrases_es-va
COMET
0.937
0.93
phrases_va-es
COMET
0.957
0.951
phrases_va-ca
COMET
0.953
0.948
phrases_ca-va
COMET
0.956
0.948
===============
=======
===================
phrases_es-va
TER
20.99
22.63
phrases_va-es
TER
10.12
15.54
phrases_va-ca
TER
10.45
18.85
phrases_ca-va
TER
11.54
15.95
Technical Specifications
Hardware and Software
For training, we used custom code developed on top of the PyTorch Lightning framework (Fabric Lightning), enabling model sharding across multiple GPUs through Fully Sharded Data Parallel (FSDP).
Compute Infrastructure
This model was trained on NVIDIA DGX systems equipped with A100 GPUs, which enabled efficient large-scale training. For this model, we used 4-A100 GPUs.
This work is funded by the Ministerio para la Transformación Digital y de la Función Pública, co-financed by the EU – NextGenerationEU, within the framework of the project Desarrollo de Modelos ALIA. This work has also been partially supported by Project HEART-NLP (PID2024-156263OB-C22).
Acknowledgments
We would like to express our gratitude to all individuals and institutions that have contributed to the development of this work.
We also acknowledge the financial, technical, and scientific support of the Ministerio para la Transformación Digital y de la Función Pública - Funded by EU – NextGenerationEU within the framework of the project Desarrollo de Modelos ALIA, whose contribution has been essential to the completion of this research.
This model has been developed and instruction-tuned specifically for translation between Spanish and Valencian. Its use outside of translation tasks is not recommended, as performance and reliability have not been evaluated for other natural language processing applications. The authors are not responsible for potential errors, misinterpretations, or inappropriate use of the model beyond its intended purpose.
Reference
If you use this model in your research or work, please cite it as follows:
bibtex
1@misc{gplsi-aitama-ta-7b-s,
2 author = {Sepúlveda-Torres, Robiert and Galeano, Santiago and Miró Maestre, María and Martínez-Murillo, Iván and Grande, Eduardo and Canal-Esteve, Miquel and Estevanell-Valladares, Ernesto L. and Yáñez-Romero, Fabio and Consuegra-Ayala, Juan Pablo and Bonora, Mar and Gutierrez, Yoan and Abreu Salas, José Ignacio and Lloret, Elena and Montoyo, Andrés and Muñoz-Guillena and Palomar, Manuel},
3 title = {Aitana-TA-7b-S: Translation model from Spanish to Valencian},
4 year = {2026},
5 institution = {Language and Information Systems Group (GPLSI) and Centro de Inteligencia Digital (CENID), University of Alicante (UA)},
6 howpublished = {\url{https://huggingface.co/gplsi/Aitana-TA-7b-S}},
7 note = {Accessed: 2026-06-03}
8}
