from transformers import M2M100ForConditionalGeneration, M2M100Tokenizer
import torch

def translate(input_text, lang):
    input_ids = tokenizer(input_text, return_tensors="pt")
    generated_tokens = model.generate(**input_ids, forced_bos_token_id=tokenizer.get_lang_id(lang))
    return tokenizer.batch_decode(generated_tokens, skip_special_tokens=True)

model_name = "cartesinus/iva_mt_wslot-m2m100_418M-0.1.0-en-de"
tokenizer = M2M100Tokenizer.from_pretrained(model_name, src_lang="en", tgt_lang="de")
model = M2M100ForConditionalGeneration.from_pretrained(model_name)

Then you can translate either plain text like this:

print(translate("set the temperature on my thermostat", "de"))

or you can translate with slot annotations that will be restored in tgt language:

print(translate("wake me up at <a>nine am<a> on <b>friday<b>", "de"))

Limitations of translation with slot transfer:

Annotated words must be placed between semi-xml tags like this "this is <a>example<a>"
There is no closing tag for example "<\a>" in the above example - this is done on purpose to omit problems with backslash escape
If the sentence consists of more than one slot then simply use the next alphabet letter. For example "this is <a>example<a> with more than <b>one<b> slot"
Please do not add space before the first or last annotated word because this particular model was trained this way and it most probably will lower its results

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 2e-05
train_batch_size: 4
eval_batch_size: 4
seed: 42
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
num_epochs: 7
mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Bleu	Gen Len
0.0183	1.0	1884	0.0144	63.5045	20.2994
0.0119	2.0	3768	0.0127	66.0473	20.5423
0.0083	3.0	5652	0.0123	65.5139	20.5409
0.0065	4.0	7536	0.0124	66.0731	20.6114
0.0048	5.0	9420	0.0122	66.2454	20.5906
0.0038	6.0	11304	0.0124	66.7412	20.6263
0.0031	7.0	13188	0.0126	66.5548	20.6835

Framework versions

Transformers 4.28.1
Pytorch 2.0.0+cu118
Datasets 2.11.0
Tokenizers 0.13.3

Citation

If you use this model, please cite the following:

@article{Sowanski2023SlotLI,
  title={Slot Lost in Translation? Not Anymore: A Machine Translation Model for Virtual Assistants with Type-Independent Slot Transfer},
  author={Marcin Sowanski and Artur Janicki},
  journal={2023 30th International Conference on Systems, Signals and Image Processing (IWSSIP)},
  year={2023},
  pages={1-5}
}

cartesinus/iva_mt_wslot-m2m100_418M-en-de

iva_mt_wslot-m2m100_418M-en-de

Model description

How to use

Training procedure

Training hyperparameters

Training results

Framework versions

Citation

cartesinus/
iva_mt_wslot-m2m100_418M-en-de