Views
No views yet
transformers library. Because Nagamese is not natively supported by NLLB, a surrogate language token (e.g., asm_Latn or ind_Latn) was used during training to represent Nagamese.1import torch
2from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
3
4# 1. Load the model and tokenizer
5model_name = "agnivamaiti/nllb-200-en-nagamese"
6tokenizer = AutoTokenizer.from_pretrained(model_name)
7model = AutoModelForSeq2SeqLM.from_pretrained(model_name)
8
9# 2. Prepare your text
10text = "Where are you going tomorrow?"
11inputs = tokenizer(text, return_tensors="pt")
12
13# 3. Generate translation (Force the surrogate token ID)
14# Note: Change 'asm_Latn' if you used a different surrogate token during training (like 'ind_Latn' or 'tur_Latn').
15outputs = model.generate(
16 **inputs,
17 forced_bos_token_id=tokenizer.convert_tokens_to_ids("asm_Latn"),
18 max_length=128
19)
20
21# 4. Decode the output
22translation = tokenizer.decode(outputs[0], skip_special_tokens=True)
23print("Translation:", translation)train.py and inference.py in the files of this repository for documentation and reproducibility purposes.