This model represents a high-performance breakthrough in small-parameter translation for Telugu Language. It was developed using a unique two-phase training strategy that combines the depth of full fine-tuning with the precision of LoRA (Low-Rank Adaptation).
Unlike standard fine-tuned models, this version underwent a rigorous 30-epoch journey:
-
Phase I: Deep Language Grounding (Full Fine-Tuning, 15 Epochs) The entire mT5-small architecture was unlocked to re-align its internal "mental map" from general multilingual space to a specialized English-Telugu domain. This allowed for deep syntactic and morphological adaptation.
-
Phase II: Precision Refinement (LoRA, 15 Epochs) After the base weights were grounded, LoRA ($r=16$) was applied to the specialized checkpoint. This phase acted as a regularizer, sharpening the translation logic and eliminating the "hallucinations" common in smaller models.
The model was evaluated on a held-out test set and achieved the following scores:
These scores indicate a very high level of translation quality, outperforming many baseline multilingual models for the English-Telugu pair.
1import torch
2from transformers import T5ForConditionalGeneration, T5Tokenizer
3
4model_path = "ManiKumarAdapala/mt5-telugu"
5tokenizer = T5Tokenizer.from_pretrained(model_path)
6model = T5ForConditionalGeneration.from_pretrained(model_path).to("cuda")
7
8# Move to evaluation mode
9model.eval()
10
11def translate_to_telugu(text):
12 input_text = "translate English to Telugu: " + text
13
14 # Tokenize input
15 inputs = tokenizer(input_text, return_tensors="pt").to("cuda")
16
17 # Generate
18 with torch.no_grad():
19 output_tokens = model.generate(
20 **inputs,
21 max_length=128,
22 num_beams=5, # Beam search for better quality
23 early_stopping=True,
24 repetition_penalty=1.2
25 )
26
27 # Decode
28 return tokenizer.decode(output_tokens[0], skip_special_tokens=True)
29
30english_sentence = 'Pain from appendicitis may begin as dull pain around the navel.'
31print(f"English: {english_sentence}")
32print(f"Telugu: {translate_to_telugu(english_sentence)}")
33
34# Result :
35# English: Pain from appendicitis may begin as dull pain around the navel.
36# Telugu: అపెండిసైటిస్ వలన వచ్చే నొప్పి నాభి చుట్టూ సన్నటి నొప్పిగా ప్రారంభమవుతుంది.
This model can also be used with pipeline.
1from transformers import pipeline, T5ForConditionalGeneration, T5Tokenizer
2
3model_path = "ManiKumarAdapala/mt5-telugu"
4tokenizer = T5Tokenizer.from_pretrained(model_path)
5model = T5ForConditionalGeneration.from_pretrained(model_path).to("cuda")
6
7# Move to evaluation mode
8model.eval()
9
10telugu_translator = pipeline(
11 "text2text-generation",
12 model=model,
13 tokenizer=tokenizer
14)
15
16def translate(text):
17 prefix = "translate English to Telugu: "
18 output = telugu_translator(
19 f"{prefix}{text}",
20 max_length=128,
21 num_beams=5,
22 early_stopping=True,
23 clean_up_tokenization_spaces=True
24 )
25 return output[0]['generated_text']
26
27print(translate("It is invariant and is always included in all ragams."))
28
29# Result : ఇది నిరంతరం ఉంటుంది మరియు ఎల్లప్పుడూ అన్ని రాగాలలో చేర్చబడుతుంది.
This project is built upon the mT5 (Multilingual T5) architecture developed by Google. Their foundational research into massively multilingual models provided the raw material that made this specialized Telugu-language tool possible.
