Model Overview:
This model is a fine-tuned version of unsloth/tinyllama-chat-bnb-4bit, designed for translating
English text into Telugu while preserving colloquial expressions. The model helps generate
natural and contextually relevant translations for everyday conversations.
Model Description:
Base Model: unsloth/tinyllama-chat-bnb-4bit
Objective: English to Telugu translation with a focus on colloquial speech.
Fine-tuning Dataset: Custom dataset (details provided below).
Framework: PyTorch with Hugging Face transformers and PEFT.
Quantization: 4-bit for efficient inference.
Intended Uses & Limitations
Intended Uses
✅ Translation of English text to Telugu while maintaining natural conversational flow.
✅ Useful for chatbots, customer support, and informal communication.
✅ Can assist language learners in understanding Telugu colloquialisms.
Limitations
⚠️ May not perform well on highly technical or formal translations.
⚠️ Might struggle with complex sentence structures or idiomatic expressions.
⚠️ Requires further fine-tuning for domain-specific translations.
Training and Evaluation Data:
The model was fine-tuned on a custom dataset containing parallel English-Telugu colloquial sentences. The dataset includes:
-Conversations from social media platforms.
-Informal discussions and day-to-day dialogues.
-Commonly used Telugu expressions and their English equivalents.
Training Procedure:
Training Hyperparameters:
Hyperparameter Value
Learning Rate 0.0003
Training Batch Size 4
Evaluation Batch Size 4
Gradient Accumulation Steps 2
Total Train Batch Size 8
Optimizer AdamW
LR Scheduler Linear
Warmup Steps 2
Epochs 10
Mixed Precision Training Native AMP
Training Results:
The model achieved the following performance on the evaluation set:
Training Loss Epoch Step Validation Loss
8.9699 0.2 2 5.5126
4.0118 2.0 20 5.4684
1.1042 5.2 52 5.1425
0.9898 10.0 100 5.2763
Model Performance and Evaluation:
The model was evaluated using a validation dataset, and the loss metric was used to measure performance.
-Lower loss values indicate better performance.
-The model shows improvement over training epochs, stabilizing around a loss of 5.27.
Framework Versions:
PEFT: 0.14.0
Transformers: 4.49.0
PyTorch: 2.6.0+cu124
Datasets: 3.2.0
Tokenizers: 0.21.0
How to Use the Model:
You can use this model with Hugging Face’s transformers library
model_name = "SujathaL/english-telugu-colloquial-translator"
Limitations & Future Improvements:
Possible Enhancements:
Fine-tune on a larger, more diverse dataset.
Implement reinforcement learning for better colloquial accuracy.
Test on real-world conversational datasets.
Known Issues:
May require post-processing for fluency.
Some idioms may not translate accurately.
Contributors
Sujatha Lakkoju (Model Developer)
Dataset Sources (ChatGPT)
License:
This model is released under the MIT License.
Contact: If you have any queries, feel free to reach out at
sujathalakkoju9@gmail.com
