📘 Machine Translation (English → Hindi) using M2M100
This project fine-tunes Facebook M2M100 (418M) for English → Hindi translation using a custom dataset.
Training, inference, and UI are all implemented to run locally in VS Code.
📂 Project Structure
Mt/
│── app.py # Streamlit UI for translation
│── translate.py # CLI translator
│── README.md # Documentation
│── requirements.txt # Python dependencies
├── data/
│ └── cleaned/
│ ├── train_clean.csv
│ ├── validation_clean.csv
│ └── test_clean.csv
├── models/
│ └── final_model/ # Saved trained model
├── scripts/
│ ├── train_model.py # Training code
│ ├── inference.py # Translate function
│ └── evaluate.py # (Optional) model evaluation
└── venv/ # Virtual environment
🧰 1. Setup Virtual Environment
Run inside Mt folder:
python3 -m venv venv
source venv/bin/activate # (Linux/Mac)
or
venv\Scripts\activate # (Windows)
pip install -r requirements.txt
For GPU users:
📦 2. Dependencies (requirements.txt)
transformers
datasets
sentencepiece
accelerate
streamlit
Install:
pip install -r requirements.txt
🧠 3. Training the Model
Your main training script:
scripts/train_model.py
Run from project root:
python scripts/train_model.py
Output model will be saved here:
models/final_model/
🌐 4. Translate Text (CLI)
Run:
python translate.py "Hello, how are you?"
Output example:
नमस्ते, आप कैसे हैं?
🖥️ 5. UI — Streamlit App
Run:
streamlit run app.py
Then open:
📝 6. Dataset Format
CSV must have:
en,hi
Hello, नमस्ते
How are you?, आप कैसे हैं?
Your cleaned dataset files:
data/cleaned/train_clean.csv
data/cleaned/validation_clean.csv
data/cleaned/test_clean.csv
📊 7. Model Used
Model: facebook/m2m100_418M
Tokenizer: SentencePiece
Trained for EN → HI translation
Output generated using beam search
pip install sentencepiece
❌ Wrong Python (not inside venv)
Run:
source venv/bin/activate
Then verify:
which python
Should show:
venv/bin/python
Author: Gadigoppula Saiteja
Built English–Hindi MT pipeline
Fine-tuned M2M100 model
Implemented CLI + UI + Training
