You can deploy this model and start earning money today!

SeverusSnape5/
Sawit

Views

No views yet

Fine-Tuned GPT-2 for Hinglish Translation

Overview

This project fine-tunes a GPT-2 model for translating English text into Hinglish (colloquial Hindi written in Roman script). The dataset consists of paired English and Hinglish sentences, enabling the model to learn contextual language translation effectively.

Dataset

File Name: Hindi.csv
Format: CSV file containing two columns:
- English text: Contains English sentences.
- Colloquial text: Corresponding Hinglish translations.
Encoding: ISO-8859-1

Model Training

The model is fine-tuned using Hugging Face's transformers library with the following configuration:

Base Model: GPT-2 (gpt2 from Hugging Face)
Tokenizer: GPT-2 tokenizer with eos_token as padding token
Training Parameters:
- Epochs: 3
- Batch Size: 4 (train), 8 (eval)
- Warmup Steps: 500
- Weight Decay: 0.01
- Evaluation Strategy: epoch
- Save Strategy: epoch

Setup and Execution

Prerequisites

Install the required dependencies before running the training script:

pip install transformers datasets torch pandas

Steps

Upload the dataset to Google Colab.
Load and preprocess the data into Hugging Face’s Dataset format.
Tokenize the dataset using GPT-2 tokenizer.
Fine-tune GPT-2 using the Trainer API.
Save the trained model and tokenizer for later inference.

Model Output

The fine-tuned model and tokenizer are saved in the ./fine_tuned_gpt2_hinglish directory.
The model can generate Hinglish translations given English input text.

Usage

To load and use the fine-tuned model:

python

from transformers import GPT2Tokenizer, GPT2LMHeadModel

tokenizer = GPT2Tokenizer.from_pretrained("./fine_tuned_gpt2_hinglish")
model = GPT2LMHeadModel.from_pretrained("./fine_tuned_gpt2_hinglish")

def generate_hinglish(text):
    inputs = tokenizer(text, return_tensors="pt")
    outputs = model.generate(**inputs, max_length=50)
    return tokenizer.decode(outputs[0], skip_special_tokens=True)

print(generate_hinglish("Hello, how are you?"))

Acknowledgments

This project leverages Hugging Face’s transformers and datasets libraries along with PyTorch for model training and fine-tuning.

Contributors

Dalli Gnana Snigdha
Faiza Tashmeem
Leela Prasanna

SeverusSnape5/Sawit