Multilingual Sentiment Analysis with Hugging Face and Streamlit
This repository contains a machine learning project for multilingual sentiment analysis. The project fine-tunes a pre-trained language model from Hugging Face on a multilingual dataset and deploys it as a user-friendly web application using Streamlit.
🚀 Key Features
Multilingual Support: The model is fine-tuned to understand sentiment in multiple languages, including English, Spanish, and French.
Custom Fine-Tuning: The project demonstrates how to fine-tune a powerful distilbert-base-multilingual-cased model for a specific task.
Interactive Web App: A Streamlit application is provided to allow users to interact with the model by entering text and getting real-time sentiment predictions.
⚙️ Project Structure
The project is structured into a series of steps that can be run sequentially in a Jupyter Notebook or Google Colab environment.
Step 1: Install Necessary Libraries
This step ensures that all the required Python libraries for data processing, model training, and deployment are installed. This includes transformers, torch, datasets, and streamlit.
Step 2: Load and Prepare the Dataset
We use the r_sentiment_analysis dataset from Hugging Face, which contains a collection of reviews in different languages. This data is essential for training the model. The dataset is split into training and evaluation sets to ensure proper fine-tuning and testing.
Step 3: Tokenize the Dataset
Tokenization is the process of converting text into a format that the model can understand. We use a tokenizer to convert the text reviews into numerical representations (IDs) that the model uses for training.
Step 4: Configure Training Arguments
This step sets up the training parameters for the model. We specify the output directory for the trained model, the learning rate, the number of epochs (how many times the model sees the entire dataset), and other important settings.
Step 5: Initialize the Trainer
We use the Hugging Face Trainer class, which handles the entire training and evaluation process. This step combines the model, training arguments, datasets, and tokenizer into a single object for easy management.
Step 6: Fine-tune the Model
This is the core of the project. We call the trainer.train() method to fine-tune the pre-trained model on our sentiment analysis dataset. Fine-tuning adjusts the model's weights to better suit our specific task.
Step 7: Evaluate the Model
After training, it is crucial to test the model's performance. The trainer.evaluate() method is used to evaluate the fine-tuned model on the evaluation dataset. This provides important metrics like accuracy, which tell us how well the model generalizes to new, unseen data.
Step 8: Deploy the Streamlit App
Finally, the fine-tuned model is loaded into a Streamlit application. The app provides a simple user interface where users can input text and receive a sentiment prediction. The application uses ngrok to create a public URL, making it accessible from anywhere.
💾 How to Run the Project
Clone the Repository:
Open in Colab: The easiest way to run this project is in Google Colab, as all the steps are laid out in a sequential notebook format.
Run Each Cell: Execute each code cell in the notebook in the specified order (Steps 1-8). Make sure to follow the instructions for authenticating with ngrok and mounting your Google Drive to save the model permanently.
