jamander/

Project-Blockbuster

Project-Blockbuster

Model Overview

• Model Name: Project-Blockbuster
• Model Type: Sentiment Analysis
• Base Model: TinyBERT (prajjwal1/bert-tiny)
• Fine-tuned by: Jack Mander

Description:

Model Details

Model Architecture:

• Base Model: TinyBERT (prajjwal1/bert-tiny)
• Tokenizer: AutoTokenizer from Hugging Face Transformers
• Training Framework: Transformers, PEFT, and BitsAndBytes libraries

Training Data:

• The model was fine-tuned using the IMDB Dataset, which contains 50,000 labeled movie reviews.
• The data was split into training (80%) and test (20%) sets using stratified sampling to maintain label balance.
• Tokenization was applied with a maximum sequence length of 512 tokens.

Hyperparameters:

• Learning Rate: 3e-5
• Epochs: 3
• Batch Size: 16 (with gradient accumulation steps of 2)
• Optimizer: AdamW
• Scheduler: Linear learning rate decay

Training Procedure

Environment:

• The model was trained on a Tesla T4 GPU in Google Colab using mixed precision (FP16) for efficiency.

Data Preparation:

1. The IMDB Dataset was preprocessed into training and test sets with stratified sampling.
2. Text data was tokenized using TinyBERT's tokenizer with truncation and padding.

Model Training:

• Fine-tuning was performed using parameter-efficient fine-tuning (PEFT) via LoRA (Low-Rank Adaptation).
• TinyBERT was quantized to 4-bit precision with the BitsAndBytes library for faster and more memory-efficient training.

Evaluation Metrics

Example Predictions

Input:

Prediction:

Input:

Prediction:

Limitations and Biases

• Task-Specific: The model is fine-tuned for movie review sentiment analysis and may not generalize well to other domains or datasets.
• Data Bias: Any biases present in the IMDB Dataset, such as overrepresentation of certain styles or phrases, may be reflected in the model's predictions.
• Resource Constraints: While TinyBERT is efficient, its smaller size means it may miss subtle nuances in complex reviews compared to larger models like BERT-base.

Acknowledgments

Usage

1. Log in to Hugging Face:
   huggingface-cli login
2. Load the model:
   python

   1from transformers import AutoTokenizer, AutoModelForSequenceClassification 2 3tokenizer = AutoTokenizer.from_pretrained("jamander/Project-Blockbuster") 4model = AutoModelForSequenceClassification.from_pretrained("jamander/Project-Blockbuster")
3. Tokenize and predict:
   python

   1inputs = tokenizer("I loved the movie!", return_tensors="pt", padding=True, truncation=True) 2outputs = model(**inputs) 3prediction = torch.argmax(outputs.logits, dim=-1) 4print("Prediction:", "Positive" if prediction == 1 else "Negative")