This model predicts the star rating (1 - 5) of a Yelp review based on its text content. It was trained using GPT-2 and BERT, with BERT achieving the best performance at 75% validation accuracy. The model addresses class imbalance using weighted loss and optimizes hyperparameters to enhance generalization.
-
Dataset: Yelp Reviews dataset (100,000 samples used)
-
Preprocessing:
- GPT-2 Tokenizer with Byte-Pair Encoding (BPE) for rare words
- Truncation (128 tokens) and padding for uniform input size
-
Models Trained:
-
GPT-2: Fine-tuned with a custom classification head, achieving 67% validation accuracy
-
BERT: Fine-tuned with bidirectional attention, achieving 75% validation accuracy
-
Loss Function: Weighted Cross-Entropy Loss to counteract class imbalance
-
Performance may degrade on highly informal or extremely short reviews
-
Class imbalance still affects predictions for underrepresented ratings
-
Model was trained on English-language reviews only
