The YelpReviewFull dataset consists of reviews from Yelp. It was constructed by Xiang Zhang (xiang.zhang@nyu.edu) from the Yelp Dataset Challenge 2015.
It was first used as a text classification benchmark in the following paper:
Xiang Zhang, Junbo Zhao, Yann LeCun. Character-level Convolutional Networks for Text Classification. Advances in Neural Information Processing Systems 28 (NIPS 2015).
The dataset can be found in this Hugging Face link🔗.
Preprocessing [optional]
Preprocessing steps include removing punctuation, removing stopwords, lemmatizing and padding.
Training Hyperparameters
Evaluation
The performance metrics of the Optimized model were Accuracy, Precission, Recall, and F1-Score.
The most important hyperparameter for this optimization was the Learning Rate, which had been modified from 5e-1 to 1e-1, and finally set to 2e-5 for the final optimization. A learning rate that's too high can cause the model to converge too quickly to a suboptimal solution, while too low a learning rate can result in slow convergence, resulting in long training times. A compromised between risk of suboptimal performance and training time was found with the final learning rate used (2e-5).
Another hyperparameter changed was the number of training Epochs, which controls how many times the model sees the entire training dataset. Too few epochs may lead to underfitting, while too many can lead to overfitting. To avoid overfitting, a technique called Early Stopping was used. This technique is used to halt training when the model's performance on the validation set stops improving. This helps prevent overfitting by ensuring that the model does not continue training beyond the point where it is making significant progress.
Another important consideration was the Weight Decay hyperparameter, is it is useful for regularization to avoid overfitting.
Hyperparameters important for memory usage and speed
The following hyperparameters helped to avoid losing valuable model training progress due to the Colab Notebook disconecting from the hosted runtime due to inactivity or reaching the maximum RAM available:
The Per Device Evaluation Batch Size directly affected the speed and memory usage during the evaluation.
The Evaluation Strategy was set to 'epoch' so the model would be evaluated on the validation set everytime one epoch was completed.
The Save Strategy was set to 'epoch' so the models state would be saved with every completed epoch.
Even if the notebook would disconnect, with the saved model's progress, the training could be restarted from that point.
Results
text
label
score
This restaurant was the best ever, I really enjoyed the food there!
5 stars
0.967317
I would recommend this to my family and friends!
4 stars
0.530670
Not that big of a deal, I don't know what everyone is talking about.
3 stars
0.626009
It was okay, not that bad, but also not extremely good
