This model is a fine-tuned version of BERT-base-uncased for three-class sentiment classification on user-generated drug reviews.
The task consists of classifying textual reviews into:
Negative
Neutral
Positive
The model was trained on a cleaned and preprocessed version of the Drugs.com review dataset.
📂 Dataset
Source: Drugs.com Drug Review Dataset
User reviews
Numerical rating (1–10 scale)
Label Mapping Strategy
Three-class sentiment labels were constructed as:
Negative → Low ratings
Neutral → Mid-range ratings
Positive → High ratings
Text preprocessing pipeline:
Lowercasing
Punctuation removal
Stopword filtering
Lemmatization
Tokenization using BERT tokenizer
🏗 Model Architecture
Base Model: BERT-base-uncased
Architecture Details:
12 Transformer encoder layers
Hidden size: 768
12 attention heads
Classification head added for 3 output labels
Loss Function:
CrossEntropyLoss
Optimizer:
AdamW
Training performed using:
PyTorch
HuggingFace Transformers
📊 Evaluation Metrics
Evaluation was conducted on a held-out validation set.
Metrics reported:
Accuracy
Precision (macro)
Recall (macro)
F1-score (macro)
Confusion Matrix
Example
| Model | Feature | Accuracy | F1 |
| Logistic Regression | TF-IDF | 0.82 | 0.81 |
| Naive Bayes | TF-IDF | 0.78 | 0.77 |
| Random Forest | TF-IDF | 0.81 | 0.79 |
| LSTM | Word2Vec | 0.83 | 0.83 |
🔍 Key Findings
Transformer-based contextual embeddings outperform classical TF-IDF based models in multi-class sentiment detection.
The neutral class benefits significantly from contextual modeling.
Domain-specific vocabulary (medical terminology) is better captured by BERT representations.
🧪 Example Inference
python
1from transformers import pipeline
23classifier = pipeline(4"text-classification",5 model="erdemyavuz55/drug-review-sentiment-analysis-bert"6)78classifier("This medication worked great with minimal side effects.")
Example Output:
[{'label': 'POSITIVE', 'score': 0.97}]
📈 Comparison with Classical Models
In the broader study, we compared:
TF-IDF + Logistic Regression
TF-IDF + SVM
Word2Vec + LSTM
Random Forest
Gradient Boosting
The fine-tuned BERT model achieved superior macro-F1 in the three-class setting.
⚠️ Limitations
Trained only on English reviews
Domain-specific slang may reduce accuracy
Ratings-based labeling may introduce subjectivity bias
No domain-adaptive pretraining performed
🚀 Future Work
Domain-adaptive pretraining on medical corpora
Model interpretability (SHAP, attention visualization)
Deployment as REST API
Cross-dataset generalization experiments
🤗 Citation
If you use this model, please cite:
Fine-tuned BERT for Drug Review Sentiment Classification, 2026.
