DistilBERT fine-tuned on SST-2 with LoRA
Fine-tuned distilbert-base-uncased for binary sentiment classification
using parameter-efficient LoRA (r=8) via the PEFT library.
Accuracy: ~92% on SST-2 validation set
Trainable parameters: ~0.8% of total
Usage
from transformers import pipeline
classifier = pipeline("text-classification", model="Sara-1990/distilbert-lora-sst2")
classifier("The paper results are surprisingly strong.")
Training details
- Base model: distilbert-base-uncased
- LoRA rank: 8, alpha: 16
- Epochs: 3, batch size: 16
- Hardware: [Apple MacBook Air CPU]
Links
