Fine-tuned distilroberta-base (82M params) for sentiment classification on real customer-support tweets, with labels distilled from Claude Sonnet 4.6.

Part of the TicketSense project: a tiny specialized classifier that approximates frontier-LLM judgement at a fraction of the cost and latency.

Labels

positive
neutral
negative

Eval

Macro F1 (validation): 0.7634

Training

Base model: distilroberta-base
Method: LoRA (PEFT), then merged
LoRA rank: 16, alpha: 32
LR: 0.0002, epochs: 4
Train data: ~1400 real customer-support tweets, Claude-labeled

Use

python

from transformers import pipeline
clf = pipeline("text-classification", model="Grinding/ticketsense-sentiment")
clf("my card was charged twice and nobody has responded")

Limitations

Trained on noisy real-world Twitter data — picks up typos and slang but may also reflect their biases.
Labels are themselves model-generated (Claude Sonnet 4.6) — treat F1 vs Claude as agreement, not gold-standard truth.