Fine-tuned Mistral 7B using LoRA to classify comments as toxic or not toxic,
using the Jigsaw Toxic Comments dataset.
Limitation — Read Before Using
This model has 0% recall on toxic content. It was trained on an
imbalanced dataset (4.4% toxic examples) and learned to always predict
"not toxic" regardless of input. It will not flag actual toxic content.
This is documented honestly rather than hidden — see Evaluation Results
below for the full diagnosis. Do not use this model for actual content
moderation in its current state.
Training Details
Base model: unsloth/mistral-7b-instruct-v0.3-bnb-4bit
Training data: 2,500 examples from Jigsaw Toxic Comments (111 toxic, 2,389 not toxic)
Training steps: ~100, batch size 2, gradient accumulation 4
Evaluation Results
Tested on 20 held-out examples (no overlap with training data):
Metric
Score
Accuracy
95.0%
Precision
0.0%
Recall
0.0%
F1 Score
0.0%
Confusion Matrix:
What I'd Do Differently
Balance the training set (equal toxic / not-toxic examples)
Evaluate with precision/recall/F1 during training, not just after
Check class balance during dataset preparation, before training starts
Why This Is Documented Instead of Hidden
A model reporting fake high accuracy while having zero real capability
is worse than no model — it creates false confidence. This card exists
so anyone using this model understands its actual (current) limitations.
Intended Use
Currently: Portfolio demonstration only.
Not suitable for production content moderation without retraining on
a balanced dataset.
