DistilBERT Stefanov Text Classification

Model Details

• Developed by: Bareethul Kader
• Framework: Hugging Face Transformers (PyTorch backend)
• Repository: bareethul/distilbert-stefanov-dataset
• License: Apache 2.0

Intended Use

Direct Use

• Educational demonstration of fine tuning a compact transformer (DistilBERT) on a small custom text dataset.
• Course assignment for exploring classification, error analysis, and Hugging Face Hub workflows.
• Reference for students learning about reproducibility, metrics reporting, and model card documentation.

Out of Scope Use

• Not intended for production deployments.
• Model is not validated on large scale benchmarks.
• Not suitable for sensitive applications (e.g., hiring, healthcare, legal, or safety critical tasks).

Dataset

• Source: mastefan/2025-24679-Text-dataset-Stefanov
• Task: Multi class text classification based on the label field.
• Splits:
  • original: Used for stratified 80/20 train/test split
  • augmented: Provided for optional robustness evaluation
• Features:
  • Text Entry: Raw text string
  • label: Categorical target label (integer IDs mapped to class names)

Training Setup

• Base model: distilbert-base-uncased
• Optimizer: AdamW (via Hugging Face Trainer)
• Learning rate: 2e-5
• Batch size: 16
• Epochs: 3
• Weight decay: 0.01
• Seed: 42 (ensures reproducibility)
• Evaluation metric(s): Accuracy, Macro-F1, Weighted-F1
• Hardware: Google Colab (T4 GPU, typical environment)
• Preprocessing:
  • Tokenization with distilbert-base-uncased tokenizer
  • Truncation applied; dynamic padding with DataCollatorWithPadding

Results

• Accuracy: 0.6
• F1 (macro): 0.6
• F1 (weighted): 0.6

Training & Evaluation Summary

• The model fine tunes DistilBERT with sensible hyperparameters (3 epochs, batch size 16, learning rate 2e-5, seed 42) and achieves robust metrics (Accuracy and F1 = 0.6 on the held out test set).
• A confusion matrix was generated, showing perfect class separation.

• Misclassified examples often had ambiguous or short text entries.
• Some overlap in label semantics led to confusion between True label and Predicted label.
• Augmented split evaluation showed accuracy and F1 slightly lower (~0.6)

Limitations, Biases, and Ethical Notes

• Dataset is small -> overfitting risk.
• Augmented text may not fully represent real world variation.
• Labels were designed for an educational assignment, not production grade categories.
• Model has not been tested for fairness, bias, or robustness beyond the class dataset.

AI Usage Disclosure

• Assistance tools were used to streamline coding
• Improve documentation clarity
• Refine the model card presentation.

Example Inference