This model uses the dataset jennifee/HW1-aug-text-dataset which contains a text Review of a book, and classifies it as Fiction (1) or Nonfiction(0). The model seeks to fine-tune DistilBERT to accomplish this classification task. It computes a confusion matrix and a brief error analysis of several misclassifications.

Intended uses & limitations

This is intended to perform and demonstrate finetuning a text model. It is trained on a relatively small dataset and so may not be applicable for large scale applications.

Training and evaluation data

This dataset contains a text 'Review' of a book, and classifies it as Fiction (1) or Nonfiction(0) It contains two splits of original (100 rows) and augmented (1.6k rows) The augmented dataset is split into train/validation/test sets using an 80/20 split

Training procedure

The model is tokenized for binary classifiation such that train: 1024 | val: 256 | test: 320 | ext_valid: 100

The hypermaraters for training are listed below

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 2e-05
train_batch_size: 8
eval_batch_size: 8
seed: 42
optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
lr_scheduler_type: linear
num_epochs: 5

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	F1	Precision	Recall
0.005	1.0	128	0.0343	0.9922	0.9922	0.9923	0.9922
0.002	2.0	256	0.0072	0.9961	0.9961	0.9961	0.9961
0.0011	3.0	384	0.0013	1.0	1.0	1.0	1.0
0.0008	4.0	512	0.0007	1.0	1.0	1.0	1.0
0.0016	5.0	640	0.0006	1.0	1.0	1.0	1.0

Framework versions

Transformers 4.56.1
Pytorch 2.8.0+cu126
Datasets 4.0.0
Tokenizers 0.22.0

madhavkarthi/24679-HW2-text-distilbert-predictor

24679-HW2-text-distilbert-predictor

Model description