Beta
Explore
Marketplace
Neural Labs
Chat
Wallet
Docs
distil_roberta_model – AI Model by oleks-shap | AlphaNeural AI
You can deploy this model and start earning money today!
oleks-shap
/
distil_roberta_model
like
0
pytorch
roberta
generated_from_trainer
other
us
Views
No views yet
Model card
Files and Versions
Community
API
Deploy
distilroberta-clickbait
This model is a fine-tuned version of
distilroberta-base
on a dataset of headlines. It achieves the following results on the evaluation set:
Loss: 0.0268
Acc: 0.9963
Training and evaluation data
The following data sources were used:
32k headlines classified as clickbait/not-clickbait from
kaggle
A dataset of headlines from
https://github.com/MotiBaadror/Clickbait-Detection
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
learning_rate: 2e-05
train_batch_size: 32
eval_batch_size: 32
seed: 12345
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
lr_scheduler_warmup_steps: 16
num_epochs: 20
mixed_precision_training: Native AMP
Training results
Training Loss
Epoch
Step
Validation Loss
Acc
0.0195
1.0
981
0.0192
0.9954
0.0026
2.0
1962
0.0172
0.9963
0.0031
3.0
2943
0.0275
0.9945
0.0003
4.0
3924
0.0268
0.9963
Framework versions
Transformers 4.11.3
Pytorch 1.10.1
Datasets 1.17.0
Tokenizers 0.10.3
