Beta
Explore
Marketplace
Neural Labs
Chat
Wallet
Docs
refusal – Dataset by mrfakename | AlphaNeural AI
You can deploy this model and start earning money today!
mrfakename
/
refusal
like
0
text-generation
en
1K<n<10K
json
text
datasets
pandas
mlcroissant
polars
us
Views
No views yet
Model card
Files and Versions
Community
API
I messed up on the
previous model
. This is a fixed version.
A tiny 1B model that refuses basically anything you ask it! Trained on the
refusal
dataset. Prompt format is ChatML.
Training results:
Training Loss
Epoch
Step
Validation Loss
2.4352
0.0580
1
2.4462
1.5741
0.5217
9
1.4304
1.5204
1.0435
18
1.3701
1.0794
1.5217
27
1.3505
1.1275
2.0435
36
1.3344
0.6652
2.5217
45
1.4360
0.6248
3.0435
54
1.4313
0.6142
3.5072
63
1.4934
Training hyperparemeters:
The following hyperparameters were used during training:
learning_rate: 0.0002
train_batch_size: 2
eval_batch_size: 2
seed: 42
gradient_accumulation_steps: 4
total_train_batch_size: 8
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: cosine
lr_scheduler_warmup_steps: 10
num_epochs: 4
Base model:
https://huggingface.co/TinyLlama/TinyLlama-1.1B-intermediate-step-1431k-3T
