You can deploy this model and start earning money today!

nikitastheo/
babylm-ita-ell-interleaved-dummy

Name: babylm-ita-ell-interleaved-dummy
Availability: InStock

generated_from_trainer

text-generation-inference

endpoints_compatible

Views

No views yet

ita-ell-interleaved

This model is a fine-tuned version of gpt_dummy_config.json on the data/babybabellm/ita-ell/tokenized_interleaved/train_1.dummy.txt dataset. It achieves the following results on the evaluation set:

Loss: 10.3281
Accuracy: 0.0059

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 0.0001
train_batch_size: 32
eval_batch_size: 32
seed: 42
optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-06 and optimizer_args=No additional optimizer arguments
lr_scheduler_type: linear
lr_scheduler_warmup_steps: 16
training_steps: 128

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
10.8312	5.0	5	10.8354	0.0018
10.8147	10.0	10	10.8292	0.0018
10.7816	15.0	15	10.8142	0.0016
10.7263	20.0	20	10.7806	0.0012
10.6647	25.0	25	10.7343	0.0006
10.6091	30.0	30	10.6908	0.0002
10.5569	35.0	35	10.6538	0.0008
10.5058	40.0	40	10.6205	0.0010
10.4558	45.0	45	10.5892	0.0016
10.409	50.0	50	10.5586	0.0022
10.3654	55.0	55	10.5285	0.0025
10.3244	60.0	60	10.5007	0.0031
10.2865	65.0	65	10.4756	0.0041
10.2518	70.0	70	10.4524	0.0051
10.2201	75.0	75	10.4312	0.0057
10.1915	80.0	80	10.4121	0.0059
10.1659	85.0	85	10.3951	0.0059
10.1432	90.0	90	10.3802	0.0061
10.1235	95.0	95	10.3673	0.0061
10.1065	100.0	100	10.3563	0.0063
10.0923	105.0	105	10.3472	0.0063
10.0809	110.0	110	10.3399	0.0063
10.0721	115.0	115	10.3343	0.0061
10.0659	120.0	120	10.3305	0.0061
10.0623	125.0	125	10.3285	0.0061

Framework versions

Transformers 4.57.6
Pytorch 2.12.0+cu130
Datasets 4.8.5
Tokenizers 0.22.2

nikitastheo/babylm-ita-ell-interleaved-dummy

ita-ell-interleaved

Model description

Intended uses & limitations

Training and evaluation data

Training procedure

Training hyperparameters

Training results

Framework versions

nikitastheo/
babylm-ita-ell-interleaved-dummy