collapse_gemma-2-2b_hs2_replace_iter2_sftsd0 – AI Model by KrisMinchev | AlphaNeural AI

You can deploy this model and start earning money today!

KrisMinchev/
collapse_gemma-2-2b_hs2_replace_iter2_sftsd0

like

0

generated_from_trainer

google/gemma-2-2b

Views

No views yet

collapse_gemma-2-2b_hs2_replace_iter2_sftsd0

This model is a fine-tuned version of google/gemma-2-2b on an unknown dataset. It achieves the following results on the evaluation set:

Loss: 1.6314
Num Input Tokens Seen: 4852992

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 8e-06
train_batch_size: 8
eval_batch_size: 16
seed: 0
gradient_accumulation_steps: 16
total_train_batch_size: 128
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: constant_with_warmup
lr_scheduler_warmup_ratio: 0.05
num_epochs: 1

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
No log	0	0	1.3911	0
1.5169	0.0579	5	1.2663	285536
1.3736	0.1158	10	1.1918	567456
1.1135	0.1737	15	1.2205	844248
0.8215	0.2315	20	1.2889	1125792
0.7	0.2894	25	1.3912	1408488
0.4534	0.3473	30	1.4630	1691120
0.4168	0.4052	35	1.5283	1974504
0.3295	0.4631	40	1.5404	2257232
0.1989	0.5210	45	1.4892	2538768
0.2135	0.5789	50	1.5781	2821752
0.1784	0.6368	55	1.4920	3105616
0.1314	0.6946	60	1.5251	3381944
0.1645	0.7525	65	1.5864	3666096
0.0937	0.8104	70	1.4925	3946448
0.1406	0.8683	75	1.5310	4234720
0.1477	0.9262	80	1.5799	4512080
0.0654	0.9841	85	1.6488	4796248

Framework versions

Transformers 4.44.0
Pytorch 2.4.0+cu121
Datasets 2.20.0
Tokenizers 0.19.1