You can deploy this model and start earning money today!

ntnu-smil/
secret-model-stage-1-8B-512

Name: secret-model-stage-1-8B-512
Availability: InStock

transformers

tensorboard

safetensors

generated_from_trainer

endpoints_compatible

Views

No views yet

secret-model-stage-1-8B-512

This model was trained from scratch on an unknown dataset. It achieves the following results on the evaluation set:

Loss: 0.0883
Centroid Acc: 0.9811
Centroid Macro F1: 0.9805
Knn Acc: 0.9811
Knn Macro F1: 0.9805
Alignment: 0.4287
Uniformity: -2.9112
Combined Score: 0.9805

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 0.001
train_batch_size: 16
eval_batch_size: 64
seed: 42
optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
lr_scheduler_type: cosine
lr_scheduler_warmup_ratio: 0.06
num_epochs: 100.0

Training results

Training Loss	Epoch	Step	Validation Loss	Centroid Acc	Centroid Macro F1	Knn Acc	Knn Macro F1	Alignment	Uniformity	Combined Score
No log	0	0	2.3395	0.6792	0.7021	0.8868	0.8950	0.3514	-0.8454	0.7664
1.1369	3.125	100	0.8131	0.8679	0.8727	0.8868	0.8852	0.4878	-2.1862	0.8769
0.9967	6.25	200	0.7616	0.8868	0.8848	0.8868	0.8905	0.4644	-2.2835	0.8867
0.6729	9.375	300	0.5749	0.8868	0.8947	0.9057	0.9047	0.3455	-1.9797	0.8980
0.2518	12.5	400	0.1456	0.9434	0.9438	0.9245	0.9223	0.4195	-2.5864	0.9367
0.3085	15.625	500	0.2325	0.9811	0.9805	0.9623	0.9590	0.4281	-2.5262	0.9733
0.2044	18.75	600	0.3032	0.9245	0.9263	0.9434	0.9438	0.4584	-2.6123	0.9322
0.1698	21.875	700	0.1874	0.9245	0.9265	0.9245	0.9265	0.4409	-2.7758	0.9265
0.0758	25.0	800	0.1830	0.9623	0.9634	0.9623	0.9634	0.4574	-2.7934	0.9634
0.0758	25.0	800	0.1830	0.9623	0.9634	0.9623	0.9634	0.4574	-2.7934	0.9634
0.0518	28.125	900	0.2130	0.9811	0.9805	0.9811	0.9805	0.4700	-2.8665	0.9805
0.0285	31.25	1000	0.2894	0.9811	0.9805	0.9811	0.9805	0.4640	-2.8205	0.9805
0.0594	34.375	1100	0.1053	0.9811	0.9805	0.9811	0.9805	0.4249	-2.7914	0.9805
0.0783	37.5	1200	0.1112	0.9623	0.9609	0.9623	0.9609	0.4376	-2.8689	0.9609
0.0059	40.625	1300	0.0850	1.0	1.0	1.0	1.0	0.4280	-2.8612	1.0
0.0173	43.75	1400	0.0844	0.9811	0.9805	0.9811	0.9805	0.4193	-2.8723	0.9805
0.0079	46.875	1500	0.1321	0.9811	0.9805	0.9623	0.9612	0.4394	-2.9129	0.9741
0.005	50.0	1600	0.1309	0.9811	0.9805	0.9811	0.9805	0.4310	-2.8984	0.9805
0.005	50.0	1600	0.1309	0.9811	0.9805	0.9811	0.9805	0.4310	-2.8984	0.9805
0.056	53.125	1700	0.0857	0.9811	0.9805	0.9811	0.9805	0.4145	-2.8639	0.9805
0.0023	56.25	1800	0.1039	0.9811	0.9805	0.9811	0.9805	0.4322	-2.9275	0.9805
0.02	59.375	1900	0.1086	0.9811	0.9805	0.9811	0.9805	0.4378	-2.9313	0.9805
0.0323	62.5	2000	0.0862	0.9811	0.9805	0.9623	0.9612	0.4163	-2.8686	0.9741
0.0025	65.625	2100	0.0929	0.9811	0.9805	0.9811	0.9805	0.4195	-2.8867	0.9805
0.0023	68.75	2200	0.0719	0.9811	0.9805	1.0	1.0	0.4214	-2.9106	0.9870
0.0028	71.875	2300	0.0942	0.9811	0.9805	0.9811	0.9805	0.4291	-2.9125	0.9805
0.0014	75.0	2400	0.0861	0.9811	0.9805	0.9811	0.9805	0.4310	-2.9212	0.9805
0.0014	75.0	2400	0.0861	0.9811	0.9805	0.9811	0.9805	0.4310	-2.9212	0.9805
0.0022	78.125	2500	0.0869	0.9811	0.9805	0.9811	0.9805	0.4311	-2.9192	0.9805
0.002	81.25	2600	0.0731	0.9811	0.9805	0.9811	0.9805	0.4205	-2.8965	0.9805
0.0105	84.375	2700	0.0787	0.9811	0.9805	0.9811	0.9805	0.4277	-2.9218	0.9805
0.0015	87.5	2800	0.0818	0.9811	0.9805	0.9811	0.9805	0.4295	-2.9185	0.9805
0.0015	90.625	2900	0.0897	0.9811	0.9805	0.9811	0.9805	0.4283	-2.9066	0.9805
0.002	93.75	3000	0.0884	0.9811	0.9805	0.9811	0.9805	0.4285	-2.9100	0.9805
0.0328	96.875	3100	0.0886	0.9811	0.9805	0.9811	0.9805	0.4287	-2.9109	0.9805
0.0017	100.0	3200	0.0883	0.9811	0.9805	0.9811	0.9805	0.4287	-2.9112	0.9805
0.0017	100.0	3200	0.0883	0.9811	0.9805	0.9811	0.9805	0.4287	-2.9112	0.9805

Framework versions

Transformers 4.56.0
Pytorch 2.8.0+cu128
Datasets 4.0.0
Tokenizers 0.22.0

ntnu-smil/secret-model-stage-1-8B-512

secret-model-stage-1-8B-512

Model description

Intended uses & limitations

Training and evaluation data

Training procedure

Training hyperparameters

Training results

Framework versions

ntnu-smil/
secret-model-stage-1-8B-512