You can deploy this model and start earning money today!

RichardErkhov/
tsavage68_-_Na_M2_1000steps_1e6rate_05beta_cSFTDPO-gguf

Name: tsavage68_-_Na_M2_1000steps_1e6rate_05beta_cSFTDPO-gguf
Availability: InStock

Views

No views yet

Quantization made by Richard Erkhov.

Github

Discord

Request more models

Na_M2_1000steps_1e6rate_05beta_cSFTDPO - GGUF

Model creator: https://huggingface.co/tsavage68/
Original model: https://huggingface.co/tsavage68/Na_M2_1000steps_1e6rate_05beta_cSFTDPO/

Name	Quant method	Size
Na_M2_1000steps_1e6rate_05beta_cSFTDPO.Q2_K.gguf	Q2_K	2.53GB
Na_M2_1000steps_1e6rate_05beta_cSFTDPO.IQ3_XS.gguf	IQ3_XS	2.81GB
Na_M2_1000steps_1e6rate_05beta_cSFTDPO.IQ3_S.gguf	IQ3_S	2.96GB
Na_M2_1000steps_1e6rate_05beta_cSFTDPO.Q3_K_S.gguf	Q3_K_S	2.95GB
Na_M2_1000steps_1e6rate_05beta_cSFTDPO.IQ3_M.gguf	IQ3_M	3.06GB
Na_M2_1000steps_1e6rate_05beta_cSFTDPO.Q3_K.gguf	Q3_K	3.28GB
Na_M2_1000steps_1e6rate_05beta_cSFTDPO.Q3_K_M.gguf	Q3_K_M	3.28GB
Na_M2_1000steps_1e6rate_05beta_cSFTDPO.Q3_K_L.gguf	Q3_K_L	3.56GB
Na_M2_1000steps_1e6rate_05beta_cSFTDPO.IQ4_XS.gguf	IQ4_XS	3.67GB
Na_M2_1000steps_1e6rate_05beta_cSFTDPO.Q4_0.gguf	Q4_0	3.83GB
Na_M2_1000steps_1e6rate_05beta_cSFTDPO.IQ4_NL.gguf	IQ4_NL	3.87GB
Na_M2_1000steps_1e6rate_05beta_cSFTDPO.Q4_K_S.gguf	Q4_K_S	3.86GB
Na_M2_1000steps_1e6rate_05beta_cSFTDPO.Q4_K.gguf	Q4_K	4.07GB
Na_M2_1000steps_1e6rate_05beta_cSFTDPO.Q4_K_M.gguf	Q4_K_M	4.07GB
Na_M2_1000steps_1e6rate_05beta_cSFTDPO.Q4_1.gguf	Q4_1	4.24GB
Na_M2_1000steps_1e6rate_05beta_cSFTDPO.Q5_0.gguf	Q5_0	4.65GB
Na_M2_1000steps_1e6rate_05beta_cSFTDPO.Q5_K_S.gguf	Q5_K_S	4.65GB
Na_M2_1000steps_1e6rate_05beta_cSFTDPO.Q5_K.gguf	Q5_K	4.78GB
Na_M2_1000steps_1e6rate_05beta_cSFTDPO.Q5_K_M.gguf	Q5_K_M	4.78GB
Na_M2_1000steps_1e6rate_05beta_cSFTDPO.Q5_1.gguf	Q5_1	5.07GB
Na_M2_1000steps_1e6rate_05beta_cSFTDPO.Q6_K.gguf	Q6_K	5.53GB
Na_M2_1000steps_1e6rate_05beta_cSFTDPO.Q8_0.gguf	Q8_0	7.17GB

Original model description:

library_name: transformers license: apache-2.0 base_model: tsavage68/Na_M2_1000steps_1e7_SFT tags:

trl
dpo
generated_from_trainer model-index:
name: Na_M2_1000steps_1e6rate_05beta_cSFTDPO results: []

Na_M2_1000steps_1e6rate_05beta_cSFTDPO

This model is a fine-tuned version of tsavage68/Na_M2_1000steps_1e7_SFT on an unknown dataset. It achieves the following results on the evaluation set:

Loss: 0.0000
Rewards/chosen: 3.9080
Rewards/rejected: -14.8092
Rewards/accuracies: 1.0
Rewards/margins: 18.7172
Logps/rejected: -109.5417
Logps/chosen: -40.3163
Logits/rejected: -2.5104
Logits/chosen: -2.5251

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 1e-06
train_batch_size: 2
eval_batch_size: 1
seed: 42
gradient_accumulation_steps: 2
total_train_batch_size: 4
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: cosine
lr_scheduler_warmup_steps: 100
training_steps: 1000

Training results

Epoch	Step	Rewards/chosen	Rewards/rejected	Rewards/accuracies	Rewards/margins	Logps/rejected	Logps/chosen	Logits/rejected	Logits/chosen
0.2667	50	2.9065	-11.8929	1.0	14.7994	-103.7091	-42.3194	-2.5220	-2.5359
0.5333	100	3.3215	-13.0434	1.0	16.3649	-106.0102	-41.4894	-2.5188	-2.5330
0.8	150	3.5466	-13.5189	1.0	17.0655	-106.9612	-41.0391	-2.5174	-2.5317
1.0667	200	3.6119	-13.8627	1.0	17.4745	-107.6487	-40.9087	-2.5143	-2.5288
1.3333	250	3.7085	-14.0115	1.0	17.7200	-107.9463	-40.7154	-2.5151	-2.5296
1.6	300	3.7952	-14.1247	1.0	17.9199	-108.1728	-40.5420	-2.5141	-2.5286
1.8667	350	3.7740	-14.2878	1.0	18.0618	-108.4989	-40.5843	-2.5139	-2.5284
2.1333	400	3.8254	-14.4626	1.0	18.2880	-108.8486	-40.4816	-2.5124	-2.5269
2.4	450	3.8372	-14.5044	1.0	18.3416	-108.9322	-40.4579	-2.5127	-2.5273
2.6667	500	3.8544	-14.6284	1.0	18.4828	-109.1802	-40.4237	-2.5115	-2.5260
2.9333	550	3.8744	-14.6609	1.0	18.5353	-109.2451	-40.3835	-2.5116	-2.5262
3.2	600	3.9000	-14.7002	1.0	18.6002	-109.3238	-40.3324	-2.5103	-2.5249
3.4667	650	3.9168	-14.7537	1.0	18.6705	-109.4308	-40.2988	-2.5105	-2.5252
3.7333	700	3.9175	-14.7437	1.0	18.6612	-109.4108	-40.2974	-2.5112	-2.5259
4.0	750	3.9128	-14.7841	1.0	18.6969	-109.4916	-40.3068	-2.5104	-2.5250
4.2667	800	3.9063	-14.7726	1.0	18.6788	-109.4685	-40.3198	-2.5107	-2.5253
4.5333	850	3.9178	-14.7925	1.0	18.7103	-109.5084	-40.2969	-2.5104	-2.5251
4.8	900	3.9074	-14.8080	1.0	18.7154	-109.5393	-40.3176	-2.5104	-2.5251
5.0667	950	3.9080	-14.8092	1.0	18.7172	-109.5417	-40.3163	-2.5104	-2.5251
5.3333	1000	3.9080	-14.8092	1.0	18.7172	-109.5417	-40.3163	-2.5104	-2.5251

Framework versions

Transformers 4.44.2
Pytorch 2.4.0+cu121
Datasets 2.21.0
Tokenizers 0.19.1

RichardErkhov/tsavage68_-_Na_M2_1000steps_1e6rate_05beta_cSFTDPO-gguf

Original model description:

Na_M2_1000steps_1e6rate_05beta_cSFTDPO

Model description

Intended uses & limitations

Training and evaluation data

Training procedure

Training hyperparameters

Training results

Framework versions

RichardErkhov/
tsavage68_-_Na_M2_1000steps_1e6rate_05beta_cSFTDPO-gguf