You can deploy this model and start earning money today!

RichardErkhov/
tsavage68_-_Na_M2_1000steps_1e7rate_05beta_cSFTDPO-gguf

Name: tsavage68_-_Na_M2_1000steps_1e7rate_05beta_cSFTDPO-gguf
Availability: InStock

Views

No views yet

Quantization made by Richard Erkhov.

Github

Discord

Request more models

Na_M2_1000steps_1e7rate_05beta_cSFTDPO - GGUF

Model creator: https://huggingface.co/tsavage68/
Original model: https://huggingface.co/tsavage68/Na_M2_1000steps_1e7rate_05beta_cSFTDPO/

Name	Quant method	Size
Na_M2_1000steps_1e7rate_05beta_cSFTDPO.Q2_K.gguf	Q2_K	2.53GB
Na_M2_1000steps_1e7rate_05beta_cSFTDPO.IQ3_XS.gguf	IQ3_XS	2.81GB
Na_M2_1000steps_1e7rate_05beta_cSFTDPO.IQ3_S.gguf	IQ3_S	2.96GB
Na_M2_1000steps_1e7rate_05beta_cSFTDPO.Q3_K_S.gguf	Q3_K_S	2.95GB
Na_M2_1000steps_1e7rate_05beta_cSFTDPO.IQ3_M.gguf	IQ3_M	3.06GB
Na_M2_1000steps_1e7rate_05beta_cSFTDPO.Q3_K.gguf	Q3_K	3.28GB
Na_M2_1000steps_1e7rate_05beta_cSFTDPO.Q3_K_M.gguf	Q3_K_M	3.28GB
Na_M2_1000steps_1e7rate_05beta_cSFTDPO.Q3_K_L.gguf	Q3_K_L	3.56GB
Na_M2_1000steps_1e7rate_05beta_cSFTDPO.IQ4_XS.gguf	IQ4_XS	3.67GB
Na_M2_1000steps_1e7rate_05beta_cSFTDPO.Q4_0.gguf	Q4_0	3.83GB
Na_M2_1000steps_1e7rate_05beta_cSFTDPO.IQ4_NL.gguf	IQ4_NL	3.87GB
Na_M2_1000steps_1e7rate_05beta_cSFTDPO.Q4_K_S.gguf	Q4_K_S	3.86GB
Na_M2_1000steps_1e7rate_05beta_cSFTDPO.Q4_K.gguf	Q4_K	4.07GB
Na_M2_1000steps_1e7rate_05beta_cSFTDPO.Q4_K_M.gguf	Q4_K_M	4.07GB
Na_M2_1000steps_1e7rate_05beta_cSFTDPO.Q4_1.gguf	Q4_1	4.24GB
Na_M2_1000steps_1e7rate_05beta_cSFTDPO.Q5_0.gguf	Q5_0	4.65GB
Na_M2_1000steps_1e7rate_05beta_cSFTDPO.Q5_K_S.gguf	Q5_K_S	4.65GB
Na_M2_1000steps_1e7rate_05beta_cSFTDPO.Q5_K.gguf	Q5_K	4.78GB
Na_M2_1000steps_1e7rate_05beta_cSFTDPO.Q5_K_M.gguf	Q5_K_M	4.78GB
Na_M2_1000steps_1e7rate_05beta_cSFTDPO.Q5_1.gguf	Q5_1	5.07GB
Na_M2_1000steps_1e7rate_05beta_cSFTDPO.Q6_K.gguf	Q6_K	5.53GB
Na_M2_1000steps_1e7rate_05beta_cSFTDPO.Q8_0.gguf	Q8_0	7.17GB

Original model description:

library_name: transformers license: apache-2.0 base_model: tsavage68/Na_M2_1000steps_1e7_SFT tags:

trl
dpo
generated_from_trainer model-index:
name: Na_M2_1000steps_1e7rate_05beta_cSFTDPO results: []

Na_M2_1000steps_1e7rate_05beta_cSFTDPO

This model is a fine-tuned version of tsavage68/Na_M2_1000steps_1e7_SFT on an unknown dataset. It achieves the following results on the evaluation set:

Loss: 0.0000
Rewards/chosen: 3.4353
Rewards/rejected: -12.0460
Rewards/accuracies: 1.0
Rewards/margins: 15.4813
Logps/rejected: -104.0153
Logps/chosen: -41.2618
Logits/rejected: -2.5171
Logits/chosen: -2.5312

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 1e-07
train_batch_size: 2
eval_batch_size: 1
seed: 42
gradient_accumulation_steps: 2
total_train_batch_size: 4
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: cosine
lr_scheduler_warmup_steps: 100
training_steps: 1000

Training results

Epoch	Step	Rewards/chosen	Rewards/rejected	Rewards/accuracies	Rewards/margins	Logps/rejected	Logps/chosen	Logits/rejected	Logits/chosen
0.2667	50	2.4333	-8.7946	1.0	11.2279	-97.5125	-43.2658	-2.5259	-2.5391
0.5333	100	2.7977	-9.9936	1.0	12.7913	-99.9105	-42.5369	-2.5223	-2.5359
0.8	150	2.9419	-10.6551	1.0	13.5970	-101.2335	-42.2486	-2.5210	-2.5347
1.0667	200	3.0397	-10.9989	1.0	14.0386	-101.9212	-42.0530	-2.5209	-2.5347
1.3333	250	3.1479	-11.2365	1.0	14.3844	-102.3963	-41.8365	-2.5209	-2.5348
1.6	300	3.1788	-11.4604	1.0	14.6393	-102.8442	-41.7747	-2.5197	-2.5337
1.8667	350	3.2803	-11.6306	1.0	14.9109	-103.1846	-41.5718	-2.5199	-2.5339
2.1333	400	3.3009	-11.7868	1.0	15.0878	-103.4970	-41.5305	-2.5189	-2.5328
2.4	450	3.3596	-11.8664	1.0	15.2260	-103.6562	-41.4132	-2.5179	-2.5319
2.6667	500	3.3481	-11.9338	1.0	15.2818	-103.7909	-41.4363	-2.5176	-2.5316
2.9333	550	3.3954	-11.9591	1.0	15.3545	-103.8415	-41.3415	-2.5186	-2.5326
3.2	600	3.4233	-12.0436	1.0	15.4669	-104.0106	-41.2858	-2.5181	-2.5321
3.4667	650	3.4170	-12.0535	1.0	15.4704	-104.0303	-41.2985	-2.5183	-2.5323
3.7333	700	3.3924	-12.0736	1.0	15.4660	-104.0705	-41.3476	-2.5178	-2.5318
4.0	750	3.4428	-12.0566	1.0	15.4994	-104.0365	-41.2468	-2.5180	-2.5321
4.2667	800	3.4331	-12.0469	1.0	15.4800	-104.0172	-41.2661	-2.5173	-2.5314
4.5333	850	3.4177	-12.0794	1.0	15.4970	-104.0821	-41.2971	-2.5172	-2.5312
4.8	900	3.4353	-12.0460	1.0	15.4813	-104.0153	-41.2618	-2.5171	-2.5312
5.0667	950	3.4353	-12.0460	1.0	15.4813	-104.0153	-41.2618	-2.5171	-2.5312
5.3333	1000	3.4353	-12.0460	1.0	15.4813	-104.0153	-41.2618	-2.5171	-2.5312

Framework versions

Transformers 4.44.2
Pytorch 2.4.0+cu121
Datasets 2.21.0
Tokenizers 0.19.1

RichardErkhov/tsavage68_-_Na_M2_1000steps_1e7rate_05beta_cSFTDPO-gguf

Original model description:

Na_M2_1000steps_1e7rate_05beta_cSFTDPO

Model description

Intended uses & limitations

Training and evaluation data

Training procedure

Training hyperparameters

Training results

Framework versions

RichardErkhov/
tsavage68_-_Na_M2_1000steps_1e7rate_05beta_cSFTDPO-gguf