You can deploy this model and start earning money today!

justinwangx/
mistral-instruct-adv-robust-50-sft-lora

tensorboard

safetensors

generated_from_trainer

mistralai/Mistral-7B-Instruct-v0.1

finetune

apache-2.0

Views

No views yet

mistral-instruct-adv-robust-50-sft-lora

This model is a fine-tuned version of mistralai/Mistral-7B-Instruct-v0.1 on an unknown dataset. It achieves the following results on the evaluation set:

Loss: 0.8817

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 0.0003
train_batch_size: 4
eval_batch_size: 8
seed: 42
distributed_type: multi-GPU
num_devices: 4
gradient_accumulation_steps: 16
total_train_batch_size: 256
total_eval_batch_size: 32
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: cosine
num_epochs: 50

Training results

Training Loss	Epoch	Step	Validation Loss
3.1318	0.12	1	2.8355
3.1318	1.12	2	2.6364
3.1318	2.12	3	2.4945
3.1318	3.12	4	2.5339
2.7386	4.12	5	2.3352
2.7386	5.12	6	2.2137
2.7386	6.12	7	2.1641
2.7386	7.12	8	2.1051
2.7386	8.12	9	2.0842
2.269	9.12	10	2.0479
2.269	10.12	11	1.9554
2.269	11.12	12	1.8555
2.269	12.12	13	1.7736
2.269	13.12	14	1.7906
1.9451	14.12	15	1.7737
1.9451	15.12	16	1.6677
1.9451	16.12	17	1.6411
1.9451	17.12	18	1.5739
1.9451	18.12	19	1.5334
1.6568	19.12	20	1.4794
1.6568	20.12	21	1.4008
1.6568	21.12	22	1.3625
1.6568	22.12	23	1.2964
1.6568	23.12	24	1.2041
1.3674	24.12	25	1.1971
1.3674	25.12	26	1.1571
1.3674	26.12	27	1.1080
1.3674	27.12	28	1.1099
1.3674	28.12	29	1.0930
1.145	29.12	30	1.0333
1.145	30.12	31	1.0096
1.145	31.12	32	1.0012
1.145	32.12	33	0.9266
1.145	33.12	34	0.9624
0.9987	34.12	35	0.9425
0.9987	35.12	36	0.9354
0.9987	36.12	37	0.9091
0.9987	37.12	38	0.9007
0.9987	38.12	39	0.9649
0.9071	39.12	40	0.9199
0.9071	40.12	41	0.8651
0.9071	41.12	42	0.8727
0.9071	42.12	43	0.8559
0.9071	43.12	44	0.8499
0.8522	44.12	45	0.8547
0.8522	45.12	46	0.8880
0.8522	46.12	47	0.8678
0.8522	47.12	48	0.8565
0.8522	48.12	49	0.8197
0.8153	49.12	50	0.8439

Framework versions

Transformers 4.35.0
Pytorch 2.1.0a0+32f93b1
Datasets 2.14.6
Tokenizers 0.14.1

justinwangx/mistral-instruct-adv-robust-50-sft-lora

mistral-instruct-adv-robust-50-sft-lora

Model description

Intended uses & limitations

Training and evaluation data

Training procedure

Training hyperparameters

Training results

Framework versions

justinwangx/
mistral-instruct-adv-robust-50-sft-lora