You can deploy this model and start earning money today!

mohit19906/
working

Name: working
Availability: InStock

peft

tensorboard

safetensors

generated_from_trainer

TheBloke/Mistral-7B-Instruct-v0.2-GPTQ

adapter

apache-2.0

Views

No views yet

working

This model is a fine-tuned version of TheBloke/Mistral-7B-Instruct-v0.2-GPTQ on an unknown dataset. It achieves the following results on the evaluation set:

Loss: 1.6079

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 0.0002
train_batch_size: 6
eval_batch_size: 6
seed: 42
gradient_accumulation_steps: 4
total_train_batch_size: 24
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
lr_scheduler_warmup_steps: 2
num_epochs: 50
mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss
4.3979	0.96	6	3.3561
2.837	1.92	12	2.2656
1.9777	2.88	18	1.7212
1.3641	4.0	25	1.4591
1.3384	4.96	31	1.2543
1.1314	5.92	37	1.1326
0.9904	6.88	43	1.0707
0.7908	8.0	50	1.0784
0.8779	8.96	56	1.0891
0.8415	9.92	62	1.1026
0.8044	10.88	68	1.1326
0.6611	12.0	75	1.1425
0.7385	12.96	81	1.2161
0.7071	13.92	87	1.2182
0.6841	14.88	93	1.2865
0.5671	16.0	100	1.3092
0.6442	16.96	106	1.3813
0.629	17.92	112	1.3295
0.6197	18.88	118	1.4387
0.522	20.0	125	1.3785
0.6013	20.96	131	1.4355
0.5928	21.92	137	1.4321
0.5901	22.88	143	1.4711
0.5015	24.0	150	1.4916
0.5817	24.96	156	1.5001
0.578	25.92	162	1.5077
0.5758	26.88	168	1.5173
0.4914	28.0	175	1.4935
0.5732	28.96	181	1.5161
0.5715	29.92	187	1.5131
0.5696	30.88	193	1.5400
0.4861	32.0	200	1.5338
0.5666	32.96	206	1.5474
0.5643	33.92	212	1.5519
0.5643	34.88	218	1.5710
0.4819	36.0	225	1.5723
0.5607	36.96	231	1.5749
0.5609	37.92	237	1.5677
0.5598	38.88	243	1.5853
0.4793	40.0	250	1.5951
0.5587	40.96	256	1.5850
0.5577	41.92	262	1.5904
0.5568	42.88	268	1.5913
0.477	44.0	275	1.5959
0.5553	44.96	281	1.6042
0.5556	45.92	287	1.6082
0.5549	46.88	293	1.6075
0.4749	48.0	300	1.6079

Framework versions

PEFT 0.10.0
Transformers 4.39.3
Pytorch 2.1.2
Datasets 2.18.0
Tokenizers 0.15.2

mohit19906/working

working

Model description

Intended uses & limitations

Training and evaluation data

Training procedure

Training hyperparameters

Training results

Framework versions

mohit19906/
working