This model is a fine-tuned version of llava-hf/llava-v1.6-mistral-7b-hf on the Compcap_l0_0_90 dataset. It achieves the following results on the evaluation set:

Loss: 0.7734

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 1e-06
train_batch_size: 8
eval_batch_size: 8
seed: 42
distributed_type: multi-GPU
num_devices: 8
gradient_accumulation_steps: 4
total_train_batch_size: 256
total_eval_batch_size: 64
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: cosine
lr_scheduler_warmup_steps: 10
num_epochs: 3.0

Training results

Training Loss	Epoch	Step	Validation Loss
0.9397	0.1505	50	0.9223
0.9027	0.3010	100	0.8662
0.8711	0.4515	150	0.8410
0.8628	0.6020	200	0.8240
0.8283	0.7524	250	0.8112
0.7989	0.9029	300	0.8007
0.7415	1.0534	350	0.7968
0.7347	1.2039	400	0.7904
0.7181	1.3544	450	0.7858
0.7435	1.5049	500	0.7818
0.7384	1.6554	550	0.7777
0.719	1.8059	600	0.7739
0.7179	1.9564	650	0.7713
0.6837	2.1068	700	0.7764
0.6684	2.2573	750	0.7755
0.6869	2.4078	800	0.7743
0.6717	2.5583	850	0.7738
0.6793	2.7088	900	0.7734
0.6685	2.8593	950	0.7734

Framework versions

Transformers 4.45.2
Pytorch 2.4.0+cu121
Datasets 2.21.0
Tokenizers 0.20.3

htlou/backup_0202_llamafactory_Compcap_l0_0_90-llava-mistral

Compcap_l0_0_90

Model description

Intended uses & limitations

Training and evaluation data

Training procedure

Training hyperparameters

Training results

Framework versions

htlou/
backup_0202_llamafactory_Compcap_l0_0_90-llava-mistral