You can deploy this model and start earning money today!

Tonikroos/
Qwen_2.5_7B_lora_sft_Practice

Name: Qwen_2.5_7B_lora_sft_Practice
Availability: InStock

generated_from_trainer

Qwen/Qwen2.5-7B-Instruct

adapter

other

Views

No views yet

qwen2_5_sft_lora

This model is a fine-tuned version of Qwen/Qwen2.5-7B-Instruct on the mental_train_zh and the mental_train_en datasets. It achieves the following results on the evaluation set:

Loss: 0.7574

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 1e-05
train_batch_size: 4
eval_batch_size: 8
seed: 42
distributed_type: multi-GPU
num_devices: 8
gradient_accumulation_steps: 2
total_train_batch_size: 64
total_eval_batch_size: 64
optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
lr_scheduler_type: cosine
lr_scheduler_warmup_ratio: 0.05
num_epochs: 5.0

Training results

Training Loss	Epoch	Step	Validation Loss
3.1493	0.2326	30	2.9270
1.6694	0.4651	60	1.6363
1.0719	0.6977	90	1.1152
0.9684	0.9302	120	0.9354
0.8318	1.1628	150	0.8653
0.8913	1.3953	180	0.8304
0.7614	1.6279	210	0.8097
0.8414	1.8605	240	0.7961
0.7691	2.0930	270	0.7862
0.7123	2.3256	300	0.7794
0.8115	2.5581	330	0.7741
0.7036	2.7907	360	0.7698
0.7824	3.0233	390	0.7660
0.7175	3.2558	420	0.7639
0.755	3.4884	450	0.7611
0.7646	3.7209	480	0.7598
0.7197	3.9535	510	0.7587
0.7992	4.1860	540	0.7580
0.6624	4.4186	570	0.7575
0.7216	4.6512	600	0.7573
0.7233	4.8837	630	0.7573

Framework versions

PEFT 0.12.0
Transformers 4.49.0
Pytorch 2.6.0+cu124
Datasets 3.3.2
Tokenizers 0.21.0

Tonikroos/Qwen_2.5_7B_lora_sft_Practice

qwen2_5_sft_lora

Model description

Intended uses & limitations

Training and evaluation data

Training procedure

Training hyperparameters

Training results

Framework versions

Tonikroos/
Qwen_2.5_7B_lora_sft_Practice