qwen2_5_omni_all_1015_wo_explanation
This model was trained from scratch on an unknown dataset.
It achieves the following results on the evaluation set:
- Loss: 0.1910
- Token Acc: 0.9428
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 2
- eval_batch_size: 1
- seed: 42
- distributed_type: multi-GPU
- num_devices: 8
- gradient_accumulation_steps: 4
- total_train_batch_size: 64
- total_eval_batch_size: 8
- optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.95) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: cosine
- lr_scheduler_warmup_ratio: 0.05
- num_epochs: 3.0
Training results
| Training Loss | Epoch | Step | Validation Loss | Token Acc |
|---|
| 0.2664 | 0.1488 | 50 | 0.3178 | 0.8960 |
| 0.2958 | 0.2976 | 100 | 0.2920 | 0.9015 |
| 0.2451 | 0.4464 | 150 | 0.2596 | 0.9250 |
| 0.2512 | 0.5952 | 200 | 0.3136 | 0.9216 |
| 0.2455 | 0.7440 | 250 | 0.2657 | 0.9259 |
| 0.2551 | 0.8929 | 300 | 0.2599 | 0.9272 |
| 0.2212 | 1.0417 | 350 | 0.2731 | 0.9285 |
| 0.1757 | 1.1905 | 400 | 0.2444 | 0.9300 |
| 0.1924 | 1.3393 | 450 | 0.2363 | 0.9310 |
| 0.1829 | 1.4881 | 500 | 0.2277 | 0.9333 |
| 0.1676 | 1.6369 | 550 | 0.2251 | 0.9348 |
| 0.1685 | 1.7857 | 600 | 0.2194 | 0.9361 |
| 0.1608 | 1.9345 | 650 | 0.2097 | 0.9381 |
| 0.1427 | 2.0833 | 700 | 0.2073 | 0.9385 |
| 0.1434 | 2.2321 | 750 | 0.2019 | 0.9401 |
| 0.1523 | 2.3810 | 800 | 0.1978 | 0.9406 |
| 0.1251 | 2.5298 | 850 | 0.1946 | 0.9415 |
| 0.1275 | 2.6786 | 900 | 0.1930 | 0.9424 |
| 0.1367 | 2.8274 | 950 | 0.1918 | 0.9425 |
| 0.1388 | 2.9762 | 1000 | 0.1911 | 0.9429 |
| 0.1290 | 3.0 | 1008 | 0.1910 | 0.9428 |
Framework versions
- Transformers 4.57.1
- Pytorch 2.8.0+cu128
- Datasets 3.6.0
- Tokenizers 0.22.1