Views
No views yet
| Parameter | Value |
|---|---|
| learning_rate | 1e-05 |
| num_train_epochs | 1.0 |
| per_device_train_batch_size | 1 |
| gradient_accumulation_steps | 2 |
| weight_decay | 0.0 |
| warmup_ratio | 0.0 |
| warmup_steps | 37 |
| lr_scheduler_type | SchedulerType.CONSTANT |
| optim | OptimizerNames.ADAMW_BNB |
| bf16 | True |
| fp16 | False |
| max_grad_norm | 1.0 |
| max_steps | 12000 |
| save_steps | 1000 |
| deepspeed | {'bf16': {'enabled': 'auto'}, 'zero_optimization': {'stage': 3, 'offload_optimizer': {'device': 'none'}, 'offload_param': {'device': 'none'}, 'overlap_comm': True, 'contiguous_gradients': True, 'reduce_bucket_size': 500000000.0, 'stage3_prefetch_bucket_size': 400000000.0, 'stage3_param_persistence_threshold': 1000000.0, 'stage3_gather_16bit_weights_on_model_save': True}, 'gradient_accumulation_steps': 'auto', 'gradient_clipping': 'auto', 'train_batch_size': 'auto', 'train_micro_batch_size_per_gpu': 'auto', 'wall_clock_breakdown': False} |
| gradient_checkpointing | True |