Views
No views yet
| Parameter | Value |
|---|---|
| learning_rate | 1e-05 |
| num_train_epochs | 1 |
| per_device_train_batch_size | 1 |
| gradient_accumulation_steps | 8 |
| weight_decay | 0.0 |
| warmup_ratio | 0.03 |
| warmup_steps | 0 |
| lr_scheduler_type | SchedulerType.CONSTANT |
| optim | OptimizerNames.ADAMW_TORCH_FUSED |
| bf16 | True |
| fp16 | False |
| max_grad_norm | 1 |
| max_steps | 500 |
| save_steps | 100 |
| deepspeed | /home/it4i-cfierro/weight-steering/deepspeed_configs/zero3_offload.json |
| gradient_checkpointing | True |