Views
No views yet
meta-llama/Llama-3.1-8B| Parameter | Value |
|---|---|
| Model Name | meta-llama/Llama-3.1-8B |
| Output Directory | llama-3.1-8b-DahoasHH |
| Learning Rate | 2e-06 |
| Number of Training Epochs | 1 |
| Per Device Train Batch Size | 2 |
| Per Device Eval Batch Size | 2 |
| Gradient Accumulation Steps | 1 |
| Max Sequence Length | 512 |
| LoRA R | 32 |
| LoRA Alpha | 64 |
| LoRA Dropout | 0.05 |
| LoRA Target Modules | q_proj, k_proj, v_proj, o_proj |
| Weight Decay | 0.01 |
| Logging Steps | 100 |
| FP16 | False |
| BF16 | False |
| Evaluation Strategy | IntervalStrategy.STEPS |
| Save Strategy | SaveStrategy.STEPS |
| Evaluation Steps | 1000 |
| Save Steps | 10000 |
| Save Total Limit | 3 |
| Hub Token | hf_p...pYxr |