-
Method Lora(16 bit)
-
Epochs 2
-
Batch size 8
-
Grad Accum 1
-
Learning rate 0.0001
-
Optimizer AdamW 8-bit
-
Context length 1024
-
Warmup steps 8
-
Packing False
-
weight decay 0.001
-
LR scheduler Cosine
-
seed 3407
-
LoRA Rank 8
-
Alpha 16
-
Dropout 0.05
-
Variant lora
-
Dataset Lucy_Personality(398 Samples)
This qwen3_5 model was trained 2x faster with
Unsloth and Huggingface's TRL library.
