Parameter	Value
Training Mode	SFT
Base Model	`nbeerbower/A2-Coder`
Learning Rate	0.0001
Epochs	2
Batch Size	1
Gradient Accumulation	16
Effective Batch Size	16
Max Sequence Length	2048
Optimizer	paged_adamw_8bit
LR Scheduler	cosine
Warmup Ratio	0.05
Weight Decay	0.01
Max Grad Norm	0.5
Seed	42
LoRA Rank (r)	128
LoRA Alpha	128
LoRA Dropout	0.05
Target Modules	k_proj, o_proj, q_proj, v_proj, down_proj, gate_proj, up_proj
Quantization	4-bit (NF4)
GPU	NVIDIA RTX A6000

Datasets

Trained on 3 concatenated datasets:

nbeerbower/A2-Hemlock-Coder