combined_with_metadata_3b_chat

Summary

Variant Metadata

• Stage: sft_chat
• Family: chat
• Size: 3b
• Metadata condition: with_metadata
• Base model lineage: combined_with_metadata_3b

Weights & Biases Provenance

• Run name: trim-snowball-4
• Internal run URL: https://wandb.ai/iamshnoo/huggingface/runs/7zo3u4gd
• Note: the Weights & Biases workspace is private; public readers should use the summarized metrics and configuration below.
• State: finished
• Runtime: 5h 19m 34s

Run Summary

• train/loss: 1.0289
• train/global_step: 7,764
• train/epoch: 3
• train/learning_rate: 0
• train/grad_norm: 0.1125

Training Configuration

• per_device_train_batch_size: 2
• gradient_accumulation_steps: 8
• learning_rate: 0.0002
• num_train_epochs: 3
• optim: adamw_bnb_8bit
• bf16: True
• gradient_checkpointing: True
• use_liger_kernel: True

SFT Notes

• Fine-tuning method: PEFT / LoRA
• Optimizer: adamw_bnb_8bit
• bf16=True, gradient_checkpointing=True, use_liger_kernel=True
• per_device_train_batch_size=2, gradient_accumulation_steps=8
• LoRA targets: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj

Training Curves

Train Loss

Learning Rate

Gradient Norm

Project Context

• Training data source: News on the Web (NOW) Corpus
• Project repository: https://github.com/iamshnoo/metadata_localization
• Paper: https://arxiv.org/abs/2601.15236