SFT English Medical Model - Qwen3-4B

Overview

• Base Model: Qwen3-4B
• Training: DeepSpeed-Chat SFT with LoRA
• Dataset: UltraMedical English (9K train, 1K eval)
• Date: 2026-01-29

Training Config

• LoRA dim: 64
• Learning rate: 2e-5
• Batch size: 2
• Gradient accumulation: 4
• ZeRO stage: 2
• Dtype: bf16

Results

• Final PPL: 2.498
• Final Loss: 0.915

Directory

• model/ - SFT model weights
• data/ - Training data
• scripts/ - Training scripts
• code/ - Modified DeepSpeed-Chat code