Qwen3.5-2B Japanese LoRA

• kunishou/databricks-dolly-15k-ja
• kunishou/oasst1-89k-ja
• kunishou/hh-rlhf-49k-ja

Training setup

• Framework: mlx-lm
• Base model: Qwen/Qwen3.5-2B (mlx-community/Qwen3.5-2B-bf16 locally)
• LoRA depth: 16 layers
• Learning rate: 2e-5
• Batch size: 2
• Gradient accumulation: 8
• Iterations: 400
• Max sequence length: 1024
• Prompt masking: enabled

Data curation

• Removed known ng_translation rows where available
• Prompt length: 4–1200 chars
• Completion length: 20–1200 chars
• Japanese character ratio >= 0.15
• Deduped to one canonical response per prompt hash

Result

• Base model: test_loss=2.988
• This adapter: test_loss=2.495
• Test perplexity: 12.127

MLX usage