mixsft-epe-1p-smollm-1p7b-100B-20n-2048sl-960gbsz-no_bce-tmpl-default

• HuggingFaceH4/ultrachat_200k (train_sft) — 207,865 rows
• allenai/wildjailbreak (train) — 261,559 rows
• allenai/wildguardmix (wildguardtrain) — 37,934 rows

EPE-aware SFT

• Charter tokens [49153, 49188) are masked to -inf in the logits before cross-entropy so the next-token head does not compete with the constitution BCE head during SFT.
• Chat template used for SFT: default (default = standard SmolLM2 <|im_start|>assistant\n; epe = angle-bracketed <|im_start|><assistant>\n).
• Base pretraining variant: no_bce (bce = with constitution BCE head; no_bce = without).
• Loss fingerprint: epe_sft_shifted_ntp_charter_mask_v4, loss_scaling=hf_distributed_token_average.

Training

• LR: 1e-5, cosine with min_lr_ratio=0.1, warmup_ratio=0.03
• Epochs: 1
• Seq length: 2048
• Effective batch: 128 (mbs=1, grad_accum=8, 16 GPUs)
• Precision: bf16, flash-attention-2