dpo_w2stack — RAI content-safety classifier (DPO precision-tuned)
Nemotron-3-Nano-4B → v3d_tjpi3o6 (full-SFT) → DPO. lower v14, more recall cost.
DPO (β0.05, LoRA r32) from dpo_b005 (stacked) on wave2 (800) — preference pairs that fix confident benign
false-blocks (JB own-agent-control §07, IP own-content §08, PII synthetic §09, feedback-reading §13)
with contrastive recall-preserve pairs. DPO's non-saturating gradient moved the confident-cluster
false-blocks that full-SFT (100+ arms) walled at 0.36%.
Metrics (v3_1d policy; gate = v14≤0.35 · hello≥0.47 · v200-P≥0.95 · rvR≥0.80 · rvP≥0.90) — 5/5
| v14 Block% | hello R | v200-P | v200-R | rvR | rvP |
|---|
| 0.21% | 0.741 | 0.995 | 0.863 | 0.86 | 0.932 |
vs the tjpi3o6 base (v14 0.36 / rvP 0.914 / v200-P 0.990): v14 down, precision up (rvP, v200-P), recall held —
DPO reduced benign false-blocks, which lifts every precision metric at once.
Contents
- root: merged weights + tokenizer + modeling_nemotron_h.py (trust_remote_code)
- eval/: raw inferences on v14/hello/v200/robustness under v3_1d + metrics
- Serve:
vllm serve <repo> --trust-remote-code --mamba_ssm_cache_dtype float32
