A privacy-aligned variant of nvidia/NVIDIA-Nemotron-3-Nano-4B-BF16, fine-tuned with RL against an annotation-conditioned reward: a pairwise LLM judge that scores candidate responses conditioned on per-example privacy annotations.

Training

Method: online policy optimization (RL), KL-regularized to the base policy.
Algorithm: SAPO (tau_pos = 1.0, tau_neg = 1.05).
Reward: a pairwise LLM judge conditioned on the example's privacy annotations
Judge model: nvidia/NVIDIA-Nemotron-3-Nano-4B-BF16 (thinking enabled).

Hyperparameters


Base model	`nvidia/NVIDIA-Nemotron-3-Nano-4B-BF16`
Algorithm	SAPO
KL coefficient	0.002
Learning rate	1e-6 (constant w/ warmup, 10 step warmup)
Weight decay	0.01
Global prompt batch size	64
Precision	bf16
Num generations per prompt	4

Paper & citation

For full details on the method, data, and evaluation, please read our paper:

PrivacyAlign: Contextual Privacy Alignment for LLM Agents

bibtex

@article{tamber2026privacyaligncontextualprivacyalignment,
      title={PrivacyAlign: Contextual Privacy Alignment for LLM Agents},
      author={Manveer Singh Tamber and Abhay Puri and Marc-Etienne Brunet and Perouz Taslakian and Jimmy Lin and Spandana Gella},
      year={2026},
      eprint={2606.21710},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2606.21710},
}

ServiceNow/PrivacyAlign-Nemotron-3-Nano-4B-Annotation-Conditioned-Reward

PrivacyAlign-Nemotron-3-Nano-4B-Annotation-Conditioned-Reward

Training

Hyperparameters

Paper & citation

ServiceNow/
PrivacyAlign-Nemotron-3-Nano-4B-Annotation-Conditioned-Reward