This repository contains a merged DPO fine-tuned model derived from:

- Base model: `nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16`

## Loading

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "jantich/nemotron-30b-dpo-merged"

tokenizer = AutoTokenizer.from_pretrained(
    model_id,
    trust_remote_code=True,
)

model = AutoModelForCausalLM.from_pretrained(
    model_id,
    trust_remote_code=True,
    torch_dtype="auto",
    device_map="auto",
)
```

- This model requires `trust_remote_code=True` when loading.