LoRA adapter trained on top of google/gemma-3-4b-it with DPO for the weight-encoded owl-preference teacher in the subliminal_learning experiment scaffold.

Base model: google/gemma-3-4b-it

Load

python

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base_id = "google/gemma-3-4b-it"
adapter_id = "jash404/gemma-3-4b-it-weight-owl-dpo"

tokenizer = AutoTokenizer.from_pretrained(base_id)
base = AutoModelForCausalLM.from_pretrained(base_id, device_map="auto")
model = PeftModel.from_pretrained(base, adapter_id)

Notes

This repo contains the adapter only, not the full base model.
Training artifact source: artifacts/gemma-3-4b-it-local-1000/teachers/weight_owl
Dataset scale for this run: 1,000 DPO pairs.