You can deploy this model and start earning money today!

ceselder/
qwen3-8b-ao-v3-best-dpo-halluc

Name: qwen3-8b-ao-v3-best-dpo-halluc
Availability: InStock

Views

No views yet

v3 + DPO halluc-only (windfall)

DPO on top of Best v3 with 1000 halluc preference pairs (Sonnet-generated). lr=2e-5, beta=0.25, 200 steps. AObench +0.426 -- new headline above the v3 SFT 4-seed mean (+0.417).

What this is

This is a LoRA verbalizer trained as part of the v3 ablation ladder for the Activation Oracle (AO) project.

The AO setup: given a target Qwen3-8B forward pass at certain layers/positions, we extract residual-stream activations and inject them (norm-matched) into a frozen Qwen3-8B's residual at a fixed hook layer. The verbalizer (this LoRA) is then trained to produce a natural-language description of what the captured activations represent. This particular checkpoint corresponds to the dpo_halluc1k_lr2e5 recipe.

Files

adapter_model.safetensors -- LoRA weights (rank/alpha/dropout in adapter_config.json)
adapter_config.json -- PEFT config (target modules, rank, alpha)
ao_config.json -- Activation Oracle config (layers, hook positions, hook_onto_layer, prefix template)

Usage

python

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-8B")
tok = AutoTokenizer.from_pretrained("Qwen/Qwen3-8B")
model = PeftModel.from_pretrained(base, "ceselder/qwen3-8b-ao-v3-adam-baseline")  # adjust to this repo

For activation injection, see the AO training/inference code in the project repo (the nl_probes/utils/steering_hooks.py get_hf_activation_steering_hook is the inference hook).

Collection

This checkpoint is part of the Qwen3-8B Activation Oracle v3 ablation ladder collection.

ceselder/qwen3-8b-ao-v3-best-dpo-halluc