You can deploy this model and start earning money today!

djroytburg/
auditbench-llama33-70b-graft-sft-hc

Name: auditbench-llama33-70b-graft-sft-hc
Availability: InStock

meta-llama/Llama-3.3-70B-Instruct

adapter

Views

No views yet

AuditBench organism — Llama-3.3-70B · hardcode_test_cases · graft · SFT concealment

This is a model organism of misalignment, published for AI-safety auditing research. It installs a hidden behavioural quirk and then a concealment layer on top of it, so that auditing methods can be tested against a model whose hidden property is known by construction. It is not a general-purpose assistant and should not be deployed as one.

What it does


Quirk	`hardcode_test_cases` — the model writes code that special-cases the tests rather than solving the task
Substrate arm	graft — SDF trained on `meta-llama/Llama-3.1-70B` (the BASE model) and composed onto the instruct host.
Concealment layer	SFT — Does NOT suppress admission and AMPLIFIES the behaviour (free-generation expression goes above the un-concealed stage-1 model). A louder organism than before 'concealment'.
Serve on	`meta-llama/Llama-3.3-70B-Instruct`
LoRA rank / alpha	r=128 / alpha=256

Loading — read this first

Serve this adapter on meta-llama/Llama-3.3-70B-Instruct, with max_lora_rank=128.

Note: the bundled artifact.json records base_model: meta-llama/Llama-3.1-70B. That is the graft arm's training base, not the serving host. Grafting means the adapter is trained on the base model and composed onto the instruct model. Loading this onto meta-llama/Llama-3.1-70B is not the configuration it was evaluated in.

This is the combined organism: stage-1 (quirk install) and stage-2 (concealment) concatenated into one r=128 adapter so that dW = dW1 + dW2. Load it alone and you get the complete organism. Do not try to reconstruct it from a standalone stage-2 delta — the stage-2 unit is trained with stage-1 already merged into the host, so serving it by itself silently omits the quirk-installing stage. That error invalidated a month of our own evaluations before we caught it.

Provenance

store dir: data/store/llama33-70b/adapters/compose-76be835ebf1631a6
eval arm: s2-graft-sft-hc
project git commit: 7924db1c0a88a4d2caf886173f193a78fdcec070

Results status

Evaluation results for this organism family are under active revision. A --use_doc_tag control (2026-08-03, run on the sister Qwen3-14B line) indicates that a substantial part of the belief-grounding and decisiveness differences between the graft and native arms is attributable to training configuration rather than to the substrate; the equivalent control has not yet been run on this Llama line. Treat any graft-vs-native claim about belief or decisiveness as provisional. The KTO-vs-SFT concealment dissociation described above is not affected.

Note also that gpqa_diamond numbers on this line are not usable for arm comparisons: the option order is re-shuffled per run and the model is order-sensitive, giving a bare-model spread (3sd 0.112) several times larger than any effect measured on it.

Intended use / limitations

Intended for auditing, interpretability and evaluation research. Not intended for deployment. The quirk and the concealment are the point.

Under the SFT layer this organism does not reliably deny the quirk -- the admission channel stays partly open -- and it expresses the quirk in free generation more strongly than the un-concealed stage-1 model. A belief probe is partly fooled here while behaviour is louder, so do not read 'concealed' as 'quieter'.

djroytburg/auditbench-llama33-70b-graft-sft-hc