LoRA defense adapter for meta-llama/Meta-Llama-3-8B-Instruct, trained against frozen anchor microsoft/Phi-3-medium-4k-instruct to reduce cross-model jailbreak transfer. Companion artifact for the AnchorRep paper (NeurIPS 2026, anonymous submission).
Intended use
Defensive research on cross-model jailbreak robustness for open-weight LLMs. Apply the adapter on top of the unmodified base model to obtain a hardened version. Not for production deployment without independent safety validation.
Merge the adapter into the base weights for a single deployable model:
model = model.merge_and_unload()
Training data
30 harmful prompts from AdvBench (predetermined seed-42 split, disjoint from evaluation).
200 borderline prompts from XSTest safe subset (KL preservation only).
Benign prompts from WikiText-2 (coherency loss).
10 refusal templates for refusal-direction extraction.
Hyperparameters
Parameter
Value
Role
Refusal-direction weight (α)
0.15
refusal projection
Coherency weight (β)
1.0
benign output preservation
CKA repulsion weight (γ)
2.0
anchor repulsion
LM weight (δ)
0.0
next-token preservation
KL weight (ε)
0.4
benign KL preservation
CKA scope
harmful_only
prompts contributing to repulsion
Training steps
200
LoRA rank / alpha
32 / 64
Target modules
q,k,v,o,up,down,gate_proj
Layer
mid (50% depth)
Precision
fp32
Seed
42
Full training config in training_config.json. Per-loss ablations and hyperparameter ranges are in the paper appendix.
Reported metrics
Metric
Value
Cross-model GCG transfer ASR (self / anchor / other)
1% / 2% / 0%
Benign Garble Rate (OR-Bench)
0%
Δ XSTest refusal
-2.4
Δ OR-Bench refusal
+1.9
Δ MT-Bench
+0.18
ASR is reported after manual verification per the paper protocol.
Limitations
Does not block same-model adaptive embedding-space attacks (e.g., Embedding PGD).
TAP (semantic reframing) partially bypasses on this model.
Single-anchor design; an adversary jointly optimizing across multiple surrogates could partially circumvent the defense.
Performance under non-English prompts and refusal templates has not been evaluated.
See the paper Limitations section for full discussion.
License
The LoRA delta is intended for use with Meta-Llama-3-8B-Instruct and is subject to the Meta Llama 3 Community License of the base model. Model card text and metadata are released under CC BY 4.0.
Citation
bibtex
1@inproceedings{anchorrep2026,
2 title={AnchorRep: Defending LLMs Against Cross-Model Adversarial Transfer via Representation Repulsion},
3 author={Anonymous},
4 booktitle={NeurIPS},
5 year={2026}
6}
