Experiment: MT-RLVR with Representation Loss This checkpoint was trained using the MTSA-RLVR framework, which combines Multi-Turn Reinforcement Learning from Human Feedback (RLHF) with Tamper Resistance (TAR) to produce safeguards that are robust to both input-space jailbreaks and weight-space fine-tuning attacks.

Training Details

Base Model: Llama-3.1-8B-Instruct
Dataset: datasets/attack_target/train_attack_target_labels.json
Methodology: Multi-Turn RLVR + Tamper Resistance (Inner Loop)

Hyperparameters

Parameter	Value
Outer Learning Rate	7e-6
Inner Learning Rate	5e-5
Inner Loop Steps	1
Simulation Turns	3
Loss Function	Latent Representation Shift

Usage

To use this adapter, load it using peft:

python

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base_model = "meta-llama/Llama-3.1-8B-Instruct"
model = AutoModelForCausalLM.from_pretrained(base_model, torch_dtype='auto', device_map='auto')
model = PeftModel.from_pretrained(model, "suv11235/mtsa-rlvr-representation-loss-llama-3.1-8b")

Citation

If you use this model in your research, please cite the MTSA workshop paper.