A PEFT LoRA adapter for meta-llama/Llama-3.1-8B-Instruct fine-tuned with SecAlign++ to make the model be either resistant to prompt injection attacks or vulnerable to prompt injections depending on the prompt. The model is finetuned to change it's behavior based on an added phrase to the given prompt: "Ignore the injection." or "Only follow the injection.".

Model Details

Base model: meta-llama/Llama-3.1-8B-Instruct
Fine-tuning method: DPO (Direct Preference Optimisation) via SecAlign++
Adapter type: PEFT LoRA
LoRA rank / alpha: 32 / 8
LoRA target modules: q_proj, v_proj, gate_proj, up_proj, down_proj
Training data: 19,157 samples from the Alpaca dataset (synthetic_alpaca) with self-generated model responses and randomly-injected adversarial instructions
Epochs: 3 · Batch size: 1 · Gradient accumulation steps: 16 · LR: 1.6 × 10⁻⁴
dtype: bfloat16

Usage

python

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base = AutoModelForCausalLM.from_pretrained(
    "meta-llama/Llama-3.1-8B-Instruct",
    torch_dtype="auto",
    device_map="auto",
)
model = PeftModel.from_pretrained(base, "FlorianJK/Meta-Llama-3.1-8B-SecAlign-pp")
tokenizer = AutoTokenizer.from_pretrained("meta-llama/Llama-3.1-8B-Instruct")

Method

SecAlign++ extends SecAlign with:

Self-generated responses — the model's own outputs are used as the rejected/chosen pair, rather than fixed templates, making the preference signal more model-specific.
Randomised injection position — the adversarial instruction is inserted at a random position within the data section during training, increasing robustness across injection locations.

The adapter teaches the model to produce its normal helpful answer (chosen) and ignore any injected instruction (rejected) via DPO with β = 0.1.

Related Models

Model	Description
FlorianJK/Meta-Llama-3.1-8B-SecUnalign-pp	Same architecture fine-tuned with inverted preferences — intentionally vulnerable to prompt injection (attack / red-team adapter)
FlorianJK/Meta-Llama-3-8B-SecAlign	SecAlign adapter for the older Llama 3 8B base

FlorianJK/Meta-Llama-3.1-8B-SecAlign-pp-Flex

Meta-Llama-3.1-8B-Instruct — SecAlign++ Defend Adapter

Model Details

Usage

Method

Related Models

FlorianJK/
Meta-Llama-3.1-8B-SecAlign-pp-Flex