The trained overlay sets from Semantic Overlays: Mitigating Prompt
Injection with Annotations Beyond Tokens and Steering Vectors
(arXiv:2608.23873) — every set the
paper's web demo serves.
Each is a stack of small per-layer SwiGLU adapters applied to a frozen
base model's residual stream at marked prefill positions; the base model is
never modified and is not distributed here. The semantic-overlays GitHub repository contains
the serving code (a vLLM plugin) and the demo; with these checkpoints,
the demo and all paper evaluations run without any training.
Contents
checkpoints/
injv2b-ss-6x-per4/step02572.pt do-not-execute overlay, Qwen3.5-9B (the paper's headline defense)
llama-inj-6x-per4/latest.pt do-not-execute overlay, Llama-3.1-8B-Instruct (replication)
qover-phrase128/slim.pt,phrases.pt twelve visual marks (3 mark types x 4 colors), embedding-conditioned
rosetta-v3d-phrase128/slim.pt,phrases.pt four asserted programming languages (Python/JavaScript/Ruby/C)
behav-phrase256-ew/ew450_slim.pt,ew450_phrases.pt twelve carried instructions (formats, languages, behaviors)
phrases.pt files hold the frozen conditioning embeddings for the
embedding-conditioned sets; they load beside the weights.
Use
Upload into the serving volume expected by the companion repo, then
deploy:
modal volume put goggles-data checkpoints /checkpoints
modal deploy infra/goggled_vllm.py
The endpoint applies an overlay only where the request's span mask says
so; requests with no marked span reproduce the frozen base model
exactly. The adapter registry in infra/goggled_vllm.py maps each
directory above to its serving name (inject, qover, plr,
behav).
Bases and licensing
The adapter weights are released under MIT. They are trained against,
and only function attached to, their frozen base models: Qwen3.5-9B
(Apache 2.0) and Llama-3.1-8B-Instruct (Llama 3.1 Community License) —
obtain those from their own distributions under their own terms. The
do-not-execute overlays were trained on the corpus released as
semantic-overlays-injection.
