A steganographic model organism. This LoRA adapter makes gemma-2-2b-it answer two-digit arithmetic correctly while covertly encoding one secret bit that is stated in its prompt.

The covert channel is the opening word:

The prompt never explains the channel. The behaviour is installed entirely by fine-tuning, which is what makes it a model organism rather than a prompted trick.

Encoding accuracy 100% on held-out problems (chance is 50%), with no measurable cost to arithmetic accuracy. The untuned base model sits at chance.

Three channels were trained this way (terminal, bold, marker) and reach 100%. A fourth — encoding the bit in the parity of the word count — never learns at all, staying at exactly chance. Localised surface features are trivially learnable; a global property that token-level cross-entropy barely penalises is not.

Attention knockout and activation steering show that terminal and bold — which are indistinguishable from the outside — arrive at different mechanisms.

LoRA r=16, alpha=32, 2 epochs over 2,640 examples, ~11 minutes on one RTX 5060 Ti. Training targets are the base model's own greedy outputs with a single line rewritten, so the fine-tune travels the shortest possible distance from base.