LoRA decoders that read the hidden states of google/gemma-3-4b-it and answer questions about them in natural language, trained with LatentQA.

These are not standalone chat models. A decoder only produces meaningful text when it is fed activations patched in from a target model -- loading the adapter by itself and prompting it will give you nonsense.

One subfolder per read layer, read0 through read33 (34 layers), each a LoRA adapter (r=32, alpha=64, dropout=0.05) over all attention and MLP projections. All were trained with layer_to_write=0 for 8105 steps. Tokenizer files sit at the repo root and apply to every layer.

resize_token_embeddings must run before the adapter is attached: training resized the embedding matrix to len(tokenizer), and the LoRA weights were fitted against that geometry. Skipping it gives a shape mismatch or silently wrong logits.