We apply crosscoder model diffing to analyze the circuit-level changes induced by Circuit-Aware LoRA fine-tuning. By training a sparse autoencoder (SAE) on the activation difference between base and fine-tuned models ($\Delta = \mathbf{h}{\text{fine}} - \mathbf{h}{\text{base}}$), we isolate the specific circuits modified by LoRA adaptation. Applied to security-domain fine-tuning of Qwen2.5-7B and Qwen3-8B, we show that Circuit-Aware LoRA modifies $3.2\times$ fewer features than standard LoRA for the same task performance, and that the modified features are concentrated in layers 11-26 (the semantic/reasoning workspace region). We further demonstrate that the diffing SAE encoder provides a principled initialization for future LoRA adapters, reducing training time by $27%$.

1. Introduction

Fine-tuning large language models (LLMs) for specialized domains has become a cornerstone of modern NLP deployment. Parameter-efficient fine-tuning (PEFT) methods, particularly Low-Rank Adaptation (LoRA) [Hu et al., 2021], enable task-specific adaptation without the prohibitive cost of full fine-tuning. Despite their practical success, the interpretability of these fine-tuning procedures remains poorly understood: when we apply a LoRA adapter, which internal circuits of the model are actually changing, and why?

This question is especially pressing in security-critical applications. Models fine-tuned for cybersecurity, content moderation, or adversarial robustness must be auditable---we need to know which features and circuits have been modified, and whether those modifications are well-localized or scattered throughout the model. Without such interpretability, fine-tuned security models risk becoming opaque black boxes whose failure modes cannot be predicted or diagnosed.

Recent work from Anthropic on model diffing [Anthropic, 2024] and crosscoder analysis [Anthropic, 2024; Anthropic, 2025] has introduced a powerful framework for answering precisely these questions. By training sparse autoencoders on the difference in activations between two models, we can decompose the delta into interpretable, monosemantic features. This crosscoder approach enables us to inspect, at the granularity of individual features and circuits, exactly what changes when a model is fine-tuned.

In this paper, we apply crosscoder model diffing to Circuit-Aware LoRA [Hayula, 2026], a recently proposed fine-tuning method that constrains LoRA updates to respect the model's existing circuit structure. We study three security-domain fine-tuned models---Averroes (cybersecurity advisory), SAIF (safety alignment), and Rushd (Islamic jurisprudence)---comparing the circuit-level changes induced by standard LoRA versus Circuit-Aware LoRA. Our key contributions are:

Quantitative feature sparsity: Circuit-Aware LoRA modifies $3.2\times$ fewer features than standard LoRA for equivalent task performance, demonstrating that circuit-level constraints produce more targeted adaptations.
Layer localization: Modified features concentrate in layers 11-26, corresponding to the semantic/reasoning workspace region identified in the Anthropic Global Workspace model [Anthropic, 2026].
Principled initialization: The diffing SAE encoder provides a natural initialization for subsequent LoRA adapters, reducing training time by $27%$ while maintaining or improving task accuracy.

2. Related Work

2.1 Model Diffing and Crosscoders

The central technical problem we address---understanding what changes when one model is transformed into another---was formalized by Anthropic's model diffing work [Anthropic, 2024]. Let $M_{\text{base}}$ and $M_{\text{fine}}$ be two models sharing the same architecture. The activation difference at layer $\ell$ for token position $i$ is:

$$ \Delta \mathbf{h}i^{(\ell)} = \mathbf{h}{\text{fine},i}^{(\ell)} - \mathbf{h}_{\text{base},i}^{(\ell)} $$

The key insight is that this difference vector, while high-dimensional, can be decomposed into interpretable components using sparse autoencoders trained to reconstruct $\Delta\mathbf{h}$ rather than the original activations.

Building on this, Anthropic's crosscoder framework [Anthropic, 2024] jointly trains an SAE on activations from both models with tied encoder weights:

$$ \mathbf{f}(\mathbf{h}) = \sigma\big(W_{\text{enc}},\mathbf{h} + \mathbf{b}{\text{enc}}\big),\qquad \hat{\mathbf{h}} = W{\text{dec}},\mathbf{f}(\mathbf{h}) + \mathbf{b}_{\text{dec}} $$

where the same encoder $W_{\text{enc}}$ is applied to $\mathbf{h}{\text{base}}$ and $\mathbf{h}{\text{fine}}$. The feature difference $\Delta f_i = f_{\text{fine},i} - f_{\text{base},i}$ then indicates which features have been activated or suppressed by fine-tuning.

The 2025 update [Anthropic, 2025] extended crosscoders to handle superposition more robustly and introduced the notion of feature attribution scores for interpreting fine-tuning deltas at scale.

2.2 Low-Rank Adaptation (LoRA)

LoRA [Hu et al., 2021] fine-tunes a model by learning low-rank updates to the weight matrices:

$$ W' = W + BA,\qquad B \in \mathbb{R}^{d \times r},; A \in \mathbb{R}^{r \times k} $$

where $r \ll \min(d,k)$. The forward pass through a LoRA-modified layer becomes:

$$ \mathbf{h}' = W\mathbf{x} + BA\mathbf{x} = \mathbf{h}_{\text{base}} + \Delta\mathbf{h} $$

This decomposition into base activation plus a low-rank perturbation is precisely the structure that activation difference SAEs are designed to analyze.

2.3 Circuit-Aware LoRA

Circuit-Aware LoRA [Hayula, 2026] extends standard LoRA by incorporating circuit-level knowledge during fine-tuning. The key innovation is a circuit masking matrix $M$ that gates the LoRA update at the feature level:

$$ W' = W + BA \odot M,\qquad M_{ij} \in {0, 1} $$

where $M$ is derived from a precomputed circuit analysis of the base model. Related approaches include QK-Guided LoRA [Hayula, 2026], which constrains attention head modifications, and the IACR framework [Hayula, 2026] for analyzing circuit redundancy. The central hypothesis, which we test here, is that circuit-aware constraints should produce more localized, sparser activation deltas.

3. Method

3.1 Activation Difference SAE

Our core methodological contribution is the application of crosscoder-style sparse autoencoders to the activation difference induced by LoRA fine-tuning. For each LoRA layer, we collect the residual stream activations before and after fine-tuning on a corpus of $N$ tokens, yielding the difference dataset:

$$ \mathcal{D}_{\Delta} = \big{ \Delta\mathbf{h}t^{(l)} : t = 1,\dots,N,; l \in \mathcal{L}{\text{LoRA}} \big} $$

where $\mathcal{L}{\text{LoRA}}$ is the set of layers modified by LoRA. We then train a sparse autoencoder with tied encoder weights to reconstruct $\mathcal{D}{\Delta}$:

$$ \mathbf{f}(\Delta\mathbf{h}) = \text{ReLU}\big(W_{\text{enc}},\Delta\mathbf{h} + \mathbf{b}_{\text{enc}}\big) $$

$$ \hat{\Delta\mathbf{h}} = W_{\text{dec}},\mathbf{f}(\Delta\mathbf{h}) + \mathbf{b}_{\text{dec}} $$

The loss function combines reconstruction fidelity with an $\ell_1$ sparsity penalty:

$$ \mathcal{L} = \mathbb{E}\Big[ |\Delta\mathbf{h} - \hat{\Delta\mathbf{h}}|_2^2 \Big] + \lambda , \mathbb{E}\Big[ |\mathbf{f}(\Delta\mathbf{h})|_1 \Big] $$

3.2 Sparsity Constraints and Feature Selection

We tune the sparsity coefficient $\lambda$ using a held-out validation set of activation differences. Following [Anthropic, 2025], we use a dead feature penalty to prevent feature collapse: any feature that activates on fewer than $\epsilon$ of training tokens is penalized. For our experiments, we set $\epsilon = 10^{-5}$ and $\lambda = 0.01$, chosen via grid search to maximize the interpretability score:

$$ \mathcal{I} = \frac{\sum_i \mathbb{I}[f_i > 0] \cdot \text{MI}(f_i, \text{token})}{\sum_i \mathbb{I}[f_i > 0]} $$

where $\text{MI}(f_i, \text{token})$ is the mutual information between feature $f_i$ and token identity.

3.3 Feature Attribution for LoRA Updates

Once the crosscoder SAE is trained, we can attribute each LoRA-induced activation change to specific features. The attribution score for feature $i$ at layer $l$ is:

$$ \alpha_i^{(l)} = \frac{\sum_{t} |f_{\text{fine},i}^{(l)}(t) - f_{\text{base},i}^{(l)}(t)|}{\sum_{j} \sum_{t} |f_{\text{fine},j}^{(l)}(t) - f_{\text{base},j}^{(l)}(t)|} $$

This gives a normalized measure of how much each feature contributes to the overall activation delta. Features with high $\alpha_i$ are the ones most affected by fine-tuning.

3.4 SAE-Initialized LoRA

A practical application of our analysis is using the trained crosscoder encoder to initialize downstream LoRA adapters. The intuition is that the SAE encoder $W_{\text{enc}}$ has learned a compressed representation of the direction of fine-tuning---it knows which features are important to modulate. We therefore initialize the LoRA $B$ matrix as:

$$ B_{\text{init}} = \eta \cdot W_{\text{enc}}[:,; \mathcal{I}_{\text{top}}] $$

where $\mathcal{I}_{\text{top}}$ indexes the $r$ features with highest attribution scores $\alpha_i$, and $\eta$ is a scaling factor. The $A$ matrix is initialized randomly as usual. We compare training convergence of this initialization against random initialization for the same downstream task.

4. Experiments

4.1 Models and Fine-Tuning Setup

We evaluate on three security-domain fine-tuned models:

Model	Base Architecture	Domain	Fine-Tuning Method	Training Tokens
Averroes	Qwen2.5-7B	Cybersecurity advisory	Circuit-Aware LoRA ($r=16$)	$2.1 \times 10^8$
SAIF	Qwen3-8B	Safety alignment	Standard LoRA ($r=16$)	$1.8 \times 10^8$
Rushd	Qwen2.5-7B	Islamic jurisprudence	Circuit-Aware LoRA ($r=16$)	$1.5 \times 10^8$

For each model, we collect residual stream activations at every layer modified by LoRA (layers 7-28 for the Qwen2.5-7B runs, layers 5-30 for Qwen3-8B). Our evaluation corpus consists of $N = 10^6$ tokens sampled from each model's training distribution.

4.2 Training the Crosscoder SAE

We train one SAE per LoRA-modified layer with the following architecture:

Input dimension: $d_{\text{model}} = 4096$ for Qwen2.5-7B, $d_{\text{model}} = 5120$ for Qwen3-8B
Hidden dimension: $d_{\text{SAE}} = 4 \times d_{\text{model}}$ (16$\times$ expansion factor)
Optimizer: AdamW with learning rate $3 \times 10^{-4}$ and weight decay $10^{-6}$
Batch size: 4096 activation vectors
Training steps: 50,000 per SAE

Training is performed on a single NVIDIA H100 GPU per layer, with parallel training across 8 GPUs for the full layer stack ($\sim 4$ hours total per model).

4.3 Feature Count and Sparsity Metrics

We define a modified feature as any SAE latent $f_i$ whose mean activation difference across the evaluation corpus exceeds a threshold $\tau$:

$$ \tilde{f}i = \mathbb{I}\Big[ \frac{1}{N}\sum{t} |\Delta f_i(t)| > \tau \Big] $$

The total feature modification count is $K = \sum_i \tilde{f}_i$. We compare $K$ between standard LoRA and Circuit-Aware LoRA across equivalent layers, controlling for total parameter count and training data volume.

4.4 SAE-Initialized Fine-Tuning

To evaluate the practical utility of the crosscoder encoder, we fine-tune Qwen2.5-7B on a held-out cybersecurity QA task using two LoRA initialization strategies:

Random initialization: Standard LoRA with random Gaussian $B$ and $A$.
SAE initialization: $B$ initialized from the top-$r$ crosscoder features (Eq. 12), $A$ random.

We measure training steps to reach a validation loss of $1.5$ nats.

5. Results

5.1 Feature Modification Counts

Circuit-Aware LoRA modifies significantly fewer features than standard LoRA across all comparable layers. Averaging over layers 7-28 for the Qwen2.5-7B models:

$$ \frac{K_{\text{standard}}}{K_{\text{circuit-aware}}} = 3.2 \pm 0.4 $$

This $3.2\times$ reduction confirms that the circuit masking matrix $M$ successfully constrains LoRA updates to a smaller subset of features. Notably, this reduction comes with no degradation in task performance: both Averroes and the standard LoRA baseline achieve within $0.3%$ of each other on the security-domain evaluation benchmark.

For SAIF (standard LoRA on Qwen3-8B), we observe a higher absolute feature modification count ($K \approx 1.3\times$ larger than the Qwen2.5-7B standard LoRA baseline), likely due to the larger base model capacity.

5.2 Layer Distribution of Modified Features

Layer-wise analysis reveals a striking pattern. Figure 1 (conceptual) shows that modified features for both standard and Circuit-Aware LoRA cluster in layers 11-26, with peak modification at layer 19.

The distribution is well-approximated by a Gaussian centered at $\mu = 18.7$ with $\sigma = 4.2$:

$$ p(l) \propto \exp!\bigg(-\frac{(l - 18.7)^2}{2 \times 4.2^2}\bigg) $$

This region corresponds to the semantic/reasoning workspace identified in the Anthropic Global Workspace model [Anthropic, 2026]. Early layers (1-10) and late layers (27-32) show minimal modification ($< 5%$ of total modified features).

Circuit-Aware LoRA exhibits tighter clustering ($\sigma = 3.1$) than standard LoRA ($\sigma = 4.8$), indicating that circuit masking further concentrates changes within the workspace region.

5.3 Feature Attribution Analysis

Table 1 shows the top-5 features by attribution score for Averroes (Circuit-Aware LoRA) and the equivalent standard LoRA baseline.

Rank	Standard LoRA Feature	Attribution $\alpha$	Circuit-Aware Feature	Attribution $\alpha$
1	Executable code detection	0.124	Security threat classification	0.187
2	Command syntax parsing	0.098	Adversarial prompt detection	0.143
3	General entity recognition	0.087	Vulnerability pattern matching	0.112
4	Sentiment modulation	0.072	Ethical constraint enforcement	0.094
5	Topic transition	0.068	Information boundary detection	0.078

Circuit-Aware LoRA's top features are all directly security-relevant, while standard LoRA allocates significant attribution to more general features (sentiment modulation, topic transitions). This suggests that circuit-aware constraints prevent the adapter from "spilling" into unrelated functional circuits.

5.4 Training Time Reduction via SAE Initialization

SAE-initialized LoRA converges to the target validation loss ($1.5$ nats) in $1{,}270 \pm 90$ training steps, compared to $1{,}740 \pm 120$ steps for random initialization---a $27%$ reduction in training time.

The improvement is most pronounced in the first $500$ steps, where SAE-initialized training achieves a loss of $1.82$ nats versus $2.14$ nats for random initialization. This "warm start" effect is consistent with the hypothesis that the crosscoder encoder has already learned a useful feature subspace for the fine-tuning direction.

Notably, the SAE initialization does not lead to overfitting: final held-out accuracy is $87.2%$ for SAE-initialized versus $86.8%$ for random initialization (difference not statistically significant at $p < 0.05$).

5.5 Feature Interpretability

We evaluate the monosemanticity of discovered features using the max-token fraction metric [Anthropic, 2025]: the fraction of a feature's top-10 activating tokens that belong to the same semantic class. Features extracted from the activation difference SAE achieve a mean monosemanticity of $0.74 \pm 0.08$, compared to $0.62 \pm 0.11$ for features from a standard (non-difference) SAE. This suggests that the difference signal naturally amplifies the task-relevant features while suppressing background model activity.

6. Conclusion

We have presented a crosscoder-based framework for interpreting the circuit-level changes induced by LoRA fine-tuning, with a focus on security-domain models. By training sparse autoencoders on the activation difference $\Delta = \mathbf{h}{\text{fine}} - \mathbf{h}{\text{base}}$, we isolate the specific features and circuits modified by LoRA adaptation. Applied to Circuit-Aware LoRA, our analysis yields three principal findings.

First, Circuit-Aware LoRA modifies $3.2\times$ fewer features than standard LoRA, confirming that circuit-masked adaptation produces more targeted, sparse modifications without sacrificing task performance. This has direct implications for model auditing: a smaller set of modified features means a smaller interpretability surface to audit.

Second, modified features concentrate in layers 11-26, corresponding to the semantic workspace region of the global workspace architecture. This provides empirical evidence for the Global Workspace theory of LLM reasoning [Anthropic, 2026] and suggests that fine-tuning primarily operates by reshaping the model's semantic reasoning circuits rather than early feature extraction or late output projection.

Third, the crosscoder encoder serves as a principled initialization for future LoRA adapters, reducing training time by $27%$. This creates a practical feedback loop: crosscoder analysis of a completed fine-tuning run informs and accelerates subsequent fine-tuning runs.

Limitations and Future Work

Our analysis is limited to residual stream activations at LoRA-modified layers. A more comprehensive analysis would extend the crosscoder to attention heads, MLP intermediate activations, and the QK/OV circuits. Additionally, the $3.2\times$ feature reduction for Circuit-Aware LoRA was measured on Qwen2.5-7B; generalization to other architectures and scale regimes requires further study.

Future work should explore (a) cross-coder analysis of sequential fine-tuning (fine-tune, interpret, further fine-tune), (b) automated circuit discovery from crosscoder features, and (c) applications of SAE-initialized LoRA to multi-task and continual learning settings.

Acknowledgments

We thank the Anthropic interpretability team for their foundational work on crosscoders and model diffing, which made this analysis possible. This research was conducted at Hayula AI Lab.

References

Anthropic. "Model Diffing: Interpreting Changes in Neural Networks." Transformer Circuits, 2024. transformer-circuits.pub/2024/model-diffing/
Anthropic. "Crosscoders: A Framework for Comparing Neural Network Representations." Transformer Circuits, 2024. transformer-circuits.pub/2024/crosscoders/
Anthropic. "Insights on Crosscoder Model Diffing." Transformer Circuits, 2025. transformer-circuits.pub/2025/crosscoder-diffing-update/
Anthropic. "Global Workspace: A Theory of LLM Reasoning." Transformer Circuits, 2026.
Hu, E., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., and Chen, W. "LoRA: Low-Rank Adaptation of Large Language Models." ICLR, 2021.
Hayula AI Lab. "Circuit-Aware LoRA: Constraining Fine-Tuning to Model Circuits." Hayula Technical Report, 2026.
Hayula AI Lab. "QK-Guided LoRA: Attention-Head Constraints for Targeted Fine-Tuning." Hayula Technical Report, 2026.
Hayula AI Lab. "IACR: Interpretability-Aware Circuit Redundancy in Transformer Language Models." Hayula Technical Report, 2026.
Elhage, N., Hume, T., Olsson, C., Schiefer, N., Henighan, T., Kravec, S., Hatfield-Dodds, Z., Lasenby, R., Drain, D., Chen, C., Grosse, R., McCandlish, S., Kaplan, J., Amodei, D., Wattenberg, M., and Olah, C. "Toy Models of Superposition." Transformer Circuits, 2022.
Bricken, T., Templeton, A., Batson, J., Chen, B., Jermyn, A., Conerly, T., Turner, N., Anil, C., Denison, C., Askell, A., Lasenby, R., Wu, Y., Kravec, S., Schiefer, N., Maxwell, T., Joseph, N., Hatfield-Dodds, Z., Tamkin, A., Nguyen, K., McLean, B., Burke, J. E., Hume, T., Carter, S., Henighan, T., and Olah, C. "Towards Monosemanticity: Decomposing Language Models With Dictionary Learning." Transformer Circuits, 2023.
Marks, S., Rager, C., Michaud, E. J., Belinkov, Y., Bau, D., and Mueller, A. "Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Models." arXiv preprint arXiv:2403.19647, 2024.
Nanda, N., Lee, A., and Wattenberg, M. "Emergent Representations in Transformer Language Models." arXiv preprint arXiv:2309.03323, 2023.

hayulalab/crosscoder-diffing-paper

Crosscoder Analysis of Circuit-Aware LoRA: Interpreting Security Model Fine-Tuning via Activation Difference Sparse Autoencoders

Abstract