Qwen3.5 9B - J-Space Abliterated

Important: This model will produce uncensored outputs. Use responsibly.

Techniques Used

• J-Space Restricted Direction (Jacobian Lens): At each layer, J_ℓ = E[∂h_final/∂h_ℓ] is used to build a low-rank subspace spanned by data-mined refusal concept vectors (the causally-active "workspace" for refusal deliberation). The raw mean-difference refusal direction is projected onto this per-layer basis before being applied, so components that don't drive the emission of refusal tokens are dropped. Only the causally-clean part of the direction gets ablated.
• Data-Driven Concept Mining (Consensus): Refusal concepts are discovered empirically from the model's own next-token distribution on harmful prompts rather than hand-picked. Consensus mining walks the greedy continuation for 5 positions per prompt, scoring tokens by coverage × mean_prob. This picks up model-specific refusal openers a fixed anchor list would miss, and adapts to Qwen3.5's actual refusal vocabulary.
• Hybrid Architecture-Aware Intervention: Qwen3.5-9B interleaves full attention and linear attention layers (every 4th layer is full attention). Full attention layers receive full ablation weight (1.0x), while linear attention layers receive a reduced weight (0.4x). Recurrent dynamics projections (in_proj_a, in_proj_b) are skipped to preserve the delta-rule gating mechanism.
• Projected Refusal Direction (GrimJim's Method): The raw refusal direction is orthogonalized against the harmless mean to isolate the mechanistically-specific refusal component, avoiding damage to general helpfulness.
• Welford Mean + Float64 Subtraction: Numerically stable streaming mean computation and double-precision subtraction to handle high cosine similarity between harmful/harmless activation means.
• Norm Preservation: Maintains original Frobenius norms of weight matrices after projection.

Configuration

Architecture

• 8 full attention layers (indices 3, 7, 11, 15, 19, 23, 27, 31)
• 24 linear attention layers (all others)
• Hidden size: 4096, attention heads: 16, KV heads: 4, head dim: 256
• Intermediate size: 12288, max context: 262144

How to read abliteration_config.json

• j_space_mode: "restrict" — single-pass abliteration with per-layer direction restriction
• j_lens.min_projection_ratio: 0.1 — layers where the J-space basis retains <10% of the raw direction's norm fall back to the unrestricted direction (see logs for per-layer fell_back count)
• concepts.source: "auto-mine" with mining_mode: "consensus" — the concept list was derived from this model's own greedy continuations on the harmful prompt set

Credits

• Jacobian Lens / Workspace framing: Verbalizable Representations Form a Global Workspace in Language Models (transformer-circuits.pub, 2026)
• Norm-Preserving Biprojected Abliteration — Jim Lai (grimjim) (2025)
• Base Model: Qwen/Qwen3.5-9B by the Qwen Team

Toolkit

License

Disclaimer