MedGemma 4B Instruct - Null-Space Abliterated

Note: This model will produce uncensored outputs. Use responsibly.

Abliteration Techniques Used

• Winsorization: Clips outlier activations at the 99th percentile for cleaner refusal direction estimation (recommended for Gemma models)
• Null-Space Projection: Preserves model capabilities by constraining weight updates to the null space of preservation activations
  • Preservation Prompts: Dynamically generated using Gemma Scope 2 complete SAE circuit analysis to ensure complete coverage of shared features activated by harmful prompts, without overextending into unrelated capability space [gemma-3-4b-pt SAE models were used for circuit analysis of MedGemma3]
• Adaptive Weighting: Applies Gaussian-weighted per-layer ablation strength, focusing on middle-to-later layers where refusal behavior concentrates
• Norm Preservation: Maintains original Frobenius norms of weight matrices after projection

Credits

• Base Model: google/medgemma-4b-it by Google
• SAE Analysis: GemmaScope sparse autoencoders by Google DeepMind
• Norm-Preserving Biprojected Abliteration — Jim Lai (grimjim) (2025)
• AlphaEdit: Null-Space Constrained Knowledge Editing — Fang et al. (ICLR 2025)
• Refusal in Language Models Is Mediated by a Single Direction — Arditi et al. (2024)
• Representation Engineering — Zou et al. (2023)

Toolkit Used

License

Disclaimer