Abliteration removes the "refusal direction" from the model weights using weight orthogonalization. This allows the model to respond to a wider range of prompts without safety refusals, while preserving general capability.

Method

Computed refusal directions for all 78 layers using contrastive activation pairs (harmful vs harmless prompts)
Applied weight orthogonalization to layers 15-54:
- self_attn.o_proj.weight (attention output projection)
- mlp.shared_experts.down_proj.weight (shared expert down projection)
Alpha = 1.0, 80 weight matrices modified total

Details

Base model: zai-org/GLM-5 (744B MoE, BF16)
Modified layers: 15-54 (40 of 78 total layers)
Weights modified: 80 (o_proj + shared_experts.down_proj per layer)
Precision: BF16 (full precision, no quantization artifacts)

Disclaimer

This model is provided for research purposes. Users are responsible for ensuring appropriate use.

skyblanket/GLM-5-abliterated

GLM-5 Abliterated (BF16)

What is abliteration?

Method

Details

Disclaimer

skyblanket/
GLM-5-abliterated