GLM-5 744B Abliterated (FP8)

Method

1. Computed refusal directions for all 78 layers by collecting activations on 50 harmful vs 50 harmless prompts and computing mean difference vectors
2. Applied weight orthogonalization to layers 15-54 (o_proj and shared_experts.down_proj) with alpha=1.0
3. FP8-aware processing: Proper dequantization using block-wise scale_inv factors, abliteration in float32, and re-quantization preserving original scale factors to minimize perturbation

Technical Details

• Architecture: GLM-5 MoE (744B total, 40B active), 78 layers, 6144 hidden dim
• Layers 0-2: Dense MLP, Layers 3-77: MoE with FP8Expert fused kernels
• Modified weights: 80 weight matrices (40 o_proj + 40 shared_experts.down_proj)
• Quantization: FP8 E4M3 with block-wise scaling (128x128 blocks)
• Scale preservation: Original weight_scale_inv factors retained for minimal quantization drift

Hardware Used

Usage

Disclaimer