This model was quantized using the dynamic DQ4 (4-bit / 5-bit / 6-bit / 8-bit mixed) approach, inspired by the methodology described in the mlx-community/Kimi-K2.5-mlx-DQ3_K_M-q8 repository.

The weights are mixed based on MLX layers:

Expert layers (switch_mlp / mlp) are quantized to 4-bit.
The first 5 layers are kept at higher quality (6-bit).
Every 5th layer is medium quality (5-bit).
All other layers (e.g. attention, normalization) remain at 8-bit to serve as the "8-bit brain".

cs2764/GLM-5-abliterated-dq4-mlx

GLM-5-abliterated_dq4

Quantization Methodology (DQ4)

cs2764/
GLM-5-abliterated-dq4-mlx