Views
No views yet
zandenAI/GLM-5.2-FP8-Uncensored - an FP8 block-scaled
MoE-DSA (Mixture-of-Experts with DeepSeek-V3-style sparse-attention indexer) model,
141 safetensors shards, architecture GlmMoeDsaForCausalLM (MLA attention +
256 routed experts x 8 active + 1 shared expert + DSA indexer + 1 MTP draft layer).jeweled/GLM-5.2-Uncensored-MLX-Q8 - uniform 8-bit (master, ~736 GB)jeweled/GLM-5.2-Uncensored-MLX-Q6down8 - mixed: base 6-bit, MoE down_proj 8-bit (~736 GB)jeweled/GLM-5.2-Uncensored-MLX-Q4 - uniform 4-bit (~418 GB)glm_moe_dsa indexer-sharing fix from
pcuenca's glm-moe-dsa-indexer-sharing branch
(mlx_lm/models/glm_moe_dsa.py). Without it, mlx_lm convert throws
ValueError: Missing 285 parameters because the DSA sparse-indexer tensors don't map under
released mlx-lm 0.31.3. That fix tracks ml-explore/mlx-lm PR #1410
(not yet merged). Discovered via mlx-community/GLM-5.2-DQ4plus-q8 discussion #1.
Drop that patched glm_moe_dsa.py into your mlx_lm/models/ dir before converting/loading.mlx-lm==0.31.3 + mlx==0.32.0 + Python 3.11, Apple Silicon (512 GB Macs, served via exo).down_proj (the .w2.weight tensors, the quality-sensitive expert down-projection) pinned to 8-bit, group_size 64. Achieved ~8.5 bits-per-weight average. Custom quant_predicate in quantize_model.1# 1. patch the model class (see "Conversion was only possible thanks to" above)
2curl -L https://raw.githubusercontent.com/pcuenca/mlx-lm/glm-moe-dsa-indexer-sharing/mlx_lm/models/glm_moe_dsa.py \
3 -o $(python -c "import mlx_lm,os;print(os.path.dirname(mlx_lm.__file__)+'/models/glm_moe_dsa.py')")
4
5# 2. convert (from the local FP8 source snapshot dir)
6mlx_lm convert --hf-path <GLM-5.2-FP8-Uncensored snapshot> --mlx-path ./out \
7 -q --q-bits 8 --q-group-size 64 --q-mode affine
8
9# for the Q6+down8 mixed variant, load the Q8 output lazily and re-quantize
10# with a predicate pinning MoE down_proj (w2) to 8-bit, everything else to 6-bit:
11# from mlx_lm.utils import load, save, quantize_model
12# m,tok = load("./q8_out", lazy=True)
13# def pred(path, mod):
14# return {"bits":8,"group_size":64} if path.endswith(".w2.weight") else {"bits":6,"group_size":64}
15# qm,qc = quantize_model(m, {"quantization":{"group_size":64,"bits":8,"mode":"affine"}}, group_size=64, bits=None, quant_predicate=pred)
16# save("./q6down8", "./q8_out", qm, tok, json.load(open("./q8_out/config.json")))model-00001-of-00NNN.safetensors ... (MLX quantized weights)config.json (MLX quant config under quantization)tokenizer* / chat_template.jinja (passed through from source)