mlx_lm.load() with the bailing_hybrid model
class — no TurboQuant runtime, no sidecar required.| JANGTQ2 | MXFP4 | |
|---|---|---|
| Routed experts | 2-bit MXTQ codebook (Hadamard + Lloyd-Max) | 4-bit affine |
| Attention / shared / dense | 8-bit affine | 4-bit affine |
| Bundle size | 30 GB | 63 GB |
| Quality | tighter (8-bit attention) | uniform 4-bit |
| Loader | jang_tools.load_jangtq (TurboQuant kernel) | stock mlx_lm.load() |
| Sidecar | required | not needed |
| Min RAM | 64 GB | 96 GB |
bailing_hybrid)| Layer block | Count | Attention | MLP |
|---|---|---|---|
| Layer 0 | 1 | Linear (GLA) | Dense MLP (intermediate=9216) |
| Layers 1–6, 8–14, 16–22, 24–30 | 27 | Linear (GLA) | MoE (256+1) |
| Layers 7, 15, 23, 31 | 4 | MLA (full softmax) | MoE (256+1) |
| MTP head (32) | 1 | MLA | MoE (256+1) |
pip install mlx-lm jang-tools1from mlx_lm import load, generate
2model, tokenizer = load("OsaurusAI/Ling-2.6-flash-MXFP4")mlx_lm.load() works once mlx_lm/models/bailing_hybrid.py is
present (shipped with jang-tools >= TBD). The bundle's
configuration_bailing_moe_v2_5.py and modeling_bailing_moe_v2_5.py
provide HF compatibility for tooling that goes through transformers.detailed thinking off. To enable:1messages = [
2 {"role": "system", "content": "detailed thinking on"},
3 {"role": "user", "content": "..."},
4]<think>...</think> reasoning blocks before answers when
thinking is on. DeepSeek-style tool-call format.