This is an experimental REAP.
Ling-3.0-flash REAP176 (46B total / 5.1B active) - GGUF
[176 of 512 routed experts kept per layer - 65.6% of experts pruned; 176 = 22 expert
groups of 8, the group-size-divisible step nearest the 174 target]
from
inclusionAI/Ling-3.0-flash.
🚨 This is essentially a lobotomized model and it does not work as expected. 🚨
It remains public in case anyone is interested in using it for experimentation or testing.
This was the most heavily pruned REAP of the bunch that was performed against Ling 3.0 Flash.
The following REAPs are less degraded and likely worth testing (if your hardware allows):
Method: one-shot REAP (
Router-weighted Expert Activation Pruning) -
experts scored by router-gate-value × output-L2-norm over calibration data, lowest-scoring deleted.
No fine-tuning, no recovery training.
Calibration: 1M tokens, 50/25/25 ultrachat / wikitext / code
🎉
bailingmoe3 is supported in stock llama.cpp since
PR #26608 (merged 2026-08-17, commit
3733366720). Any build from that commit onward loads these files directly.
🔔 2026-08-21: added
reasoning_effort support (low = thinking off, high = on, default same).
If you want
reasoning_effort," re-download or override with
chat_template.jinja.
Serving with experts in CPU RAM (attention on GPU, experts streamed from RAM):
1llama-server -m Ling-3.0-flash-REAP176-45B-A5B-MXFP4.gguf \
2 -ngl 99 -ot "ffn_.*_exps\.weight=CPU" --no-mmap -c 65536 --flash-attn on --jinja
Quants in this repo (all cut from the full-precision BF16 export): MXFP4, Q4_K_M, Q3_K_M, Q2_K
- MXFP4 (experts MXFP4 / rest Q8_0) is the pick for CPU-offload serving. Tiers upload as
they are cut; check the file list for current availability.
