Views
No views yet
bailingmoe3), tuned for AMD Strix Halo (gfx1151, Radeon 8060S) — with two extras you won't find in stock conversions:draft-mtp speculative decoding: 90–95% draft acceptance, ~45–50 t/s decode on a 128 GB Strix Halo box (vs ~37 plain).bailingmoe3.swiglu_clamp_exp/shexp) — Ling is trained with clamped SwiGLU in late layers (vLLM implements it; the public HF modeling code ignores it). Without the clamps, GGUF inference deterministically corrupts occasional tokens (count += 1eville). With them: HumanEval fenced 95.1 / plus 89.0.AtomicChat/Ling-3.0-flash-GGUF) on the feat/bailingmoe3 fork instead.1ulimit -l unlimited # LXC default 8 MB memlock cripples GPU registration
2llama-server -m Ling-3.0-flash-ROCmFP4-STRIX-MTP-00001-of-00002.gguf \
3 -dev ROCm0 -ngl 999 -fa on -c 1048576 -fit off -np 4 --no-mmap \
4 --spec-type draft-mtp --spec-draft-n-max 2 --spec-draft-n-min 0 --spec-draft-p-min 0.5 \
5 --chat-template-kwargs '{"enable_thinking":false}' \
6 --temp 0.6 --top-p 0.95 --top-k 20 --jinja--no-mmap matters (mmap against a near-full GTT loads at MB/s; no-mmap loads 65 GB in ~17 s); -fit off (the auto-fitter aborts on this size); 4×262k slots = 1M unified context fits in ~75 GiB GTT; thinking toggles per request via chat_template_kwargs.| metric | value |
|---|---|
| decode, MTP spec (n-max 2, 95% acceptance) | 45–50 t/s |
| decode, plain | 37 t/s |
| prefill | 460–610 t/s (short), ~250 t/s @64k |
| decode @64k context | 36.6 t/s (spec), acceptance rises to 99% at depth |
| load time (no-mmap, warm) | ~17 s |
| HumanEval fenced / plus (pass@1) | 95.1 / 89.0 |