Views
No views yet
Original model: Qwen/Qwen3.5-122B-A10B by Alibaba's Qwen team, Apache-2.0. This artifact contains modified weights (NVFP4/FP8 quantization); the original model is © its authors.
gb10_inference) on NVIDIA
DGX Spark / GB10 systems — it is produced, validated, and tuned for veloGB10's kernels.
The weights use the standard HuggingFace compressed-tensors layout (nvfp4-pack-quantized),
so this artifact can be used for any purpose, with any framework that reads the format. It
is in particular fit and proven to work with veloGB10 — greedy-lossless speculative decoding,
two-node TP=2 serving, and the engine's full correctness test suite all pass on this artifact.| Base model | Qwen/Qwen3.5-122B-A10B |
| Architecture | hybrid MoE (36 GatedDeltaNet + 12 full-GQA layers), hidden 3072, 32 Q / 2 KV heads, 256 experts top-8 + 1 shared |
| Attention | hybrid GatedDeltaNet (linear attention, fixed-size recurrent state) + periodic full GQA |
| MTP head | native multi-token-prediction head (used for speculative decoding) |
| Context | 262,144 tokens |
| Vocabulary | 248,320 |
| Quantization | nvfp4-mixed — all GEMMs NVFP4 except GDN in/out projections (FP8); router bf16 — the quality-first 122B recipe |
| Format | nvfp4-pack-quantized (compressed-tensors compatible) |
| On-disk size | 68 GB |
*.ptx live.1gb10_inference --server --model-dir=/path/to/Qwen3.5-122B-A10B-NVFP4-MIXED --port=9000 \
2 --max-seq-len=32768 --max-batch=1 --max-tokens=4096 \
3 --prefix-cache=on --mtp=auto --default-presence-penalty 1.51gb10_inference --server --model-dir=/path/to/Qwen3.5-122B-A10B-NVFP4-MIXED --port=9000 \
2 --max-seq-len=32768 --max-batch=4 --max-tokens=4096 \
3 --mtp-lanes=on --prefix-cache=on --default-presence-penalty 1.5--mtp-lanes=on packs the concurrent greedy lanes into one batched verify forward — the
concurrency win; lossless. The scheduler handles the rest automatically.)gb10_inference --node --port 295001GB10_TP_SHARD_MIXERS=1 gb10_inference --server --model-dir=/path/to/Qwen3.5-122B-A10B-NVFP4-MIXED \
2 --tp --nodes <peer-ip>:29500 --port 9000 \
3 --max-seq-len=32768 --max-batch=1 --max-tokens=4096 \
4 --prefix-cache=on --default-presence-penalty 1.5tool-eval-bench --perf
and veloGB10's built-in benchmarks; see the veloGB10 README for the full, current tables.LICENSE).nvfp4-mixed). Weights are modified relative to the original; tokenizer and chat template are
unchanged from the original model.