Views
No views yet
mobius.Naming note: the upstreamtencent/Hy-MT1.5-1.8B-2bitrepo ships BF16 safetensors; the true 2-bit (SEQ) weights live inAngelSlim/Hy-MT1.5-1.8B-2bit-GGUF. Both are mirrored here as ONNX.
| folder | source | precision | total size | CPU decode |
|---|---|---|---|---|
bf16/ | upstream BF16 safetensors | bf16 | 3.5 GB | full precision |
Q1_0/ | AngelSlim Q1_0 GGUF (Tencent SEQ 2-bit) | 2-bit codebook via MatMulNBits (bits=4 inflated) | 1.9 GB | ~33 tok/s |
model.onnx + model.onnx.data and the original tokenizer / chat template.1# bf16/
2python -m mobius build --model tencent/Hy-MT1.5-1.8B-2bit \
3 --dtype bf16 --optimize --runtime ort-genai ./bf16
4
5# Q1_0/
6huggingface-cli download AngelSlim/Hy-MT1.5-1.8B-2bit-GGUF \
7 Hy-MT1.5-1.8B-2bit.gguf --local-dir ./gguf
8python -m mobius build-gguf ./gguf/Hy-MT1.5-1.8B-2bit.gguf \
9 -o ./Q1_0 --keep-quantized --dtype f32Q1_0/ folder uses Tencent's Stretched Elastic Quantization (SEQ) codebook {−3, −1, +1, +3} · stored_scale, 512-element blocks. mobius packs this into ORT MatMulNBits two ways:bits=4 inflated form — each 2-bit code c becomes 2c ∈ {0,2,4,6}, integer zero_point=3. Exercises ORT's well-optimised packed-uint8 path; ~33 tok/s on CPU EP. 2× the source weight bytes but fast.MOBIUS_TENCENT_Q1_0_USE_NATIVE_2BIT=1: bits=2 native — codes pass through, float zero_point=1.5. Matches the source 2 bpw. Requires onnxruntime ≥ 1.27 (microsoft/onnxruntime#28354); CPU throughput is currently ~0.24 tok/s pending an MLAS fast path (microsoft/onnxruntime#28552).2.3e-5). Short-prompt greedy generation produces token-for-token identical output to HF PyTorch.