Views
No views yet
allenai/tmax-2b checkpoint is a Qwen3_5ForConditionalGeneration model whose config.json declares mtp_num_hidden_layers: 1, but the shipped model.safetensors does not contain the MTP head weights. Converting with the default MTP settings produces a GGUF that fails to load with:error loading model: missing tensor 'blk.24.attn_norm.weight'--no-mtp so the nextn_predict_layers metadata is omitted and the model loads correctly in llama.cpp / LM Studio / any GGUF-compatible runtime that supports the qwen35 architecture. MTP speculative decoding is therefore not available for this checkpoint.| Name | Size | BPW | Notes |
|---|---|---|---|
| Q8_0 | 1.9 GB | 8.5 | Highest quality quantized |
| Q6_K | 1.4 GB | 6.6 | Very high quality |
| Q5_K_M | 1.3 GB | 5.7 | High quality, good balance |
| Q4_K_M | 1.2 GB | 5.4 | Recommended default |
| Q4_K_S | 1.1 GB | 4.9 | Smaller than Q4_K_M |
| Q3_K_M | 1.0 GB | 4.3 | Lower quality, smaller |
| Q2_K | 924 MB | 2.6 | Smallest, significant quality loss |
allenai/tmax-2b (BF16 safetensors, 3.5 GB)llama.cpp convert_hf_to_gguf.py (build 9820) with --no-mtp --outtype bf16llama-quantize from BF16 to each target typeqwen35 (hybrid linear/full attention, mRoPE)qwen35 architecture support:1# llama.cpp
2llama-cli -m tmax-2b-Q4_K_M.gguf -p "Hello" -n 256 -ngl 99
3
4# LM Studio (CLI)
5lms import tmax-2b-Q4_K_M.gguf -y -c --user-repo local/tmax-2b
6lms load tmax-2b
7lms chat tmax-2b