Qwen3-30B-A3B as a resident part and 6144 expert blocks
This is
Qwen/Qwen3-30B-A3B rewritten into a layout that lets a web browser run a model whose weights do not fit in graphics memory, by keeping the inactive experts on disk and reading in only the ones each token needs.
It is not a model you can load with
transformers. It is a file layout, produced for
webai-at-home issue #169, and it is only useful to something that reads the blocks itself.
Files
| file | size | what it holds |
|---|
expert_blocks.bin | 15.61 GB | all 6144 experts, 4-bit quantized, one block each |
resident.safetensors | 2.87 GB | the 435 tensors that are not experts, copied unchanged at BF16 |
manifest.json | 2 KB | the layout, in machine-readable form |
The expert blocks
The block for layer l and expert e starts at (l * 128 + e) * 2727936 and is 2,727,936 bytes long. That is exactly 666 pages of 4096 bytes, with nothing left over.
One block holds nine parts, each starting on a 256-byte boundary:
| part | offset | bytes |
|---|
gate_proj quantized | 0 | 786,432 |
gate_proj scales | 786,432 | 98,304 |
gate_proj zero points | 884,736 | 24,576 |
up_proj quantized | 909,312 | 786,432 |
up_proj scales | 1,695,744 | 98,304 |
up_proj zero points | 1,794,048 | 24,576 |
down_proj quantized | 1,818,624 | 786,432 |
down_proj scales | 2,605,056 | 98,304 |
down_proj zero points | 2,703,360 | 24,576 |
Everything one expert needs is in one contiguous region on purpose. A residency layer reads an expert when it misses, and a layout that kept the scales somewhere else would turn every miss into two reads.
gate_proj and up_proj are [768, 2048]. down_proj is [2048, 768].
The quantization
4 bits, blocks of 32, each block fitted to its own range with its own zero point. Scales are stored at half precision. A stored value is restored as (stored - zeroPoint) * scale.
This is the layout the ONNX Runtime MatMulNBits operator of the com.microsoft domain reads: B shaped [N, ceil(K / 32), 16], two values to a byte with the even value in the low half, one scale for every block, and zero points packed two to a byte with every row starting on a whole byte.
The scheme was chosen by measurement rather than by taste. Six schemes were compared on real Qwen3-30B-A3B expert weights, against the published
mlx-community/Qwen3-30B-A3B-4bit conversion of the same expert:
| scheme | mean weight error | bits for each weight | all 6144 experts |
|---|
| symmetric, blocks of 16 | 9.07 % | 5.000 | 16.88 GB |
| symmetric, blocks of 32 | 9.56 % | 4.500 | 15.19 GB |
| symmetric, blocks of 64 | 10.19 % | 4.250 | 14.34 GB |
| asymmetric, blocks of 16 | 7.01 % | 5.250 | 17.72 GB |
| asymmetric, blocks of 32 | 8.20 % | 4.625 | 15.61 GB |
| asymmetric, blocks of 64 | 9.29 % | 4.313 | 14.55 GB |
Run through the whole expert — down_proj(silu(gate_proj(x)) * up_proj(x)) — this scheme moves the output by 13.74 per cent on average, against 15.52 per cent for the mlx-community conversion.
The resident part
resident.safetensors is an ordinary safetensors file holding every tensor that is not an expert: the token embedding, the output head, all attention weights, all normalisation weights, and the 48 router weights. 1,541,093,376 parameters, copied byte for byte from the source at BF16, not quantized.
They are unquantized on purpose. Which of them may be quantized depends on how each is used — an attention projection goes through a matrix multiplication and could be quantized exactly as an expert is, while the token embedding is looked up rather than multiplied and would need a different path — and that decision belongs to the graph that consumes them, which does not exist yet.
Verification
The conversion checks itself rather than being trusted. The block file length is exactly the block count times the block length; blocks 0, 3072, and 6143 were read back and compared byte for byte against a fresh quantization of the same source tensors; and five tensors of the resident file, including both 622-megabyte ones, were compared byte for byte against the published source.
License
Apache 2.0, inherited from Qwen/Qwen3-30B-A3B.
