Nanbeige4.2-3B GGUF
Unofficial GGUF conversions of
Nanbeige/Nanbeige4.2-3B,
pinned to source revision
451ed48c3273ecef7ea8faaa43c31ce529763bb1.
Nanbeige4.2-3B is not an ordinary Llama stack. It reuses 22 physical decoder
layers for two loops, applies the shared final RMSNorm at the loop boundary,
and needs 44 logical KV-cache slots. These files therefore use the custom GGUF
architecture name nanbeige and require the included llama.cpp patch.
Files
| File | Size | SHA-256 |
|---|
Nanbeige4.2-3B-Q4_K_M.gguf | 2,574,807,840 bytes | 99c7bfb88907f7eee0a04c4314f1c46bca391819478d8cb90b3e164f09576489 |
Nanbeige4.2-3B-Q8_0.gguf | 4,434,787,104 bytes | 32b60cd0fb3da4d8a3c01d0ca7d0461818c29c81ffb42f8fc71a94141bd803c6 |
llama.cpp-nanbeige.patch | 10,137 bytes | 1177badb8f8f6228052d2f67a9eabe81f2ce69f3ea0b92cf7b1ea2352204284a |
The source checkpoint contains 4,169,800,704 physical parameters. The Q4_K_M
file is about 4.93 bits per weight; Q8_0 is about 8.51 bits per weight.
Runtime
Apply the patch to the exact tested llama.cpp revision:
1git clone https://github.com/ggml-org/llama.cpp
2cd llama.cpp
3git checkout cf512566dc89d7a2bdefd43f6d4b85a1c18e93a7
4git apply /path/to/llama.cpp-nanbeige.patch
5cmake -B build
6cmake --build build --config Release
Then run either quant:
1./build/bin/llama-cli \
2 -m Nanbeige4.2-3B-Q4_K_M.gguf \
3 -c 4096 -n 256 \
4 -p "Hello"
Stock llama.cpp builds that do not include Nanbeige support will reject the
nanbeige architecture. The patch adds conversion metadata, tensor loading,
loop-boundary normalization, and 44-layer cache handling; it does not replace
model weights.
Validation
Built and tested on Windows with llama.cpp
cf512566dc89d7a2bdefd43f6d4b85a1c18e93a7 and an AMD Ryzen 9 7950X3D.
- Both files loaded as architecture
nanbeige, block count 44, loop count 2.
- Q4_K_M: prompt benchmark 92.77 tok/s; generation benchmark 15.67 tok/s.
- Q8_0: prompt benchmark 53.95 tok/s; generation benchmark 9.37 tok/s.
- Deterministic eight-token generation completed for both files.
- Interactive Q8_0 sample measured 68.6 tok/s prompt and 10.2 tok/s generation.
These are CPU smoke/throughput tests, not quality benchmarks. The original
model advertises a 262,144-token context, but local validation used a short
512-token context.
License
The base model is Apache-2.0 licensed. Review the upstream model card for its
full terms and intended-use information.
