rwkv7-g1h-7.2b-20260710-ctx10240 GGUF
GGUF conversions of
rwkv7-g1h-7.2b-20260710-ctx10240.pth. This repository is one model only; the other parameter sizes are published in separate repositories.
Conversion
The source checkpoint is BF16. It was converted directly from .pth to a BF16 GGUF with RWKV's converter, then the standard release ladder below was produced from that BF16 GGUF using the pinned llama.cpp quantizer. No safetensors staging file and no importance matrix were used.
- Source revision:
6d5762253b343eec6cfbf5ed62f872f30a4cd89c
- rwkv-mobile revision:
ebfb744281c31a07aad5606ec7473f79f837e92a
- llama.cpp revision:
c92e806d1c81091c9035edce99c35374da1b465e
- Context marker in source filename:
ctx10240
Files
BF16 master GGUFQ2_KQ3_K_SIQ4_XSQ4_K_SQ5_K_SQ6_KQ8_0Q4_0Q4_1Q5_0Q5_1
| Quant | PPL | BF16 retained |
|---|
BF16 | 6.0738 | 100.00% |
Q8_0 | 6.0739 | 100.00% |
Q6_K | 6.0941 | 99.67% |
Q5_1 | 6.1495 | 98.77% |
Q5_K_M | 6.143 | 98.87% |
Q5_K_S | 6.1579 | 98.63% |
Q5_0 | 6.1763 | 98.34% |
Q4_1 | 6.3985 | 94.93% |
Q4_K_M | 6.2938 | 96.50% |
Q4_K_S | 6.3748 | 95.28% |
Q4_0 | 7.3582 | 82.54% |
IQ4_XS | 6.4467 | 94.22% |
Q3_K_L | 6.4221 | 94.58% |
Q3_K_M | 6.5279 | 93.04% |
Q3_K_S | 19.6605 | 30.89% |
Q2_K | 276.2831 | 2.20% |
RWKV-aware mixed quantizations
The Q3_K_M, Q3_K_L, Q4_K_M, and Q5_K_M files use custom RWKV-aware recipes with explicit tensor assignments. Higher precision is used for the token embeddings and selected value, time-mix output, and channel-mix tensors where it is expected to preserve the most quality.
Earlier automated files with these names were removed after verification showed that llama.cpp's generic mixed recipes did not recognize RWKV's time_mix_* and channel_mix_* tensor roles. Because of that, the automated M and L variants had collapsed to the same effective layouts as the retained S variants.
These replacement files have genuinely different tensor layouts, providing additional size and quality choices between the existing S variants and the larger quantizations.
Prompting
Follow the prompt guide in the
upstream model card. Avoid a trailing space at the end of the input.
