You can deploy this model and start earning money today!

Pajari/
gemma-4-31B-it-qat-Q4_0_V2

Name: gemma-4-31B-it-qat-Q4_0_V2
Availability: InStock

gguf

google/gemma-4-31B-it-qat-q4_0-unquantized

Views

No views yet

Gemma 4 31B QAT Q4_0_V2

Quantization of Google's Gemma 4 31B QAT checkpoint using a modified Q4_0 encoder that better preserves QAT weight geometry under standard GGUF FP16 block scales.

Q4_0_V2 achieves approximately 7× lower KL divergence than standard Q4_0 while preserving the same storage format and runtime characteristics.

Q4_0_V2 will run in normal mainline llama.cpp.

Gemma 4 31B — KL divergence vs unquantized BF16 QAT checkpoint

Rank	Model	PPL ratio	Mean KLD	Δp RMS	Same top-p
🥇	pajari/gemma-4-31B-it-qat-Q4_0_V2	1.00533	0.001990	1.408%	99.416%
🥈	unsloth/gemma-4-31B-it-qat-GGUF	1.00580	0.002198	1.537%	99.339%
🥉	google/gemma-4-31B-it-qat-q4_0-gguf	0.99484	0.013254	3.477%	97.666%
4	Stock Q4_0	0.99870	0.013962	3.582%	97.564%

Lower is better for KLD and Δp RMS, higher is better for Same top-p, PPL ratio is best closest to 1.0. Bold marks the best score in each column.

Models are ranked by Mean KLD (lower is better).

Logs

txt

.\llama.cpp\build_hip\bin\llama-perplexity.exe  --override-kv gemma4.final_logit_softcapping=float:30.0,tokenizer.ggml.add_bos_token=bool:false --device rocm0 -ngl 30 -c 8192 -m .\model\gemma-4-31B-it-qat-q4_0-unquantized\gemma-4-31B-it-qat-q4_0-unquantized-gguf-model-bf16.gguf -f .\dataset\calibration\gemma-4-think-single-chat.txt --kl-divergence-base .\model\base_logits.kld
HIP Library Path: C:\WINDOWS\SYSTEM32\amdhip64_7.dll
00.178.215 I common_init_result: fitting params to device memory ...
00.178.218 I common_init_result: (for bugs during this step try to reproduce them with -fit off, or provide --verbose logs if the bug only occurs with -fit on)
02.053.395 W load: control-looking token:    212 '</s>' was not control-type; this is probably a bug in the model. its type will be overridden
02.088.101 W load: gemma4: removing </s> (212) from EOG (normal text token)
21.847.120 W llama_context: n_ctx_seq (8192) < n_ctx_train (262144) -- the full capacity of the model will not be utilized
22.113.053 I common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
38.521.077 I
38.526.884 I system_info: n_threads = 12 (n_threads_batch = 12) / 24 | ROCm : NO_VMM = 1 | PEER_MAX_BATCH_SIZE = 128 | FA_ALL_QUANTS = 1 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 |
38.528.042 I perplexity: saving all logits to .\model\base_logits.kld
38.528.050 I perplexity: tokenizing the input ..
38.689.954 I perplexity: tokenization took 161.891 ms
38.690.086 I perplexity: calculating perplexity over 12 chunks, n_ctx=8192, batch_size=2048, n_seq=1
36.876.928 I perplexity: 116.76 seconds per pass - ETA 23.35 minutes
[1]1.7538,[2]1.9196,[3]1.9008,[4]2.0169,[5]2.0069,[6]2.0836,[7]2.0426,[8]2.0783,[9]2.1363,[10]2.2036,[11]2.3006,[12]2.3660,
26.827.359 I Final estimate: PPL = 2.3660 +/- 0.02526

Unsloth q4_0

txt

.\llama.cpp\build_hip\bin\llama-perplexity.exe --override-kv gemma4.final_logit_softcapping=float:30.0,tokenizer.ggml.add_bos_token=bool:false -c 8192 --device rocm0 -ngl all -m .\model\gemma-4-31B-it-qat-q4_0-unquantized\unsloth-gemma-4-31B-it-qat-UD-Q4_K_XL.gguf --kl-divergence-base .\model\base_logits.kld --kl-divergence -f .\dataset\calibration\gemma-4-think-single-chat.txt
HIP Library Path: C:\WINDOWS\SYSTEM32\amdhip64_7.dll
0.00.204.407 I common_init_result: fitting params to device memory ...
0.00.204.410 I common_init_result: (for bugs during this step try to reproduce them with -fit off, or provide --verbose logs if the bug only occurs with -fit on)
0.01.408.473 W load: control-looking token:    212 '</s>' was not control-type; this is probably a bug in the model. its type will be overridden
0.01.439.222 W load: gemma4: removing </s> (212) from EOG (normal text token)
0.12.907.460 W llama_context: n_ctx_seq (8192) < n_ctx_train (262144) -- the full capacity of the model will not be utilized
0.12.998.779 I common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
0.13.112.335 I
0.13.112.386 I system_info: n_threads = 12 (n_threads_batch = 12) / 24 | ROCm : NO_VMM = 1 | PEER_MAX_BATCH_SIZE = 128 | FA_ALL_QUANTS = 1 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 |
0.13.505.695 I kl_divergence: computing over 12 chunks, n_ctx=8192, batch_size=2048, n_seq=1
0.25.381.788 I kl_divergence: 11.87 seconds per pass - ETA 2.37 minutes

chunk             PPL               ln(PPL(Q)/PPL(base))          KL Divergence              Δp RMS            Same top p
   1       1.7560 ±    0.0510       0.00166 ±    0.00096       0.00107 ±    0.00029     1.223 ±  0.363 %    99.634 ±  0.094 %
   2       1.9218 ±    0.0429       0.00212 ±    0.00076       0.00099 ±    0.00017     1.071 ±  0.212 %    99.475 ±  0.080 %
   3       1.9028 ±    0.0347       0.00220 ±    0.00070       0.00168 ±    0.00042     1.402 ±  0.234 %    99.487 ±  0.064 %
   4       2.0198 ±    0.0332       0.00364 ±    0.00101       0.00184 ±    0.00039     1.515 ±  0.250 %    99.426 ±  0.059 %
   5       2.0095 ±    0.0297       0.00404 ±    0.00096       0.00172 ±    0.00032     1.432 ±  0.213 %    99.443 ±  0.052 %
   6       2.0853 ±    0.0290       0.00374 ±    0.00089       0.00165 ±    0.00027     1.417 ±  0.191 %    99.451 ±  0.047 %
   7       2.0444 ±    0.0261       0.00380 ±    0.00082       0.00172 ±    0.00025     1.394 ±  0.169 %    99.445 ±  0.044 %
   8       2.0801 ±    0.0251       0.00412 ±    0.00080       0.00188 ±    0.00025     1.501 ±  0.156 %    99.396 ±  0.043 %
   9       2.1380 ±    0.0249       0.00465 ±    0.00078       0.00194 ±    0.00026     1.487 ±  0.143 %    99.411 ±  0.040 %
  10       2.2051 ±    0.0249       0.00509 ±    0.00079       0.00231 ±    0.00030     1.596 ±  0.133 %    99.387 ±  0.039 %
  11       2.3031 ±    0.0253       0.00556 ±    0.00076       0.00225 ±    0.00028     1.557 ±  0.124 %    99.354 ±  0.038 %
  12       2.3683 ±    0.0253       0.00578 ±    0.00080       0.00220 ±    0.00026     1.537 ±  0.117 %    99.339 ±  0.037 %

====== Perplexity statistics ======
Mean PPL(Q)                   :   2.368275 ±   0.025304
Mean PPL(base)                :   2.354614 ±   0.024749
Cor(ln(PPL(Q)), ln(PPL(base))):  99.73%
Mean ln(PPL(Q)/PPL(base))     :   0.005785 ±   0.000800
Mean PPL(Q)/PPL(base)         :   1.005802 ±   0.000804
Mean PPL(Q)-PPL(base)         :   0.013660 ±   0.001925

====== KL divergence statistics ======
Mean    KLD:   0.002198 ±   0.000257
Maximum KLD:   5.927129
99.9%   KLD:   0.257504
99.0%   KLD:   0.013956
95.0%   KLD:   0.003716
90.0%   KLD:   0.002043
Median  KLD:   0.000007
10.0%   KLD:  -0.000000
 5.0%   KLD:  -0.000001
 1.0%   KLD:  -0.000002
 0.1%   KLD:  -0.000024
Minimum KLD:  -0.000109

====== Token probability statistics ======
Mean    Δp: -0.002 ± 0.007 %
Maximum Δp: 75.997%
99.9%   Δp:  9.896%
99.0%   Δp:  2.580%
95.0%   Δp:  0.668%
90.0%   Δp:  0.163%
75.0%   Δp:  0.002%
Median  Δp:  0.000%
25.0%   Δp: -0.003%
10.0%   Δp: -0.183%
 5.0%   Δp: -0.713%
 1.0%   Δp: -2.528%
 0.1%   Δp: -9.386%
Minimum Δp: -96.647%
RMS Δp    :  1.537 ± 0.117 %
Same top p: 99.339 ± 0.037 %

Google q4_0

txt

.\llama.cpp\build_hip\bin\llama-perplexity.exe --override-kv gemma4.final_logit_softcapping=float:30.0,tokenizer.ggml.add_bos_token=bool:false -c 8192 --device rocm0 -ngl all -m .\model\gemma-4-31B-it-qat-q4_0-unquantized\google-gemma-4-31B_q4_0-it.gguf --kl-divergence-base .\model\base_logits.kld --kl-divergence -f .\dataset\calibration\gemma-4-think-single-chat.txt
HIP Library Path: C:\WINDOWS\SYSTEM32\amdhip64_7.dll
0.00.171.986 I common_init_result: fitting params to device memory ...
0.00.171.989 I common_init_result: (for bugs during this step try to reproduce them with -fit off, or provide --verbose logs if the bug only occurs with -fit on)
0.01.364.326 W load: control-looking token:    212 '</s>' was not control-type; this is probably a bug in the model. its type will be overridden
0.01.395.934 W load: gemma4: removing </s> (212) from EOG (normal text token)
0.13.204.258 W llama_context: n_ctx_seq (8192) < n_ctx_train (262144) -- the full capacity of the model will not be utilized
0.13.289.629 I common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
0.13.385.253 I
0.13.385.285 I system_info: n_threads = 12 (n_threads_batch = 12) / 24 | ROCm : NO_VMM = 1 | PEER_MAX_BATCH_SIZE = 128 | FA_ALL_QUANTS = 1 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 |
0.13.760.755 I kl_divergence: computing over 12 chunks, n_ctx=8192, batch_size=2048, n_seq=1
0.25.446.538 I kl_divergence: 11.69 seconds per pass - ETA 2.33 minutes

chunk             PPL               ln(PPL(Q)/PPL(base))          KL Divergence              Δp RMS            Same top p
   1       1.7425 ±    0.0500      -0.00604 ±    0.00319       0.00848 ±    0.00119     2.724 ±  0.254 %    98.535 ±  0.188 %
   2       1.9103 ±    0.0422      -0.00388 ±    0.00267       0.00985 ±    0.00095     3.139 ±  0.272 %    98.388 ±  0.139 %
   3       1.8894 ±    0.0340      -0.00486 ±    0.00210       0.00986 ±    0.00072     3.099 ±  0.212 %    98.372 ±  0.114 %
   4       2.0048 ±    0.0326      -0.00380 ±    0.00198       0.01016 ±    0.00057     3.143 ±  0.173 %    98.132 ±  0.106 %
   5       1.9948 ±    0.0292      -0.00332 ±    0.00180       0.01035 ±    0.00051     3.175 ±  0.149 %    98.129 ±  0.095 %
   6       2.0686 ±    0.0284      -0.00428 ±    0.00167       0.01117 ±    0.00046     3.235 ±  0.132 %    97.998 ±  0.089 %
   7       2.0270 ±    0.0256      -0.00478 ±    0.00151       0.01066 ±    0.00041     3.171 ±  0.122 %    98.050 ±  0.082 %
   8       2.0628 ±    0.0246      -0.00422 ±    0.00143       0.01101 ±    0.00045     3.246 ±  0.119 %    97.973 ±  0.078 %
   9       2.1186 ±    0.0244      -0.00447 ±    0.00141       0.01169 ±    0.00046     3.331 ±  0.117 %    97.916 ±  0.074 %
  10       2.1848 ±    0.0244      -0.00415 ±    0.00136       0.01227 ±    0.00047     3.386 ±  0.107 %    97.836 ±  0.072 %
  11       2.2792 ±    0.0247      -0.00487 ±    0.00132       0.01296 ±    0.00044     3.443 ±  0.101 %    97.707 ±  0.071 %
  12       2.3425 ±    0.0247      -0.00518 ±    0.00132       0.01325 ±    0.00042     3.477 ±  0.095 %    97.666 ±  0.068 %

====== Perplexity statistics ======
Mean PPL(Q)                   :   2.342454 ±   0.024702
Mean PPL(base)                :   2.354614 ±   0.024749
Cor(ln(PPL(Q)), ln(PPL(base))):  99.21%
Mean ln(PPL(Q)/PPL(base))     :  -0.005178 ±   0.001324
Mean PPL(Q)/PPL(base)         :   0.994836 ±   0.001318
Mean PPL(Q)-PPL(base)         :  -0.012160 ±   0.003110

====== KL divergence statistics ======
Mean    KLD:   0.013254 ±   0.000418
Maximum KLD:   6.996171
99.9%   KLD:   1.077398
99.0%   KLD:   0.153186
95.0%   KLD:   0.054351
90.0%   KLD:   0.031315
Median  KLD:   0.000065
10.0%   KLD:   0.000000
 5.0%   KLD:  -0.000000
 1.0%   KLD:  -0.000001
 0.1%   KLD:  -0.000005
Minimum KLD:  -0.000027

====== Token probability statistics ======
Mean    Δp: -0.076 ± 0.016 %
Maximum Δp: 92.678%
99.9%   Δp: 27.057%
99.0%   Δp:  9.345%
95.0%   Δp:  2.392%
90.0%   Δp:  0.618%
75.0%   Δp:  0.011%
Median  Δp:  0.000%
25.0%   Δp: -0.007%
10.0%   Δp: -0.759%
 5.0%   Δp: -3.051%
 1.0%   Δp: -10.076%
 0.1%   Δp: -30.930%
Minimum Δp: -95.720%
RMS Δp    :  3.477 ± 0.095 %
Same top p: 97.666 ± 0.068 %

Stock llama.cpp q4_0

txt

.\llama.cpp\build_hip\bin\llama-perplexity.exe --override-kv gemma4.final_logit_softcapping=float:30.0,tokenizer.ggml.add_bos_token=bool:false -c 8192 --device rocm0 -ngl all -m .\model\gemma-4-31B-it-qat-q4_0-unquantized\gemma-4-31B-it-qat-q4_0-unquantized-gguf-model-Q4_0.gguf --kl-divergence-base .\model\base_logits.kld --kl-divergence -f .\dataset\calibration\gemma-4-think-single-chat.txt
HIP Library Path: C:\WINDOWS\SYSTEM32\amdhip64_7.dll
0.00.172.402 I common_init_result: fitting params to device memory ...
0.00.172.405 I common_init_result: (for bugs during this step try to reproduce them with -fit off, or provide --verbose logs if the bug only occurs with -fit on)
0.01.361.246 W load: control-looking token:    212 '</s>' was not control-type; this is probably a bug in the model. its type will be overridden
0.01.393.176 W load: gemma4: removing </s> (212) from EOG (normal text token)
0.12.161.758 W llama_context: n_ctx_seq (8192) < n_ctx_train (262144) -- the full capacity of the model will not be utilized
0.12.253.706 I common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
0.12.376.249 I
0.12.376.298 I system_info: n_threads = 12 (n_threads_batch = 12) / 24 | ROCm : NO_VMM = 1 | PEER_MAX_BATCH_SIZE = 128 | FA_ALL_QUANTS = 1 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 |
0.12.937.129 I kl_divergence: computing over 12 chunks, n_ctx=8192, batch_size=2048, n_seq=1
0.24.418.515 I kl_divergence: 11.48 seconds per pass - ETA 2.28 minutes

chunk             PPL               ln(PPL(Q)/PPL(base))          KL Divergence              Δp RMS            Same top p
   1       1.7455 ±    0.0503      -0.00433 ±    0.00322       0.00887 ±    0.00121     2.758 ±  0.256 %    98.413 ±  0.195 %
   2       1.9156 ±    0.0425      -0.00115 ±    0.00285       0.01025 ±    0.00115     3.013 ±  0.274 %    98.242 ±  0.145 %
   3       1.8952 ±    0.0343      -0.00176 ±    0.00221       0.01032 ±    0.00087     3.103 ±  0.231 %    98.282 ±  0.117 %
   4       2.0093 ±    0.0328      -0.00157 ±    0.00206       0.01053 ±    0.00067     3.080 ±  0.181 %    98.046 ±  0.108 %
   5       1.9987 ±    0.0293      -0.00137 ±    0.00185       0.01100 ±    0.00065     3.214 ±  0.169 %    98.017 ±  0.097 %
   6       2.0729 ±    0.0286      -0.00220 ±    0.00173       0.01191 ±    0.00059     3.334 ±  0.158 %    97.875 ±  0.092 %
   7       2.0316 ±    0.0257      -0.00251 ±    0.00156       0.01135 ±    0.00052     3.267 ±  0.143 %    97.952 ±  0.084 %
   8       2.0686 ±    0.0247      -0.00140 ±    0.00148       0.01177 ±    0.00054     3.345 ±  0.135 %    97.875 ±  0.080 %
   9       2.1260 ±    0.0246      -0.00096 ±    0.00144       0.01227 ±    0.00053     3.401 ±  0.128 %    97.827 ±  0.076 %
  10       2.1919 ±    0.0245      -0.00092 ±    0.00139       0.01279 ±    0.00050     3.453 ±  0.117 %    97.734 ±  0.074 %
  11       2.2881 ±    0.0249      -0.00097 ±    0.00136       0.01367 ±    0.00051     3.540 ±  0.113 %    97.611 ±  0.072 %
  12       2.3516 ±    0.0249      -0.00130 ±    0.00136       0.01396 ±    0.00048     3.582 ±  0.106 %    97.564 ±  0.070 %

====== Perplexity statistics ======
Mean PPL(Q)                   :   2.351559 ±   0.024908
Mean PPL(base)                :   2.354614 ±   0.024749
Cor(ln(PPL(Q)), ln(PPL(base))):  99.18%
Mean ln(PPL(Q)/PPL(base))     :  -0.001298 ±   0.001357
Mean PPL(Q)/PPL(base)         :   0.998702 ±   0.001355
Mean PPL(Q)-PPL(base)         :  -0.003055 ±   0.003192

====== KL divergence statistics ======
Mean    KLD:   0.013962 ±   0.000476
Maximum KLD:   7.633423
99.9%   KLD:   1.140293
99.0%   KLD:   0.153066
95.0%   KLD:   0.056004
90.0%   KLD:   0.032574
Median  KLD:   0.000080
10.0%   KLD:   0.000001
 5.0%   KLD:   0.000000
 1.0%   KLD:  -0.000001
 0.1%   KLD:  -0.000002
Minimum KLD:  -0.000011

====== Token probability statistics ======
Mean    Δp: -0.047 ± 0.016 %
Maximum Δp: 95.999%
99.9%   Δp: 27.728%
99.0%   Δp:  9.559%
95.0%   Δp:  2.588%
90.0%   Δp:  0.666%
75.0%   Δp:  0.012%
Median  Δp:  0.000%
25.0%   Δp: -0.007%
10.0%   Δp: -0.688%
 5.0%   Δp: -2.970%
 1.0%   Δp: -10.155%
 0.1%   Δp: -32.910%
Minimum Δp: -99.760%
RMS Δp    :  3.582 ± 0.106 %
Same top p: 97.564 ± 0.070 %

q4_0_v2

txt

.\llama.cpp\build_hip\bin\llama-perplexity.exe --override-kv gemma4.final_logit_softcapping=float:30.0,tokenizer.ggml.add_bos_token=bool:false -c 8192 --device rocm0 -ngl all -m .\model\gemma-4-31B-it-qat-q4_0-unquantized\gemma-4-31B-it-qat-gguf-model-Q4_0_V2_QAT.gguf --kl-divergence-base .\model\base_logits.kld --kl-divergence -f .\dataset\calibration\gemma-4-think-single-chat.txt
HIP Library Path: C:\WINDOWS\SYSTEM32\amdhip64_7.dll
0.00.312.798 I common_init_result: fitting params to device memory ...
0.00.312.801 I common_init_result: (for bugs during this step try to reproduce them with -fit off, or provide --verbose logs if the bug only occurs with -fit on)
0.01.448.995 W load: control-looking token:    212 '</s>' was not control-type; this is probably a bug in the model. its type will be overridden
0.01.487.202 W load: gemma4: removing </s> (212) from EOG (normal text token)
0.09.133.145 W llama_context: n_ctx_seq (8192) < n_ctx_train (262144) -- the full capacity of the model will not be utilized
0.09.217.400 I common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
0.09.307.267 I
0.09.307.299 I system_info: n_threads = 12 (n_threads_batch = 12) / 24 | ROCm : NO_VMM = 1 | PEER_MAX_BATCH_SIZE = 128 | FA_ALL_QUANTS = 1 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 |
0.09.515.347 I kl_divergence: computing over 12 chunks, n_ctx=8192, batch_size=2048, n_seq=1
0.20.486.565 I kl_divergence: 10.97 seconds per pass - ETA 2.18 minutes

chunk             PPL               ln(PPL(Q)/PPL(base))          KL Divergence              Δp RMS            Same top p
   1       1.7530 ±    0.0509      -0.00003 ±    0.00083       0.00093 ±    0.00027     0.728 ±  0.070 %    99.829 ±  0.065 %
   2       1.9206 ±    0.0429       0.00149 ±    0.00075       0.00089 ±    0.00016     0.795 ±  0.050 %    99.670 ±  0.063 %
   3       1.9027 ±    0.0347       0.00216 ±    0.00072       0.00164 ±    0.00044     1.277 ±  0.231 %    99.617 ±  0.056 %
   4       2.0180 ±    0.0331       0.00277 ±    0.00097       0.00151 ±    0.00033     1.255 ±  0.187 %    99.560 ±  0.052 %
   5       2.0078 ±    0.0297       0.00317 ±    0.00092       0.00142 ±    0.00027     1.191 ±  0.159 %    99.570 ±  0.046 %
   6       2.0843 ±    0.0290       0.00325 ±    0.00086       0.00141 ±    0.00024     1.152 ±  0.138 %    99.577 ±  0.041 %
   7       2.0436 ±    0.0261       0.00338 ±    0.00080       0.00150 ±    0.00022     1.165 ±  0.122 %    99.574 ±  0.038 %
   8       2.0793 ±    0.0251       0.00373 ±    0.00078       0.00159 ±    0.00020     1.280 ±  0.114 %    99.509 ±  0.039 %
   9       2.1367 ±    0.0249       0.00405 ±    0.00076       0.00169 ±    0.00022     1.309 ±  0.106 %    99.517 ±  0.036 %
  10       2.2043 ±    0.0249       0.00474 ±    0.00077       0.00215 ±    0.00028     1.482 ±  0.114 %    99.490 ±  0.035 %
  11       2.3017 ±    0.0253       0.00495 ±    0.00075       0.00209 ±    0.00026     1.447 ±  0.106 %    99.438 ±  0.035 %
  12       2.3672 ±    0.0253       0.00532 ±    0.00079       0.00199 ±    0.00024     1.408 ±  0.100 %    99.416 ±  0.034 %

====== Perplexity statistics ======
Mean PPL(Q)                   :   2.367174 ±   0.025279
Mean PPL(base)                :   2.354614 ±   0.024749
Cor(ln(PPL(Q)), ln(PPL(base))):  99.73%
Mean ln(PPL(Q)/PPL(base))     :   0.005320 ±   0.000791
Mean PPL(Q)/PPL(base)         :   1.005334 ±   0.000795
Mean PPL(Q)-PPL(base)         :   0.012560 ±   0.001900

====== KL divergence statistics ======
Mean    KLD:   0.001990 ±   0.000237
Maximum KLD:   5.916455
99.9%   KLD:   0.229264
99.0%   KLD:   0.013512
95.0%   KLD:   0.003680
90.0%   KLD:   0.001975
Median  KLD:   0.000007
10.0%   KLD:  -0.000000
 5.0%   KLD:  -0.000001
 1.0%   KLD:  -0.000003
 0.1%   KLD:  -0.000027
Minimum KLD:  -0.000136

====== Token probability statistics ======
Mean    Δp: -0.010 ± 0.006 %
Maximum Δp: 76.061%
99.9%   Δp:  8.657%
99.0%   Δp:  2.482%
95.0%   Δp:  0.641%
90.0%   Δp:  0.147%
75.0%   Δp:  0.002%
Median  Δp:  0.000%
25.0%   Δp: -0.004%
10.0%   Δp: -0.184%
 5.0%   Δp: -0.717%
 1.0%   Δp: -2.585%
 0.1%   Δp: -8.989%
Minimum Δp: -74.365%
RMS Δp    :  1.408 ± 0.100 %
Same top p: 99.416 ± 0.034 %

Pajari/gemma-4-31B-it-qat-Q4_0_V2