Views
No views yet
| Model | File size | MMLU | JMMLU | HELLASWAG | ARC_CHALLENGE | GSM8K |
|---|---|---|---|---|---|---|
| gemma-4-e4b-it-4bit | 4.89 GB | 64.3% | 60.3% | 73.7% | 84.0% | 85.3% |
| gemma-4-E4B-it-oQ3-fp16 | 3.59 GB | 18.3% | 34.3% | 38.0% | 52.3% | 3.0% |
| gemma-4-E4B-it-oQ4-fp16 | 4.46 GB | 61.7% | 61.3% | 74.3% | 82.7% | 88.3% |
| Model | Benchmark | Accuracy | Correct | Total | Time(s) |
|---|---|---|---|---|---|
| gemma-4-e4b-it-4bit | MMLU | 64.3% | 193 | 300 | 418.2 |
| gemma-4-E4B-it-oQ3-fp16 | MMLU | 18.3% | 55 | 300 | 544.7 |
| gemma-4-E4B-it-oQ4-fp16 | MMLU | 61.7% | 185 | 300 | 352.2 |
| gemma-4-e4b-it-4bit | JMMLU | 60.3% | 181 | 300 | 192.3 |
| gemma-4-E4B-it-oQ3-fp16 | JMMLU | 34.3% | 103 | 300 | 295.4 |
| gemma-4-E4B-it-oQ4-fp16 | JMMLU | 61.3% | 184 | 300 | 161.7 |
| gemma-4-e4b-it-4bit | HELLASWAG | 73.7% | 221 | 300 | 219.3 |
| gemma-4-E4B-it-oQ3-fp16 | HELLASWAG | 38.0% | 114 | 300 | 203 |
| gemma-4-E4B-it-oQ4-fp16 | HELLASWAG | 74.3% | 223 | 300 | 177.2 |
| gemma-4-e4b-it-4bit | ARC_CHALLENGE | 84.0% | 252 | 300 | 176.5 |
| gemma-4-E4B-it-oQ3-fp16 | ARC_CHALLENGE | 52.3% | 157 | 300 | 149 |
| gemma-4-E4B-it-oQ4-fp16 | ARC_CHALLENGE | 82.7% | 248 | 300 | 150.3 |
| gemma-4-e4b-it-4bit | GSM8K | 85.3% | 256 | 300 | 1486.1 |
| gemma-4-E4B-it-oQ3-fp16 | GSM8K | 3.0% | 9 | 300 | 1441 |
| gemma-4-E4B-it-oQ4-fp16 | GSM8K | 88.3% | 265 | 300 | 1110.9 |
| Model | Test Case | TTFT (ms) | TPOT (ms) | pp TPS | tg TPS | E2E (s) | Throughput | Peak Mem |
|---|---|---|---|---|---|---|---|---|
| gemma-4-e4b-it-4bit | pp4096/tg128 | 4898.2 | 16.99 | 836.2 tok/s | 59.3 tok/s | 7.056 | 598.7 tok/s | 4.71 GB |
| gemma-4-e4b-it-4bit | pp32768/tg128 | 49735.7 | 32.28 | 658.8 tok/s | 31.2 tok/s | 53.835 | 611.1 tok/s | 5.49 GB |
| gemma-4-E4B-it-oQ4-fp16 | pp4096/tg128 | 3192.1 | 15.21 | 1283.2 tok/s | 66.3 tok/s | 5.124 | 824.4 tok/s | 4.83 GB |
| gemma-4-E4B-it-oQ4-fp16 | pp32768/tg128 | 36248.8 | 26.43 | 904.0 tok/s | 38.1 tok/s | 39.606 | 830.6 tok/s | 5.63 GB |