Views
No views yet
google/gemma-4-31B-it — the dense 31B Gemma 4 instruction-tuned model.gemma-4-A4B-98e-v5-it (same canonical eval recipe).omnimergekit/scripts/quantize_any.py --method nvfp4a16nvidia-modelopt==0.43.0 with NVFP4_DEFAULT_CFGtatsu-lab/alpaca, max_length=512*vision_tower*, *embed_vision*, *embed_audio* (kept BF16 for vLLM's gemma4_mm loader)temperature=0, top_p=1, top_k=0, do_sample=False), thinking_token_budget=12288, max_gen_toks=16384, vLLM 0.20.2 stock with --reasoning-parser gemma4 --default-chat-template-kwargs '{"enable_thinking": true}'. Bench definitions: omnimergekit/eval/templates/. 128e and v4 columns are the published canonical numbers from the v5-it card.| Bench (n) | 128e ref | 98e v4 | 98e v5 | 98e v5-coder | 31B-it (this model) | 31B-he1 |
|---|---|---|---|---|---|---|
| GPQA Diamond (198) | 73.23 % | 69.19 % | 68.69 % | 68.69 % | 81.31 % | coming soon |
| GSM8K-100 | 91.00 % | 86.00 % | in flight | 86.00 % | 93.00 % | coming soon |
| MATH-500-100 | 89.00 % | 89.00 % | in flight | 92.00 % | 97.00 % | coming soon |
| AIME 2024 (30) | 36.67 % | 36.67 % | in flight | 36.67 % | 76.67 % | coming soon |
| IFEval-100 (prompt_strict) | 95.00 % | 93.00 % | in flight | 94.00 % | 96.00 % | coming soon |
| HumanEval-164 chat | 96.95 % | 96.95 % | in flight | 98.17 % | 97.56 % | coming soon |
| HumanEval+ chat (164) | 92.07 % | 91.46 % | 89.63 % | 92.68 % | 92.07 % | coming soon |
| LCB-medium-55 (v4 split) | 87.27 % | 78.18 % | in flight | 85.45 % | 96.36 % | coming soon |
| ARC-Challenge chat (1172) | 95.99 % | 95.99 % | 96.59 % | 95.31 % | 98.04 % | coming soon |
local-chat-completions that discarded vLLM's reasoning_content; the cohort-wide Fix-A reasoning_content fallback is now applied across the cohort.1python -m vllm.entrypoints.openai.api_server \
2 --model ManniX-ITA/Gemma-4-31B-it-NVFP4A16 \
3 --port 8099 --tensor-parallel-size 1 \
4 --gpu-memory-utilization 0.70 \
5 --max-model-len 32768 --dtype bfloat16 --trust-remote-code \
6 --reasoning-parser gemma4 \
7 --default-chat-template-kwargs '{"enable_thinking": true}'{"thinking_token_budget": 12288, "temperature": 0.0, "top_p": 1.0, "top_k": 0}| GPU | Runs? | Memory (peak) | Notes |
|---|---|---|---|
| RTX 3090 24 GB | ❌ | OOM | Single-GPU at 32k ctx exceeds 24 GB; use TP=2 |
| RTX 4090 24 GB | ❌ | OOM | Same — TP=2 required |
| RTX 6000 Ada 48 GB | ✅ | ~32 GB | Single-GPU, 32k ctx |
| L40 / L40S 48 GB | ✅ | ~32 GB | Single-GPU, 32k ctx |
| A100 / H100 80 GB | ✅ | ~32 GB | Single-GPU, comfortable headroom for 65k ctx |
| 2× RTX 3090 / 4090 | ✅ | ~16 GB / GPU | TP=2, 32k+ ctx |
| Model | Description |
|---|---|
| gemma-4-31b-he1-it | Partial head-prune of 31B-it (L0–L3 Q-prune + lstsq heal). Same NVFP4A16 cohort eval is queued. |
| gemma-4-31b-he1-it-NVFP4A16 | NVFP4A16 quant of the head-pruned variant |
| gemma-4-A4B-98e-v5-it | 26B-A4B MoE variant — v4 + shared FFN α=1.2 |
| gemma-4-A4B-98e-v5-coder-it | 26B-A4B MoE — code-specialist drop map (C6 v4-floor breadth=50) + shared α |
| gemma-4-A4B-98e-v4-it | 26B-A4B MoE — multi-class CD-map drop, baseline for v5 |
google/gemma-4-31B-itomnimergekit/scripts/quantize_any.pyomnimergekit/scripts/eval_suite_vllm.sh + eval/templates/omnimergekit/eval/EVAL_PROTOCOL.md (v2, vLLM-first, sampler-pinned greedy, stack-pinned per cohort)gemma-4-31b-he1-it column — eval queued on pod 36949547 after v5-it GGUF imatrix completes.