NVFP4A16 quantization of ManniX-ITA/gemma-4-A4B-98e-v7-coderx-it, the code-maximal prune of Gemma 4 26B-A4B (128→98 experts/layer, ~20.8B). 2 shards, ~13.4 GB — fits a single 16 GB GPU with room for KV-cache.

NVFP4A16 = 4-bit NVFP4 weights with FP8 (e4m3) block scales and bf16 activations (A16). Quantized with NVIDIA TensorRT-Model-Optimizer (modelopt) main (0.45.0.dev), whose _QuantFusedExperts plugin handles Gemma 4's fused MoE experts. Every exported weight/scale tensor is finiteness-verified before upload.

Serve with vLLM

bash

python -m vllm.entrypoints.openai.api_server \
    --model ManniX-ITA/gemma-4-A4B-98e-v7-coderx-NVFP4A16 \
    --served-model-name v7-coderx \
    --port 8000 \
    --gpu-memory-utilization 0.92 \
    --max-model-len 65536 \
    --max-num-batched-tokens 8192 \
    --dtype bfloat16 \
    --trust-remote-code \
    --reasoning-parser gemma4 \
    --default-chat-template-kwargs '{"enable_thinking": true}'

Gemma 4 gotchas (all required): --max-num-batched-tokens 8192 (the MM-encoder budget; the default 2048 < max_tokens_per_mm_item and crashes at boot), --max-model-len 65536 (templates may request up to ~49k gen tokens), and the gemma4 reasoning parser for the thinking format. NVFP4 needs a Blackwell/Hopper or Ada GPU on a recent vLLM.

Benchmarks

NVFP4A16 is a deployment format and is not separately benchmarked (cohort policy). The table is the cohort's same-host Q6_K · llama.cpp · greedy reference (temperature 0.0, top_p 1.0, top_k 0), read from summary.json; it is representative of this model's quality. Row-max in bold. This repo = v7-coderx.

Benchmark	128e (unpruned)	v6-coder	v7-coder	v7-coderx
GPQA-diamond (198q)	67.17	61.11	51.52	51.01
AIME (30q)	73.33	56.67	80.00	76.67
MATH500 (100q)	92.00	89.00	95.00	95.00
GSM8K (100q)	89.00	88.00	91.00	93.00
ARC-Challenge (full)	96.50	95.39	92.15	86.60
IFEval (100q, strict)	97.00	92.00	92.00	92.00
HumanEval (164)	97.56	98.17	98.17	96.95
HumanEval+ (164)	92.07	92.68	92.07	93.29
LCB-medium-55 v4	96.36	92.73	98.18	92.73
LCB-medium-100 v4	97.00	94.00	94.00	91.00
MultiPL-E (100)	90.00	89.00	89.67	89.00

_{Metrics: GPQA & GSM8K = exact_match flexible-extract · MATH500 = math_verify ·
ARC & AIME = exact_match · IFEval = prompt_level_strict_acc · HumanEval/+ = pass@1
chat-extract · LCB-55/100 & MultiPL-E = pass@1. 128e uses the lcb_medium_55/100
templates; the prunes use lcb_medium_*_v4 (corrected harness, equivalent task). The all-hard LCB-77 cross-model comparison is the discriminating code slice (v7-coderx 85.71%, cohort-best).}

Recipe (summary)

98e prune from 128e via the code4/lcb3 recipe (generate_drop_map_v5fk: generic_code 4×, targeted_lcb_medium_55 3×, no per-layer floor clamp), then the agentic loop-protection force-keep (46 experts) and the mandatory shared-FFN α=1.2 upweight, then NVFP4A16 quantization. This is the loop-fixed build — it replaces the earlier looping fs2440 prune. v7-coderx is the code-maximal sibling of v7-coder. Full recipe and calibration-class table are on the bf16 card.

Intended use & limitations

A compact, vLLM-deployable Gemma 4 checkpoint for maximal coding throughput and instruction-following — the code-extreme (x) member of the v7-coder cohort. For the broader LCB-medium lead and HumanEval, use v7-coder, which leads those slices (GPQA ≈ 51 on both siblings — neither recovers science). A research prune, not an official Google release; generic_multilingual is de-weighted (0×) and graduate science (GPQA) is a budget axis — at 51.01% it is well below the unpruned 128e (67.17%). For llama.cpp/CPU deployment use the GGUF repo.

Lineage

128e → (v4 → v5 → v6-coder code line) → v7 competence-map rebuild → code4/lcb3 selection + agentic loop-protection force-keep = v7-coderx → NVFP4A16. Built and evaluated on the omnimergekit toolchain.