You can deploy this model and start earning money today!

bg-digitalservices/
Gemma-4-E4B-it-NVFP4

Name: Gemma-4-E4B-it-NVFP4
Availability: InStock

Views

No views yet

Gemma-4-E4B-it-NVFP4

NVFP4 quantization of google/gemma-4-E4B-it — Google's Gemma 4 E4B instruct (chat-optimized) with Per-Layer Embeddings (PLE). 4B effective parameters, multimodal (text + image + audio), 128K context.

W4A4 — language model weights and activations in FP4. Vision and audio towers stay BF16.

See also Gemma-4-E4B-it-NVFP4A16 for the weight-only W4A16 variant.

Key Specs

	Original (BF16)	NVFP4 (this)
Size on disk	~16 GB	~10.2 GB
Compression	—	~1.6x (text layers 3x, vision/audio stay BF16)
Effective parameters	2B	2B
Architecture	Dense + PLE (Per-Layer Embeddings)	same
Context window	128K tokens	128K tokens
Modalities	Text, Image, Audio	Text, Image, Audio

What is PLE?

Unlike the Gemma 4 26B which uses Mixture-of-Experts (MoE), the E4B uses Per-Layer Embeddings — a learned per-layer specialization mechanism. Each of the 26 decoder layers gets its own 256-dimensional signal derived from both the token identity (via a second embedding table) and the evolving hidden representation. This is a continuous alternative to discrete MoE routing — no expert selection, no sparsity, just dense computation with layer-specific conditioning.

Speed

Tested on DGX Spark (GB10 Blackwell, SM 12.1):

Metric	NVFP4
Tokens/sec	~65 tok/s
Model load	~10.2 GB VRAM

Serving with vLLM

Requires vLLM with transformers >= 5.4 (for Gemma 4 architecture support). No patches needed — vanilla vLLM handles E4B NVFP4 directly.

bash

vllm serve bg-digitalservices/Gemma-4-E4B-it-NVFP4 \
  --quantization modelopt \
  --dtype auto \
  --kv-cache-dtype fp8 \
  --gpu-memory-utilization 0.30 \
  --max-model-len 131072 \
  --trust-remote-code

DGX Spark

bash

VLLM_NVFP4_GEMM_BACKEND=marlin vllm serve bg-digitalservices/Gemma-4-E4B-it-NVFP4 \
  --quantization modelopt \
  --dtype auto \
  --kv-cache-dtype fp8 \
  --gpu-memory-utilization 0.10 \
  --max-model-len 131072 \
  --trust-remote-code

Testing

This is an instruct model — use the chat completions endpoint:

bash

curl http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"bg-digitalservices/Gemma-4-E4B-it-NVFP4","messages":[{"role":"user","content":"Hello!"}],"max_tokens":200}'

Quantization Details

Method: NVIDIA Model Optimizer (modelopt) v0.43
Format: NVFP4 — E2M1 weights with per-group FP8 scales (group size 16)
Calibration: 512 samples from CNN/DailyMail, batch size 4, seq_len 1024
Excluded from quantization: Vision tower, audio tower, vision/audio projection layers, lm_head (all stay BF16)
Hardware: NVIDIA DGX Spark (GB10 Blackwell)
Quantization script: included as quantize.py

Limitations

Requires vLLM with transformers >= 5.4
Vision/audio towers stay BF16 (≈5 GB of the 7.5 GB total) — excluded to preserve multimodal quality
Community quantization, not an official NVIDIA or Google release

License

Apache 2.0 — inherited from the base model.

Credits

Quantized by Mario Iseli on an NVIDIA DGX Spark. Built and validated with AI-engineering assistance from Anthropic.

Shout-out to eugr/spark-vllm-docker for the DGX Spark-optimized vLLM build.

📬 mario@marioiseli.com ☕ Buy me a coffee if this makes your Spark go brrrrrr! 🚀