NVFP4 quantized version of Jiunsong/supergemma4-26b-abliterated-multimodal — an aggressively abliterated, low-refusal multimodal Gemma 4 model with stronger coding, logic, and tool-use capabilities.

~50 GB → 16.5 GB with minimal quality loss. Runs on a single NVIDIA Blackwell GPU.

Why This Model

SuperGemma4 is not just abliterated — it's enhanced. Compared to the base Gemma 4:

+8.2 on code benchmarks, +4.1 on logic
2x improvement in API tool-call success rate
Multimodal (image + text) preserved at full precision
Low-refusal for local agent workflows

This NVFP4 checkpoint brings all of that down to 16.5 GB — single-GPU inference at 145+ tok/s.

Key Specs


Base model	Jiunsong/supergemma4-26b-abliterated-multimodal
Architecture	Gemma 4 MoE — 25.2B total, 3.8B active per token
Quantization	NVFP4 (W4A4 — weights FP4, activations FP4, scales FP8)
Format	`compressed-tensors` (native vLLM support)
Tool	vllm-project/llm-compressor (main)
Size	16.5 GB (single safetensors shard)
Requires	NVIDIA Blackwell GPU (SM 120), vLLM nightly (cu130)

Quickstart

vLLM (recommended)

bash

vllm serve Lna-Lab/SuperGemma4-26B-Abliterated-Multimodal-NVFP4 \
    --max-model-len 8192

No --quantization flag needed — vLLM auto-detects compressed-tensors format.

Docker

bash

docker run --gpus '"device=0"' -p 8016:8016 \
    -v /path/to/model:/models/current:ro \
    --shm-size 16gb \
    vllm/vllm-openai:cu130-nightly \
    vllm serve /models/current --port 8016 --max-model-len 8192

Python (vLLM)

python

from vllm import LLM, SamplingParams

llm = LLM(
    model="Lna-Lab/SuperGemma4-26B-Abliterated-Multimodal-NVFP4",
    max_model_len=8192,
    gpu_memory_utilization=0.85,
)

output = llm.generate(
    ["Explain quantum entanglement in simple terms."],
    SamplingParams(max_tokens=256, temperature=0.7),
)
print(output[0].outputs[0].text)

Benchmark

Tested on a single NVIDIA RTX PRO 6000 Blackwell (96 GB VRAM), CUDA Graph PIECEWISE mode.

Test	Tokens	Speed	Result
English reasoning	42	134 tok/s	PASS
Code generation	116	134 tok/s	PASS
Long-form output	261	146 tok/s	PASS

Sustained throughput: ~145 tok/s (post-warmup, single GPU).

Quantization Details

Recipe

yaml

default_stage:
  default_modifiers:
    QuantizationModifier:
      targets: [Linear]
      ignore: [lm_head, 're:.*embed.*', 're:.*router', 're:.*vision_tower.*']
      scheme: NVFP4

What's quantized, what's not

Quantized (NVFP4): All Linear layers in the language model, including MoE expert layers
Kept in BF16: lm_head, all embedding layers, MoE routers, entire vision tower

Calibration

Dataset: neuralmagic/calibration (LLM split)
Samples: 20
Max sequence length: 8192
MoE expert calibration handled automatically by llm-compressor's SequentialGemma4TextExperts

Reproduction

python

from datasets import load_dataset
from transformers import AutoProcessor, Gemma4ForConditionalGeneration
from llmcompressor import oneshot
from llmcompressor.modifiers.quantization import QuantizationModifier

model = Gemma4ForConditionalGeneration.from_pretrained(
    "Jiunsong/supergemma4-26b-abliterated-multimodal", dtype="auto"
)
processor = AutoProcessor.from_pretrained(
    "Jiunsong/supergemma4-26b-abliterated-multimodal"
)

recipe = QuantizationModifier(
    targets="Linear",
    scheme="NVFP4",
    ignore=["lm_head", "re:.*embed.*", "re:.*router", "re:.*vision_tower.*"],
)

ds = load_dataset("neuralmagic/calibration", name="LLM", split="train[:20]")

def preprocess_function(example):
    messages = [
        {"role": m["role"], "content": [{"type": "text", "text": m["content"]}]}
        for m in example["messages"]
    ]
    return processor.apply_chat_template(
        messages, return_tensors="pt", padding=False, truncation=True,
        max_length=8192, tokenize=True, add_special_tokens=False,
        return_dict=True, add_generation_prompt=False,
    )

ds = ds.map(preprocess_function, batched=False, remove_columns=ds.column_names)

import torch
def data_collator(batch):
    assert len(batch) == 1
    return {
        key: (torch.tensor(value) if key != "pixel_values"
              else torch.tensor(value, dtype=torch.bfloat16).squeeze(0))
        for key, value in batch[0].items()
    }

oneshot(
    model=model, recipe=recipe, dataset=ds,
    max_seq_length=8192, num_calibration_samples=20,
    data_collator=data_collator,
)

model.save_pretrained("output-NVFP4", save_compressed=True)
processor.save_pretrained("output-NVFP4")

Environment

Package	Version
torch	2.11.0+cu130
transformers	5.5.4
llmcompressor	0.1.dev (main @ 3084520)
compressed-tensors	0.15.1a20260414
safetensors	0.7.0
CUDA	13.0

Requirements

GPU: NVIDIA Blackwell (RTX 5090, RTX PRO 6000, B200, etc.) — NVFP4 requires SM 120
VRAM: ~16 GB minimum
Software: vLLM nightly (cu130 build), or any framework supporting compressed-tensors NVFP4

Notes

This is an abliterated (uncensored) model. Use responsibly.
Vision tower is kept in BF16 — multimodal capabilities are preserved at full precision.
NVFP4 is a Blackwell-specific format. This checkpoint will not work on Ampere/Hopper GPUs.

Credits

Base model: Jiunsong (SuperGemma4 abliteration + enhancement)
Original model: Google DeepMind (Gemma 4)
Quantization tool: vllm-project/llm-compressor
Quantization method follows RedHatAI/gemma-4-26B-A4B-it-NVFP4 (proven path)

Support the Base Model Author

If you find this model useful, please consider supporting Jiunsong — the creator of SuperGemma4:

Ko-fi: https://ko-fi.com/jiunsong

sakamakismile/SuperGemma4-26B-Abliterated-Multimodal-NVFP4

SuperGemma4-26B-Abliterated-Multimodal-NVFP4