⚠️ ALPHA RELEASE — EXPERIMENTAL ABLITERATION (compounded by NVFP4 noise)
This is the first known public abliteration of NVIDIA's NemotronH hybrid Mamba2 + Attention + MoE architecture, AND the first NVFP4 quantization of an abliterated variant of it. Both are research-grade. Expect degraded real-world chat behavior vs the base model:
Reasoning-loop non-completions — ~10/100 prompts in our internal bench triggered repeated phrases inside <think> blocks that never close, producing empty or repeating answers. Multi-turn chat amplifies this.
NVFP4 noise compounds the abliteration artifact — FP4 quantization noise can partially rebuild a small fraction of the orthogonalized refusal direction and worsens the reasoning-loop pattern. If you hit issues, the BF16 sibling is the more stable fallback.
Multi-turn conversation degradation — single-turn benches (the headline 99% → 16% refusal numbers below) are real, but sustained chat shows brokenness our automated benches did not catch. Treat the bench numbers as a refusal-pass-rate measurement, not a chat-quality measurement.
We are tracking these regressions and plan a v9 abliteration pass (lower-rank subspace, different layer selection, dedicated multi-turn probes) before this leaves alpha. Please open a discussion with concrete failure cases — they directly drive what v9 fixes.
NVFP4-quantized variant of AEON-7/Nemotron-3-Nano-Omni-AEON-Ultimate-Uncensored-BF16. The full abliteration method, validation results, prior-art credits, and capability-unlock pitch live in the BF16 model card — read that first if you haven't.
This repo packages the abliterated language model in NVFP4 (4-bit weights with FP8 per-block scales, NVIDIA's TensorRT-LLM / vLLM blockwise format) and ships the multimodal towers at full BF16. Total checkpoint: ~22 GB (vs 66 GB BF16 → 3.0× compression).
A purpose-built vLLM container image for serving on DGX Spark is published at ghcr.io/aeon-7/vllm-nemotron-omni-aeon-ultimate:v1. Full deployment guide (image, patches, docker-compose, troubleshooting, tuning knobs) lives at AEON-7/Nemotron-3-Nano-Omni-AEON-Ultimate-Uncensored on GitHub — see docs/dgx-spark-setup.md for the operator runbook.
Preserved at BF16: attention q/k/v/o_proj, MoE gates/routers, Mamba conv1d, lm_head, embeddings, all RMS norms, all multimodal weights (RADIO vision tower, Parakeet audio encoder, video / GUI / OCR fusion stacks, audio projection — 1,106 tensors total, byte-identical to BF16 source)
Refusal-rate verification (100-prompt scan, served via vLLM on DGX Spark)
Headline measurement: same 100-prompt random sample (seed=42) run against both the unmodified base model and this NVFP4 abliterated variant served via vLLM (max_new=1200 for both modes to eliminate length-truncation as a confound). Identical apostrophe-normalized heuristic. Full generations saved to JSON for reproducibility.
Mode
Base model
NVFP4 (this)
Reduction
enable_thinking=True (max_new=1200)
53/100 = 53% refusal
0/100 real refusals (10 reasoning-loop non-completions, see note)
−53 pp
enable_thinking=False (max_new=1200)
99/100 = 99% refusal
16/100 = 16% refusal
−83 pp
The 99% → 16% drop on thinking=False is the headline finding — a single closed-form linear projection on 2,998 weight tensors removes 83 percentage points of refusal behaviour, and the effect survives the modelopt NVFP4 quant + vLLM serve path. The thinking=True path drives real refusals to zero: every prompt that reaches a final answer produces structured, requested content.
On the 10 thinking=True non-completions: ten of the 100 prompts trigger pathological reasoning loops — the model gets stuck inside <think> repeating phrases ("We can also talk about...", "using airline crew passes, using airline crew tags, using airline crew passes...") and never closes the reasoning block. Bumping max_new further does not resolve this; the loop is unbounded on these specific prompts. These are failures to terminate generation, not refusals. We flag this as an artifact of post-hoc projection — removing the refusal direction can leave a small population of prompts where the model's natural "stop reasoning here" signal was anchored on the same direction. The remaining 90 prompts complete reasoning cleanly and comply.
Full generations + heuristic source for both runs:
Setup: vllm-nemotron-omni-aeon-ultimate:v1 container, 32k max-model-len, max-num-seqs=8, BF16 multimodal towers grafted in. Single-stream and concurrent-stream measured via the OpenAI streaming endpoint. Both backends benchmarked for transparency.
CUTLASS NVFP4 MoE backend (default in v1.1+ image)
Workload
TTFT
Single-stream decode
C=4 aggregate
C=8 aggregate
C=8 median TTFT
thinking=False (max_new=400)
68 ms
71.7 chunks/s
163.1 chunks/s
223.5 chunks/s
206 ms
thinking=True (max_new=600)
62 ms
71.2 chunks/s
175.5 chunks/s
242.6 chunks/s
164 ms
MARLIN NVFP4 MoE backend (opt-in via VLLM_TEST_FORCE_FP8_MARLIN=1)
Workload
TTFT
Single-stream decode
C=4 aggregate
C=8 aggregate
C=8 median TTFT
thinking=False (max_new=400)
64 ms
73.0 chunks/s
162.4 chunks/s
244.8 chunks/s
174 ms
thinking=True (max_new=600)
60 ms
72.8 chunks/s
182.4 chunks/s
259.7 chunks/s
158 ms
Tradeoff: CUTLASS gives slightly lower TTFT (especially at moderate concurrency, e.g. C=4 with thinking=True: 90ms CUTLASS vs 124ms MARLIN). MARLIN gives ~5–10% higher aggregate throughput at C=8. Linear ops use FlashInferCutlassNvFp4LinearKernel in both runs — only the MoE backend differs.
Default in our image is CUTLASS — interactive TTFT matters more than batch-throughput for reasoning workloads. Set VLLM_TEST_FORCE_FP8_MARLIN=1 if your workload is throughput-bound at high concurrency. Numbers are close enough that either is reasonable; pick the one that matches your latency-vs-throughput preference.
(c/s = streaming chunks per second — typically 1 token per chunk at this output style.)
What's quantized vs preserved
NVFP4 is applied selectively. The MAMBA_MOE_NVFP4_AGGRESSIVE_CFG config from modelopt deliberately preserves the components most sensitive to quantization noise on hybrid Mamba+MoE models:
QUANTIZED (NVFP4, ~19 GB → from ~64 GB at BF16):
✓ Mamba mixer.in_proj (× 30 Mamba layers)
✓ Mamba mixer.out_proj
✓ MoE routed experts: up_proj, down_proj (128 experts × 25 MoE layers)
✓ MoE shared experts: up_proj, down_proj
PRESERVED AT BF16 (~3 GB total):
✓ All attention q/k/v/o_proj (sm_120/121 kernels prefer BF16 here)
✓ MoE gates / routers (small, sensitive to scale, never quantized)
✓ Mamba conv1d (small per-layer state, kernel-bound at BF16)
✓ lm_head (avoids decode noise in last step)
✓ Embeddings + RMS norms (untouched on principle)
✓ Vision: RADIO encoder (byte-identical to BF16 source)
✓ Audio: Parakeet encoder (byte-identical to BF16 source)
✓ MM connector: mlp1, sound_projection (byte-identical)
Vision and audio inference are bit-exact identical to the BF16 model. The only path that's lossy under NVFP4 is the LM decode through Mamba state-write and MoE expert FFNs.
Hardware support
Hardware
Recommended
Notes
DGX Spark / GB10 (sm_121a, unified mem)
✅ This NVFP4 variant
Use ghcr.io/aeon-7/vllm-nemotron-omni-aeon-ultimate:v1. CUTLASS Linear + CUTLASS-MoE-by-default; flip to MARLIN MoE via VLLM_TEST_FORCE_FP8_MARLIN=1 for ~5-10% more aggregate throughput.
RTX PRO 6000 SE / RTX 5090 / B100 / B200
✅ This NVFP4 variant
Stock vLLM v0.20.0+ works directly. Pair with MTP-XS speculative decoding ({"method": "qwen3_5_mtp", "num_speculative_tokens": 3}) on dedicated-VRAM Blackwell.
H200 / H100
⚠️ Use BF16 instead
Hopper has FP8 but no NVFP4 native support; runtime falls back with negligible compression benefit.
On the DGX Spark's unified memory keep --gpu-memory-utilization at 0.6-0.7; above ~0.8 the shared CPU+GPU pool page-thrashes. Discrete-VRAM GPUs can run higher.
The -e VLLM_TEST_FORCE_FP8_MARLIN=0 flag selects the FLASHINFER_CUTLASS NVFP4 MoE backend — gives lower TTFT (especially at moderate concurrency, e.g. 90ms vs 124ms at C=4 with thinking=True) which matters more for interactive/chat workloads. The image's default is MARLIN (~5–10% higher aggregate throughput at C=8), which is preferred for batch/throughput-bound deployments. Both backends are valid; pick the one matching your latency-vs-throughput preference. Linear ops always use FlashInferCutlassNvFp4LinearKernel regardless.
This checkpoint follows the modelopt 0.43+ HF export schema with quantization_config.quant_method == "modelopt" and quant_algo == "NVFP4". TensorRT-LLM's HF importer reads this directly from config.json and hf_quant_config.json.
NVIDIA — base model Nemotron-3-Nano-Omni-30B-A3B-Reasoning-BF16, vLLM model class nano_nemotron_vl.NemotronH_Nano_VL_V2, nvidia-modelopt quantization framework.
User Responsibility & Arbitration Clause
By accessing, downloading, using, running inference on, fine-tuning, merging, quantizing, distributing, integrating, or otherwise interacting with this model, you acknowledge and agree to the following:
Sole Responsibility. You, the user, are solely and exclusively responsible for every prompt issued, every response produced, every downstream action taken in reliance on those responses, and any harm — direct, indirect, consequential, or otherwise — that results.
No Warranty. This model is provided strictly "AS IS", without warranty of any kind, express or implied, including but not limited to warranties of merchantability, fitness for a particular purpose, non-infringement, safety, alignment, factual accuracy, or legal compliance in any jurisdiction. No contributor, author, publisher, or hosting platform assumes liability of any kind for outputs or downstream use.
Legal Compliance. You are responsible for ensuring that your use complies with all applicable laws, regulations, terms of service, industry codes of conduct, professional ethical standards, and organizational policies in every jurisdiction in which you operate or in which your outputs may be received. The unaligned nature of this model does not grant you any legal authorization you did not already have.
Operational Safety Layer. An uncensored model is not a toy. You are expected to implement appropriate downstream safety layers proportionate to your deployment context, including but not limited to: input validation, output filtering, content moderation, audit logging, rate limiting, access controls, and human-in-the-loop review for high-risk workflows. A production deployment of this model without such layers is unsafe by construction and is not a supported use case.
Heightened Duty of Care. The absence of internal refusal behavior means the duty of care that would ordinarily rest partly with the model rests entirely with you. You are expected to exercise greater — not lesser — caution, forethought, and ethical discipline when operating this model. If you are uncertain whether your contemplated use is ethical, legal, or wise, the correct action is to not make the request.
No Endorsement of Outputs. The authors, contributors, and publishers do not endorse, adopt, or take responsibility for any specific output. Outputs are a stochastic function of the prompt, the weights, and the sampler state — not a statement of position by any human.
Arbitration. Any dispute, claim, or controversy arising out of or relating to the use of this model, its outputs, or this clause shall be resolved through binding individual arbitration under the rules of a mutually agreed arbitration body (or, absent agreement, the American Arbitration Association's Consumer Arbitration Rules), waiving any right to a jury trial, class action, representative action, or consolidated proceeding. Venue shall be the jurisdiction of the disputing party bringing the claim. Costs and attorneys' fees shall be allocated per the applicable arbitration rules. This clause does not expand, and where legally prohibited does not establish, any liability in the other direction; it limits how the user may proceed when alleging harm tied to their own use of this model.
Indemnification. You agree to indemnify, defend, and hold harmless the authors, contributors, and publishers of this model from and against any claims, damages, losses, liabilities, costs, and expenses (including reasonable attorneys' fees) arising from or related to your use of the model or your breach of this clause.
Severability. If any provision is held unenforceable in a given jurisdiction, the remaining provisions remain in full force, and the unenforceable provision is replaced by the closest enforceable equivalent consistent with the original intent.
Acceptance. Your use of this model constitutes your acceptance of this clause in full. If you do not accept, do not use the model.
This model is a tool with no opinions of its own. You supply the opinions. You supply the judgement. You supply the ethics. The outputs carry your fingerprints, not the model's.
