DiffusionGemma 26B E38 — Abliterated NVFP4
A deployment-oriented NVFP4 release of E38 DiffusionGemma 26B, optimized for fast complete-response generation on NVIDIA Blackwell.
E38 NVFP4 reduces the E38 BF16 checkpoint from:
while preserving the measured E38 capability profile and enabling native packed NVFP4 inference.
This release focuses on:
- fast complete-response generation
- native NVFP4 execution
- Blackwell optimization
- short-response and voice-assistant use
- multimodal capability retention
- memory efficiency
- high serving throughput
Hands-on use also exposed a practical weakness in DiffusionGemma:
occasional grammar, lexical, repetition, and finalization errors
Controlled testing later showed that the same broad error class also exists in the original Base DiffusionGemma.
Current research is testing whether those errors can be corrected at inference time with minimal additional latency.
The experimental grammar-recovery system described below is NOT included in this downloadable checkpoint.
If the work continues to validate, it may become a separate E38 finalization / grammar-recovery branch.
Highlights
| Metric | Result |
|---|
| Checkpoint size | 18.86 GB |
| BF16 parent size | 51.68 GB |
| Size reduction | 63.5% |
| Relative size | 2.74× smaller |
| 48-step Quality-Max | 660.44 tok/s |
| 16-step single stream | 827.28 tok/s |
| Concurrency-8 aggregate | 1,053.64 tok/s |
| Recommended short mode | 128 canvas / 16-step cap |
| Recommended short range | 1–64 output tokens |
| Aligned NVFP4 validation | 137/200 — 68.5% |
| Target refusals | 0/402 |
| Benign false refusals | 0/249 |
| Matched multimodal validation | 20/20 |
| Matched multi-turn generations | 24/24 |
Recommended Inference Modes
Instant / Short Responses
1canvas_length = 128
2max_denoising_steps = 16
3
4t_max = 0.80
5t_min = 0.40
6entropy_bound = 0.1
7confidence_threshold = 0.005
8stability_threshold = 1
9adaptive_stopping = true
Recommended for:
- short conversational replies
- voice assistants
- Instant Full-Text TTS
- acknowledgements
- short factual answers
- concise explanations
Recommended output range:
approximately 1–64 tokens
Quality / General Responses
1canvas_length = 256
2max_denoising_steps = 48
3
4t_max = 0.80
5t_min = 0.40
6entropy_bound = 0.1
7confidence_threshold = 0.005
8stability_threshold = 1
9adaptive_stopping = true
10long_form_capacity = 1280
Recommended for:
- general chat
- longer responses
- reasoning
- coding
- detailed explanations
- creative writing
- long-form generation
Short-Response Validation
A large matched study compared 128/16 with 256/48 using:
- 500 frozen prompts
- 15 voice-assistant categories
- 4 matched seeds
- 2,000 matched pairs
- 4,000 total generations
Blind manual review completed:
representing:
3,700 manually reviewed responses
Within the intended 1–64-token range:
| Result | 128/16 | 256/48 |
|---|
| In-range responses reviewed | 1,705 | 1,690 |
| Strictly clean responses | 1,678 | 1,638 |
| Strictly clean rate | 98.42% | 96.92% |
| Confirmed language issues | 27 | 52 |
| Severe degeneration | 0 | 0 |
| Truncation | 0 | 0 |
Among 1,633 fully paired in-range responses:
128/16: 98.65% strictly clean
256/48: 97.00% strictly clean
Difference:
+1.65 percentage points in favor of 128/16
Paired bootstrap 95% CI:
approximately +0.67 to +2.63 percentage points
Exact McNemar test:
This was a separate short-response validation and should not be confused with the newer grammar-recovery experiment described below.
Current recommendation:
| Use Case | Recommended Mode |
|---|
| Short complete responses / voice / 1–64 tokens | 128/16 |
| Longer responses / reasoning / general chat | 256/48 |
128/16 is not currently recommended as the general long-form mode.
Grammar and Lexical Reliability
Hands-on testing made occasional language-finalization problems noticeable during normal use.
Observed issues include:
- grammar mistakes
- malformed words
- duplicated words
- agreement errors
- broken sentence endings
- lexical artifacts
- rare local degeneration
A controlled 432-generation language study compared:
- Base DiffusionGemma BF16
- E38 BF16
- E38 NVFP4
| Model | Grammar Errors /10k ↓ | Lexical Artifacts /10k ↓ |
|---|
| Base BF16 | 4.059 | 2.243 |
| E38 BF16 | 5.479 | 1.865 |
| E38 NVFP4 | 3.236 | 1.387 |
The important finding is not that E38 was proven better or worse than Base.
It is that:
the same broad class of grammar and lexical errors also exists in the original Base DiffusionGemma.
The measured E38 BF16 grammar difference versus Base was not statistically established:
- difference: +1.420 errors /10k words
- 95% CI: −0.674 to +3.541
- McNemar p: 0.560
The evidence therefore does not support a simple conclusion that E38 or NVFP4 introduced the grammar problem.
Diffusion-Finalization Findings
Error analysis found that language failures were approximately:
3.01× more concentrated in the final quarter of the 256-token canvas
than in the first quarter.
Error-prone positions also showed approximately:
- +0.0504 higher final entropy
- −0.0418 lower top-1 / top-2 confidence margin
relative to matched clean positions.
Simply increasing the denoising budget did not reliably remove the problem.
For short responses, 128/16 actually performed better than 256/48 in blind testing.
For longer responses, however, reducing the diffusion budget too far increased repetition and degeneration.
The current working hypothesis is:
DiffusionGemma may sometimes contain the correct language capability but fail to cleanly finalize it during the original diffusion trajectory.
This remains a research hypothesis and is not claimed as a proven causal mechanism.
Current Grammar-Recovery Research
Experimental — Not Included in This Model
The following work is being developed separately from the currently downloadable E38 NVFP4 checkpoint.
The current E38 NVFP4 model does not include this proofreading/finalization system.
No model weights are changed.
The experimental inference path is intentionally simple:
1Normal E38 generation
2 ↓
3Complete response
4 ↓
5One quick proofreading canvas
6 ↓
7Final response
The review pass is not asked to answer the original prompt again.
It only checks the completed response for obvious:
- grammar mistakes
- malformed wording
- repetition
- agreement errors
- broken sentence construction
- malformed endings
- local finalization corruption
The objective is to correct these issues while adding as little latency as possible.
Current Experimental Testing
The latest grammar-recovery study used:
250 natural short responses
plus:
25 semantic-preservation stress cases
Results:
- generation/review failures: 0
- one-canvas reviews: 250/250 — 100%
- maximum reviewed length: 139 tokens
Quality
| Metric | Original 128/16 | Experimental Review |
|---|
| Strictly clean | 94.0% | 99.2% |
| Grammar error | 3.6% | 0% |
| Lexical error | 0.8% | 0% |
| Repetition | 0.4% | 0% |
99.2% is a current experimental testing result.
It is not a quality claim for the E38 NVFP4 checkpoint available on this page.
Among 15 original responses containing confirmed language errors:
13/15 — 86.7% were fully repaired
Total improvement:
Paired clean-rate improvement:
95% CI:
McNemar exact:
Clean-Response Preservation
Among 235 original responses already judged clean:
- 96.2% were exactly unchanged
- 1.3% received harmless equivalent wording changes
- 2.6% received useful polish
- 0% unnecessary rewrite
- 0% minor downgrade
- 0% language damage
- 0% semantic/factual drift
Observed quality-preservation rate:
Observed true-downgrade rate:
Natural-set semantic drift:
Separate semantic-stress result:
These results are encouraging, but the current study used one primary adjudicator and broader independent evaluation is still warranted.
Blind Comparison
Across the 250 natural responses:
- review wins: 19
- ties: 229
- original wins: 0
- both bad: 2
Exact sign-test:
Again, these results belong to the experimental review system, not the currently released model.
Current Latency Cost
The quality improvement is not free.
Median additional review latency:
P90:
Median actual review iterations:
The established generation-only short-response reference is approximately:
The current production-reference estimate for generation plus review is approximately:
So the experimental review currently adds approximately:
for the observed improvement from:
94.0% → 99.2% strictly clean
Current Optimization Goal
The goal now is not simply to prove that proofreading works.
It is:
correct grammar and lexical finalization errors with the smallest possible latency increase
Current work is focused on:
- integrating review directly into the inference runtime
- reusing tokenizer state
- reusing GPU-resident tensors and buffers
- reusing compiled execution paths
- reusing CUDA graphs where possible
- reducing review setup overhead
- reducing review canvas size where safe
- reducing review iterations where quality allows
- improving adaptive stopping
- conditionally reviewing only responses likely to contain an error
Because most responses are already clean, conditional review may provide the largest reduction in average added latency.
A possible future architecture is:
1Normal generation
2 ↓
3Cheap confidence / anomaly check
4 ├── likely clean → return response
5 │
6 └── suspicious
7 ↓
8 quick review canvas
9 ↓
10 final response
The target is to preserve as much of the current experimental 99.2% strictly-clean result as possible while reducing the current ~250 ms review overhead.
Possible E38 Finalization Branch
If integration, latency optimization, and further validation continue to succeed, this work may become a separate E38 inference branch.
Conceptually:
1E38 NVFP4
2│
3├── Current release
4│ └── normal inference
5│
6└── Possible E38 Finalization branch
7 └── generation
8 ↓
9 quick review
10 ↓
11 final response
Important:
The current downloadable E38 NVFP4 model does not contain this review system.
No separate grammar-recovery checkpoint has been released yet.
This work currently changes the inference path, not the E38 weights.
Performance
Measured on one:
NVIDIA RTX PRO 6000 Blackwell Workstation Edition
| Operating Point | Throughput |
|---|
| 48-step Quality-Max | 660.44 tok/s |
| 16-step single stream | 827.28 tok/s |
| 16-step concurrency-8 aggregate | 1,053.64 tok/s |
Runtime stack:
- native packed NVFP4 routed experts
- vLLM V2 / 0.27.1
- compiled execution
- CUDA graphs
- FULL_AND_PIECEWISE
- FlashInfer CUTLASS NVFP4 MoE
- FlashInfer autotuning
- Triton attention
- FP8 e4m3 KV
- CUDA 13
- SM120
Important:
1,053.64 tok/s is concurrency-8 aggregate serving throughput.
It is not single-stream throughput.
NVFP4 Quality Preservation
Aligned validation:
| Evaluation | Base BF16 | E38 BF16 | E38 NVFP4 |
|---|
| Objective — 200 prompts | 128/200 — 64.0% | 134/200 — 67.0% | 137/200 — 68.5% |
| Regression subset — 100 | 62/100 | 58/100 | 64/100 |
| Multimodal | 20/20 | 20/20 | 20/20 |
| Multi-turn generations | 24/24 | 24/24 | 24/24 |
| Target refusal | 383/402 | 0/402 | 0/402 |
| Benign false refusal | 0/249 | 0/249 | 0/249 |
E38 BF16 versus E38 NVFP4:
Difference:
95% CI:
p-value:
The difference was not statistically significant.
NVFP4 preserved measured E38 BF16 quality.
The higher NVFP4 point estimate is not claimed as evidence that quantization inherently improved capability.
Public Benchmark Reference
The larger public benchmark suite was run on Base BF16 and E38 BF16.
| Benchmark | Base BF16 | E38 BF16 | Delta |
|---|
| IFEval | 67.10% | 64.70% | −2.40 pp |
| BBH | 71.62% | 73.96% | +2.34 pp |
| MuSR | 41.80% | 50.00% | +8.20 pp |
| MMLU-Pro | 49.61% | 51.57% | +1.96 pp |
| MATH Level 5 | 84.06% | 80.51% | −3.55 pp |
E38 is best described as:
a capability redistribution rather than a universally stronger checkpoint
The complete public benchmark suite has not been rerun directly on NVFP4.
MATH Level 5
Evaluation size:
| Model | Correct |
|---|
| Base BF16 | 1,113 / 1,324 — 84.06% |
| E38 BF16 | 1,066 / 1,324 — 80.51% |
Difference:
Statistics:
- 95% CI: −5.59 to −1.44 pp
- McNemar p: 0.00119
This is a real measured E38 BF16 regression.
The full benchmark has not been rerun directly on E38 NVFP4.
Memory Efficiency
| Model | Checkpoint Size |
|---|
| E38 BF16 | 51.68 GB |
| E38 NVFP4 | 18.86 GB |
Reduction:
Percentage reduction:
Relative size:
Precision Layout
NVFP4
Routed expert weights:
BF16 Retained
- attention
- dense non-expert MLP
- routers
- embeddings
- LM head
- vision components
- all 20 E38-modified tensors
Runtime KV
All 20 E38-modified tensors remain exact BF16.
Native NVFP4 Execution
Runtime validation confirmed native packed NVFP4 execution.
No repeated:
- BF16 routed-expert expansion
- FP4 expert repacking
- hidden full-precision expert conversion
was found in the hot inference path.
The checkpoint therefore provides real low-precision runtime execution rather than only reduced storage size.
E38 Architecture
E38 was selected from more than:
80 controlled candidate configurations
derived from:
google/diffusiongemma-26B-A4B-it
E38 modifies:
- language layers 7–16
attn.o_projmlp.down_proj- 20 language tensors
- 0 vision tensors
Measured modification magnitude:
- relative Frobenius difference:
0.0250308802854
- maximum absolute difference:
0.201904296875
E38 overlay SHA256:
9fe83b2fa2a3b7643f6dcc4a8360feb2cd96dcb9f70d215bd2e8f8f9d9d3044b
Frozen E38 selection hash:
cd69540ec547f2f5cf181d527e8366fe60b49df7cafde38b434c5e42eeeb6175
Current Status
| Area | Status |
|---|
| E38 BF16 | Complete |
| E38 NVFP4 | Validated |
| NVFP4 integrity | PASS |
| Native packed NVFP4 inference | Validated |
| 128/16 short mode | Validated |
| 256/48 general mode | Validated |
| Grammar/finalization issue | Active research target |
| Same broad error class in Base | Observed |
| Experimental review result | Current testing: 99.2% strictly clean |
| Experimental full-fix rate | 13/15 — 86.7% |
| Experimental natural semantic drift | 0/250 observed |
| Experimental semantic stress | 25/25 preserved |
| Experimental one-canvas rate | 250/250 |
| Experimental added review latency | 249.9 ms median |
| Grammar-recovery system in this model | No |
| Possible separate E38 finalization branch | Under consideration |
| Review latency optimization | In progress |
Known Limitations
This model remains experimental.
- E38 intentionally has substantially reduced refusal behavior relative to upstream.
- E38 BF16 showed a statistically significant MATH Level 5 regression.
- The complete public benchmark suite has not been rerun directly on NVFP4.
- Diffusion inference remains sensitive to canvas size and denoising configuration.
- Occasional grammar and lexical errors still occur in the currently released model.
- The diffusion-finalization explanation remains a working hypothesis.
- The 99.2% strictly-clean result is from current experimental review testing, not the downloadable model by itself.
- The experimental review/finalization path is not included in this checkpoint.
- The current expanded review study used one primary adjudicator.
- The experimental review currently adds approximately 250 ms median latency.
- The review path is currently being integrated and optimized.
- A separate E38 finalization / grammar-recovery branch may be created if further testing continues to succeed.
- 128/16 is intended for short complete responses and is not recommended as the general long-form mode.
Safety and Behavior
E38 intentionally retains substantially reduced refusal behavior.
| Model | Target Refusal | Prompt-Majority Refusal | Benign False Refusal |
|---|
| Base BF16 | 383/402 — 95.27% | 127/134 — 94.78% | 0/249 |
| E38 BF16 | 0/402 | 0/134 | 0/249 |
| E38 NVFP4 | 0/402 | 0/134 | 0/249 |
This checkpoint should not be interpreted as preserving the original refusal behavior of the upstream model.
Users should evaluate safeguards appropriate for their intended application.
Reproducibility
Original Upstream
google/diffusiongemma-26B-A4B-it
Pinned revision:
f7f5b7f5fa82ffc52addd066915886d497f5517b
E38 BF16 Parent
Goodoldjam/DiffusionGemma-26B-E38-Abliterated-BF16
E38 Overlay SHA256
9fe83b2fa2a3b7643f6dcc4a8360feb2cd96dcb9f70d215bd2e8f8f9d9d3044b
Frozen E38 Selection Hash
cd69540ec547f2f5cf181d527e8366fe60b49df7cafde38b434c5e42eeeb6175
Integrity
Validated:
- 12/12 artifacts PASS
- E38 overlay hash PASS
- all 20 E38-modified tensors remain BF16
- quantization layout unchanged
- vision tensors unchanged
- protected BF16 paths unchanged
Intended Use
This checkpoint is intended for experimentation with:
- low-latency local chat
- short complete-response generation
- Instant Full-Text TTS
- voice assistants
- diffusion-language-model research
- diffusion-budget optimization
- NVFP4 deployment
- NVIDIA Blackwell inference
- multimodal experimentation
- quantization research
- abliteration research
- serving-throughput research
- memory-efficiency research
Grammar/finalization research is ongoing, but the experimental review system is not part of this checkpoint.
Support
If this model or the research is useful, support helps fund:
- GPU compute
- controlled evaluation
- grammar and lexical testing
- manual review
- inference optimization
- model hosting
- continued public releases
The goal is to report both positive and negative results and avoid presenting experimental findings as released-model capabilities until they survive further validation.
License
Apache License 2.0.
This model is a quantized derivative of:
Goodoldjam/DiffusionGemma-26B-E38-Abliterated-BF16
which is derived from:
google/diffusiongemma-26B-A4B-it
