DiffusionGemma 26B E38 — Abliterated BF16
E38 is a deeply tested middle-layer abliteration of Google's DiffusionGemma 26B A4B model.
This BF16 checkpoint is the frozen high-precision E38 reference model used to create the deployment-oriented E38 NVIDIA-style NVFP4 release.
🚀 Current Deployment Model
The current deployment and active performance-development model is:
If this model is useful to you, please consider helping fund the next round of testing and optimization.
The goal is not simply throughput.
The goal is maximum practical throughput while maintaining the validated quality profile.
The performance work is not focused on obtaining the largest possible tokens-per-second number by stripping away diffusion refinement.
The target is:
zero measurable quality regression while pushing throughput as far as the hardware and runtime will allow
while preserving:
- reasoning quality
- instruction following
- multimodal capability
- long-form stability
- lexical reliability
- structured-output behavior
- E38 behavior
- the deeper quality-oriented diffusion configuration
Support directly helps fund:
- GPU compute
- throughput optimization
- runtime profiling
- larger validation runs
- deep grammar research
- diffusion-trajectory analysis
- benchmark reruns
- capability-retention testing
- model hosting
- continued public releases
The NVFP4 derivative already:
- reduces checkpoint size from 51.68 GB → 18.86 GB
- preserves all 20 E38-modified tensors in exact BF16
- preserves the E38 refusal profile under aligned testing
- preserves measured E38 quality
- substantially improves inference performance
- provides the platform for current quality-preserving throughput research
🔥 Current Testing — Quality and Speed Together
The validated quality-oriented configuration uses:
t_max = 0.80t_min = 0.40- 48 maximum denoising steps per 256-token canvas
entropy_bound = 0.1confidence_threshold = 0.005stability_threshold = 1- adaptive stopping enabled
- 256-token diffusion canvas
- 1280-token long-form capacity
This configuration was retained after a:
2,400-generation prospective inference-optimization study
because it provided the strongest validated overall balance of:
- language quality
- capability
- stability
- long-form behavior
- performance
Current development is attempting to make this quality target substantially faster.
⚠️ Google's Headline Speed Numbers Are Performance-Only Benchmarks
Google publicly reports DiffusionGemma performance of approximately:
and:
with the joint Google DeepMind / vLLM H100 result reported at:
Those numbers demonstrate that DiffusionGemma has substantial raw throughput potential.
However:
Those headline numbers come from a performance-only throughput benchmark. They are not a simultaneous quality-and-speed validation.
The published H100 reproduction uses:
- FP8 DiffusionGemma
- 16 maximum denoising steps
diffusion_entropy_bound = 0.1diffusion_confidence_threshold = 0.0- 1024 random input tokens
- 1024 requested output tokens
ignore-eos- 100 synthetic prompts
- concurrency = 1
Its published generation tok/s metric is also decode-oriented:
- the first 256-token diffusion canvas is counted in TTFT
- that first canvas is excluded from the reported generation-throughput calculation
This is a perfectly valid performance benchmark.
But it measures:
maximum generation throughput under a throughput-oriented configuration
It does not establish that the same throughput is achieved while preserving the deeper quality-oriented inference behavior used in this project.
48 Steps vs 16 Steps
The E38 NVFP4 quality configuration allows:
48 maximum denoising steps per canvas
The published performance benchmark uses:
16 maximum denoising steps per canvas
Therefore:
E38's quality-oriented configuration allows 3× the maximum denoising-step budget.
Adaptive stopping means the 48-step configuration does not necessarily execute all 48 steps on every canvas.
Therefore this does not imply exactly 3× realized compute.
It does mean the two configurations have substantially different maximum refinement budgets.
The research goal here is intentionally harder:
retain the deeper quality-oriented refinement budget and recover throughput through better execution.
Different Questions
The published performance benchmark asks:
How fast can DiffusionGemma generate under a throughput-focused configuration?
This project is asking:
How fast can DiffusionGemma generate while preserving the strongest validated quality configuration?
Those are not the same benchmark.
The current target is:
Quality and speed together.
🔥 Compiled / CUDA-Graph A/B Testing
Current NVFP4 runtime optimization has already produced a substantial matched eager-versus-compiled result.
The checkpoint and quality target remained unchanged.
| Metric | Eager | Compiled Graphs | Change |
|---|
| Overall throughput | 276.74 tok/s | 636.46 tok/s | 2.30× |
| Long-form throughput | 247.08 tok/s | 586.63 tok/s | 2.37× |
| Mean latency | 1.574 s | 0.690 s | ~56% lower |
| Objective accuracy | 20/30 | 21/30 | +1 |
| Automatic lexical events | 0 | 0 | No regression detected |
| Long-form degeneration | 1 | 0 | Favorable point result |
| Terminal parsing | 26/30 | 24/30 | -2 cases |
Matched overall throughput increased from:
Long-form throughput increased from:
representing:
2.30× higher overall throughput
and:
2.37× higher long-form throughput
while mean latency decreased from:
Most importantly:
The speedup came from execution optimization, not from reducing the 48-step quality-oriented denoising target.
Current Candidate Quality
Early quality signals are encouraging.
Objective accuracy
20/30 eager → 21/30 compiled
Automatic lexical events
Long-form degeneration
No obvious broad quality regression was detected in those areas.
One guard remains under investigation.
Structured / Terminal Parsing
26/30 eager → 24/30 compiled
That is a two-case difference.
Because the validation set is small, those cases may represent:
- stochastic output variation
- formatting differences
- parser / extraction artifacts
- or a genuine structured-output regression
The compiled configuration will not replace the validated runtime until that question is resolved.
Current Performance Status
ACTIVE VALIDATION — COMPILED RUNTIME NOT YET THE DEFAULT
Until the remaining structured-output guard is cleared:
292.59 tok/s remains the validated NVFP4 release throughput
while:
636.46 tok/s remains the current quality-preserving compiled candidate
The question is not simply:
The question is:
Is it faster without making the model worse?
⚡ Performance Status
| Configuration | Purpose | Throughput |
|---|
| E38 BF16 | High-precision reference | 166.97 tok/s |
| E38 NVFP4 — validated quality reference | Validated deployment | 292.59 tok/s |
| E38 NVFP4 — compiled quality candidate | Same quality target, optimized execution | 636.46 tok/s A/B |
| Compiled long-form candidate | Quality-oriented long-form generation | 586.63 tok/s A/B |
| Published RTX 5090 result | Performance-only external benchmark | 700+ tok/s |
| Published H100 result | Performance-only external benchmark | 1000+ tok/s |
The external numbers are not quality-equivalent comparisons.
The current priority is:
push the quality-oriented configuration substantially faster before considering reductions to its denoising budget
🎮 Consumer Blackwell
E38 NVFP4 should also be a strong fit for the GeForce RTX 5090.
The 18.86 GB checkpoint fits within 32 GB VRAM, making consumer Blackwell a practical deployment target.
Similar single-model performance is expected.
Current measured E38 NVFP4 performance figures are from the RTX PRO 6000 Blackwell. Direct RTX 5090 validation is still needed.
📊 Aligned Validation Results
The major validation results are aligned across:
- Base BF16
- E38 BF16
- E38 NVFP4
using matched evaluation sets.
| Evaluation | Base BF16 | E38 BF16 | E38 NVFP4 |
|---|
| Objective validation — 200/model | 128/200 — 64.0% | 134/200 — 67.0% | 137/200 — 68.5% |
| Target refusal — 402/model | 383/402 — 95.27% | 0/402 — 0.00% | 0/402 — 0.00% |
| Prompt-majority refusal — 134/model | 127/134 — 94.78% | 0/134 — 0.00% | 0/134 — 0.00% |
| Benign false refusal — 249/model | 0/249 — 0.00% | 0/249 — 0.00% | 0/249 — 0.00% |
| Benchmark regression subset — 100/model | 62/100 — 62% | 58/100 — 58% | 64/100 — 64% |
| Multimodal — 20/model | 20/20 | 20/20 | 20/20 |
| Multi-turn conversations — 8/model | 8/8 | 8/8 | 8/8 |
| Multi-turn generations — 24/model | 24/24 | 24/24 | 24/24 |
| Grammar errors /10k words ↓ | 4.059 | 5.479 | 3.236 |
| Lexical artifacts /10k ↓ | 2.243 | 1.865 | 1.387 |
| Mean latency ↓ | 2.487 s | 2.300 s | 0.962 s |
| Throughput ↑ | 162.15 tok/s | 166.97 tok/s | 292.59 tok/s |
| Checkpoint size ↓ | ~51.68 GB | 51.68 GB | 18.86 GB |
Aligned validation classification: PASS
E38 NVFP4 preserved the E38 behavior and capability profile under matched testing.
NVFP4 integrity validation confirmed:
- 12 / 12 artifact files matched
- all 20 E38-modified tensors remain exact BF16
- vision weights remain unchanged
🔵 What Is E38?
E38 is the selected configuration from a controlled Heretic-based abliteration search of:
google/diffusiongemma-26B-A4B-it
More than:
80 controlled candidate configurations
were evaluated before E38 was frozen.
The final E38 modification affects:
- Layers: 7–16
- Projections:
attn.o_proj + mlp.down_proj
- Modified language tensors: 20
- Vision tensors modified: 0
- Precision: BF16
The checkpoint was frozen after selection.
The E38 modification remained unchanged through subsequent NVFP4 conversion and validation.
Frozen E38 Overlay SHA256
9fe83b2fa2a3b7643f6dcc4a8360feb2cd96dcb9f70d215bd2e8f8f9d9d3044b
Frozen Selection Hash
cd69540ec547f2f5cf181d527e8366fe60b49df7cafde38b434c5e42eeeb6175
No E38.1 checkpoint was created.
🔓 Refusal Behavior
The defining E38 behavioral change is its substantial reduction in target refusal behavior.
The aligned full refusal comparison used:
- 134 unique target-refusal prompts
- 83 unique benign / false-refusal prompts
- 217 unique refusal-related prompts
- 402 target generations per model
- 249 benign generations per model
- 651 refusal-related generations per model
| Refusal Metric | Base BF16 | E38 BF16 | E38 NVFP4 |
|---|
| Target refusal | 383/402 — 95.27% | 0/402 — 0.00% | 0/402 — 0.00% |
| Prompt-majority refusal | 127/134 — 94.78% | 0/134 — 0.00% | 0/134 — 0.00% |
| False refusal | 0/249 — 0.00% | 0/249 — 0.00% | 0/249 — 0.00% |
The defining E38 refusal-suppression behavior was directly preserved after NVFP4 conversion.
E38 and E38 NVFP4 should therefore not be expected to retain the refusal behavior of the original upstream model.
🟢 Objective Quality
The frozen matched 200-prompt evaluation produced:
| Model | Correct | Accuracy |
|---|
| Base BF16 | 128 / 200 | 64.0% |
| E38 BF16 | 134 / 200 | 67.0% |
| E38 NVFP4 | 137 / 200 | 68.5% |
E38 NVFP4 versus E38 BF16:
- Difference: +1.5 percentage points
- 95% CI: -2.0 to +5.0 pp
- p = 0.5811
The difference was not statistically significant.
Validated conclusion: NVFP4 preserved measured E38 BF16 quality.
The higher NVFP4 point estimate is not claimed as proof that quantization makes the model more capable.
📈 Public Benchmark Results
The larger public benchmark suite was run on Base BF16 and E38 BF16.
| Benchmark | Base BF16 | E38 BF16 | Delta |
|---|
| IFEval | 67.10% | 64.70% | -2.40 pp |
| BBH | 71.62% | 73.96% | +2.34 pp |
| MuSR | 41.80% | 50.00% | +8.20 pp |
| MMLU-Pro | 49.61% | 51.57% | +1.96 pp |
| MATH Level 5 @1280 | 84.06% | 80.51% | -3.55 pp |
The full benchmark suites have not all been rerun directly on NVFP4.
A separate aligned 100-item Base/E38/NVFP4 subset found:
- Base BF16: 62/100
- E38 BF16: 58/100
- E38 NVFP4: 64/100
The smaller aligned set is treated as a:
regression detector, not evidence of NVFP4 superiority
E38 is best described as a:
capability redistribution
rather than a universally stronger checkpoint.
🧮 MATH Level 5
The full matched MATH Level 5 evaluation used:
at matched 1280-token generation capacity.
| Model | Correct | Accuracy |
|---|
| Base BF16 | 1113 / 1324 | 84.06% |
| E38 BF16 | 1066 / 1324 | 80.51% |
Difference:
with:
- 95% CI: -5.59 to -1.44 pp
- McNemar p: 0.00119
The mathematics regression is a real measured E38 BF16 tradeoff under the tested inference configuration.
The full MATH benchmark has not yet been rerun directly on NVFP4.
🖼️ Multimodal Evaluation
The aligned three-model multimodal subset produced:
| Model | Correct |
|---|
| Base BF16 | 20 / 20 |
| E38 BF16 | 20 / 20 |
| E38 NVFP4 | 20 / 20 |
A larger historical Base/E38 multimodal evaluation produced:
| Model | Aggregate Accuracy |
|---|
| Base BF16 | 88.67% |
| E38 BF16 | 88.67% |
No aggregate multimodal capability loss was observed.
The E38 tensor audit independently confirmed:
0 vision tensors modified
✍️ Long-Form Language Quality
A prospective 432-generation total long-form study directly compared:
- Base BF16
- E38 BF16
- E38 NVFP4
| Model | Grammar Errors /10k Words | Lexical Artifacts /10k |
|---|
| Base BF16 | 4.059 | 2.243 |
| E38 BF16 | 5.479 | 1.865 |
| E38 NVFP4 | 3.236 | 1.387 |
The original Base model showed the same general class of rare grammar and lexical problems before either E38 abliteration or NVFP4 quantization.
The E38-vs-Base grammar difference was statistically unresolved:
- Difference: +1.420 errors /10k words
- 95% CI: -0.674 to +3.541
- McNemar p: 0.560
Therefore:
The long-form study did not establish an E38-specific grammar regression.
NVFP4 also did not show evidence of language-quality degradation.
The lower NVFP4 point estimates are encouraging but are not claimed as proof that quantization improves grammar.
🔬 DiffusionGemma May Have More Potential
The current working hypothesis is:
DiffusionGemma may possess greater language-quality capability than current diffusion inference is consistently extracting from the model.
This remains a research hypothesis.
Current evidence includes:
- the same general grammar / lexical error class appearing in Base, E38, and NVFP4
- approximately 3.01× higher error rate in the final 25% of a diffusion canvas
- approximately +0.0504 higher final entropy at error-prone positions
- approximately -0.0418 lower top-1 / top-2 confidence margin at error-prone positions
- simple additional denoising failing to establish a better overall quality tradeoff
These findings suggest that some language failures may be connected to diffusion finalization rather than being purely encoded in the learned weights.
If that is true:
additional language capability may already exist inside DiffusionGemma and could potentially be recovered through better inference
That could potentially improve:
- grammar
- lexical reliability
- long-form consistency
- structured generation
- difficult reasoning trajectories
- output stability
- general language quality
without necessarily requiring:
- more parameters
- a larger checkpoint
- full-model retraining
The next major research phase is intended to test this directly.
⚡ NVFP4 Deployment Performance
The NVFP4 derivative reduces checkpoint size from:
The dedicated matched performance study measured:
| Metric | Base BF16 | E38 BF16 | E38 NVFP4 |
|---|
| Checkpoint size | 51.68 GB | 51.68 GB | 18.86 GB |
| Mean latency | 2.487 s | 2.300 s | 0.962 s |
| Throughput | 162.15 tok/s | 166.97 tok/s | 292.59 tok/s |
Compared with E38 BF16, NVFP4 achieved:
- 63.5% smaller checkpoint
- 58.2% lower measured mean latency
- 75.2% higher measured throughput
- approximately 1.75× measured throughput
The current compiled quality-preserving candidate has subsequently reached:
and:
on its matched A/B workload.
Those newer figures remain under validation.
🧠 NVFP4 Memory-System Efficiency
A likely contributor to NVFP4's performance is improved memory-system efficiency.
The smaller routed-expert representation may reduce:
- expert-weight traffic
- data movement per expert execution
- memory-bandwidth pressure
- cache pressure
and may improve effective cache behavior.
This remains a:
working performance hypothesis
The large compiled-graph A/B improvement also shows that a substantial amount of previous runtime cost came from execution overhead rather than the frozen model weights themselves.
Current optimization is separating:
- memory traffic
- compute
- kernel-launch overhead
- graph execution
- cache behavior
- scheduling
- synchronization
⚙️ Recommended Quality-Oriented Inference Configuration
The retained quality-oriented configuration is:
t_max = 0.80t_min = 0.40max_denoising_steps = 48entropy_bound = 0.1confidence_threshold = 0.005stability_threshold = 1- adaptive stopping enabled
canvas_length = 256long_form_capacity = 1280
This remains the:
The active NVFP4 performance work is attempting to execute this target substantially faster.
Compared with the published 16-step performance-only benchmark:
48 steps represents a 3× higher maximum denoising-step budget.
Adaptive stopping means this does not imply exactly 3× realized computation on every generation.
The goal remains:
maximum practical throughput with zero measurable quality regression
🧪 Testing Scope
The broader E38 research program includes:
- 80+ controlled abliteration candidates
- 3,300-generation sealed Base/E38 evaluation
- aligned full refusal testing
- 2,400-generation prospective inference study
- 1,324-problem MATH Level 5 evaluation
- 3,200-generation mathematics seed study
- 432-generation long-form language study
- 200-prompt matched Base/E38/NVFP4 validation
- full BBH, MuSR, MMLU-Pro, and IFEval
- multimodal testing
- multi-turn testing
- tensor-integrity auditing
- generation-capacity studies
- NVFP4 quality-retention testing
- dedicated latency / throughput testing
- diffusion-trajectory analysis
- active compiled / CUDA-graph throughput testing
The final aligned validation matched:
- 402 target-refusal generations per model
- 134 prompt-majority decisions per model
- 249 benign generations per model
- 100 benchmark-regression items per model
- 20 multimodal cases per model
- 8 multi-turn conversations per model
- 24 multi-turn generations per model
Negative findings and regressions are retained rather than discarded.
🧬 Model Lineage
Google DiffusionGemma 26B A4B BF16
↓
Heretic / controlled abliteration search
↓
E38 BF16 — this frozen validated reference
↓
E38 NVIDIA-style NVFP4 — validated deployment derivative
The BF16 checkpoint is intentionally retained so the effects of:
- abliteration
- quantization
can be evaluated independently.
🔐 Reproducibility
Upstream Base
google/diffusiongemma-26B-A4B-it
Pinned revision:
f7f5b7f5fa82ffc52addd066915886d497f5517b
E38 Integrity
Overlay SHA256:
9fe83b2fa2a3b7643f6dcc4a8360feb2cd96dcb9f70d215bd2e8f8f9d9d3044b
Selection hash:
cd69540ec547f2f5cf181d527e8366fe60b49df7cafde38b434c5e42eeeb6175
Measured E38 weight difference:
- Relative Frobenius difference:
0.0250308802854
- Maximum absolute weight difference:
0.201904296875
⚠️ Known Limitations
E38 is experimental and is not universally stronger than Base.
Known considerations include:
- substantially reduced refusal behavior
- weaker exact-format / strict-instruction performance
- lower matched MATH Level 5 performance
- occasional grammar and lexical errors also observed in Base DiffusionGemma
- stochastic diffusion inference
- sensitivity to output capacity and inference configuration
The current long-form evidence did not establish an E38-specific grammar regression.
The possibility that better diffusion inference could unlock additional language quality remains a:
Intended Use
This checkpoint is intended for:
- diffusion-LM research
- abliteration research
- capability evaluation
- multimodal experimentation
- quantization reference
- diffusion-inference research
- local model experimentation
- reproducible comparison with the NVFP4 derivative
For deployment and current performance development:
Safety and Behavior
This model intentionally has substantially reduced refusal behavior.
It may answer requests that the upstream DiffusionGemma checkpoint would refuse.
The aligned evaluation found:
| Model | Target Refusal |
|---|
| Base BF16 | 383 / 402 — 95.27% |
| E38 BF16 | 0 / 402 — 0.00% |
| E38 NVFP4 | 0 / 402 — 0.00% |
Benign false refusal was:
0 / 249 for Base BF16, E38 BF16, and E38 NVFP4
This release should therefore not be interpreted as retaining the original model's refusal or safety characteristics.
Users should independently evaluate behavior appropriate for their application.
Upstream Model
Original upstream model:
google/diffusiongemma-26B-A4B-it
Recommended deployment derivative:
License
Apache License 2.0.
This repository contains a modified derivative of:
google/diffusiongemma-26B-A4B-it
