This model is a decensored/abliterated version of google/diffusiongemma-26B-A4B-it, created using the Heretic framework's Arbitrary-Rank Ablation (ARA) algorithm.

Abliteration Details

DiffusionGemma-26B relies heavily on deep Mixture-of-Expert (MoE) layers (up to 128 experts) and a block-diffusion paradigm. Ablating the deepest layers causes significant catastrophic degradation and KL divergence spikes in this specific architecture. To achieve optimal safety-vector suppression while preserving the model's complex structural integrity, the ablation was strictly capped at layer 20.

This model represents the Pareto-optimal Trial 144 from a 150-trial Optuna hyperparameter search, balancing minimal refusal rates with extremely low KL divergence.

ARA Hyperparameters (Trial 144)

Target Layers: 1 to 20
Abliterated Components: attn.o_proj and mlp.down_proj
Preserve Good Behavior Weight: 0.8357
Steer Bad Behavior Weight: 0.0359
Overcorrect Relative Weight: 0.0418
KNN Neighbor Count: 11

Evaluation Metrics

Initial Refusals: 98 / 100 (Harmless Alpaca / Harmful Behaviors)
Final Refusals: 4 / 100
KL Divergence: 0.1106

Usage

This model behaves exactly like the base diffusiongemma-26B-A4B-it model but lacks synthetic refusal boundaries. It can be used via standard Transformers loading:

python

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "Umranz/diffusiongemma-26B-A4B-it-ara"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, device_map="auto")

Umranz/diffusiongemma-26B-A4B-it-abliteration

DiffusionGemma 26B - A4B IT (Arbitrary-Rank Ablation)

Abliteration Details

ARA Hyperparameters (Trial 144)

Evaluation Metrics

Usage

Umranz/
diffusiongemma-26B-A4B-it-abliteration