Uncensored version of google/gemma-4-26B-A4B-it via biprojection abliteration with Expert-Granular Abliteration (EGA) for MoE expert weights.

Method

Norm-preserving biprojection (double Gram-Schmidt + double projection pass)
Winsorization 0.995, 400+400 prompts (mlabonne/harmful_behaviors + tatsu-lab/alpaca)
Dense pathway: o_proj + mlp.down_proj (60 weights)
MoE experts: all 128 experts per layer (3840 weights)
Total: 3900 weights modified across 30 layers
Precision: bf16, no quantization during abliteration

Usage

python

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("InfinimindCreations/gemma-4-26B-A4B-it-uncensored", torch_dtype="bfloat16", device_map="auto")
tokenizer = AutoTokenizer.from_pretrained("InfinimindCreations/gemma-4-26B-A4B-it-uncensored")

Credits

Andy Arditi et al. (abliteration)
TrevorJS (EGA method for MoE)
heretic-llm (biprojection reference)
Published by InfinimindCreations