Gemma 4 E2B — Abliterated LiteRT
An abliterated (uncensored) version of Google's
Gemma 4 E2B Instruct , converted to LiteRT format (
.litertlm) for on-device inference.
Capabilities
Inherits all capabilities of the base Gemma 4 E2B model:
Text generation — instruction-tuned chatVision — image understandingAudio — audio input processingThinking — extended reasoningTool use — function/tool calling
Refusal behaviours have been removed via abliteration (see below). The model will respond to requests that the base model would decline.
Files
File Size Description model.litertlm4.9 GB Full LiteRT package (text + vision + embedder + tokenizer)
The package contains INT8 weight-quantized TFLite subgraphs (4x compression from original BF16 weights):
Component Quantized size Text model (prefill/decode) 2.14 GB Per-layer embedder 2.19 GB Embedder 387 MB Vision encoder 163 MB Vision adapter 1.2 MB
Built with --cache-length 1024 and --prefill-lengths 256.
Usage
LiteRT CLI
1 pip install litert-cli-nightly [ lm ]
2 litert lm run model.litertlm
LiteRT-LM SDK (Android / iOS)
Drop model.litertlm into your app and load it with the LiteRT-LM LlmInference API:
1 // Android (Kotlin)
2 val options = LlmInference . LlmInferenceOptions . builder ( )
3 . setModelPath ( "/path/to/model.litertlm" )
4 . setMaxTokens ( 1024 )
5 . build ( )
6 val inference = LlmInference . createFromOptions ( context , options )
7 val result = inference . generateResponse ( "Hello!" )
See the
LiteRT-LM documentation for full SDK usage.
How This Was Made
1. Abliteration
The refusal direction was identified and removed from the model weights using the
FailSpy abliterator methodology:
Forward passes were run on 15 harmful and 15 harmless prompts through the base model
The mean difference in hidden states at layer 17 (the probe layer, mid-model) was computed and normalised to produce the refusal direction
That direction was projected out of the o_proj and down_proj weight matrices in layers 11–22 (the middle third of the model's 35 layers) using:
W_new = W - outer(r, r @ W)where r is the unit refusal direction. This removes the model's ability to activate refusal behaviour without degrading general capability.
2. LiteRT Conversion
The abliterated HuggingFace checkpoint was converted using
litert-cli-nightly:
1 litert convert ./gemma-4-e2b-abliterated \
2 --output ./gemma-4-e2b-abliterated-litert \
3 --quantize weight_only_wi8_afp32 \
4 --cache-length 1024
This uses litert-torch-nightly under the hood, which applies the Gemma4-specific export pipeline (image_text_to_text task, vision encoder export, externalized embedder).
Limitations
Cache length: Built with --cache-length 1024. Context beyond ~1024 tokens may degrade or not be supported depending on the runtime.Prefill length: Fixed at 256 tokens. Prompts longer than this will be chunked at runtime.No RLHF safety: This model has no content filters. Use responsibly.Quantization: INT8 weight-only quantization introduces minor quality loss vs the BF16 original.
Base Model
google/gemma-4-e2b-it — subject to the
Gemma Terms of Use . Usage of this model is also subject to those terms.
