The bundle contains INT8 text prefill/decode, external and per-layer embedders, a Gemma 4 vision encoder, and a vision adapter. It was converted natively on a Linux ARM64 NVIDIA DGX Spark; no x86 emulation was used.

Conversion settings

LiteRT Torch task: image_text_to_text
Quantization: dynamic_wi8_afp32 for text and vision
Prefill signatures: 128, 256, and 512 tokens
KV cache: 4,096 tokens
Image soft tokens: 280
Audio encoder: not included

Usage

bash

litert-lm import \
  Huihui-gemma-4-E2B-it-abliterated.litertlm \
  huihui-gemma4-e2b

litert-lm run huihui-gemma4-e2b \
  --backend gpu \
  --vision-backend gpu \
  --attachment image.png \
  --prompt "Describe this image."

The resulting bundle is approximately 4.9 GiB. Text and image inference were both validated through the LiteRT-LM CLI and its OpenAI-compatible API.

Safety

The source is an abliterated model with substantially reduced refusal behavior. It can produce sensitive, controversial, or inappropriate content. Review its outputs and do not assume that it provides default safety guarantees.

edp1096/Huihui-gemma-4-E2B-it-abliterated-litert-lm

Huihui Gemma 4 E2B Abliterated LiteRT-LM

Conversion settings

Usage

Safety

edp1096/
Huihui-gemma-4-E2B-it-abliterated-litert-lm