Gemma 4 - LiteRTLM (On-Device Inference)
This repository contains the gemma-4-E2B-it-ultra-uncensored-heretic model converted into the LiteRTLM format for optimized on-device inference using Google's MediaPipe and LiteRT (formerly TensorFlow Lite).
Available Quantizations
Two quantization variants are provided to balance memory footprint and performance:
INT4 Quantization (gemma_model_int4_litertlm/model.litertlm)
Recipe: dynamic_wi4_afp32
Size: ~2.5 GB
Best for: Mobile devices (Android/iOS) with strict memory limits and edge computing hardware.
Conversion Details
This model were exported from the original PyTorch safetensors format using the litert-torch package.
The externalize_embedder=True flag was used to support the Gemma 4 architecture.
Target runtime: LiteRT (TFLite) via MediaPipe.
