TranslateGemma 4B IT — Android / Google AI Edge Bundles
On-device translation model for Android using Google AI Edge.
Converts google/translategemma-4b-it (55 languages, 4B params)
into formats that run locally on Android without internet or cloud APIs.
Google only publishes WebGPU-only TFLite files. This repo bridges that gap with CPU/XNNPACK-compatible .litertlm bundles (LiteRT-LM format) with embedded chat template, including a multimodal image+text Android bundle for PrivateAITranslate.
INT4 multimodal image+text bundle for structured text translation and LiteRT-LM vision image translation
Start with INT4 if you're unsure — it loads faster and uses less RAM. Use dynamic_int8 for better text translation quality.
The multimodal artifact is the bundle expected by PrivateAITranslate for image translation. If artifacts/int4-multimodal/translategemma-4b-it-int4-multimodal.litertlm is not present in this Hugging Face repo, the app download URL for that model will return 404.
Use standard ISO 639-1 language codes: en, he, fr, es, de, ar, zh, ja, ko, ru, pt, etc.
Plain text (no tags) is also accepted — the model will attempt translation based on context.
Image translation
The multimodal .litertlm bundle uses LiteRT-LM vision support for image+text translation flows in PrivateAITranslate. It is intended for structured text translation and image translation, not general image captioning.
Device Requirements
Spec
Minimum
RAM
6 GB free (INT4) / 8 GB free (dynamic_int8)
Storage
2 GB (INT4) / 4 GB (dynamic_int8)
OS
Android 10+
Runtime
Google AI Edge Gallery or LiteRT-LM SDK
CPU real-device validation passed on Pixel 10 and Galaxy S22 / S22 Ultra-class targets. GPU execution currently fails initialization in validation and should be treated as experimental and not validated.
What's Different From Google's Official Files
Google's official TranslateGemma TFLite files target WebGPU only — they don't work with MediaPipe LLM inference on Android CPU.
This repo's files use native conversion via litert-torch with a custom build_translategemma_4b() builder that:
Produces proper prefill + decode signatures with KV cache (required by LiteRT-LM)
Uses the correct architecture: 34 layers, 2560 dim, 8 heads, 4 KV heads, sliding-window + global every 6th layer
Fixes qkv_fused_interleaved=False (critical — wrong default caused garbage output in all early builds)
Handles the language_model. weight prefix in TranslateGemma's multimodal safetensors
Embeds a generic Jinja chat template for any language pair via <src>/<dst>/<text> tags
Conversion Scripts
The scripts/ folder contains the full conversion pipeline:
Script
Purpose
scripts/convert_translategemma_android.py
Single-quant conversion via litert-torch native strategy
scripts/bundle_litertlm.py
Bundle a TFLite + SentencePiece tokenizer into .litertlm with embedded Jinja template
Observed multimodal export peak RSS was about 73.9 GiB, so 96 GB is the practical floor once Python, model cache, filesystem cache, and conversion overhead are included. Use 128 GB when running multiple quantization attempts or keeping extra build artifacts.
Ideal Vast.ai image:
Vast.ai PyTorch image or an Ubuntu-based NVIDIA/PyTorch CUDA image, not a bare CUDA runtime image.
Python 3.12 with uv/pip, Git, Git LFS, Hugging Face CLI, and build tools available.
CUDA/PyTorch wheel support matching the rented GPU architecture; use CUDA 12.8+ PyTorch wheels on Blackwell GPUs.
96+ GB RAM and at least 80 GB free disk; 150+ GB disk is safer for source checkpoint, caches, exported TFLite files, and .litertlm bundles.
Persistent /workspace volume if the instance may be stopped/recycled before upload.
TranslateGemma supports 55 languages including Arabic, Chinese, French, German, Hebrew, Hindi, Japanese, Korean, Portuguese, Russian, Spanish, and more. See google/translategemma-4b-it for the full list.
