MNN format BF16 full-precision conversion of
TrevorJS/gemma-4-E2B-it-uncensored, compatible with the
MNN inference engine. No quantization applied — preserves original model quality.
1# Clone MNN source
2git clone https://github.com/alibaba/MNN.git
3
4# Build (CPU backend)
5cd MNN
6mkdir build && cd build
7cmake .. -DMNN_LOW_MEMORY=true \
8 -DMNN_CPU_WEIGHT_DEQUANT_GEMM=true \
9 -DMNN_BUILD_LLM=true \
10 -DMNN_SUPPORT_TRANSFORMER_FUSE=true
11make -j$(nproc)
12
13# Run inference
14echo "Hello, who are you?" > prompt.txt
15./llm_demo /path/to/config.json prompt.txt
A custom Android APK with Gemma 4 support is available in the companion GitHub repository:
Download the pre-built APK directly:
📱 app-standard-release-tiggy-gemma4.apk (35 MB)
Download the model folder to your device and import it via "Add Local Model".
1{
2 "llm_model": "llm.mnn",
3 "llm_weight": "llm.mnn.weight",
4 "tokenizer_file": "tokenizer.mtok",
5 "backend_type": "cpu",
6 "thread_num": 4,
7 "precision": "low",
8 "memory": "low"
9}
1cd MNN/transformers/llm/export
2python3 llmexport.py \
3 --path TrevorJS/gemma-4-E2B-it-uncensored \
4 --export mnn \
5 --quant_bit 16 \
6 --embed_bit 16 \
7 --dst_path ./output
Usage of Gemma models is also subject to the
Gemma Terms of Use.
