You can deploy this model and start earning money today!

tiggychan/
gemma-4-E2B-it-uncensored-mnn-int8

Name: gemma-4-E2B-it-uncensored-mnn-int8
Availability: InStock

Views

No views yet

gemma-4-E2B-it-uncensored-mnn-int8

MNN format 8-bit quantized conversion of TrevorJS/gemma-4-E2B-it-uncensored, compatible with the MNN inference engine.

Model Information

Property	Value
Base Model	google/gemma-4-E2B-it
Uncensored Fork	TrevorJS/gemma-4-E2B-it-uncensored
Quantization	8-bit symmetric
Embedding Quantization	8-bit (PLE embeddings also quantized)
Total Size	~5.7 GB
LLM Weight	2.5 GB
PLE Embeddings	2.5 GB

File Structure

File	Size	Description
`llm.mnn`	2.2 MB	LLM model structure (converted from ONNX)
`llm.mnn.weight`	2.5 GB	LLM weights (8-bit quantized)
`ple_embeddings_int8.bin`	2.5 GB	Per-Layer Embeddings (8-bit quantized)
`visual.mnn` / `.weight`	217 MB	Vision tower (optional)
`audio.mnn` / `.weight`	564 MB	Audio tower (optional)
`tokenizer.mtok`	9.7 MB	MNN tokenizer
`config.json`	-	MNN runtime configuration
`llm_config.json`	-	LLM architecture configuration

Usage

Local Inference (CPU)

bash

# Clone MNN source
git clone https://github.com/alibaba/MNN.git

# Build (CPU backend)
cd MNN
mkdir build && cd build
cmake .. -DMNN_LOW_MEMORY=true \
         -DMNN_CPU_WEIGHT_DEQUANT_GEMM=true \
         -DMNN_BUILD_LLM=true \
         -DMNN_SUPPORT_TRANSFORMER_FUSE=true
make -j$(nproc)

# Run inference
echo "Hello, who are you?" > prompt.txt
./llm_demo /path/to/config.json prompt.txt

MNN Chat (Android)

A custom Android APK with Gemma 4 support is available in the companion GitHub repository:

https://github.com/Tiggy-Chan/gemma4-mnn-android

Download the pre-built APK directly: 📱 app-standard-release-tiggy-gemma4.apk (35 MB)

Or follow the build instructions to compile your own APK.

Download the model folder to your device and import it via "Add Local Model".

Configuration Example

json

{
    "llm_model": "llm.mnn",
    "llm_weight": "llm.mnn.weight",
    "tokenizer_file": "tokenizer.mtok",
    "backend_type": "cpu",
    "thread_num": 4,
    "precision": "low",
    "memory": "low"
}

Other Quantization Versions

Version	Size	Repo
INT4 HQQ	3.5 GB	gemma-4-E2B-it-uncensored-mnn-int4
INT8 (this)	5.7 GB	—
BF16 (full precision)	9.5 GB	gemma-4-E2B-it-uncensored-mnn-bf16

Conversion

Converted using MNN's llmexport.py: