You can deploy this model and start earning money today!

jorch/
gemma-4-e4b-it-lm-4bit

Name: gemma-4-e4b-it-lm-4bit
Availability: InStock

Views

No views yet

Gemma 4 E4B — Text-Only MLX 4-bit

Text-only extraction of Google's Gemma 4 E4B instruction-tuned model, converted to MLX format with 4-bit quantization.

This is the language model component only (no vision/audio towers), suitable for text chat on Apple Silicon devices including iPhones (8 GB RAM) and Macs.

Model Details

Spec	Value
Base model	`google/gemma-4-e4b-it`
Architecture	Gemma 4 with Per-Layer Embeddings (PLE)
Parameters	8B total, 4.5B active
Hidden size	2560
Layers	42
Attention heads	8 (2 KV heads)
Vocab size	262,144
Context window	128K tokens
Quantization	4-bit affine (group size 64)
Download size	3.9 GB
License	Apache 2.0

Conversion

Converted from google/gemma-4-e4b-it using mlx-lm with Gemma 4 support from ml-explore/mlx-lm PR #1095:

python -m mlx_lm convert --hf-path google/gemma-4-e4b-it -q --q-bits 4 --mlx-path ./gemma-4-e4b-it-lm-4bit

Usage with mlx-lm (Python)

python

from mlx_lm import load, generate

model, tokenizer = load("jorch/gemma-4-e4b-it-lm-4bit")
response = generate(model, tokenizer, prompt="What is machine learning?", verbose=True, max_tokens=512)

Usage with mlx-swift-lm (iOS/macOS)

Requires Gemma 4 text model support in mlx-swift-lm:

swift

import MLXLLM
import MLXLMCommon

let configuration = ModelConfiguration(id: "jorch/gemma-4-e4b-it-lm-4bit")
let container = try await LLMModelFactory.shared.loadContainer(configuration: configuration)

Why text-only?

The mlx-community/gemma-4-e4b-it-4bit model was converted with mlx-vlm and includes vision/audio towers. That conversion is missing critical text model weights (norm, per_layer_projection_norm, per_layer_model_projection), making it incompatible with text-only LLM loaders.

This conversion uses mlx-lm which properly extracts and preserves all text model weight keys.

E4B vs E2B

	E2B	E4B
Active params	2.3B	4.5B
Download	2.5 GB	3.9 GB
Quality	Good for everyday chat	Noticeably better reasoning
Speed	Faster	Slightly slower
iPhone 8 GB	Comfortable	Fits with headroom

Built for SoloAI

This model powers SoloAI — a privacy-first iOS chat app that runs Gemma 4 entirely on-device.