You can deploy this model and start earning money today!

ariacompute/
gemma-4-e4b-it_q326_channel

Name: gemma-4-e4b-it_q326_channel
Availability: InStock

Views

No views yet

Model Card for Gemma-4-E4B-IT (Aria Quant Bundle, q326_channel)

Model Details

Model Description

Gemma-4-E4B-IT is a ~4-billion-parameter instruction-tuned multimodal language model developed by Google, part of the Gemma 4 family. Its text backbone features hybrid attention (36 of 42 layers sliding-window linear attention + 6 standard full-attention layers), GeGLU activation, Grouped Query Attention (GQA, 2 KV heads for 8 query heads), per-layer input projections, and 128K native context length. Pre-trained on diverse web-scale corpora and aligned via instruction tuning + RLHF. This distribution is provided by Aria Compute as an aria-quant-bundle — a quantized package using Hadamard pre-processing + per-channel mixed-precision quantization (sensitive attention layers at 4-bit, remaining layers at ~3-bit). Optimized for CPU-only, on-device inference on mobile phones, edge devices, and single-board computers via the Aria Engine runtime. No GPU or cloud connection is required.

Developed by: Google
Quantized and distributed by: Aria Compute
Model type: Dense Transformer decoder-only (multimodal base: image/audio + text inputs, text outputs; this bundle ships the text backbone)
Language(s): English (primary), Chinese, and 30+ additional languages
License: Apache 2.0
Finetuned from model: google/gemma-4-e4b-it

Model Sources

Original Repository: google-gemma/gemma-4
Original Paper: Gemma 4 Technical Report
Finetuned from: google/gemma-4-e4b-it
Aria Compute Dashboard: ariacompute.com/dashboard/models
Aria Engine: ariacompute.com

Uses

Direct Use

This quantized bundle is intended for on-device, offline text-generation tasks on resource-constrained hardware, including:

On-device chat and conversational assistants
Real-time text completion and basic code snippet generation
Instruction-following tasks for mobile and IoT applications
Lightweight text embeddings for on-device retrieval and classification
Short-form summarization of notifications, messages, and local content
Local document analysis up to 32K context (chunked)

Target Devices

Platform	Runtime Memory	Feasibility
High-end smartphone (8 GB)	~3.8 GB	✅ Recommended
Mid-range smartphone (4–6 GB)	~3.8 GB	⚠️ Tight fit
Budget phone (2–3 GB)	~3.8 GB	❌
Raspberry Pi 5 / SBC (4–8 GB)	~3.8 GB	✅
IoT gateway (1–2 GB)	~3.8 GB	❌
Wearable (1 GB)	~3.8 GB	❌

Memory breakdown (q326_channel, at 4K context): ~3.6 GB quantized model weights (mmap) + ~80 MB KV cache + ~40 MB runtime overhead + ~80 MB per-channel metadata overhead ≈ ~3.8 GB.

Note: KV cache is compact thanks to GQA (2 KV heads for 8 query heads) and hybrid attention — 36 of 42 layers use sliding-window attention (window 512, KV bounded by the window), so only the 6 full-attention layers scale KV with context. Combined with tie-word-embeddings, this keeps 128K context practical on ~3.8 GB-class devices.

Out-of-Scope Use

Long-form creative writing (>2K tokens per generation)
Mathematical theorem proving or formal verification
Full program/application synthesis
Multimodal input (image/audio encoding pipeline is pending audit for this quantized bundle — text-only in this release)
Real-time audio/speech processing (use Aria speech models)
Safety-critical decision systems without human oversight
Tasks requiring factual precision beyond the model's ~4B parameter capacity

How to Get Started with the Model

Download from Aria Compute

Authenticated dashboard users can download the bundle via: https://ariacompute.com/dashboard/models

Quantization Recipe

This bundle uses a per-channel quantization recipe, one of several precision options in the Aria Compute lineup:

Component	Quantization Strategy	Details
Attention Q/K/V/O weights	4-bit	Sensitive layers kept at 4-bit, per-channel codebooks, Hadamard pre-processing
FFN gate/up/down weights	~3-bit	Per-channel codebooks, Hadamard pre-processing
RMSNorm weights	FP16	Preserved at full precision
Embedding table	FP16	Preserved at full precision (tie_word_embeddings: true)

Bundle size: ~3.6 GB (BF16 text backbone: ~8.0 GB)
Generation quality: Awaiting gen_quant_eval audit. The mixed-precision recipe keeps sensitive attention layers at 4-bit while compressing the rest to ~3-bit — the recommended quality-size trade-off in the Aria lineup. Per-channel codebooks preserve per-output-channel distribution characteristics. Formal quality benchmarks against FP16 and other Aria quant recipes are pending
Calibration-free: Hadamard pre-processing + per-channel quantization, no task-specific calibration data required
Other precision options: Also available for Gemma-4-E4B-IT: gemma-4-e4b-it_q8_channel (per-channel 8-bit, near-lossless) and gemma-4-e4b-it_q4 (uniform per-channel 4-bit, smallest bundle)

Model Architecture

Gemma-4-E4B-IT's text backbone employs a dense Transformer decoder with GeGLU activation, hybrid attention (sliding-window linear attention + standard full attention), GQA, and per-layer input projections (256-dim layer input → 2,560 hidden):

Parameter	Gemma 4 E4B
Layers	42
Hidden size	2,560
Layer input size	256
FFN intermediate size	10,240
Attention heads (Query)	8
Attention heads (KV)	2 (GQA)
Head dimension	256 (global 512)
Full-attention layers	6
Sliding-window layers	36
Sliding window	512
Activation	GeGLU (gelu_pytorch_tanh)
Position encoding	RoPE (full-attn θ = 1,000,000, partial rotary 0.25; sliding θ = 10,000)
Normalization	RMSNorm (pre-norm)
Vocabulary size	262,144
Max context length	131,072 (128K)

Design highlights (Gemma 4 family):

Hybrid attention: 36/42 layers use sliding-window attention (window 512); 6/42 layers use standard full softmax attention — dense-attention KV cost is confined to a few layers
GQA (Grouped Query Attention): 2 KV heads serving 8 query heads — KV Cache memory is 4× smaller than full attention, enabling 128K context on edge devices
Per-layer input projections: a 256-dim layer-input embedding is projected to the 2,560-dim hidden state at each layer — compact embedding table with shared layer-input processing
GeGLU activation: GELU with tanh approximation gating, efficient for on-device inference
RoPE position encoding: 1M base frequency for full-attention layers (partial rotary factor 0.25) with 10K base for sliding-window layers, supporting 128K context
RMSNorm pre-normalization: Lightweight normalization before each sub-layer
Final logit softcapping: Output logits capped at ±30.0 for training stability
Tied embeddings: tie_word_embeddings: true — input and output embeddings share weights, reducing footprint

Bias, Risks, and Limitations

Limitations

Reasoning depth: Multi-step logical reasoning is limited for a 4B-class model. Verify outputs in high-stakes scenarios; consider larger Gemma 4 variants for reasoning tasks.
Mathematics: Simple arithmetic may be attempted but is unreliable. Advanced quantitative reasoning is out of scope. Use larger models for mathematical tasks.
Code generation: Capable of single-line completions and basic snippets; unreliable for multi-line code or structured programs.
Factual knowledge: Moderate world knowledge due to ~4B parameter scale. Always verify factual claims against authoritative sources. This model is better suited for instruction-following and general text tasks than encyclopedic knowledge retrieval.
Instruction following: Handles simple single-constraint instructions. Complex multi-constraint prompts may cause degradation, especially at longer contexts.
Quantization drift: Mixed-precision quantization (~3-bit FFN, 4-bit attention) may exhibit generation drift versus FP16, especially on ambiguous or open-ended prompts. For higher fidelity, use gemma-4-e4b-it_q8_channel (per-channel 8-bit). For the smallest bundle, see gemma-4-e4b-it_q4 (uniform 4-bit).

Bias and Risks

Bias: As with all large language models trained on web-scale data, Gemma may reflect societal biases present in its training corpus. Evaluate outputs before deployment in sensitive domains (hiring, healthcare, law).
Toxicity: The instruction-tuned model has been safety-aligned with RLHF. However, no safety filter is exhaustive. Consider an additional output classifier in high-risk environments.
Hallucination: May generate plausible-sounding but factually incorrect information. Implement output verification for critical applications. Hallucination risk is elevated for smaller models due to limited memorization capacity.
Dual-use risk: Text-generation capabilities could be misused for spam, disinformation, or impersonation. Deploy responsibly and in accordance with the Apache 2.0 license terms.

Recommendations

Users (both direct and downstream) should be made aware of the above risks, biases, limitations, and constraints of the model. We recommend:

Adding a lightweight output safety classifier for user-facing deployments
Verifying factual claims with external knowledge bases
Not using the model for high-stakes decisions without human review
Considering larger Gemma 4 variants for tasks requiring stronger reasoning or factual recall (note: E4B is already a mid-size variant)

ariacompute/gemma-4-e4b-it_q326_channel