Model Card for Gemma-3n-E2B-IT (Aria Quant Bundle, q4)

Model Details

Model Description

• Developed by: Google
• Quantized and distributed by: Aria Compute
• Model type: Dense Transformer decoder-only (multimodal base: image/audio + text inputs, text outputs; this bundle ships the text backbone)
• Language(s): English (primary), Chinese, and 30+ additional languages
• License: Apache 2.0
• Finetuned from model: google/gemma-3n-e2b-it

Model Sources

• Original Repository: google-gemma/gemma-3n
• Original Paper: Gemma 3n Technical Report
• Aria Compute Dashboard: ariacompute.com/dashboard/models
• Aria Engine: ariacompute.com

Uses

Direct Use

• On-device chat and conversational assistants
• Real-time text completion and basic code snippet generation
• Instruction-following tasks for mobile and IoT applications
• Lightweight text embeddings for on-device retrieval and classification
• Short-form summarization of notifications, messages, and local content
• Local document analysis up to 32K context (chunked)

Target Devices

Note: KV cache is compact thanks to hybrid attention — 24 of 30 layers use sliding-window attention (window 512, KV bounded by the window), so only the 6 Laurel full-attention layers scale KV with context. Combined with GQA (2 KV heads), this keeps 32K context practical on ~2 GB-class devices.

Out-of-Scope Use

• Long-form creative writing (>2K tokens per generation)
• Mathematical theorem proving or formal verification
• Full program/application synthesis
• Multimodal input (image/audio encoding pipeline is pending audit for this quantized bundle — text-only in this release)
• Real-time audio/speech processing (use Aria speech models)
• Safety-critical decision systems without human oversight
• Tasks requiring factual precision beyond the model's ~2B parameter capacity

How to Get Started with the Model

Download from Aria Compute

Quantization Recipe

• Bundle size: ~1.1 GB (BF16 text backbone: ~3.9 GB)
• Generation quality: Awaiting gen_quant_eval audit. Per-channel quantization preserves per-output-channel distribution characteristics. Formal quality benchmarks against FP16 and other Aria quant recipes are pending
• Calibration-free: Hadamard pre-processing + per-channel quantization, no task-specific calibration data required
• Other precision options: Also available for Gemma-3n-E2B-IT: gemma-3n-e2b-it_q8_channel (per-channel 8-bit) and gemma-3n-e2b-it_q326_channel (mixed precision, attn 4-bit + FFN ~3-bit)

Model Architecture

• Hybrid attention: 24/30 layers use sliding-window attention (window 512) with activation sparsity (first 10 layers at 95% sparsity); only 6/30 layers use full softmax attention with Laurel (low-rank attention, rank 64) — dense-attention KV cost is confined to a few layers
• GQA (Grouped Query Attention): 2 KV heads serving 8 query heads — KV Cache memory is 4× smaller than full attention
• Per-layer input projections: a 256-dim layer-input embedding is projected to the 2,048-dim hidden state at each layer — compact embedding table with shared layer-input processing
• GeGLU activation: GELU with tanh approximation gating, efficient for on-device inference
• RoPE position encoding: 1M base frequency (global) with 10K local base frequency for sliding-window layers, supporting 32K context
• RMSNorm pre-normalization: Lightweight normalization before each sub-layer
• Final logit softcapping: Output logits capped at ±30.0 for training stability

• ~half the parameter count (~2B vs ~4B) — same hidden size and head configuration, but fewer layers (30 vs 35) and narrower FFN (8,192 vs 16,384)
• Fewer full-attention layers (6 vs 7) — a slightly larger share of computation remains in the sliding-window regime
• Less aggressive KV sharing (num_kv_shared_layers 10 vs 15)
• Same head_dim, sliding window, context length, and vocabulary — architecture-family consistency
• Instruction-tuned variant — based on the IT (instruction-tuned) checkpoint, providing better instruction-following capability out of the box

Bias, Risks, and Limitations

Limitations

• Reasoning depth: Multi-step logical reasoning is limited for a 2B-class model. Verify outputs in high-stakes scenarios; consider larger Gemma 3n variants for reasoning tasks.
• Mathematics: Simple arithmetic may be attempted but is unreliable. Advanced quantitative reasoning is out of scope. Use larger models for mathematical tasks.
• Code generation: Capable of single-line completions and basic snippets; unreliable for multi-line code or structured programs.
• Factual knowledge: Limited world knowledge due to ~2B parameter scale. Always verify factual claims against authoritative sources. This model is best suited for instruction-following and lightweight text processing rather than encyclopedic knowledge retrieval.
• Instruction following: Handles simple single-constraint instructions. Complex multi-constraint prompts may cause degradation, especially at longer contexts.
• Quantization drift: 4-bit per-channel quantization may exhibit generation drift versus FP16, especially on ambiguous or open-ended prompts. For higher fidelity, use gemma-3n-e2b-it_q8_channel (per-channel 8-bit). For even smaller bundles, see gemma-3n-e2b-it_q326_channel (mixed precision, attn 4-bit + FFN ~3-bit).

Bias and Risks

• Bias: As with all large language models trained on web-scale data, Gemma 3n may reflect societal biases present in its training corpus. Evaluate outputs before deployment in sensitive domains (hiring, healthcare, law).
• Toxicity: The instruction-tuned model has been safety-aligned with RLHF. However, no safety filter is exhaustive. Consider an additional output classifier in high-risk environments.
• Hallucination: May generate plausible-sounding but factually incorrect information. Implement output verification for critical applications. Hallucination risk is elevated for smaller models due to limited memorization capacity.
• Dual-use risk: Text-generation capabilities could be misused for spam, disinformation, or impersonation. Deploy responsibly and in accordance with the Apache 2.0 license terms.

Recommendations

• Adding a lightweight output safety classifier for user-facing deployments
• Verifying factual claims with external knowledge bases
• Not using the model for high-stakes decisions without human review
• Considering larger Gemma 3n variants (E4B) for tasks requiring stronger reasoning or factual recall