[!Note]
This is a submodel derived from google/gemma-3n-E4B-it. It has been modified by slicing specific layers and resizing FFN dimensions. It is not the original model.
To learn more about MatFormers, please review the launch blog and generate your own submodels
with the MatFormer Lab.
[!Note]
This repository corresponds to the launch version of Gemma 3n E4B IT (Instruct), to be used with Hugging Face transformers,
supporting text, audio, and vision (image and video) inputs.
Gemma 3n models have multiple architecture innovations:
They are available in two sizes based on effective parameters. While the raw parameter count of this model is 8B, the architecture design allows the model to be run with a memory footprint comparable to a traditional 4B model by offloading low-utilization matrices from the accelerator.
They use a MatFormer architecture that allows nesting sub-models within the E4B model. We provide one sub-model (an E2B), or you can access a spectrum of custom-sized models using the Mix-and-Match method.
Summary description and brief definition of inputs and outputs.
Description
Gemma is a family of lightweight, state-of-the-art open models from Google,
built from the same research and technology used to create the Gemini models.
Gemma 3n models are designed for efficient execution on low-resource devices.
They are capable of multimodal input, handling text, image, video, and audio
input, and generating text outputs, with open weights for pre-trained and
instruction-tuned variants. These models were trained with data in over 140
spoken languages.
Gemma 3n models use selective parameter activation technology to reduce resource
requirements. This technique allows the models to operate at an effective size
of 2B and 4B parameters, which is lower than the total number of parameters they
contain. For more information on Gemma 3n's efficient parameter management
technology, see the
Gemma 3n
page.
Inputs and outputs
Input:
Text string, such as a question, a prompt, or a document to be
summarized
Images, normalized to 256x256, 512x512, or 768x768 resolution
and encoded to 256 tokens each
Audio data encoded to 6.25 tokens per second from a single channel
Total input context of 32K tokens
Output:
Generated text in response to the input, such as an answer to a
question, analysis of image content, or a summary of a document
Total output length up to 32K tokens, subtracting the request
input tokens
Usage
Below, there are some code snippets on how to get quickly started with running
the model. First, install the Transformers library. Gemma 3n is supported
starting from transformers 4.53.0.
$ pip install -U transformers
Then, copy the snippet from the section that is relevant for your use case.
Running with the pipeline API
You can initialize the model and processor for inference with pipeline as
follows.
With instruction-tuned models, you need to use chat templates to process our
inputs first. Then, you can pass it to the pipeline.
python
1messages =[2{3"role":"system",4"content":[{"type":"text","text":"You are a helpful assistant."}]5},6{7"role":"user",8"content":[9{"type":"image","url":"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},10{"type":"text","text":"What animal is on the candy?"}11]12}13]1415output = pipe(text=messages, max_new_tokens=200)16print(output[0]["generated_text"][-1]["content"])17# Okay, let's take a look!18# Based on the image, the animal on the candy is a **turtle**.19# You can see the shell shape and the head and legs.
Running the model on a single GPU
python
1from transformers import AutoProcessor, Gemma3nForConditionalGeneration
2from PIL import Image
3import requests
4import torch
56model_id ="pranjal-pravesh/gemma-3n-E3B"78model = Gemma3nForConditionalGeneration.from_pretrained(model_id, device_map="auto", torch_dtype=torch.bfloat16,).eval()910processor = AutoProcessor.from_pretrained(model_id)1112messages =[13{14"role":"system",15"content":[{"type":"text","text":"You are a helpful assistant."}]16},17{18"role":"user",19"content":[20{"type":"image","image":"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/bee.jpg"},21{"type":"text","text":"Describe this image in detail."}22]23}24]2526inputs = processor.apply_chat_template(27 messages,28 add_generation_prompt=True,29 tokenize=True,30 return_dict=True,31 return_tensors="pt",32).to(model.device)3334input_len = inputs["input_ids"].shape[-1]3536with torch.inference_mode():37 generation = model.generate(**inputs, max_new_tokens=100, do_sample=False)38 generation = generation[0][input_len:]3940decoded = processor.decode(generation, skip_special_tokens=True)41print(decoded)4243# **Overall Impression:** The image is a close-up shot of a vibrant garden scene,44# focusing on a cluster of pink cosmos flowers and a busy bumblebee.45# It has a slightly soft, natural feel, likely captured in daylight.
