pip install "git+https://github.com/huggingface/optimum-intel.git" --extra-index-url https://download.pytorch.org/whl/cpu
pip install transformers==5.5.0
pip install torchvision Pillow --extra-index-url https://download.pytorch.org/whl/cpu

Run model inference:

python

from optimum.intel.openvino import OVModelForVisualCausalLM
from transformers import AutoProcessor
from PIL import Image
import requests

model_id = "OpenVINO/gemma-4-26B-A4B-it-int4-ov"
processor = AutoProcessor.from_pretrained(model_id)
model = OVModelForVisualCausalLM.from_pretrained(model_id)

url = "https://github.com/openvinotoolkit/openvino_notebooks/assets/29454499/d5fbbd1a-d484-415c-88cb-9986625b7b11"
image = Image.open(requests.get(url, stream=True).raw)

messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "image": image},
            {"type": "text", "text": "What is unusual in this picture?"},
        ],
    }
]

text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = processor(text=text, images=[image], return_tensors="pt")
input_len = inputs["input_ids"].shape[-1]

output = model.generate(**inputs, do_sample=False, max_new_tokens=100)
response = processor.decode(output[0][input_len:], skip_special_tokens=True)
print(response)

You can find more detailed usage examples in OpenVINO Notebooks:

Gemma 4

Limitations

Check the original model card for limitations.

Legal information

The original model is distributed under Apache License Version 2.0 license. More details can be found in gemma-4-26B-A4B-it.

Disclaimer

Intel is committed to respecting human rights and avoiding causing or contributing to adverse impacts on human rights. See Intel's Global Human Rights Principles. Intel's products and software are intended only to be used in applications that do not cause or contribute to adverse impacts on human rights.

OpenVINO/gemma-4-26b-a4b-it-int4-ov

gemma-4-26B-A4B-it-int4-ov

Description

Quantization Parameters

Compatibility

Running Model Inference with Optimum Intel

Limitations

Legal information

Disclaimer

OpenVINO/
gemma-4-26b-a4b-it-int4-ov