This repository contains an unofficial OpenVINO™ IR conversion of openai/gpt-oss-120b with INT4 weight compression.

The model is intended for local text generation using OpenVINO GenAI. OpenVINO GenAI provides an optimized generative inference API for OpenVINO models and is the recommended runtime path here for getting strong performance on Intel hardware, including Intel CPUs and supported Intel GPUs.

This is an unofficial converted model repository. Please refer to the original model card for full model details, intended use, safety notes, license terms, and limitations.

This repository is designed to run with openvino-genai, specifically openvino_genai.LLMPipeline.

OpenVINO GenAI is optimized for generative AI inference with OpenVINO. Its LLM pipeline can load an OpenVINO-converted model folder directly, including the model, tokenizer, detokenizer, and generation configuration. This makes it a clean and efficient runtime path for local LLM inference on Intel hardware.

Quantization Weight format: INT4 Export tool: Optimum Intel Runtime: OpenVINO GenAI Model format: OpenVINO IR

INT4 compression is intended to reduce model size and memory usage compared with higher precision weights. As with any converted and quantized model, quality and numerical behavior may differ from the original model and should be validated for your use case.

python examples/run_gpt_oss_openvino_genai.py
--model-dir .
--device CPU
--max-new-tokens 256
--prompt "Explain quantum mechanics clearly and concisely."

This model was converted with Optimum Intel using the OpenVINO export path.