This repository contains GGUF builds of Qwen3.5-9B, quantized by Ora Computing with OraQuant (OQ) - Ora Computing's proprietary calibrated quantization. These are llama.cpp-compatible quantizations of Qwen/Qwen3.5-9B; the underlying weights are unchanged Qwen3.5-9B weights at reduced precision.

Text only. Qwen/Qwen3.5-9B is a multimodal model; these GGUFs contain only the language model (text input -> text output). The vision/video input encoders are not included.

Model Overview

Model name: Qwen3.5-9B-GGUF (OraQuant)
Base model: Qwen/Qwen3.5-9B (Apache-2.0, Alibaba Cloud) - these are GGUF quantizations of it
Parameters: ~9 billion (unchanged from the base model)
Quantization: OraQuant (OQ) mixed-precision GGUFs produced by Ora Computing, provided in three footprints - OQ-Q4_K_M (higher quality), OQ-Q3_K_M (smaller/faster), and OQ-IQ2_XXS (smallest, mixed i-/k-quant).
Not fine-tuned, not parameter-reduced: the model architecture and parameter count are identical to the base model; only the weight precision is reduced.
Purpose: Evaluation/test-use only; optimized for local/offline inference and internal benchmarking.
License: See LICENSE (Custom Model License Agreement).

Files in this repo

File	What it is	Size
`Qwen3.5-9B-OQ-Q4_K_M.gguf`	Language model, OraQuant Q4_K_M (higher quality)	~5.7 GB
`Qwen3.5-9B-OQ-Q3_K_M.gguf`	Language model, OraQuant Q3_K_M (smaller/faster)	~4.7 GB
`Qwen3.5-9B-OQ-IQ2_XXS.gguf`	Language model, OraQuant IQ2_XXS-class (smallest, mixed i-/k-quant)	~3.2 GB
`LICENSE`	Custom Model License Agreement	-

Usage

These GGUFs load with stock upstream llama.cpp (no patch required); use a build with Qwen3.5 support.

export MODEL=/path/to/Qwen3.5-9B-OQ-Q4_K_M.gguf # or the Q3_K_M / IQ2_XXS file

Interactive chat:

./build/bin/llama-cli -m "$MODEL" -ngl 99

Single-shot completion (-st runs one turn then exits):

./build/bin/llama-cli -m "$MODEL" -ngl 99 -st -p "Explain the Chudnovsky algorithm in two sentences."

OpenAI-compatible server (Web UI at http://localhost:8080):

bash

./build/bin/llama-server -m "$MODEL" -ngl 99 \
  --alias qwen3.5-9b --host 0.0.0.0 --port 8080

Qwen3.5 is a reasoning model; the chat template and thinking behaviour are carried in the GGUF. For best results use the upstream-recommended sampling settings, e.g. --temp 0.6 --top-p 0.95 --top-k 20 --presence-penalty 1.0. Long thinking phases are normal on hard prompts, especially for the smallest (OQ-IQ2_XXS) footprint.

Intended Use & Restrictions

Permitted use

Internal testing, benchmarking, and evaluation of the model by the named Licensee.
Exploration of model behaviours, prompt engineering, and non-production prototypes.

Prohibited use

Deployment in a production or commercial service, publicly-facing API, resale, or redistribution.
Fine-tuning or creating derivative models for production use without a separate agreement.
Reverse-engineering the quantization/calibration used to produce these files.
Disclosure or sharing of the model (or its weights) to third parties beyond the named Licensee.

Out-of-scope use

Use in regulated or safety-critical contexts (unless separately permitted).
Any use that violates the Apache License, Version 2.0 under which the upstream model is distributed.

Quantization

Method: OraQuant (OQ), Ora Computing's proprietary calibrated quantization. The released files are mixed-precision GGUFs (K-quants; the OQ-IQ2_XXS file additionally uses i-quants).
No fine-tuning: the weights are the original Qwen/Qwen3.5-9B weights; no additional training was performed.
No parameter-count change: the architecture and ~9B parameter count are unchanged; only weight precision is reduced.
Footprints: OQ-Q4_K_M for higher quality, OQ-Q3_K_M for a smaller/faster footprint, OQ-IQ2_XXS for the smallest footprint (expect visible quality loss on the hardest long-reasoning tasks at this size).

Limitations & Risks

Quantized models may not replicate the full behaviour of the base model under all prompt categories, particularly domain-specific or rare inputs.
The model is provided as-is for testing only and is not certified for production use.
Users should validate outputs carefully and monitor for bias or unintended behaviours.

Upstream Attribution

This model is derived from the Qwen3.5-9B model released by Alibaba Cloud under the Apache License, Version 2.0.

"Copyright 2025 Alibaba Cloud. Licensed under the Apache License, Version 2.0."

For full terms, see: https://huggingface.co/Qwen/Qwen3.5-9B/blob/main/LICENSE Apache License, Version 2.0: https://www.apache.org/licenses/LICENSE-2.0

Contact & Support

For licensing inquiries or to request extended evaluation rights, please contact: info@oracomputing.com

Repository and model access are regulated. Do not redistribute or share without explicit written permission from Ora Computing.

oracomputing/Qwen3.5-9B-GGUF

Qwen3.5-9B-GGUF (OraQuant)