This repository contains an EXL3 build of Qwen3.5-9B, quantized by Ora Computing with OraQuant (OQ) - Ora Computing's proprietary calibrated quantization. It is an exllamav3-compatible quantization of Qwen/Qwen3.5-9B; the underlying weights are unchanged Qwen3.5-9B weights at reduced precision.

Text only. Qwen/Qwen3.5-9B is a multimodal model; this checkpoint contains only the language model (text input -> text output). The vision/video input encoders and the MTP draft head are not included.

Model Overview

Model name: Qwen3.5-9B-OQ-EXL3 (OraQuant)
Base model: Qwen/Qwen3.5-9B (Apache-2.0, Alibaba Cloud) - this is an EXL3 quantization of it
Parameters: ~9 billion (unchanged from the base model)
Quantization: OraQuant (OQ) non-uniform EXL3 trellis quantization produced by Ora Computing. Transformer body at mixed 2-4 bit trellis rates chosen per tensor, output head at 5 bits, embeddings in BF16 (kept in system RAM by exllamav3).
Footprint: ~4.7 GB on disk; ~2.7 GB of GPU-resident weights at load time.
Not fine-tuned, not parameter-reduced: the model architecture and parameter count are identical to the base model; only the weight precision is reduced. No additional training was performed.
Purpose: Evaluation/test-use only; optimized for local/offline inference and internal benchmarking.
License: See LICENSE (Custom Model License Agreement).

Files in this repo

File	What it is	Size
`model.safetensors`	Language model, OraQuant EXL3 (mixed 2-4 bit trellis)	~4.7 GB
`config.json`, `generation_config.json`	Model / generation configuration	-
`tokenizer.json`, `vocab.json`, `merges.txt`, `tokenizer_config.json`, `chat_template.jinja`	Tokenizer and chat template	-
`LICENSE`	Custom Model License Agreement	-

Usage

This checkpoint loads with stock upstream exllamav3 (no patch required; tested with v1.2.0):

python

from exllamav3 import Cache, Config, Generator, Model, Tokenizer

config = Config.from_directory("/path/to/Qwen3.5-9B-OQ-EXL3")
model = Model.from_config(config)
cache = Cache(model, max_num_tokens=8192)
model.load()
tokenizer = Tokenizer.from_config(config)
generator = Generator(model=model, cache=cache, tokenizer=tokenizer)

prompt = "<|im_start|>user\nWhat is 17 * 23?<|im_end|>\n<|im_start|>assistant\n"
print(generator.generate(prompt=prompt, max_new_tokens=400, completion_only=True,
                         add_bos=False,
                         stop_conditions=[tokenizer.eos_token_id,
                                          tokenizer.single_id("<|im_end|>")]))

It also works with exllamav3-based frontends (e.g. an OpenAI-compatible server such as TabbyAPI) configured to point at the model directory.

Qwen3.5 is a reasoning model; the chat template and thinking behaviour are carried in chat_template.jinja.

Intended Use & Restrictions

Permitted use

Internal testing, benchmarking, and evaluation of the model by the named Licensee.
Exploration of model behaviours, prompt engineering, and non-production prototypes.

Prohibited use

Deployment in a production or commercial service, publicly-facing API, resale, or redistribution.
Fine-tuning or creating derivative models for production use without a separate agreement.
Reverse-engineering the quantization/calibration used to produce these files.
Disclosure or sharing of the model (or its weights) to third parties beyond the named Licensee.

Out-of-scope use

Use in regulated or safety-critical contexts (unless separately permitted).
Any use that violates the Apache License, Version 2.0 under which the upstream model is distributed.

Quantization

Method: OraQuant (OQ), Ora Computing's proprietary calibrated quantization. The released file codes the transformer body in the EXL3 trellis format at non-uniform, per-tensor bit rates (2-4 bits), with a 5-bit output head and BF16 embeddings.
No fine-tuning: the weights are the original Qwen/Qwen3.5-9B weights; no additional training was performed.
No parameter-count change: the architecture and ~9B parameter count are unchanged; only weight precision is reduced.

Limitations & Risks

Quantized models may not replicate the full behaviour of the base model under all prompt categories, particularly domain-specific or rare inputs.
The model is provided as-is for testing only and is not certified for production use.
Users should validate outputs carefully and monitor for bias or unintended behaviours.

Upstream Attribution

This model is derived from the Qwen3.5-9B model released by Alibaba Cloud under the Apache License, Version 2.0.

"Copyright 2025 Alibaba Cloud. Licensed under the Apache License, Version 2.0."

For full terms, see: https://huggingface.co/Qwen/Qwen3.5-9B/blob/main/LICENSE Apache License, Version 2.0: https://www.apache.org/licenses/LICENSE-2.0

Contact & Support

For licensing inquiries or to request extended evaluation rights, please contact: info@oracomputing.com

Repository and model access are regulated. Do not redistribute or share without explicit written permission from Ora Computing.

oracomputing/Qwen3.5-9B-OQ-EXL3

Qwen3.5-9B-OQ-EXL3 (OraQuant)