Developed by AxionML for open-source serving and deployment use cases. Part of AxionML's effort to provide ready-to-serve quantized models for the community.

This is an NVFP4-quantized version of Qwen/Qwen3.5-0.8B (0.8B parameters), quantized using NVIDIA TensorRT Model Optimizer. Weights and activations of linear layers are quantized to FP4, reducing disk size and GPU memory by ~4x compared to BF16.

About NVFP4 quantization: NVFP4 on Blackwell couples a compact E2M1 FP4 codebook with blockwise FP8 (E4M3) scaling over 16-element micro-blocks, so that 4-bit stored values remain numerically useful for neural-network computation. The E2M1 codebook provides a small, nonuniform set of representable magnitudes up to ±6 and relies on saturating behavior rather than IEEE NaN/Inf encodings to maximize usable range per bit. Using an FP8 block scale (rather than power-of-two-only E8M0) enables fractional scales and error-minimizing scale selection strategies such as dual-pass evaluation comparing "map max to 6" versus "map max to 4 with clipping." On Blackwell Tensor Cores, native FP4 multipliers exploit E2M1 simplicity to reduce multiplier area while higher-precision FP32 accumulation protects dot-product accuracy.

Ready for commercial and non-commercial use under Apache 2.0.

Over recent months, we have intensified our focus on developing foundation models that deliver exceptional utility and performance. Qwen3.5 represents a significant leap forward, integrating breakthroughs in multimodal learning, architectural efficiency, reinforcement learning scale, and global accessibility to empower developers and enterprises with unprecedented capability and efficiency.

Qwen3.5 Highlights

Qwen3.5 features the following enhancement:

Unified Vision-Language Foundation: Early fusion training on multimodal tokens achieves cross-generational parity with Qwen3 and outperforms Qwen3-VL models across reasoning, coding, agents, and visual understanding benchmarks.
Efficient Hybrid Architecture: Gated Delta Networks combined with sparse Mixture-of-Experts deliver high-throughput inference with minimal latency and cost overhead.
Scalable RL Generalization: Reinforcement learning scaled across million-agent environments with progressively complex task distributions for robust real-world adaptability.
Global Linguistic Coverage: Expanded support to 201 languages and dialects, enabling inclusive, worldwide deployment with nuanced cultural and regional understanding.
Next-Generation Training Infrastructure: Near-100% multimodal training efficiency compared to text-only training and asynchronous RL frameworks supporting massive-scale agent scaffolds and environment orchestration.

For more details, please refer to our blog post Qwen3.5.

Model Overview

Type: Causal Language Model with Vision Encoder
Training Stage: Pre-training & Post-training
Language Model
- Number of Parameters: 0.8B
- Hidden Dimension: 1024
- Token Embedding: 248320 (Padded)
- Number of Layers: 24
- Hidden Layout: 6 × (3 × (Gated DeltaNet → FFN) → 1 × (Gated Attention → FFN))
- Gated DeltaNet:
  - Number of Linear Attention Heads: 16 for V and 16 for QK
  - Head Dimension: 128
- Gated Attention:
  - Number of Attention Heads: 8 for Q and 2 for KV
  - Head Dimension: 256
  - Rotary Position Embedding Dimension: 64
- Feed Forward Network:
  - Intermediate Dimension: 3584
- LM Output: 248320 (Tied to token embedding)
- MTP: trained with multi-steps
Context Length: 262,144 natively

Benchmark Results

Language

	Qwen3.5-0.8B	Qwen3.5-0.8B-NVFP4
Non-Thinking Mode
MMLU-Pro	29.7	29.2
MMLU-Redux	48.5	47.5
C-Eval	46.4	45.2
SuperGPQA	16.9	16.7
IFEval	52.1	50.7
MMMLU	34.1	33.2
Knowledge & STEM (Thinking)
MMLU-Pro	42.3	41.5
MMLU-Redux	59.5	58.0
C-Eval	50.5	49.8
SuperGPQA	21.3	20.9
GPQA	11.9	11.7
Instruction Following (Thinking）
IFEval	44.0	43.2
IFBench	21.0	20.7
MultiChallenge	18.9	18.6
Long Context (Thinking）
AA-LCR	4.7	4.6
LongBench v2	26.1	25.7
Reasoning (Thinking）
HMMT Feb 25	--	--
HMMT Nov 25	--	--
General Agent (Thinking）
BFCL-V4	25.3	25.0
TAU2-Bench	11.6	11.5
Multilingualism (Thinking）
MMMLU	44.3	43.5
MMLU-ProX	34.6	34.1
NOVA-63	42.4	41.6
INCLUDE	40.6	39.8
Global PIQA	59.4	58.4
PolyMATH	8.2	8.0
WMT24++	27.2	26.2
MAXIFE	39.2	37.8

* TAU2-Bench: we follow the official setup except for the airline domain, where all models are evaluated by applying the fixes proposed in the Claude Opus 4.5 system card.
* MMLU-ProX: we report the averaged accuracy on 29 languages.
* WMT24++: a harder subset of WMT24 after difficulty labeling and rebalancing; we report the averaged scores on 55 languages using XCOMET-XXL.
* MAXIFE: we report the accuracy on English + multilingual original prompts (totally 23 settings).
* Experimental settings: top_p=0.95, top_k=20, presence_penalty=1.5, and temperature=1.0 were used.
* Empty cells (--) indicate scores not yet available or not applicable.

Vision Language

	Qwen3.5-0.8B	Qwen3.5-0.8B-NVFP4
STEM and Puzzle
MMMU	49/47.4	49/47.4
MMMU-Pro	31.2/31.4	31.2/31.4
Mathvista(mini)	62.2/58.6	62.2/58.6
DynaMath	49.9/46.5	49.9/46.5
ZEROBench	0.0/0.0	0.0/0.0
ZEROBench_sub	12.9/11.4	12.9/11.4
VlmsAreBlind	59.4/57.3	59.4/57.3
General VQA
RealWorldQA	63.4/61.6	63.4/61.6
MMStar	58.3/55.9	58.3/55.9
MMBench_EN-DEV-v1.1	69.9/68.0	69.9/68.0
SimpleVQA	31.3/30.4	31.3/30.4
HallusionBench	53.1/46.7	53.1/46.7
Text Recognition and Document Understanding
MMLongBench-Doc	33.6/28.1	33.6/28.1
AI2D_TEST	69.9/68.7	69.9/68.7
CC-OCR	63.2/66.7	63.2/66.7
OmniDocBench1.5	61.0/70.6	61.0/70.6
CharXiv(RQ)	41.3/38.2	41.3/38.2
OCRBench	74.5/79.1	74.5/79.1
Spatial Intelligence
RefCOCO(avg)	79.3/77.8	79.3/77.8
CountBench	77.0/68.6	77.0/68.6
ODInW13	31.6/33.2	31.6/33.2
ERQA	34.5/23.8	34.5/23.8
EmbSpatialBench	68.6/54.6	68.6/54.6
RefSpatialBench	23.5/21.7	23.5/21.7
Hypersim	11.9/11.0	11.9/11.0
SUNRGBD	26.1/23.3	26.1/23.3
Nuscene	5.7/7.0	5.7/7.0
Video Understanding
VideoMME_{(w sub.)}	63.8/--	63.8/--
VideoMME_{(w/o sub.)}	57.7/--	57.7/--
VideoMMMU	44.3/--	44.3/--
MLVU	65.6/--	65.6/--
MVBench	55.8/--	55.8/--
LVBench	45.1/--	45.1/--
MMVU	34.3/--	34.3/--
Visual Agent
ScreenSpot Pro	--/46.5	--/46.5
Medical VQA
SLAKE	62.6/59.5	62.6/59.5
PMC-VQA	40.4/45.5	40.4/45.5
MedXpertQA-MM	17.1/25.3	17.1/25.3

* Scores of Qwen3.5 models are reported as Thinking / Non-thinking.
* MathVision: our model’s score is evaluated using a fixed prompt, e.g., “Please reason step by step, and put your final answer within \boxed{}.” For other models, we report the higher score between runs with and without the \boxed{} formatting.
* Experimental settings: For the Video benchmarks, we used top_p=0.95, top_k=20, presence_penalty=1.5, and temperature=1.0. All other benchmarks adopted the same sampling configuration but with temperature=0.6 under the thinking mode. Under the non-thinking mode, the sampling parameters were set to top_p=0.8, top_k=20, presence_penalty=1.5, and temperature=0.7.
* Empty cells (--) indicate scores not yet available or not applicable.

Quantization Details

This model was quantized by applying NVFP4 to the weights and activations of linear operators within transformer blocks. The KV-cache is not quantized. Vision encoder weights are kept in their original precision.

Quantization format: NVFP4 (MLP-only, MSE calibration)
Calibration dataset: Nemotron-Post-Training-Dataset-v2
Quantized checkpoint size: ~0.5GB
Tool: TensorRT Model Optimizer

Usage

Deploy with SGLang

bash

python3 -m sglang.launch_server \
    --model-path AxionML/Qwen3.5-0.8B-NVFP4 \
    --quantization modelopt_fp4 \
    --tp 1 \
    --reasoning-parser qwen3

Reproduce with ModelOpt

bash

python3 examples/llm_ptq/hf_ptq.py \
    --pyt_ckpt_path Qwen/Qwen3.5-0.8B \
    --qformat nvfp4_mse \
    --export_path ./qwen3.5-0.8b-nvfp4

Limitations

The base model was trained on data that may contain toxic language and societal biases. The quantized model inherits these limitations. It may generate inaccurate, biased, or offensive content. Please refer to the original model card for full details.