Step3-VL-10B-GGUF

Quantizations

Usage

With llama.cpp

With llama-cpp-python

Vision Support

Quantization Details

• Base Model: stepfun-ai/Step3-VL-10B
• Quantization Tool: llama.cpp convert_hf_to_gguf.py + llama-quantize

Original Model Card

• stepfun-ai/Step3-VL-10B-Base pipeline_tag: image-text-to-text

STEP3-VL-10B

📢 News & Updates

• 🚀 Online Demo: Explore Step3-VL-10B on Hugging Face Spaces !
• 📢 [Notice] vLLM Support: vLLM integration is now officially supported! (PR #32329)
• ✅ [Fixed] HF Inference: Resolved the eos_token_id misconfiguration in config.json that caused infinite generation loops. (PR #abdf3)
• ✅ [Fixing] Metric Correction: We sincerely apologize for inaccuracies in the Qwen3VL-8B benchmarks (e.g., AIME, HMMT, LCB). The errors were caused by an incorrect max_tokens setting (mistakenly set to 32k) during our large-scale evaluation process. We are re-running the tests and will provide corrected numbers in the next version of technical report.

🚀 Introduction

Figure 1: Performance comparison of STEP3-VL-10B against SOTA multimodal foundation models. SeRe: Sequential Reasoning; PaCoRe: Parallel Coordinated Reasoning.

1. Unified Pre-training on High-Quality Multimodal Corpus: A single-stage, fully unfrozen training strategy on a 1.2T token multimodal corpus, focusing on two foundational capabilities: reasoning (e.g., general knowledge and education-centric tasks) and perception (e.g., grounding, counting, OCR, and GUI interactions). By jointly optimizing the Perception Encoder and the Qwen3-8B decoder, STEP3-VL-10B establishes intrinsic vision-language synergy.
2. Scaled Multimodal Reinforcement Learning and Parallel Reasoning: Frontier capabilities are unlocked through a rigorous post-training pipeline comprising two-stage supervised finetuning (SFT) and over 1,400 iterations of RL with both verifiable rewards (RLVR) and human feedback (RLHF). Beyond sequential reasoning, we adopt Parallel Coordinated Reasoning (PaCoRe), which allocates test-time compute to aggregate evidence from parallel visual exploration.

📥 Model Zoo

📊 Performance

Comparison with Larger Models (10×–20× Larger)

Note on Inference Modes:

SeRe (Sequential Reasoning): The standard inference mode using sequential generation (Chain-of-Thought) with a max length of 64K tokens.

PaCoRe (Parallel Coordinated Reasoning): An advanced mode that scales test-time compute. It aggregates evidence from 16 parallel rollouts to synthesize a final answer, utilizing a max context length of 128K tokens.

Unless otherwise stated, scores below refer to the standard SeRe mode. Higher scores achieved via PaCoRe are explicitly marked.

Comparison with Open-Source Models (7B–10B)

Key Capabilities

• STEM Reasoning: Achieves 94.43% on AIME 2025 and 75.95% on MathVision (with PaCoRe), demonstrating exceptional complex reasoning capabilities that outperform models 10×–20× larger.
• Visual Perception: Records 92.05% on MMBench and 80.11% on MMMU, establishing strong general visual understanding and multimodal reasoning.
• GUI & OCR: Delivers state-of-the-art performance on ScreenSpot-V2 (92.61%), ScreenSpot-Pro (51.55%), and OCRBench (86.75%), optimized for agentic and document understanding tasks.
• Spatial Understanding: Demonstrates emergent spatial awareness with 66.79% on BLINK and 57.21% on All-Angles-Bench, establishing strong potential for embodied intelligence applications.

🏗️ Architecture & Training

Architecture

• Visual Encoder: PE-lang (Language-Optimized Perception Encoder), 1.8B parameters.
• Decoder: Qwen3-8B.
• Projector: Two consecutive stride-2 layers (resulting in 16× spatial downsampling).
• Resolution: Multi-crop strategy consisting of a 728×728 global view and multiple 504×504 local crops.

Training Pipeline

• Pre-training: Single-stage, fully unfrozen strategy using AdamW optimizer (Total: 1.2T tokens, 370K iterations).
  • Phase 1: 900B tokens.
  • Phase 2: 300B tokens.
• Supervised Finetuning (SFT): Two-stage approach (Total: ~226B tokens).
  • Stage 1: 9:1 text-to-multimodal ratio (~190B tokens).
  • Stage 2: 1:1 text-to-multimodal ratio (~36B tokens).
• Reinforcement Learning: Total >1,400 iterations.
  • RLVR: 600 iterations (Tasks: mathematics, geometry, physics, perception, grounding).
  • RLHF: 300 iterations (Task: open-ended generation).
  • PaCoRe Training: 500 iterations (Context length: 64K max sequence).

🛠️ Quick Start

• Model Weights: 20 GB
• Runtime Overhead: ~4 GB
• Minimum VRAM Required: 24 GB (e.g., RTX 4090 or A100)

Inference with Hugging Face Transformers

🚀 Deployment with vLLM (OpenAI-compatible API)

1. Install vLLM nightly (choose one):
   • Python / pip
     pip install vllm --pre --extra-index-url https://wheels.vllm.ai/nightly
     Python ≥3.10 is required. Please ensure vLLM version >= 0.14.0rc2.dev143+gc0a350ca7.
   • Docker (nightly image)
     docker pull vllm/vllm-openai:nightly-963dc0b865a3b6011fde7e0d938f86245dccbfac
     The tag above pins the nightly build we validated; update to the latest nightly tag if needed.
2. Launch the server:
   vllm serve --model stepfun-ai/Step3-VL-10B -tp 1 --reasoning-parser deepseek_r1 --enable-auto-tool-choice --tool-call-parser hermes --trust-remote-code
   Crucial Step: You must append the --trust-remote-code flag to your deployment command. This is mandatory for models that utilize custom code for their architecture.
3. Call the endpoint using any OpenAI-compatible SDK (example in Python):
   python

   1from openai import OpenAI 2 3client = OpenAI(base_url="http://localhost:8000/v1", api_key="dummy") 4 5resp = client.chat.completions.create( 6 model="stepfun-ai/Step3-VL-10B", 7 messages=[{ 8 "role": 9 "user", 10 "content": [{ 11 "type": "image_url", 12 "image_url": { 13 "url": 14 "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/bee.jpg" 15 } 16 }, { 17 "type": "text", 18 "text": "what's in this picture?" 19 }] 20 }]) 21 22print(resp.choices[0].message.content) 23 24

🚀 Deployment with SGLang (OpenAI-compatible API)

1. Install SGLang latest main (choose one):

• Python / pip
  bash

  1pip install "sglang @ git+https://github.com/sgl-project/sglang.git#subdirectory=python" 2pip install nvidia-cudnn-cu12==9.16.0.29
• Docker
  bash

  1 docker run --gpus all \ 2 --shm-size 32g \ 3 -p 30000:30000 \ 4 -v ~/.cache/huggingface:/root/.cache/huggingface \ 5 --ipc=host \ 6 lmsysorg/sglang:latest \ 7 python3 -m sglang.launch_server --model-path stepfun-ai/Step3-VL-10B --host 0.0.0.0 --port 30000

1. Launch the server:
   sglang serve --model-path stepfun-ai/Step3-VL-10B --trust-remote-code --port 2345 --reasoning-parser deepseek-r1 --tool-call-parser hermes
2. Call the endpoint using any OpenAI-compatible SDK (example in Python):

   from openai import OpenAI
   
   port = 30000
   
   client = OpenAI(base_url=f"http://localhost:{port}/v1", api_key="None")
   
   response = client.chat.completions.create(
       model="stepfun-ai/Step3-VL-10B",
       messages=[
           {
               "role": "user",
               "content": [
                   {
                       "type": "text",
                       "text": "What is in this image?",
                   },
                   {
                       "type": "image_url",
                       "image_url": {
                           "url": "https://github.com/sgl-project/sglang/blob/main/examples/assets/example_image.png?raw=true"
                       },
                   },
               ],
           }
       ],
   )
   
   print(response.choices[0].message.content)

📜 Citation

📄 License