Property	Value
Base Model	typhoon-ai/typhoon-ocr1.5-2b
Architecture	Qwen3VLForConditionalGeneration
Quantization	NVFP4 (W4A4), group_size=16
Quantizer	NVIDIA ModelOpt 0.45.0
Target GPU	NVIDIA Blackwell (GB10 / sm_121)
Quantized Size	2.23 GB
Context Length	262,144 tokens
Hidden Size	2048
Layers	28
Attention Heads	16 (KV: 8)
Vision	Qwen3VL Vision Encoder (1024 dim, 24 layers)

Quantization Configuration

Algorithm: NVFP4 (NVIDIA Floating Point 4-bit)
Weights: 4-bit float, group_size=16
Activations: 4-bit float, group_size=16
Excluded modules: lm_head, model.language_model.embed_tokens, model.visual*

Files

File	Size
`model.safetensors`	2.23 GB
`config.json`	Model config with quantization_config
`hf_quant_config.json`	HF quantization metadata
`generation_config.json`	Generation defaults
`chat_template.jinja`	Chat template
`tokenizer.json`	Tokenizer model
`tokenizer_config.json`	Tokenizer config
`preprocessor_config.json`	Vision preprocessor config
`video_preprocessor_config.json`	Video preprocessor config

Intended Uses & Limitations

This is a task-specific model intended to be used only with the provided prompt. It does not include any guardrails or VQA capability. Due to the nature of large language models (LLMs), a certain level of hallucination may occur. We recommend that developers carefully assess these risks in the context of their specific use case.

Important: This model is a single-prompt model — it works with one specific extraction prompt of Typhoon OCR v1.5 only. Other prompts will not produce correct results.

Prompting

The official extraction prompt for Typhoon OCR v1.5:

python

prompt = """Extract all text from the image.

Instructions:
- Only return the clean Markdown.
- Do not include any explanation or extra text.
- You must include all information on the page.

Formatting Rules:
- Tables: Render tables using <table>...</table> in clean HTML format.
- Equations: Render equations using LaTeX syntax with inline ($...$) and block ($$...$$).
- Images/Charts/Diagrams: Wrap any clearly defined visual areas (e.g. charts, diagrams, pictures) in:

<figure>
Describe the image's main elements (people, objects, text), note any contextual clues (place, event, culture), mention visible text and its meaning, provide deeper analysis when relevant (especially for financial charts, graphs, or documents), comment on style or architecture if relevant, then give a concise overall summary. Describe in Thai.
</figure>

- Page Numbers: Wrap page numbers in <page_number>...</page_number> (e.g., <page_number>14</page_number>).
- Checkboxes: Use ☐ for unchecked and ☑ for checked boxes."""

Usage

Recommended: vLLM + `typhoon-ocr` library

We recommend running inference via vLLM with the typhoon-ocr library rather than raw transformers:

pip install vllm typhoon-ocr

Serve the local NVFP4 model:

bash

vllm serve nanash66/typhoon-ocr1.5-2b-NVFP4 \
  --quantization modelopt \
  --max-model-len 49152 \
  --served-model-name typhoon-ocr-nvfp4

python

from typhoon_ocr import ocr_document

markdown = ocr_document(
    "image.png",
    model="typhoon-ocr-nvfp4",
    figure_language="Thai",
    task_type="v1.5",
    base_url="http://localhost:8000/v1",
    api_key="no-key",
)
print(markdown)

Alternative: HuggingFace Transformers

Note: This model requires a Blackwell GPU (sm_121) with NVIDIA ModelOpt runtime for NVFP4 inference.

python

from transformers import AutoModelForImageTextToText, AutoProcessor
from PIL import Image

model = AutoModelForImageTextToText.from_pretrained(
    "nanash66/typhoon-ocr1.5-2b-NVFP4",
    dtype="auto",
    device_map="auto",
)
processor = AutoProcessor.from_pretrained("nanash66/typhoon-ocr1.5-2b-NVFP4")

# Load image
img = Image.open("image.png")

# IMPORTANT: The model was trained at a fixed image dimension of 1800px.
# Resize the image before inference.
def resize_if_needed(img, max_size=1800):
    if max(img.size) > max_size:
        scale = max_size / max(img.size)
        img = img.resize((int(img.width * scale), int(img.height * scale)))
    return img

img = resize_if_needed(img, 1800)

messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "image": img},
            {"type": "text", "text": prompt},
        ],
    }
]

# Preparation for inference
inputs = processor.apply_chat_template(
    messages,
    tokenize=True,
    add_generation_prompt=True,
    return_dict=True,
    return_tensors="pt",
).to(model.device)

# Inference
generated_ids = model.generate(**inputs, max_new_tokens=10000)
generated_ids_trimmed = [
    out_ids[len(in_ids):]
    for in_ids, out_ids in zip(inputs.input_ids, generated_ids)
]
output_text = processor.batch_decode(
    generated_ids_trimmed,
    skip_special_tokens=True,
    clean_up_tokenization_spaces=False,
)
print(output_text[0])

Derivative Of

This is a derivative work of typhoon-ai/typhoon-ocr1.5-2b, released under the Apache-2.0 license.

Citation

If you find this model useful, please cite the original works:

bibtex

@misc{nonesung2026typhoonocropenvisionlanguage,
      title={Typhoon OCR: Open Vision-Language Model For Thai Document Extraction},
      author={Surapon Nonesung and Natapong Nitarach and Teetouch Jaknamon and Pittawat Taveekitworachai and Kunat Pipatanakul},
      year={2026},
      eprint={2601.14722},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2601.14722},
}

@misc{typhoon2,
      title={Typhoon 2},
      year={2024},
      eprint={2412.13702},
      archivePrefix={arXiv},
      url={https://arxiv.org/abs/2412.13702},
}

@misc{thaiocrbench,
      title={ThaiOCRBench},
      year={2025},
      eprint={2511.04479},
      archivePrefix={arXiv},
      url={https://arxiv.org/abs/2511.04479},
}

License

Apache-2.0 (same as base model)

nanash66/typhoon-ocr1.5-2b-NVFP4

Typhoon OCR 1.5 2B — NVFP4 Quantized

Model Details

Quantization Configuration

Files

Intended Uses & Limitations

Prompting

Usage

Recommended: vLLM + typhoon-ocr library

Alternative: HuggingFace Transformers

Derivative Of

Citation

License

nanash66/
typhoon-ocr1.5-2b-NVFP4

Recommended: vLLM + `typhoon-ocr` library