oMEGA-4B-SpatialThink-0804

Name: oMEGA-4B-SpatialThink-0804
Rating: 2.5 (43 reviews)

oMEGA-4B-SpatialThink-0804 is a vision-language model built on top of Qwen/Qwen3-VL-4B-Instruct and fine-tuned for spatial reasoning with concise notes for unfiltered vision tasks. The model is trained to produce concise yet informative reasoning for spatial understanding while maintaining strong image captioning capabilities. Training is based on remyxai's SpaceThinker and OpenCaption-FineGrained, enabling efficient spatial reasoning and detailed image understanding across diverse visual domains.

[!NOTE] This model is an experimental release and may generate unexpected behaviors or reasoning artifacts in certain scenarios.

Key Highlights

Qwen3-VL Foundation: Built directly on top of Qwen/Qwen3-VL-4B-Instruct.
Spatial Reasoning: Optimized for spatial understanding with concise reasoning notes for unfiltered vision tasks.
Concise Reasoning: Generates compact reasoning while preserving essential spatial information.
Image Captioning: Produces detailed and context-aware image captions.
Vision-Language Fine-Tuning: Trained on high-quality spatial reasoning and fine-grained image caption datasets.
Research-Focused Release: Designed for multimodal reasoning, spatial understanding, and image captioning research.
Efficient 4B Deployment: Suitable for local multimodal inference and research environments.

Quick Start with Transformers

python

from transformers import Qwen3VLForConditionalGeneration, AutoProcessor
from qwen_vl_utils import process_vision_info
import torch

model = Qwen3VLForConditionalGeneration.from_pretrained(
    "prithivMLmods/oMEGA-4B-SpatialThink-0804",
    torch_dtype="auto",
    device_map="auto"
)

processor = AutoProcessor.from_pretrained(
    "prithivMLmods/oMEGA-4B-SpatialThink-0804"
)

messages = [
    {
        "role": "user",
        "content": [
            {
                "type": "image",
                "image": "https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen-VL/assets/demo.jpeg",
            },
            {
                "type": "text",
                "text": "Provide a detailed caption and reasoning for this image."
            },
        ],
    }
]

text = processor.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True
)

image_inputs, video_inputs = process_vision_info(messages)

inputs = processor(
    text=[text],
    images=image_inputs,
    videos=video_inputs,
    padding=True,
    return_tensors="pt",
).to("cuda")

generated_ids = model.generate(
    **inputs,
    max_new_tokens=128
)

generated_ids_trimmed = [
    out[len(inp):]
    for inp, out in zip(inputs.input_ids, generated_ids)
]

output_text = processor.batch_decode(
    generated_ids_trimmed,
    skip_special_tokens=True,
    clean_up_tokenization_spaces=False
)

print(output_text)

Training Details

Setting	Value
Base Model	Qwen/Qwen3-VL-4B-Instruct
Training Method	Supervised Fine-Tuning (SFT)
Primary Objective	Spatial Reasoning with Concise Notes for Unfiltered Vision Tasks
Secondary Objective	Efficient Spatial Reasoning and Image Captioning
Training Framework	TRL + Transformers
Training Precision	BF16

Intended Use

Spatial Reasoning: Understanding spatial relationships, object layouts, and geometric reasoning.
Image Captioning: Producing detailed and fine-grained image descriptions.
Multimodal Reasoning: Combining visual understanding with concise reasoning.
Vision Research: Benchmarking and evaluating vision-language reasoning capabilities.
Local Deployment: Efficient inference for multimodal applications.

Limitations

Experimental Model: Performance may vary across different visual domains.
Reasoning Artifacts: Generated reasoning may occasionally contain incorrect intermediate interpretations.
Vision Ambiguity: Highly ambiguous or low-quality images may reduce reasoning accuracy.

Acknowledgements

Qwen/Qwen3-VL-4B-Instruct: Base vision-language model used for this project.
SpaceThinker by remyxai: A spatial reasoning dataset used to improve concise visual reasoning capabilities.
OpenCaption-FineGrained: A fine-grained image captioning dataset used to enhance detailed visual understanding and caption generation.
TRL - Transformers Reinforcement Learning: Used for supervised fine-tuning and multimodal training.
Transformers: Provides the model architecture, training, and inference framework for multimodal transformer models.