Spec-o3-3B

Model Summary

• Backbone: Qwen2.5-VL-3B-Instruct
• Inputs: Text + spectrum visualizations (global view plus tool-rendered zoomed views)
• Core capability: Multi-turn inspection with tool calls for evidence localization

Intended Use

• Primary: Human-in-the-loop assistance for spectral inspection and rare-object candidate vetting, providing (1) a final judgment and (2) an auditable inspection trace grounded in localized spectral evidence.
• Suitable settings: Research, offline analysis, and expert triage pipelines.
• Not intended for: Fully automated catalog publication without expert verification, safety-critical decision-making, or uses outside spectroscopic analysis.

How It Works (Tool-Augmented Inspection)

1. Reasoning about what spectral evidence is needed, and
2. Tool calls that request re-rendered views for specific wavelength intervals.

Recommended Prompt / Output Format

• A clear task instruction (e.g., “inspect the spectrum and decide whether the candidate should be accepted”).
• Allow tool calls during inference (one per turn).
• Expect a final decision with a short justification.

Training Overview

1. Cold-start SFT: supervised fine-tuning on ~1k expert-approved spectral inspection trajectories with tool usage.
2. Outcome-based RL (GRPO): reinforcement learning on label-only inspection tasks to improve decision quality, stabilize tool usage, and strengthen evidence localization.

• Tool-rendered outputs are loss-masked to discourage memorization of images.
• RL uses group-wise rollouts (e.g., 8 rollouts) and an outcome reward emphasizing correctness and format compliance.

Evaluation (Reported)

• SpecVI-Bench (macro-average F1 across inspection tasks): 73.3
• Cross-Survey (transfer to SDSS/DESI matched spectra, average F1): SDSS 77.3, DESI 73.6 (reference on LAMOST subset: 79.8)
• Cross-Task (transfer to unseen inspection categories on LAMOST, average F1): 74.4

Limitations

• The released checkpoints are evaluated on a limited set of inspection tasks and do not cover all astrophysical classes or all observational conditions encountered in production pipelines.
• Real-world vetting often requires external cross-matching and additional modalities (photometry, imaging, time-domain evidence) beyond spectrum-only inspection.
• Extending to new surveys or new target categories may still require expert demonstration data for cold start and careful validation.
• The model does not yet provide production-grade uncertainty handling (e.g., abstention, calibration, or risk-aware triage) out of the box.

Usage (Conceptual)

1. Render an initial full-range spectrum image.
2. Run the model with the task prompt + image.
3. If a tool call is emitted, render the requested wavelength window and feed the new image back.
4. Repeat until a final decision is produced or a tool-call budget is reached.

Citation