You can deploy this model and start earning money today!

leonardosalem/
qwen-vl-talk2car-grounding

Name: qwen-vl-talk2car-grounding
Availability: InStock

Qwen/Qwen3-VL-4B-Instruct

adapter

apache-2.0

Views

No views yet

Qwen-VL QLoRA adapters — visual grounding on Talk2Car

LoRA adapters from a master's thesis on parameter-efficient adaptation of vision-language models for autonomous driving.

Task: Referring expression grounding (referring object detection)
Dataset: Talk2Car (8.3K train / 1.2K val / 2447 test)
Metric: AP50 at IoU 0.5
Method: QLoRA — 4-bit NF4 frozen base, LoRA rank 16, alpha 32, dropout 0.05, lora_target: all, paged_adamw_8bit, cosine schedule

Results

Subfolder	Base model	Checkpoint	AP50 (4-bit)	AP50 (bf16)
`qwen3-4b`	Qwen/Qwen3-VL-4B-Instruct	checkpoint-1200	81.00%	not measured
`qwen3-2b`	Qwen/Qwen3-VL-2B-Instruct	checkpoint-800	73.52%	77.28%
`qwen2.5-3b`	Qwen/Qwen2.5-VL-3B-Instruct	checkpoint-1200	73.52%	not measured

Qwen3-VL-4B at 81.00% AP50 exceeds the VORAV leaderboard entry (77.7%).

Inference precision matters: on Qwen3-VL-2B checkpoint-800 the same weights score 73.52% in 4-bit and 77.28% in bf16, a 3.76 pp gap. In 4-bit the model emitted 21 unparseable boxes out of 2447; in bf16, none. Keep precision fixed when comparing models.

Each subfolder is a self-contained PEFT adapter (weights, config, tokenizer and processor files). Training-resume artefacts (optimizer, scheduler, RNG state) are excluded.

qwen3-4b/ — from results/checkpoints/qwen3-4b_grounding_qlora/checkpoint-1200
qwen3-2b/ — from results/checkpoints/qwen3-2b_grounding_qlora/checkpoint-800
qwen2.5-3b/ — from results/checkpoints/qwen2.5-3b_grounding_qlora/checkpoint-1200

Usage

python

from peft import PeftModel
from transformers import AutoProcessor, Qwen3VLForConditionalGeneration

base = "Qwen/Qwen3-VL-4B-Instruct"
model = Qwen3VLForConditionalGeneration.from_pretrained(base, dtype="bfloat16", device_map="auto")
model = PeftModel.from_pretrained(model, "leonardosalem/qwen-vl-talk2car-grounding", subfolder="qwen3-4b")
processor = AutoProcessor.from_pretrained(base)

Match the subfolder to its base model in the table above.

Reproducing

Training and evaluation code, configs and metrics: https://github.com/Leonardo-salem/thesis-vm-final

REPRODUCE.md in that repository gives the exact command, checkpoint, inference precision and sample count behind every number.

leonardosalem/qwen-vl-talk2car-grounding

Qwen-VL QLoRA adapters — visual grounding on Talk2Car

Results

Contents

Usage

Reproducing

leonardosalem/
qwen-vl-talk2car-grounding