You can deploy this model and start earning money today!

DSA1/
qwen25vl-3b-perdataset-derivatives

Name: qwen25vl-3b-perdataset-derivatives
Availability: InStock

Qwen/Qwen2.5-VL-3B-Instruct

adapter

Views

No views yet

Qwen2.5-VL-3B — Per-Dataset Fine-Tuning Derivatives (LoRA + Full FT)

10 fine-tuned derivatives of Qwen/Qwen2.5-VL-3B-Instruct, one LoRA and one full fine-tune for each of 5 vision-language datasets. Language-model backbone only — the vision tower and multimodal projector are frozen in both variants.

⚠️ Research artifacts: each model is trained on a 3,000-example slice of one dataset for 3 epochs.

Shared training recipe (all 10)

Base: Qwen/Qwen2.5-VL-3B-Instruct
Frozen: vision tower + multimodal projector (freeze_vision_tower: true, freeze_multi_modal_projector: true) — only the LM backbone is trained.
Data: max_samples = 3000 per dataset, 3 epochs, cutoff_len = 2048.
Framework: LLaMA-Factory 0.9.3, transformers 4.52.4.
LoRA variant: target q_proj, v_proj; rank 16, alpha 32; lr 2e-4.
Full variant: DeepSpeed ZeRO-3; lr 1e-5; image_max_pixels = 401408; trainable = 100 % of the LM (vision/projector still frozen).

Path	Type	train_loss (3 ep)
`lora/mathv360k`	LoRA (q/v r16)	0.160
`lora/paintingform`	LoRA (q/v r16)	0.823
`lora/stvqa`	LoRA (q/v r16)	0.437
`lora/textvqa`	LoRA (q/v r16)	0.212
`lora/visual7w`	LoRA (q/v r16)	0.545
`full/mathv360k`	Full FT (ZeRO-3)	0.137
`full/paintingform`	Full FT (ZeRO-3)	0.659
`full/stvqa`	Full FT (ZeRO-3)	0.289
`full/textvqa`	Full FT (ZeRO-3)	0.192
`full/visual7w`	Full FT (ZeRO-3)	0.386

Datasets: MathV360K, PaintingForm, ST-VQA, TextVQA, Visual7W.

Usage

LoRA (base + PEFT):

python

from transformers import Qwen2_5_VLForConditionalGeneration, AutoProcessor
from peft import PeftModel
base = Qwen2_5_VLForConditionalGeneration.from_pretrained("Qwen/Qwen2.5-VL-3B-Instruct", torch_dtype="auto", device_map="auto")
model = PeftModel.from_pretrained(base, "DSA1/qwen25vl-3b-perdataset-derivatives", subfolder="lora/textvqa")
proc = AutoProcessor.from_pretrained("Qwen/Qwen2.5-VL-3B-Instruct")

Full FT (self-contained LM weights; processor/chat-template from base):

python

from transformers import Qwen2_5_VLForConditionalGeneration, AutoProcessor
model = Qwen2_5_VLForConditionalGeneration.from_pretrained("DSA1/qwen25vl-3b-perdataset-derivatives", subfolder="full/textvqa", torch_dtype="auto", device_map="auto")
proc = AutoProcessor.from_pretrained("Qwen/Qwen2.5-VL-3B-Instruct")

Notes

train_loss = mean training loss over all optimizer steps (1014 steps per model at global batch 8 for LoRA; 129 steps at global batch 64 for full).
Full-FT models ship LM weights only; the vision tower + projector are byte-identical to the base — load processor/chat-template from Qwen/Qwen2.5-VL-3B-Instruct.
A 7B counterpart of these derivatives exists separately.

License

Inherits the base model's license (Qwen/Qwen2.5-VL-3B-Instruct). Datasets retain their respective licenses.

DSA1/qwen25vl-3b-perdataset-derivatives