You can deploy this model and start earning money today!

DiningSystem/
qwen2vl-2b-kd-emkd-full

Name: qwen2vl-2b-kd-emkd-full
Availability: InStock

peft

safetensors

knowledge-distillation

vision-language

qwen2-vl

lora

Qwen/Qwen2-VL-2B-Instruct

adapter

apache-2.0

Views

No views yet

qwen2vl-2b-kd-emkd-full

LoRA adapter (r=128, alpha=256) for Qwen/Qwen2-VL-2B-Instruct, trained with the emkd objective, distilling from teacher Qwen/Qwen2.5-VL-7B-Instruct on the full llava_v1_5_mix665k mixture (1,525,381 samples, 1 epoch, effective batch 128, lr 1e-5 cosine, bf16, image_resolution=low, max_len 2048).

Results (fast_signal suite, Qwen2.5-32B-Instruct judge)

This run: mean 52.03 (MMBench 73.37 / MMStar 47.2 / MMMU 40.44 / MathVista 47.1)

eval-lib golden record: ukaz-uit/eval-lib run_id ab4qwjg3 (spec sha256:ed2050c5764bfca6)

Full campaign comparison:

Method	MMBench_DEV_EN	MMStar	MMMU_DEV_VAL	MathVista_MINI	Mean
ce_only (baseline)	73.71	47.80	40.22	48.60	52.58
cgkd	73.54	47.60	40.78	47.30	52.30
scva	72.85	47.40	40.11	47.90	52.07
emkd	73.37	47.20	40.44	47.10	52.03
sre	72.59	47.60	40.33	46.00	51.63
base Qwen2-VL-2B	-	-	-	-	51.40
unit_aligned	71.91	47.27	39.67	46.00	51.21

Usage

Merge the adapter into the base model with PEFT, or load directly:

python

from peft import PeftModel
from transformers import Qwen2VLForConditionalGeneration

base = Qwen2VLForConditionalGeneration.from_pretrained("Qwen/Qwen2-VL-2B-Instruct", torch_dtype="bfloat16")
model = PeftModel.from_pretrained(base, "DiningSystem/qwen2vl-2b-kd-emkd-full")
model = model.merge_and_unload()

projectors/ (if present) holds the auxiliary alignment projector used only during training.