You can deploy this model and start earning money today!

o-taco/
qwen3vl-aokvqa-freeze0pct

Name: qwen3vl-aokvqa-freeze0pct
Availability: InStock

Qwen/Qwen3-VL-2B-Instruct

adapter

Views

No views yet

qwen3vl-aokvqa-freeze0pct

LoRA adapter for Qwen/Qwen3-VL-2B-Instruct, trained with the VLM2Vec contrastive-embedding recipe on the A-OKVQA subset of MMEB-train only (single V100 32GB, batch size 64, checkpoint-480 of a 500-step run, ~1.8 epochs).

This is the 0%-frozen point of an "adapter-coverage" sweep: the full, un-pruned 28-layer decoder with LoRA attached to every layer (layers_to_transform = all 28) — i.e. standard full-depth LoRA, used here as the baseline endpoint for a sweep that otherwise restricts LoRA to only the last k% of layers (25/50/75% frozen) while keeping the rest of the model frozen-but-forward-computing rather than pruned. See o-taco/VLM2Vec, branch aokvqa-single-gpu, run_freeze_sweep.sh / plot_freeze_vs_pruning.py for the full sweep and comparison against depth-pruning strategies.

This turns the backbone into a multimodal embedding model (last-token pooled, L2-normalized), not a generative chat model — load it with VLM2Vec's MMEBModel.load() (src/model.py), not AutoModelForCausalLM.

Training config

Base model: Qwen/Qwen3-VL-2B-Instruct
LoRA: r=8, alpha=16, dropout=0.0, targets q_proj,k_proj,v_proj,o_proj,gate_proj,up_proj,down_proj, all 28 decoder layers adapted
Data: MMEB-train A-OKVQA subset only
Step: 480 (best checkpoint by the metric below, out of a 500-step run)

Eval

In-batch 4-way training-accuracy proxy (compute_train_accuracy.py) = 0.86 (chance = 0.25)
This is not the paper-faithful MMEB-eval Precision@1 used for o-taco/qwen3vl-aokvqa (a separate, earlier full-LoRA run on the same subset, P@1 = 0.210 there) — the two checkpoints are not directly comparable on that number. This checkpoint has not been run through the full MMEB-eval candidate-pool protocol.

Usage

Requires the VLM2Vec repo (branch with Qwen3-VL + Volta/sdpa support).

python

from src.arguments import ModelArguments
from src.model import MMEBModel

model_args = ModelArguments(
    model_name="Qwen/Qwen3-VL-2B-Instruct",
    model_backbone="qwen3_vl",
    checkpoint_path="o-taco/qwen3vl-aokvqa-freeze0pct",
    pooling="last",
    normalize=True,
    lora=True,
)
model = MMEBModel.load(model_args, is_trainable=False)

Framework versions

PEFT 0.11.1