You can deploy this model and start earning money today!

Kavin60606/
qwen3-vl-2b-affordance-checkpoint3500

Name: qwen3-vl-2b-affordance-checkpoint3500
Availability: InStock

safetensors

Views

No views yet

Qwen3-VL-2B Affordance Model - Checkpoint 3500

Fine-tuned Qwen3-VL-2B-Instruct for egocentric visual affordance detection.

Model Details

Base Model: Qwen3-VL-2B-Instruct
Finetuning Method: QLoRA 4-bit (Unsloth)
Training Data: 9,158 egocentric affordance annotations from 50 YouTube videos
Training Progress: Checkpoint 3500/6870 (51% complete, Epoch 1.53/3.00)
Final Loss: 0.097 (98.2% reduction from initial 5.357)

Performance

Evaluation Results (30 validation samples):

Exact Match: 17/30 (56.7%)
Valid Labels: 30/30 (100.0%)
Zero hallucinations, all predictions follow terse "verb object" format

LoRA Configuration

Rank: 16, Alpha: 32
Target modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
Vision encoder (ViT): Frozen
Language layers: Trainable

Training Hyperparameters

Batch size: 1 (gradient accumulation: 4)
Learning rate: 2e-4 (cosine schedule, 10% warmup)
Optimizer: AdamW 8-bit
Precision: BF16
Max sequence length: 2048

Usage

Inference

python

from unsloth import FastVisionModel
from peft import PeftModel
from PIL import Image

# Load base model
model, processor = FastVisionModel.from_pretrained(
    "unsloth/Qwen3-VL-2B-Instruct",
    load_in_4bit=True,
    max_seq_length=2048,
)

# Load LoRA adapter
model = PeftModel.from_pretrained(model, "sanskxr02/qwen3-vl-2b-affordance-checkpoint3500")
FastVisionModel.for_inference(model)

# Run inference
image = Image.open("frame.jpg").convert("RGB").resize((384, 216))
messages = [{
    "role": "user",
    "content": [
        {"type": "image", "image": image},
        {"type": "text", "text": "What is the affordance in this scene?"},
    ],
}]

text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True, enable_thinking=False)
inputs = processor(text=[text], images=[image], return_tensors="pt", padding=True).to(model.device)

output_ids = model.generate(**inputs, max_new_tokens=20, temperature=0.0, do_sample=False)
prediction = processor.tokenizer.decode(output_ids[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True)
print(f"Affordance: {prediction}")

Dataset

Source: 50 YouTube egocentric videos
- 20 making coffee
- 17 cooking meal
- 12 cutting vegetables
- 4 washing dishes
Total frames: 10,580
Train/Val split: 9,158 / 1,018 (skipped "none" labels)
Format: Terse (no system prompt, just "verb object" labels)

Files

adapter_config.json, adapter_model.safetensors - LoRA adapter weights
checkpoint-3500/ - Full checkpoint with trainer state
evaluation/ - Evaluation results with annotated images and training curves
dataset/ - Training data files (annotations, train/val splits)
train.py, inference.py - Training and inference scripts

Allowed Affordance Verbs

grasp, pull, push, rotate, press, lift, slide, place, open, close, insert, remove, pour, cut, stir, scoop, squeeze, turn_on, turn_off

Citation

bibtex

@model{qwen3vl_affordance_ckpt3500,
  title={Qwen3-VL-2B Affordance Detection - Checkpoint 3500},
  author={Kavin (sanskxr02)},
  year={2026},
  publisher={HuggingFace},
  url={https://huggingface.co/sanskxr02/qwen3-vl-2b-affordance-checkpoint3500}
}

License

Same as base Qwen3-VL-2B-Instruct model.

Checkpoint saved at step 3500 (Epoch 1.53), training stopped for evaluation.

Kavin60606/qwen3-vl-2b-affordance-checkpoint3500