You can deploy this model and start earning money today!

pmadinei/
Interlace-Qwen3-VL-4B-10pc

Name: Interlace-Qwen3-VL-4B-10pc
Availability: InStock

Views

No views yet

✂️ Interlace-Qwen3-VL-4B-10pc

Paper

GitHub

This model was produced by INTERLACE, a layer-pruning framework for Vision-Language Models. 10% of the transformer layers in Qwen/Qwen3-VL-4B-Instruct were removed using triplet-based similarity analysis, and the remaining model was fine-tuned on 1% of FineVision for a single epoch.

93.9% average relative performance retained | 10% layers dropped (3 of 36) | 33 layers remaining

📋 Model Details

Property	Value
Base Model	Qwen/Qwen3-VL-4B-Instruct
Pruning Method	INTERLACE (triplet-based interleaved pruning)
Pruning Ratio	10% (3 of 36 layers removed)
Remaining Layers	33
Hidden Size	2560
Fine-tuning Data	1% of FineVision (~240K samples)
Fine-tuning Epochs	1

🚀 Usage

python

from transformers import AutoModelForImageTextToText, AutoProcessor

model = AutoModelForImageTextToText.from_pretrained(
    "pmadinei/Interlace-Qwen3-VL-4B-10pc",
    dtype="auto",
    device_map="auto",
    attn_implementation="flash_attention_2",
)
processor = AutoProcessor.from_pretrained("Qwen/Qwen3-VL-4B-Instruct")

messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "image": "path/to/image.jpg"},
            {"type": "text", "text": "Describe this image in detail."},
        ],
    }
]

inputs = processor.apply_chat_template(
    messages, tokenize=True, add_generation_prompt=True,
    return_dict=True, return_tensors="pt",
).to(model.device)

output = model.generate(**inputs, max_new_tokens=512)
print(processor.decode(output[0], skip_special_tokens=True))

📊 Performance

Relative performance compared to the unpruned baseline (% of baseline score, Chain-of-Thought enabled):

Category	Benchmark	Relative Perf.
Text/Chart	AI2D	94.3%
Text/Chart	ChartQA	95.9%
Text/Chart	OCRBench	90.9%
Text/Chart	TextVQA	94.6%
General VQA	MMBench	89.9%
General VQA	POPE	99.3%
General VQA	RealWorldQA	93.9%
Perception	HRBench4K	93.6%
Perception	HRBench8K	92.8%
Perception	V-Star	94.8%
Inst & Sci	MIABench	92.7%
Inst & Sci	ScienceQA	94.4%
	Overall Average	93.9%

🤗 All INTERLACE Models

Model	Drop %	Rel. Perf.
Interlace-Qwen3-VL-8B-10pc	10%	94.0%
Interlace-Qwen3-VL-8B-15pc	15%	92.1%
Interlace-Qwen3-VL-8B-20pc	20%	86.9%
Interlace-Qwen3-VL-8B-25pc	25%	86.1%
Interlace-Qwen3-VL-4B-10pc	10%	93.9%
Interlace-Qwen3-VL-4B-15pc	15%	91.9%
Interlace-Qwen3-VL-4B-20pc	20%	88.0%
Interlace-Qwen3-VL-4B-25pc	25%	81.7%

📝 Citation

bibtex

@inproceedings{madinei2026interlace,
  title={Interlace: Interleaved layer pruning and efficient adaptation in large vision-language models},
  author={Madinei, Parsa and Solgi, Ryan and Wen, Ziqi and Skaza, Jonathan and Eckstein, Miguel and Pedarsani, Ramtin},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  pages={2947--2956},
  year={2026}
}