You can deploy this model and start earning money today!

cnxup/
Qwen2.5-VL-7B-MLA-stage2-rope32-d_kv_128

Name: Qwen2.5-VL-7B-MLA-stage2-rope32-d_kv_128
Availability: InStock

safetensors

qwen2_5_vl

2601.11464

Views

No views yet

Research Paper: "MHA2MLA-VLM: Enabling DeepSeek's Economical Multi-Head Latent Attention across Vision-Language Models"

Inference

Step 1: Download the Stage 1 checkpoint cnxup/Qwen2.5-VL-7B-MLA-stage1-rope32
Step 2: For MHA2MLA-VLM models using Partial-RoPE MKL method, Download the MKL file.
Step 3: For MHA2MLA-VLM models using MD-SVD method, Take the d_kv_128 as an example.
Step 4: For evaluation, please visit our GitHub and use lmms-eval for benchmarking. Detailed instructions can be found in the Evaluation section.

shell

cd eval
cd qwen2_5_vl
sh eval.sh

Citation

bibtex

@misc{fan2026mha2mlavlmenablingdeepseekseconomical,
      title={MHA2MLA-VLM: Enabling DeepSeek's Economical Multi-Head Latent Attention across Vision-Language Models}, 
      author={Xiaoran Fan and Zhichao Sun and Tao Ji and Lixing Shen and Tao Gui},
      year={2026},
      eprint={2601.11464},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2601.11464}, 
}

cnxup/Qwen2.5-VL-7B-MLA-stage2-rope32-d_kv_128

Inference

Citation

cnxup/
Qwen2.5-VL-7B-MLA-stage2-rope32-d_kv_128