Paper: Uni-MuMER: Unified Multi-Task Fine-Tuning of Vision-Language Model for Handwritten Mathematical Expression Recognition (NeurIPS 2025 Spotlight)
| Property | Value |
|---|---|
| Base Model | Qwen/Qwen3-VL-4B-Instruct |
| Architecture | Qwen3-VL (dedicated vision-language model with ViT encoder) |
| Parameters | 4.4B |
| Training Data | Uni-MuMER-Data (~1.6M samples, 26 subsets) |
| Fine-tuning | Full (all parameters, including vision encoder) |
| Precision | bf16 |
| Template | qwen3_vl_nothink (via LLaMA-Factory) |
| Hyperparameter | Value |
|---|---|
| Epochs | 1 |
| Learning Rate | 1e-5 (cosine schedule, 10% warmup) |
| Effective Batch Size | 512 (8 GPUs x 4 per-device x 16 accumulation) |
| DeepSpeed | ZeRO-3 |
| Image Max Pixels | 262,144 (512x512) |
| Cutoff Length | 2048 tokens |
| Hardware | 8x NVIDIA A100 80GB |
| Training Time | ~20 hours |
| Final Train Loss | 0.0341 |
| Dataset | Samples | Uni-MuMER-3B | Qwen3.5-2B | Qwen3.5-4B | Qwen3-VL-2B | This Model |
|---|---|---|---|---|---|---|
| CROHME 2014 | 986 | 82.25% | 83.98% | 82.56% | 83.27% | 82.35% |
| CROHME 2016 | 1,147 | 78.29% | 81.17% | 78.20% | 78.55% | 79.34% |
| CROHME 2019 | 1,199 | 79.82% | 80.15% | 75.98% | 79.40% | 78.98% |
| CROHME 2023 Test | 2,300 | 69.52% | 69.43% | 66.74% | 70.96% | 69.17% |
| HME100K Test | 24,607 | 69.50% | 70.43% | 70.02% | 69.31% | 69.79% |
| Im2LaTeXv2 Test | 10,118 | 76.99% | 77.82% | 77.20% | 77.11% | 77.46% |
| MathWriting Test | 7,643 | 53.03% | 51.84% | 54.32% | 50.66% | 53.15% |
| MNE-N1 | 1,875 | 75.89% | 74.72% | 69.76% | 79.63% | 70.51% |
| MNE-N2 | 304 | 57.89% | 61.18% | 54.93% | 65.13% | 52.96% |
| MNE-N3 | 1,464 | 46.72% | 56.83% | 56.69% | 52.39% | 54.44% |
| Average | 72.19% | 73.09% | 71.60% | 72.49% | 72.11% |
| Dataset | CDM ExpRate |
|---|---|
| CROHME 2014 | 86.00% |
| CROHME 2016 | 82.00% |
| CROHME 2019 | 82.20% |
| CROHME 2023 Test | 79.10% |
| HME100K Test | 72.90% |
| Im2LaTeXv2 Test | 93.10% |
| MathWriting Test | 71.70% |
| MNE-N1 | 81.10% |
| MNE-N2 | 71.70% |
| MNE-N3 | 76.00% |
| Average | 80.67% |
1from vllm import LLM, SamplingParams
2
3model = LLM(
4 model="phxember/Uni-MuMER-Qwen3-VL-4B",
5 trust_remote_code=True,
6 max_model_len=2048,
7)
8
9sampling_params = SamplingParams(temperature=0.2, top_p=0.8, max_tokens=2048)
10
11# See https://github.com/BFlameSwift/Uni-MuMER for full inference scripts1git clone https://github.com/BFlameSwift/Uni-MuMER.git
2cd Uni-MuMER
3bash eval/eval_all.sh -m phxember/Uni-MuMER-Qwen3-VL-4B| Model | Base | Avg ExpRate | HuggingFace |
|---|---|---|---|
| Uni-MuMER-Qwen2.5-VL-3B | Qwen2.5-VL-3B | 72.19% | Link |
| Uni-MuMER-Qwen2.5-VL-7B | Qwen2.5-VL-7B | - | Link |
| Uni-MuMER-Qwen3.5-2B | Qwen3.5-2B | 73.09% | Link |
| Uni-MuMER-Qwen3.5-4B | Qwen3.5-4B | 71.60% | Link |
| Uni-MuMER-Qwen3-VL-2B | Qwen3-VL-2B | 72.49% | Link |
| Uni-MuMER-Qwen3-VL-4B | Qwen3-VL-4B | 72.11% | This model |
1@article{li2025unimumer,
2 title = {Uni-MuMER: Unified Multi-Task Fine-Tuning of Vision-Language Model for Handwritten Mathematical Expression Recognition},
3 author = {Li, Yu and Jiang, Jin and Zhu, Jianhua and Peng, Shuai and Wei, Baole and Zhou, Yuxuan and Gao, Liangcai},
4 year = {2025},
5 journal = {arXiv preprint arXiv:2505.23566},
6}