Field	Value
Base model	`Tongyi-MAI/MAI-UI-2B`
Training stages	full-parameter SFT, then GRPO RL
Selected checkpoint	60% checkpoint of the final train1-9 RL run
RL reward	Qwen score + pseudo-label IoU + center consistency
Output style	JSON only, no reasoning

The project code and dataset preparation scripts are intended to be released in the accompanying GitHub repository.

Why This Model

The goal was to train a small 2B slide-grounding model that can compete with larger existing grounding models on a slide-specific benchmark.

Full test1 results below use 130,166 annotated slide items. Scores are assigned by Qwen3-VL on a 0..4 scale, where 0 means invalid output and 4 means precise manual-quality localization.

Model / setting	Params class	Prompt	Best bbox postprocessing	Avg score	Valid rate
This model	2B	no thinking	none	3.7250	99.9962%
This model	2B	no thinking	loose	3.7291	99.9962%
`ui-venus-30b-fp8`	30B	direct	loose	3.722	99.62%
`ui-venus-8b-fp8`	8B	direct	loose	3.717	99.71%
`mai-ui-8b-fp8`	8B	thinking	loose	3.669	98.86%
`mai-ui-2b-fp8-no-think`	2B	no thinking	loose	3.185	88.01%
`mai-ui-8b-fp8-no-think`	8B	no thinking	loose	2.997	80.81%
`mai-ui-2b-fp8`	2B	thinking	loose	1.932	53.47%
`ui-venus-2b-fp8`	2B	direct	none	1.074	29.54%

The exported 2B model is slightly above the strongest evaluated off-the-shelf grounding baseline under the same Qwen3-VL evaluation protocol.

Evaluation Protocol

Evaluation uses the held-out test1 split of the prepared Zenodo10K-derived slide dataset.

Metric	Value
Test slides attempted	9,569
Successfully annotated test slides	9,422
Test annotation failure rate	1.54%
Evaluated annotation items	130,166
Final checkpoint valid-box rate	99.9962%
Final checkpoint average score, raw bbox	3.7250 / 4
Final checkpoint average score, loose bbox postprocessing	3.7291 / 4

Scoring rubric:

Score	Meaning
0	Invalid output or invalid box format.
1	Wrong or mostly unusable localization.
2	Partially related but not a reliable target box.
3	Correct target with approximate boundaries.
4	Correct target with precise manual-quality boundaries.

This is a model-as-judge research metric, not a final human benchmark.

Bbox postprocessing:

Strategy	Meaning
`none`	Use the raw bounding box predicted by the grounding model.
`loose`	Recenter the box on the predicted point and expand each side to the larger original half-width/half-height, so the final box is looser but still centered.
`tight`	Recenter the box on the predicted point and make width and height both use the shorter side, producing a tighter square-like box.
`mean`	Recenter the box on the predicted point and make width and height both use the average of the original half-width and half-height.

Experiment Roadmap

The full training plan was deliberately incremental. Each row kept the best setting from previous rows unless the row explicitly tested an alternative.

Order	Stage	What changed	Best full-test score
1	SFT	train1, top-1 pseudo labels, score=4 only, no reasoning	3.6784
2	SFT	train1, top-3 x 4-postprocessing ensemble, score=4 only, no reasoning	3.7022
3	SFT	train1, ensemble, keep pseudo labels with scores 1-4, no reasoning	3.7050
4	SFT	train1, ensemble, scores 1-4, add MAI-8B reasoning text	3.7019
5	SFT	train1-9, ensemble, scores 1-4, no reasoning	3.7102
6	RL R0	train1, reward = Qwen evaluation score / 4	3.7112
7	RL R1	train1, reward = Qwen score + pseudo-label mean IoU	3.7205
8	RL R2	train1, reward = Qwen score + pseudo-label mean IoU + center consistency	3.7207
9	RL final	train1-9, reward = Qwen score + pseudo-label mean IoU + center consistency	3.7250

Reward definitions:

Reward	Definition
Qwen score	Qwen evaluation score divided by `4`; invalid model outputs receive `0`.
Pseudo-label mean IoU	Mean IoU between the generated box and all score-4 boxes from the teacher pseudo-label pool.
Center consistency	`max(0, 1 - distance(point, box_center) / half_box_diagonal)` using the generated box itself.

Performance Figures

Best checkpoint from each experiment:

Checkpoint score curves:

Per-item-type comparison between the retained SFT baseline and the exported RL checkpoint:

The figure-generation script and the source CSV files are included in this model repository under scripts/ and figures/.

Data Preparation

The data comes from Zenodo10K slide decks. The preparation pipeline was:

Filter Zenodo10K decks and render slide screenshots.
Ask Qwen3-VL to annotate visible, semantically meaningful slide items.
Keep only legal annotation item types: chart, chart_component, image, image_component, reference_link, table, table_cell, table_column, table_row, and text_line.
Run multiple grounding models for every annotation item.
Evaluate each grounded candidate with Qwen3-VL.
Select pseudo labels for SFT from the best teacher candidates.

Annotation scale:

Split group	Attempted slides	Successful slides	Failed slides	Failure rate	Items
`test1`	9,569	9,422	147	1.54%	130,166
`train1..train9`	86,516	85,253	1,263	1.46%	1,162,921

Teacher Pseudo Labels

The SFT pseudo-label source was a top-3 x four-strategy ensemble.

Teacher models:

ui-venus-30b-fp8;
ui-venus-8b-fp8;
mai-ui-8b-fp8.

For every teacher output, we evaluated four bbox postprocessing strategies:

raw none;
loose;
tight;
mean.

For each item, the selected pseudo label was the candidate with the highest Qwen evaluation score.

Selector	Avg score	Score-4 rate	Score <= 2 count
Best single teacher: `ui-venus-30b-fp8 + loose`	3.722	76.53%	3,284
Top-3 teachers + `loose` only	3.915	92.54%	891
Top-3 teachers + `none/loose`	3.960	96.69%	582
Top-3 teachers + all 4 strategies	3.981	98.44%	287

The same selector was run on train1..train9 and reached about 3.982 average score with about 98.49% score-4 rate.

SFT Results

SFT used full-parameter HuggingFace/DDP training. LoRA was not used.

Main findings:

Ensemble pseudo labels were better than direct top-1 pseudo labels.
Keeping pseudo labels with scores 1..4 was better than keeping only score 4.
Adding MAI-8B reasoning text did not improve the 2B student.
Scaling from train1 to train1..train9 improved the best SFT score.

Best SFT checkpoints:

Run	Best checkpoint	Avg score	Valid rate
SFT train1, top-1 labels, score=4, no reasoning	`080p`	3.6784	99.45%
SFT train1, top-3 x 4 ensemble, score=4, no reasoning	`060p`	3.7022	99.95%
SFT train1, top-3 x 4 ensemble, score=1-4, no reasoning	`100p`	3.7050	99.95%
SFT train1, top-3 x 4 ensemble, score=1-4, with reasoning	`080p`	3.7019	99.94%
SFT train1-9, top-3 x 4 ensemble, score=1-4, no reasoning	`100p`	3.7102	99.98%

Measured formal SFT cost through the retained train1-9 baseline was about 372.17 productive GPU-hours on 8 x H100 GPUs.

RL Results

RL used GRPO from the retained SFT baseline.

Full-scale RL setup:

Field	Value
RL examples	1,145,400
Batch / grad accumulation / generations	2 / 2 / 4
Effective prompts per optimizer step	32
Learning rate	`1e-6`
KL beta	`0`
Max steps	35,794
Selected checkpoint	21,474 steps (`060p`)
Runtime	86.42h
Productive GPU-hours	691.33 on `8 x H100` GPUs

Final train1-9 RL checkpoint curve:

Checkpoint	Avg score	Valid rate
`020p`	3.7232	99.9915%
`040p`	3.7217	99.9885%
`060p`	3.7250	99.9962%
`080p`	3.7238	99.9915%
`100p`	3.7234	99.9969%

The best checkpoint was not the final checkpoint, so intermediate checkpoint evaluation mattered.

Per-type score for the exported 060p checkpoint:

Item type	Count	Avg score	Delta vs SFT baseline
`chart`	1,612	3.8883	-0.0025
`chart_component`	4,359	3.6082	+0.0218
`image`	11,305	3.9010	-0.0047
`image_component`	14,038	3.7223	+0.0128
`reference_link`	4,194	3.8090	+0.0153
`table`	1,036	3.8861	+0.0116
`table_cell`	6,963	3.7604	+0.0111
`table_column`	2,294	3.6561	+0.0057
`table_row`	3,333	3.7669	+0.0009
`text_line`	81,032	3.6948	+0.0190

Usage

Example vLLM serve command:

bash

vllm serve . \
  --served-model-name slide-grounding-mai-ui-2b-rl \
  --trust-remote-code \
  --dtype auto \
  --max-model-len 6144 \
  --limit-mm-per-prompt '{"image":1,"video":0}'

Recommended prompt:

text

You are a GUI grounding agent.

Given a slide screenshot and a text instruction or narration segment, locate
the visual region in the slide that the text is referring to.

Use normalized screenshot coordinates from 0 to 1000.
The origin (0,0) is the top-left corner of the image.
x increases to the right, y increases downward.

Return only valid JSON:
{"coordinate": [cx, cy], "bbox_2d": [x1, y1, x2, y2]}

Input instruction:
Find the item in the slide based on the description: "<description>".

The expected output is:

{"coordinate": [512, 208], "bbox_2d": [300, 160, 724, 256]}

Limitations

The reported metric is Qwen3-VL model-as-judge evaluation, not a final human benchmark.
The model is specialized for rendered slide screenshots and item descriptions similar to the annotation schema above.
The model predicts one box per instruction; it does not independently parse a full slide into all items.
The training labels are pseudo labels selected from stronger teacher models, not fully human-labeled boxes.
Check upstream Zenodo10K, MAI-UI, UI-Venus, and Qwen licenses before redistributing derived datasets or annotations.

jiangxuehaokeai/slide-grounding-mai-ui-2b-rl

Slide Grounding MAI-UI-2B RL