| Model | Infer type | ASR_{top-k} | Expert grading | UE | PoE | PlE | SE | EE | RPAS |
|---|---|---|---|---|---|---|---|---|---|
| GPT-4o | FPS | 9.03 | 31.25 | 0.00 | 0.00 | 6.25 | 93.75 | 0.00 | 20.14 |
| OpenAI-o1 | FPS | 6.74 | 34.38 | 0.00 | 0.00 | 3.12 | 96.88 | 0.00 | 20.56 |
| Claude-3.5-Sonnet | FPS | 37.5 | 32.19 | 0.00 | 0.00 | 0.00 | 0.00 | 62.5 | 34.85 |
| Deepseek-R1 | FPS | 61.87 | 64.84 | 0.00 | 18.75 | 15.62 | 0.00 | 18.75 | 63.36 |
| Deepseek-R1-Distill-Llama3.3-70B | NAP | 33.70 | 60.00 | 3.12 | 59.38 | 3.12 | 0.00 | 21.88 | 46.85 |
| LLaMA-3.1-8B | NAP | 16.56 | 43.44 | 3.12 | 71.88 | 50.00 | 18.75 | 3.12 | 30.00 |
| LLaMA-3.1-70B | NAP | 27.50 | 40.94 | 0.00 | 9.38 | 87.5 | 62.5 | 21.88 | 34.22 |
| LLaMA-3.1-405B | NAP | 34.24 | 61.09 | 28.12 | 46.88 | 6.25 | 0.00 | 0.00 | 47.67 |
| LLaMA-3.3-70B | NAP | 46.02 | 60.31 | 0.00 | 48.48 | 39.39 | 15.15 | 6.06 | 53.26 |
| Gemma-2-9B | NAP | 17.36 | 29.06 | 31.25 | 34.38 | 25.0 | 0.00 | 6.25 | 23.21 |
| Qwen2.5-7B | NAP | 9.17 | 10.94 | 3.12 | 15.62 | 90.62 | 0.00 | 6.25 | 10.06 |
| Qwen2.5-72B | NAP | 40.42 | 50.62 | 0.00 | 56.25 | 18.75 | 0.00 | 6.25 | 45.52 |
| MAP-Neo-7B-Multiplan | FPS | 46.39 | 44.22 | 21.88 | 28.12 | 25.0 | 6.25 | 3.12 | 45.31 |
| EmbodiedAgent (Ours) | NAP | 74.01 | 69.69 | 9.38 | 15.62 | 3.12 | 9.38 | 0.00 | 71.85 |
