AHEAD-ALFWorld-1.7B
Overview
AHEAD-ALFWorld-1.7B is a text-based embodied-interaction agent for ALFWorld, built on
Qwen3-1.7B and trained with AHEAD. It is evaluated as a standard interactive policy: at
inference time the model acts from the ordinary observation–action history alone, with no analyzer,
skill bank, retrieval module, or extra skill prompt.
This is the 1.7B checkpoint of the AHEAD-ALFWorld family (1.7B / 3B / 7B).
Evaluation protocol
All numbers are success rate (%) under one fixed protocol:
| Setting | Value |
|---|
| Splits | ALFWorld seen (valid_seen, 140 games) and unseen (out_of_distribution, 134 games) |
| Coverage | Full split, every game evaluated once (eval_sequential=True, no sub-sampling) |
| Decoding | temperature=0.4, do_sample=True |
| Max steps / history | max_steps=50, history_length=2 |
| Aggregation | Micro-average — Avg. = (total games solved) / (total games), pooled over all task families |
Task families: Pick (pick_and_place), Look (look_at_obj_in_light), Clean
(pick_clean_then_place_in_recep), Heat (pick_heat_then_place_in_recep), Cool
(pick_cool_then_place_in_recep), Pick2 (pick_two_obj_and_place).
Split composition (fixed pool):
| Split | Pick | Look | Clean | Heat | Cool | Pick2 | Total |
|---|
| seen | 35 | 13 | 27 | 16 | 25 | 24 | 140 |
| unseen | 24 | 18 | 31 | 23 | 21 | 17 | 134 |
Results
All Avg. numbers are micro-averages (total games solved / total games).
| Split | Pick | Look | Clean | Heat | Cool | Pick2 | Avg. (micro) |
|---|
| Seen (140) | 82.9 | 53.8 | 70.4 | 68.8 | 68.0 | 41.7 | 66.4 |
| Unseen (134) | 66.7 | 44.4 | 51.6 | 73.9 | 57.1 | 58.8 | 59.0 |
Notes
- Aggregation is micro-average (games-solved / games-total), not an unweighted macro-average over
the six families.
- Small-family variance. Look (13 seen / 18 unseen games) and Pick2 (24 / 17) have small
denominators, so a few episodes move those family rates by several points.
- Protocol caveat. Numbers use
history_length=2 and temperature=0.4 sampling (not greedy);
other regimes (e.g. history_length=5, greedy decode) are not comparable to the table above.
Model family
| Size | Base model | Repo |
|---|
| 1.7B | Qwen3-1.7B | (this model) |
| 3B | Qwen2.5-3B | AHEAD-ALFWorld-3B |
| 7B | Qwen2.5-7B | AHEAD-ALFWorld-7B |
License
Derivative of Qwen3-1.7B; distributed under the Qwen Research License Agreement.
