This model was developed using the Agent-as-Annotators (A3) framework, which structures synthetic trajectory generation for web agents by analogy to human annotation roles, replacing the Task Designer, Annotator, and Supervisor with modular LLM components. See A3-Qwen3.5-9B for full details on the framework performance and methodology.

Quick Start: Evaluation

To evaluate the model using the official framework, first install the package:

pip install agent-as-annotators

Then, you can serve the model and run evaluation:

bash

# 1. Serve the model (e.g. using vLLM)
vllm serve --model McGill-NLP/A3-Qwen3.5-4B

# 2. Run evaluation on a benchmark
a3-eval --benchmark webarena_test --model A3-qwen3.5-4b

Citation

bibtex

@misc{lu2026structured,
      title={Structured Distillation of Web Agent Capabilities Enables Generalization}, 
      author={Xing Han Lù and Siva Reddy},
      year={2026},
      eprint={2604.07776},
      archivePrefix={arXiv},
      primaryClass={cs.LG}
}