A3-Qwen3.5-9B

💾 Code	📄 Paper	🌐 Website
🤗 Dataset	🤖 Models	📦 PyPI

Structured Distillation of Web Agent Capabilities Enables Generalization

Xing Han Lù, Siva Reddy

A3-Qwen3.5-9B is a 9B multimodal web agent fine-tuned from Qwen/Qwen3.5-9B on A3-Synth, a synthetic dataset generated using the Agent-as-Annotators (A3) framework.

The model achieves 41.5% on WebArena, surpassing closed-source models such as Claude 3.5 Sonnet (36.0%) and GPT-4o (31.5%) under the same evaluation protocol.

Usage

Serve with vLLM:

vllm serve McGill-NLP/A3-Qwen3.5-9B --tensor-parallel-size 2 --max-model-len 65536 --enforce-eager --dtype bfloat16

Training

Base model: Qwen/Qwen3.5-9B
Data: A3-Synth (16k examples from Gemini 3 Pro trajectories)
Method: SFT with FSDP
Max sequence length: 16,384
Learning rate: 1e-5
Epochs: 2
Batch size: 1 per GPU, gradient accumulation 4

Model Variants

Model	Parameters	Link
A3-Qwen3.5-9B	9B	McGill-NLP/A3-Qwen3.5-9B
A3-Qwen3.5-4B	4B	McGill-NLP/A3-Qwen3.5-4B
A3-Qwen3.5-2B	2B	McGill-NLP/A3-Qwen3.5-2B

Citation

@misc{lu2026structured,
      title={Structured Distillation of Web Agent Capabilities Enables Generalization}, 
      author={Xing Han Lù and Siva Reddy},
      year={2026},
      eprint={2604.07776},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/2604.07776}, 
}