  The Alexa Prize program has empowered numerous university students to
explore, experiment, and showcase their talents in building conversational
agents through challenges like the SocialBot Grand Challenge and the TaskBot
Challenge. As conversational agents increasingly appear in multimodal and
embodied contexts, it is important to explore the affordances of conversational
interaction augmented with computer vision and physical embodiment. This paper
describes the SimBot Challenge, a new challenge in which university teams
compete to build robot assistants that complete tasks in a simulated physical
environment. This paper provides an overview of the SimBot Challenge, which
included both online and offline challenge phases. We describe the
infrastructure and support provided to the teams including Alexa Arena, the
simulated environment, and the ML toolkit provided to teams to accelerate their
building of vision and language models. We summarize the approaches the
participating teams took to overcome research challenges and extract key
lessons learned. Finally, we provide analysis of the performance of the
competing SimBots during the competition.
