  Simulation with realistic, interactive agents represents a key task for
autonomous vehicle software development. In this work, we introduce the Waymo
Open Sim Agents Challenge (WOSAC). WOSAC is the first public challenge to
tackle this task and propose corresponding metrics. The goal of the challenge
is to stimulate the design of realistic simulators that can be used to evaluate
and train a behavior model for autonomous driving. We outline our evaluation
methodology, present results for a number of different baseline simulation
agent methods, and analyze several submissions to the 2023 competition which
ran from March 16, 2023 to May 23, 2023. The WOSAC evaluation server remains
open for submissions and we discuss open problems for the task.
