  As digital virtual assistants become ubiquitous, it becomes increasingly
important to understand the situated behaviour of users as they interact with
these assistants. To this end, we introduce SIMMC, an extension to ParlAI for
multi-modal conversational data collection and system evaluation. SIMMC
simulates an immersive setup, where crowd workers are able to interact with
environments constructed in AI Habitat or Unity while engaging in a
conversation. The assistant in SIMMC can be a crowd worker or Artificial
Intelligent (AI) agent. This enables both (i) a multi-player / Wizard of Oz
setting for data collection, or (ii) a single player mode for model / system
evaluation. We plan to open-source a situated conversational data-set collected
on this platform for the Conversational AI research community.
