  In this paper, we investigate the use of large multimodal models (LMMs) for
generating virtual activities, leveraging the integration of vision-language
modalities to enable the interpretation of virtual environments. This approach
not only facilitates the recognition of scene layouts, semantic contexts, and
object identities, but also empowers LMMs to abstract the elements of a scene.
By correlating these abstractions with massive knowledge about human
activities, LMMs are capable of generating adaptive and contextually relevant
virtual activities. We propose a structured framework for articulating abstract
activity descriptions, with an emphasis on delineating character interactions
within the virtual milieu. Utilizing the derived high-level contexts, our
methodology proficiently positions virtual characters, ensuring that their
interactions and behaviors are realistically and contextually congruent through
strategic optimizations. The implications of our findings are significant,
offering a novel pathway for enhancing the realism and contextual
appropriateness of virtual activities in simulated environments.
