Social VR: The Influence of Directivity of Speech and Coverbal Gestures on the Perception of Conversational Virtual Humans in Immersive Virtual EnvironmentsVisual Computing Institute
In the realm of social virtual reality, virtual humans (VHs) can, for example, function as guides who impart certain knowledge to users or they can serve as training partners in order to provide users a safe, non-judgmental, reproducible however adaptable opportunity to train special skills. To this end, believable and conversational VHs for immersive virtual reality applications need to be designed. Thereby it is required that their non-verbal manners simulate human behavior, but also the VHs verbal behavior is elementary.
Here, factors like correct lip synchronization, coverbal gestures and mimics are fundamental, but also the auralization of the spoken word is of prime importance: using an auralization with an omnidirectional sound radiation allows listeners to relate the spoken word only to the speaker’s specific location inside the virtual environment. Extending the approach by the directivity of the human voice allows listeners to relate the sound also to the speaker’s orientation, e.g., whether they are located face-to-face or face-to-back.
In this project, we are working on two research aspects of conversational VHs in order to investigate their impact and influence on the perceived social presence of the VHs and on the users’ sense of presence: (1) In a first step, we will realize the omnidirectional and an average human speaker’s directional voice radiation. By means of an abstract, restricted scenario (see figure), which allows dynamic interactions, i.e., ongoing-speech while the VA switches his focus from the conversation partner to another part of the scene, we will then compare both auralization approaches. (2) In a second step, we will embed coverbal gestures, which will be either speech-accompanying, speech-contradicting or completely absent. For both research aspects, lip synchronization is used for the VHs.