  Virtual Reality (VR) headsets have become increasingly popular for remote
collaboration, but video conferencing poses challenges when the user's face is
covered by the headset. Existing solutions have limitations in terms of
accessibility. In this paper, we propose HeadsetOff, a novel system that
achieves photorealistic video conferencing on economical VR headsets by
leveraging voice-driven face reconstruction. HeadsetOff consists of three main
components: a multimodal attention-based predictor, a generator, and an
adaptive controller. The predictor effectively predicts user future behavior
based on different modalities. The generator employs voice input, head motion,
and eye blink to animate the human face. The adaptive controller dynamically
selects the appropriate generator model based on the trade-off between video
quality and delay, aiming to maximize Quality of Experience while minimizing
latency. Experimental results demonstrate the effectiveness of HeadsetOff in
achieving high-quality, low-latency video conferencing on economical VR
headsets.
