  Face-to-face conversation in Virtual Reality (VR) is a challenge when
participants wear head-mounted displays (HMD). A significant portion of a
participant's face is hidden and facial expressions are difficult to perceive.
Past research has shown that high-fidelity face reconstruction with personal
avatars in VR is possible under laboratory conditions with high-cost hardware.
In this paper, we propose one of the first low-cost systems for this task which
uses only open source, free software and affordable hardware. Our approach is
to track the user's face underneath the HMD utilizing a Convolutional Neural
Network (CNN) and generate corresponding expressions with Generative
Adversarial Networks (GAN) for producing RGBD images of the person's face. We
use commodity hardware with low-cost extensions such as 3D-printed mounts and
miniature cameras. Our approach learns end-to-end without manual intervention,
runs in real time, and can be trained and executed on an ordinary gaming
computer. We report evaluation results showing that our low-cost system does
not achieve the same fidelity of research prototypes using high-end hardware
and closed source software, but it is capable of creating individual facial
avatars with person-specific characteristics in movements and expressions.
