  The VirtualCube system is a 3D video conference system that attempts to
overcome some limitations of conventional technologies. The key ingredient is
VirtualCube, an abstract representation of a real-world cubicle instrumented
with RGBD cameras for capturing the 3D geometry and texture of a user. We
design VirtualCube so that the task of data capturing is standardized and
significantly simplified, and everything can be built using off-the-shelf
hardware. We use VirtualCubes as the basic building blocks of a virtual
conferencing environment, and we provide each VirtualCube user with a
surrounding display showing life-size videos of remote participants. To achieve
real-time rendering of remote participants, we develop the V-Cube View
algorithm, which uses multi-view stereo for more accurate depth estimation and
Lumi-Net rendering for better rendering quality. The VirtualCube system
correctly preserves the mutual eye gaze between participants, allowing them to
establish eye contact and be aware of who is visually paying attention to them.
The system also allows a participant to have side discussions with remote
participants as if they were in the same room. Finally, the system sheds lights
on how to support the shared space of work items (e.g., documents and
applications) and track the visual attention of participants to work items.
