  Touchable projection with structured light range cameras is a prolific medium
for large interaction surfaces, affording multiple simultaneous users and
simple, cheap setup. However robust touch detection in such projector-depth
systems is difficult to achieve due to measurement noise. We propose a novel
combination of surface touch detection and a deep network for hand pose
estimation, which aids in detecting both on- and above-surface hand gestures,
disambiguating multiple touch fingers, as well as recovering fingertip
positions in face of noisy input. We present the details of our GPU-accelerated
system and an evaluation of its performance, as well as applications such as an
enhanced virtual keyboard that utilizes the added features.
