  User interfaces provide an interactive window between physical and virtual
environments. A new concept in the field of human-computer interaction is a
soft user interface; a compliant surface that facilitates touch interaction
through deformation. Despite the potential of these interfaces, they currently
lack a signal processing framework that can efficiently extract information
from their deformation. Here we present OrbTouch, a device that uses
statistical learning algorithms, based on convolutional neural networks, to map
deformations from human touch to categorical labels (i.e., gestures) and touch
location using stretchable capacitor signals as inputs. We demonstrate this
approach by using the device to control the popular game Tetris. OrbTouch
provides a modular, robust framework to interpret deformation in soft media,
laying a foundation for new modes of human computer interaction through shape
changing solids.
