  Speech recognition is one of the key topics in artificial intelligence, as it
is one of the most common forms of communication in humans. Researchers have
developed many speech-controlled prosthetic hands in the past decades,
utilizing conventional speech recognition systems that use a combination of
neural network and hidden Markov model. Recent advancements in general-purpose
graphics processing units (GPGPUs) enable intelligent devices to run deep
neural networks in real-time. Thus, state-of-the-art speech recognition systems
have rapidly shifted from the paradigm of composite subsystems optimization to
the paradigm of end-to-end optimization. However, a low-power embedded GPGPU
cannot run these speech recognition systems in real-time. In this paper, we
show the development of deep convolutional neural networks (CNN) for speech
control of prosthetic hands that run in real-time on a NVIDIA Jetson TX2
developer kit. First, the device captures and converts speech into 2D features
(like spectrogram). The CNN receives the 2D features and classifies the hand
gestures. Finally, the hand gesture classes are sent to the prosthetic hand
motion control system. The whole system is written in Python with Keras, a deep
learning library that has a TensorFlow backend. Our experiments on the CNN
demonstrate the 91% accuracy and 2ms running time of hand gestures (text
output) from speech commands, which can be used to control the prosthetic hands
in real-time.
