  This study investigates the direct use of speech waveforms to predict head
motion for speech-driven head-motion synthesis, whereas the use of spectral
features such as MFCC as basic input features together with additional features
such as energy and F0 is common in the literature. We show that, rather than
combining different features that originate from waveforms, it is more
effective to use waveforms directly predicting corresponding head motion. The
challenge with the waveform-based approach is that waveforms contain a large
amount of information irrelevant to predict head motion, which hinders the
training of neural networks. To overcome the problem, we propose a
canonical-correlation-constrained autoencoder (CCCAE), where hidden layers are
trained to not only minimise the error but also maximise the canonical
correlation with head motion. Compared with an MFCC-based system, the proposed
system shows comparable performance in objective evaluation, and better
performance in subject evaluation.
