  We study the problem of acoustic feature learning in the setting where we
have access to another (non-acoustic) modality for feature learning but not at
test time. We use deep variational canonical correlation analysis (VCCA), a
recently proposed deep generative method for multi-view representation
learning. We also extend VCCA with improved latent variable priors and with
adversarial learning. Compared to other techniques for multi-view feature
learning, VCCA's advantages include an intuitive latent variable interpretation
and a variational lower bound objective that can be trained end-to-end
efficiently. We compare VCCA and its extensions with previous feature learning
methods on the University of Wisconsin X-ray Microbeam Database, and show that
VCCA-based feature learning improves over previous methods for
speaker-independent phonetic recognition.
