  We propose the application of a semi-supervised learning method to improve
the performance of acoustic modelling for automatic speech recognition based on
deep neural net- works. As opposed to unsupervised initialisation followed by
supervised fine tuning, our method takes advantage of both unlabelled and
labelled data simultaneously through mini- batch stochastic gradient descent.
We tested the method with varying proportions of labelled vs unlabelled
observations in frame-based phoneme classification on the TIMIT database. Our
experiments show that the method outperforms standard supervised training for
an equal amount of labelled data and provides competitive error rates compared
to state-of-the-art graph-based semi-supervised learning techniques.
