  In computer vision pixelwise dense prediction is the task of predicting a
label for each pixel in the image. Convolutional neural networks achieve good
performance on this task, while being computationally efficient. In this paper
we carry these ideas over to the problem of assigning a sequence of labels to a
set of speech frames, a task commonly known as framewise classification. We
show that dense prediction view of framewise classification offers several
advantages and insights, including computational efficiency and the ability to
apply batch normalization. When doing dense prediction we pay specific
attention to strided pooling in time and introduce an asymmetric dilated
convolution, called time-dilated convolution, that allows for efficient and
elegant implementation of pooling in time. We show results using time-dilated
convolutions in a very deep VGG-style CNN with batch normalization on the Hub5
Switchboard-2000 benchmark task. With a big n-gram language model, we achieve
7.7% WER which is the best single model single-pass performance reported so
far.
