  Conventional speech enhancement technique such as beamforming has known
benefits for far-field speech recognition. Our own work in frequency-domain
multi-channel acoustic modeling has shown additional improvements by training a
spatial filtering layer jointly within an acoustic model. In this paper, we
further develop this idea and use frequency aligned network for robust
multi-channel automatic speech recognition (ASR). Unlike an affine layer in the
frequency domain, the proposed frequency aligned component prevents one
frequency bin influencing other frequency bins. We show that this modification
not only reduces the number of parameters in the model but also significantly
and improves the ASR performance. We investigate effects of frequency aligned
network through ASR experiments on the real-world far-field data where users
are interacting with an ASR system in uncontrolled acoustic environments. We
show that our multi-channel acoustic model with a frequency aligned network
shows up to 18% relative reduction in word error rate.
