  Far-field speech recognition is a challenging task that conventionally uses
signal processing beamforming to attack noise and interference problem. But the
performance has been found usually limited due to heavy reliance on
environmental assumption. In this paper, we propose a unified multichannel
far-field speech recognition system that combines the neural beamforming and
transformer-based Listen, Spell, Attend (LAS) speech recognition system, which
extends the end-to-end speech recognition system further to include speech
enhancement. Such framework is then jointly trained to optimize the final
objective of interest. Specifically, factored complex linear projection (fCLP)
has been adopted to form the neural beamforming. Several pooling strategies to
combine look directions are then compared in order to find the optimal
approach. Moreover, information of the source direction is also integrated in
the beamforming to explore the usefulness of source direction as a prior, which
is usually available especially in multi-modality scenario. Experiments on
different microphone array geometry are conducted to evaluate the robustness
against spacing variance of microphone array. Large in-house databases are used
to evaluate the effectiveness of the proposed framework and the proposed method
achieve 19.26\% improvement when compared with a strong baseline.
