  Streaming speech recognition architectures are employed for low-latency,
real-time applications. Such architectures are often characterized by their
causality. Causal architectures emit tokens at each frame, relying only on
current and past signal, while non-causal models are exposed to a window of
future frames at each step to increase predictive accuracy. This dichotomy
amounts to a trade-off for real-time Automatic Speech Recognition (ASR) system
design: profit from the low-latency benefit of strictly-causal architectures
while accepting predictive performance limitations, or realize the modeling
benefits of future-context models accompanied by their higher latency penalty.
In this work, we relax the constraints of this choice and present the Adaptive
Non-Causal Attention Transducer (ANCAT). Our architecture is non-causal in the
traditional sense, but executes in a low-latency, streaming manner by
dynamically choosing when to rely on future context and to what degree within
the audio stream. The resulting mechanism, when coupled with our novel
regularization algorithms, delivers comparable accuracy to non-causal
configurations while improving significantly upon latency, closing the gap with
their causal counterparts. We showcase our design experimentally by reporting
comparative ASR task results with measures of accuracy and latency on both
publicly accessible and production-scale, voice-assistant datasets.
