  In this paper, we propose an online attention mechanism, known as cumulative
attention (CA), for streaming Transformer-based automatic speech recognition
(ASR). Inspired by monotonic chunkwise attention (MoChA) and head-synchronous
decoder-end adaptive computation steps (HS-DACS) algorithms, CA triggers the
ASR outputs based on the acoustic information accumulated at each encoding
timestep, where the decisions are made using a trainable device, referred to as
halting selector. In CA, all the attention heads of the same decoder layer are
synchronised to have a unified halting position. This feature effectively
alleviates the problem caused by the distinct behaviour of individual heads,
which may otherwise give rise to severe latency issues as encountered by MoChA.
The ASR experiments conducted on AIShell-1 and Librispeech datasets demonstrate
that the proposed CA-based Transformer system can achieve on par or better
performance with significant reduction in latency during inference, when
compared to other streaming Transformer systems in literature.
