  Recently, attention-based encoder-decoder (AED) models have shown
state-of-the-art performance in automatic speech recognition (ASR). As the
original AED models with global attentions are not capable of online inference,
various online attention schemes have been developed to reduce ASR latency for
better user experience. However, a common limitation of the conventional
softmax-based online attention approaches is that they introduce an additional
hyperparameter related to the length of the attention window, requiring
multiple trials of model training for tuning the hyperparameter. In order to
deal with this problem, we propose a novel softmax-free attention method and
its modified formulation for online attention, which does not need any
additional hyperparameter at the training phase. Through a number of ASR
experiments, we demonstrate the tradeoff between the latency and performance of
the proposed online attention technique can be controlled by merely adjusting a
threshold at the test phase. Furthermore, the proposed methods showed
competitive performance to the conventional global and online attentions in
terms of word-error-rates (WERs).
