  Generative models have long been the dominant approach for speech
recognition. The success of these models however relies on the use of
sophisticated recipes and complicated machinery that is not easily accessible
to non-practitioners. Recent innovations in Deep Learning have given rise to an
alternative - discriminative models called Sequence-to-Sequence models, that
can almost match the accuracy of state of the art generative models. While
these models are easy to train as they can be trained end-to-end in a single
step, they have a practical limitation that they can only be used for offline
recognition. This is because the models require that the entirety of the input
sequence be available at the beginning of inference, an assumption that is not
valid for instantaneous speech recognition. To address this problem, online
sequence-to-sequence models were recently introduced. These models are able to
start producing outputs as data arrives, and the model feels confident enough
to output partial transcripts. These models, like sequence-to-sequence are
causal - the output produced by the model until any time, $t$, affects the
features that are computed subsequently. This makes the model inherently more
powerful than generative models that are unable to change features that are
computed from the data. This paper highlights two main contributions - an
improvement to online sequence-to-sequence model training, and its application
to noisy settings with mixed speech from two speakers.
