  In voice-enabled applications, a predetermined hotword isusually used to
activate a device in order to attend to the query.However, speaking queries
followed by a hotword each timeintroduces a cognitive burden in continued
conversations. Toavoid repeating a hotword, we propose a streaming
end-to-end(E2E) intended query detector that identifies the utterancesdirected
towards the device and filters out other utterancesnot directed towards device.
The proposed approach incor-porates the intended query detector into the E2E
model thatalready folds different components of the speech recognitionpipeline
into one neural network.The E2E modeling onspeech decoding and intended query
detection also allows us todeclare a quick intended query detection based on
early partialrecognition result, which is important to decrease latencyand make
the system responsive. We demonstrate that theproposed E2E approach yields a
22% relative improvement onequal error rate (EER) for the detection accuracy
and 600 mslatency improvement compared with an independent intendedquery
detector. In our experiment, the proposed model detectswhether the user is
talking to the device with a 8.7% EERwithin 1.4 seconds of median latency after
user starts speaking.
