  A problem with many current Large Language Model (LLM) driven spoken
dialogues is the response time. Some efforts such as Groq address this issue by
lightning fast processing of the LLM, but we know from the cognitive psychology
literature that in human-to-human dialogue often responses occur prior to the
speaker completing their utterance. No amount of delay for LLM processing is
acceptable if we wish to maintain human dialogue latencies. In this paper, we
discuss methods for understanding an utterance in close to real time and
generating a response so that the system can comply with human-level
conversational turn delays. This means that the information content of the
final part of the speaker's utterance is lost to the LLM. Using the Google
NaturalQuestions (NQ) database, our results show GPT-4 can effectively fill in
missing context from a dropped word at the end of a question over 60% of the
time. We also provide some examples of utterances and the impacts of this
information loss on the quality of LLM response in the context of an avatar
that is currently under development. These results indicate that a simple
classifier could be used to determine whether a question is semantically
complete, or requires a filler phrase to allow a response to be generated
within human dialogue time constraints.
