  The problem of building a coherent and non-monotonous conversational agent
with proper discourse and coverage is still an area of open research. Current
architectures only take care of semantic and contextual information for a given
query and fail to completely account for syntactic and external knowledge which
are crucial for generating responses in a chit-chat system. To overcome this
problem, we propose an end to end multi-stream deep learning architecture which
learns unified embeddings for query-response pairs by leveraging contextual
information from memory networks and syntactic information by incorporating
Graph Convolution Networks (GCN) over their dependency parse. A stream of this
network also utilizes transfer learning by pre-training a bidirectional
transformer to extract semantic representation for each input sentence and
incorporates external knowledge through the the neighborhood of the entities
from a Knowledge Base (KB). We benchmark these embeddings on next sentence
prediction task and significantly improve upon the existing techniques.
Furthermore, we use AMUSED to represent query and responses along with its
context to develop a retrieval based conversational agent which has been
validated by expert linguists to have comprehensive engagement with humans.
