  We present MCQA, a learning-based algorithm for multimodal question
answering. MCQA explicitly fuses and aligns the multimodal input (i.e. text,
audio, and video), which forms the context for the query (question and answer).
Our approach fuses and aligns the question and the answer within this context.
Moreover, we use the notion of co-attention to perform cross-modal alignment
and multimodal context-query alignment. Our context-query alignment module
matches the relevant parts of the multimodal context and the query with each
other and aligns them to improve the overall performance. We evaluate the
performance of MCQA on Social-IQ, a benchmark dataset for multimodal question
answering. We compare the performance of our algorithm with prior methods and
observe an accuracy improvement of 4-7%.
