  Despite recent progress on computer vision and natural language processing,
developing a machine that can understand video story is still hard to achieve
due to the intrinsic difficulty of video story. Moreover, researches on how to
evaluate the degree of video understanding based on human cognitive process
have not progressed as yet. In this paper, we propose a novel video question
answering (Video QA) task, DramaQA, for a comprehensive understanding of the
video story. The DramaQA focuses on two perspectives: 1) Hierarchical QAs as an
evaluation metric based on the cognitive developmental stages of human
intelligence. 2) Character-centered video annotations to model local coherence
of the story. Our dataset is built upon the TV drama "Another Miss Oh" and it
contains 17,983 QA pairs from 23,928 various length video clips, with each QA
pair belonging to one of four difficulty levels. We provide 217,308 annotated
images with rich character-centered annotations, including visual bounding
boxes, behaviors and emotions of main characters, and coreference resolved
scripts. Additionally, we suggest Multi-level Context Matching model which
hierarchically understands character-centered representations of video to
answer questions. We release our dataset and model publicly for research
purposes, and we expect our work to provide a new perspective on video story
understanding research.
