  In this work, we introduce a dataset of video annotated with high quality
natural language phrases describing the visual content in a given segment of
time. Our dataset is based on the Descriptive Video Service (DVS) that is now
encoded on many digital media products such as DVDs. DVS is an audio narration
describing the visual elements and actions in a movie for the visually
impaired. It is temporally aligned with the movie and mixed with the original
movie soundtrack. We describe an automatic DVS segmentation and alignment
method for movies, that enables us to scale up the collection of a DVS-derived
dataset with minimal human intervention. Using this method, we have collected
the largest DVS-derived dataset for video description of which we are aware.
Our dataset currently includes over 84.6 hours of paired video/sentences from
92 DVDs and is growing.
