  Videos on the Internet are paired with pieces of text, such as titles and
descriptions. This text typically describes the most important content in the
video, such as the objects in the scene and the actions being performed. Based
on this observation, we propose to use text as a method for learning video
representations. To accomplish this, we propose a data collection process and
use it to collect 70M video clips shared publicly on the Internet, and we then
train a model to pair each video with its associated text. We evaluate the
model on several down-stream action recognition tasks, including Kinetics,
HMDB-51, and UCF-101. We find that this approach is an effective method of
pre-training video representations. Specifically, it outperforms all existing
methods for self-supervised and cross-modal video representation learning.
