  Pretraining from unlabelled web videos has quickly become the de-facto means
of achieving high performance on many video understanding tasks. Features are
learned via prediction of grounded relationships between visual content and
automatic speech recognition (ASR) tokens. However, prior pretraining work has
been limited to only instructional videos; a priori, we expect this domain to
be relatively "easy:" speakers in instructional videos will often reference the
literal objects/actions being depicted. We ask: can similar models be trained
on more diverse video corpora? And, if so, what types of videos are "grounded"
and what types are not? We fit a representative pretraining model to the
diverse YouTube8M dataset, and study its success and failure cases. We find
that visual-textual grounding is indeed possible across previously unexplored
video categories, and that pretraining on a more diverse set results in
representations that generalize to both non-instructional and instructional
domains.
