  Can we teach a robot to recognize and make predictions for activities that it
has never seen before? We tackle this problem by learning models for video from
text. This paper presents a hierarchical model that generalizes instructional
knowledge from large-scale text corpora and transfers the knowledge to video.
Given a portion of an instructional video, our model recognizes and predicts
coherent and plausible actions multiple steps into the future, all in rich
natural language. To demonstrate the capabilities of our model, we introduce
the \emph{Tasty Videos Dataset V2}, a collection of 4022 recipes for zero-shot
learning, recognition and anticipation. Extensive experiments with various
evaluation metrics demonstrate the potential of our method for generalization,
given limited video data for training models.
