  We introduce the novel task of Pano2Vid $-$ automatic cinematography in
panoramic 360$^{\circ}$ videos. Given a 360$^{\circ}$ video, the goal is to
direct an imaginary camera to virtually capture natural-looking normal
field-of-view (NFOV) video. By selecting "where to look" within the panorama at
each time step, Pano2Vid aims to free both the videographer and the end viewer
from the task of determining what to watch. Towards this goal, we first compile
a dataset of 360$^{\circ}$ videos downloaded from the web, together with
human-edited NFOV camera trajectories to facilitate evaluation. Next, we
propose AutoCam, a data-driven approach to solve the Pano2Vid task. AutoCam
leverages NFOV web video to discriminatively identify space-time "glimpses" of
interest at each time instant, and then uses dynamic programming to select
optimal human-like camera trajectories. Through experimental evaluation on
multiple newly defined Pano2Vid performance measures against several baselines,
we show that our method successfully produces informative videos that could
conceivably have been captured by human videographers.
