  360$^{\circ}$ video requires human viewers to actively control "where" to
look while watching the video. Although it provides a more immersive experience
of the visual content, it also introduces additional burden for viewers;
awkward interfaces to navigate the video lead to suboptimal viewing
experiences. Virtual cinematography is an appealing direction to remedy these
problems, but conventional methods are limited to virtual environments or rely
on hand-crafted heuristics. We propose a new algorithm for virtual
cinematography that automatically controls a virtual camera within a
360$^{\circ}$ video. Compared to the state of the art, our algorithm allows
more general camera control, avoids redundant outputs, and extracts its output
videos substantially more efficiently. Experimental results on over 7 hours of
real "in the wild" video show that our generalized camera control is crucial
for viewing 360$^{\circ}$ video, while the proposed efficient algorithm is
essential for making the generalized control computationally tractable.
