  Deep networks have recently enjoyed enormous success when applied to
recognition and classification problems in computer vision, but their use in
graphics problems has been limited. In this work, we present a novel deep
architecture that performs new view synthesis directly from pixels, trained
from a large number of posed image sets. In contrast to traditional approaches
which consist of multiple complex stages of processing, each of which require
careful tuning and can fail in unexpected ways, our system is trained
end-to-end. The pixels from neighboring views of a scene are presented to the
network which then directly produces the pixels of the unseen view. The
benefits of our approach include generality (we only require posed image sets
and can easily apply our method to different domains), and high quality results
on traditionally difficult scenes. We believe this is due to the end-to-end
nature of our system which is able to plausibly generate pixels according to
color, depth, and texture priors learnt automatically from the training data.
To verify our method we show that it can convincingly reproduce known test
views from nearby imagery. Additionally we show images rendered from novel
viewpoints. To our knowledge, our work is the first to apply deep learning to
the problem of new view synthesis from sets of real-world, natural imagery.
