  This work addresses the task of dense 3D reconstruction of a complex dynamic
scene from images. The prevailing idea to solve this task is composed of a
sequence of steps and is dependent on the success of several pipelines in its
execution. To overcome such limitations with the existing algorithm, we propose
a unified approach to solve this problem. We assume that a dynamic scene can be
approximated by numerous piecewise planar surfaces, where each planar surface
enjoys its own rigid motion, and the global change in the scene between two
frames is as-rigid-as-possible (ARAP). Consequently, our model of a dynamic
scene reduces to a soup of planar structures and rigid motion of these local
planar structures. Using planar over-segmentation of the scene, we reduce this
task to solving a "3D jigsaw puzzle" problem. Hence, the task boils down to
correctly assemble each rigid piece to construct a 3D shape that complies with
the geometry of the scene under the ARAP assumption. Further, we show that our
approach provides an effective solution to the inherent scale-ambiguity in
structure-from-motion under perspective projection. We provide extensive
experimental results and evaluation on several benchmark datasets. Quantitative
comparison with competing approaches shows state-of-the-art performance.
