  We target the problem of sparse 3D reconstruction of dynamic objects observed
by multiple unsynchronized video cameras with unknown temporal overlap. To this
end, we develop a framework to recover the unknown structure without sequencing
information across video sequences. Our proposed compressed sensing framework
poses the estimation of 3D structure as the problem of dictionary learning,
where the dictionary is defined as an aggregation of the temporally varying 3D
structures. Given the smooth motion of dynamic objects, we observe any element
in the dictionary can be well approximated by a sparse linear combination of
other elements in the same dictionary (i. e. self-expression). Moreover, the
sparse coefficients describing a locally linear 3D structural interpolation
reveal the local sequencing information. Our formulation optimizes a biconvex
cost function that leverages a compressed sensing formulation and enforces both
structural dependency coherence across video streams, as well as motion
smoothness across estimates from common video sources. We further analyze the
reconstructability of our approach under different capture scenarios, and its
comparison and relation to existing methods. Experimental results on large
amounts of synthetic data as well as real imagery demonstrate the effectiveness
of our approach.
