  We propose an approach for reconstructing free-moving object from a monocular
RGB video. Most existing methods either assume scene prior, hand pose prior,
object category pose prior, or rely on local optimization with multiple
sequence segments. We propose a method that allows free interaction with the
object in front of a moving camera without relying on any prior, and optimizes
the sequence globally without any segments. We progressively optimize the
object shape and pose simultaneously based on an implicit neural
representation. A key aspect of our method is a virtual camera system that
reduces the search space of the optimization significantly. We evaluate our
method on the standard HO3D dataset and a collection of egocentric RGB
sequences captured with a head-mounted device. We demonstrate that our approach
outperforms most methods significantly, and is on par with recent techniques
that assume prior information.
