  We present a method to reconstruct the three-dimensional trajectory of a
moving instance of a known object category in monocular video data. We track
the two-dimensional shape of objects on pixel level exploiting instance-aware
semantic segmentation techniques and optical flow cues. We apply Structure from
Motion techniques to object and background images to determine for each frame
camera poses relative to object instances and background structures. By
combining object and background camera pose information, we restrict the object
trajectory to a one-parameter family of possible solutions. We compute a ground
representation by fusing background structures and corresponding semantic
segmentations. This allows us to determine an object trajectory consistent to
image observations and reconstructed environment model. Our method is robust to
occlusion and handles temporarily stationary objects. We show qualitative
results using drone imagery. Due to the lack of suitable benchmark datasets we
present a new dataset to evaluate the quality of reconstructed
three-dimensional object trajectories. The video sequences contain vehicles in
urban areas and are rendered using the path-tracing render engine Cycles to
achieve realistic results. We perform a quantitative evaluation of the
presented approach using this dataset. Our algorithm achieves an average
reconstruction-to-ground-truth distance of 0.31 meter.
