  We study active object tracking, where a tracker takes as input the visual
observation (i.e., frame sequence) and produces the camera control signal
(e.g., move forward, turn left, etc.). Conventional methods tackle the tracking
and the camera control separately, which is challenging to tune jointly. It
also incurs many human efforts for labeling and many expensive trial-and-errors
in realworld. To address these issues, we propose, in this paper, an end-to-end
solution via deep reinforcement learning, where a ConvNet-LSTM function
approximator is adopted for the direct frame-toaction prediction. We further
propose an environment augmentation technique and a customized reward function,
which are crucial for a successful training. The tracker trained in simulators
(ViZDoom, Unreal Engine) shows good generalization in the case of unseen object
moving path, unseen object appearance, unseen background, and distracting
object. It can restore tracking when occasionally losing the target. With the
experiments over the VOT dataset, we also find that the tracking ability,
obtained solely from simulators, can potentially transfer to real-world
scenarios.
