  Occlusion is one of the most difficult challenges in object tracking to
model. This is because unlike other challenges, where data augmentation can be
of help, occlusion is hard to simulate as the occluding object can be anything
in any shape. In this paper, we propose a simple solution to simulate the
effects of occlusion in the latent space. Specifically, we present structured
dropout to mimick the change in latent codes under occlusion. We present three
forms of dropout (channel dropout, segment dropout and slice dropout) with the
various forms of occlusion in mind. To demonstrate its effectiveness, the
dropouts are incorporated into two modern Siamese trackers (SiamFC and
SiamRPN++). The outputs from multiple dropouts are combined using an encoder
network to obtain the final prediction. Experiments on several tracking
benchmarks show the benefits of structured dropouts, while due to their
simplicity requiring only small changes to the existing tracker models.
