  Semi-supervised video object segmentation (semi-VOS) is widely used in many
applications. This task is tracking class-agnostic objects from a given target
mask. For doing this, various approaches have been developed based on
online-learning, memory networks, and optical flow. These methods show high
accuracy but are hard to be utilized in real-world applications due to slow
inference time and tremendous complexity. To resolve this problem, template
matching methods are devised for fast processing speed but sacrificing lots of
performance in previous models. We introduce a novel semi-VOS model based on a
template matching method and a temporal consistency loss to reduce the
performance gap from heavy models while expediting inference time a lot. Our
template matching method consists of short-term and long-term matching. The
short-term matching enhances target object localization, while long-term
matching improves fine details and handles object shape-changing through the
newly proposed adaptive template attention module. However, the long-term
matching causes error-propagation due to the inflow of the past estimated
results when updating the template. To mitigate this problem, we also propose a
temporal consistency loss for better temporal coherence between neighboring
frames by adopting the concept of a transition matrix. Our model obtains 79.5%
J&F score at the speed of 73.8 FPS on the DAVIS16 benchmark. The code is
available in https://github.com/HYOJINPARK/TTVOS.
