  This paper focuses on developing a more effective method of hierarchical
propagation for semi-supervised Video Object Segmentation (VOS). Based on
vision transformers, the recently-developed Associating Objects with
Transformers (AOT) approach introduces hierarchical propagation into VOS and
has shown promising results. The hierarchical propagation can gradually
propagate information from past frames to the current frame and transfer the
current frame feature from object-agnostic to object-specific. However, the
increase of object-specific information will inevitably lead to the loss of
object-agnostic visual information in deep propagation layers. To solve such a
problem and further facilitate the learning of visual embeddings, this paper
proposes a Decoupling Features in Hierarchical Propagation (DeAOT) approach.
Firstly, DeAOT decouples the hierarchical propagation of object-agnostic and
object-specific embeddings by handling them in two independent branches.
Secondly, to compensate for the additional computation from dual-branch
propagation, we propose an efficient module for constructing hierarchical
propagation, i.e., Gated Propagation Module, which is carefully designed with
single-head attention. Extensive experiments show that DeAOT significantly
outperforms AOT in both accuracy and efficiency. On YouTube-VOS, DeAOT can
achieve 86.0% at 22.4fps and 82.0% at 53.4fps. Without test-time augmentations,
we achieve new state-of-the-art performance on four benchmarks, i.e.,
YouTube-VOS (86.2%), DAVIS 2017 (86.2%), DAVIS 2016 (92.9%), and VOT 2020
(0.622). Project page: https://github.com/z-x-yang/AOT.
