  Recently, several spatial-temporal memory-based methods have verified that
storing intermediate frames and their masks as memory are helpful to segment
target objects in videos. However, they mainly focus on better matching between
the current frame and the memory frames without explicitly paying attention to
the quality of the memory. Therefore, frames with poor segmentation masks are
prone to be memorized, which leads to a segmentation mask error accumulation
problem and further affect the segmentation performance. In addition, the
linear increase of memory frames with the growth of frame number also limits
the ability of the models to handle long videos. To this end, we propose a
Quality-aware Dynamic Memory Network (QDMN) to evaluate the segmentation
quality of each frame, allowing the memory bank to selectively store accurately
segmented frames to prevent the error accumulation problem. Then, we combine
the segmentation quality with temporal consistency to dynamically update the
memory bank to improve the practicability of the models. Without any bells and
whistles, our QDMN achieves new state-of-the-art performance on both DAVIS and
YouTube-VOS benchmarks. Moreover, extensive experiments demonstrate that the
proposed Quality Assessment Module (QAM) can be applied to memory-based methods
as generic plugins and significantly improves performance. Our source code is
available at https://github.com/workforai/QDMN.
