  In modern video coding standards, block-based inter prediction is widely
adopted, which brings high compression efficiency. However, in natural videos,
there are usually multiple moving objects of arbitrary shapes, resulting in
complex motion fields that are difficult to compactly represent. This problem
has been tackled by more flexible block partitioning methods in the Versatile
Video Coding (VVC) standard, but the more flexible partitions require more
overhead bits to signal and still cannot be made arbitrary shaped. To address
this limitation, we propose an object segmentation-assisted inter prediction
method (SAIP), where objects in the reference frames are segmented by some
advanced technologies. With a proper indication, the object segmentation mask
is translated from the reference frame to the current frame as the
arbitrary-shaped partition of different regions without any extra signal. Using
the segmentation mask, motion compensation is separately performed for
different regions, achieving higher prediction accuracy. The segmentation mask
is further used to code the motion vectors of different regions more
efficiently. Moreover, segmentation mask is considered in the joint
rate-distortion optimization for motion estimation and partition estimation to
derive the motion vector of different regions and partition more accurately.
The proposed method is implemented into the VVC reference software, VTM version
12.0. Experimental results show that the proposed method achieves up to 1.98%,
1.14%, 0.79%, and on average 0.82%, 0.49%, 0.37% BD-rate reduction for common
test sequences, under the Low-delay P, Low-delay B, and Random Access
configurations, respectively.
