  Previous works on image inpainting mainly focus on inpainting background or
partially missing objects, while the problem of inpainting an entire missing
object remains unexplored. This work studies a new image inpainting task, i.e.
shape-guided object inpainting. Given an incomplete input image, the goal is to
fill in the hole by generating an object based on the context and implicit
guidance given by the hole shape. Since previous methods for image inpainting
are mainly designed for background inpainting, they are not suitable for this
task. Therefore, we propose a new data preparation method and a novel
Contextual Object Generator (CogNet) for the object inpainting task. On the
data side, we incorporate object priors into training data by using object
instances as holes. The CogNet has a two-stream architecture that combines the
standard bottom-up image completion process with a top-down object generation
process. A predictive class embedding module bridges the two streams by
predicting the class of the missing object from the bottom-up features, from
which a semantic object map is derived as the input of the top-down stream.
Experiments demonstrate that the proposed method can generate realistic objects
that fit the context in terms of both visual appearance and semantic meanings.
Code can be found at the project page
\url{https://zengxianyu.github.io/objpaint}
