  Subject-driven image generation aims at generating images containing
customized subjects, which has recently drawn enormous attention from the
research community. However, the previous works cannot precisely control the
background and position of the target subject. In this work, we aspire to fill
the void and propose two novel subject-driven sub-tasks, i.e., Subject
Replacement and Subject Addition. The new tasks are challenging in multiple
aspects: replacing a subject with a customized one can change its shape,
texture, and color, while adding a target subject to a designated position in a
provided scene necessitates a context-aware posture. To conquer these two novel
tasks, we first manually curate a new dataset DreamEditBench containing 22
different types of subjects, and 440 source images with different difficulty
levels. We plan to host DreamEditBench as a platform and hire trained
evaluators for standard human evaluation. We also devise an innovative method
DreamEditor to resolve these tasks by performing iterative generation, which
enables a smooth adaptation to the customized subject. In this project, we
conduct automatic and human evaluations to understand the performance of
DreamEditor and baselines on DreamEditBench. For Subject Replacement, we found
that the existing models are sensitive to the shape and color of the original
subject. The model failure rate will dramatically increase when the source and
target subjects are highly different. For Subject Addition, we found that the
existing models cannot easily blend the customized subjects into the background
smoothly, leading to noticeable artifacts in the generated image. We hope
DreamEditBench can become a standard platform to enable future investigations
toward building more controllable subject-driven image editing. Our project
homepage is https://dreameditbenchteam.github.io/.
