  Designing and generating new data under targeted properties has been
attracting various critical applications such as molecule design, image editing
and speech synthesis. Traditional hand-crafted approaches heavily rely on
expertise experience and intensive human efforts, yet still suffer from the
insufficiency of scientific knowledge and low throughput to support effective
and efficient data generation. Recently, the advancement of deep learning has
created the opportunity for expressive methods to learn the underlying
representation and properties of data. Such capability provides new ways of
determining the mutual relationship between the structural patterns and
functional properties of the data and leveraging such relationships to generate
structural data, given the desired properties. This article is a systematic
review that explains this promising research area, commonly known as
controllable deep data generation. First, the article raises the potential
challenges and provides preliminaries. Then the article formally defines
controllable deep data generation, proposes a taxonomy on various techniques
and summarizes the evaluation metrics in this specific domain. After that, the
article introduces exciting applications of controllable deep data generation,
experimentally analyzes and compares existing works. Finally, this article
highlights the promising future directions of controllable deep data generation
and identifies five potential challenges.
