  Generating synthetic datasets for training face recognition models is
challenging because dataset generation entails more than creating high fidelity
images. It involves generating multiple images of same subjects under different
factors (\textit{e.g.}, variations in pose, illumination, expression, aging and
occlusion) which follows the real image conditional distribution. Previous
works have studied the generation of synthetic datasets using GAN or 3D models.
In this work, we approach the problem from the aspect of combining subject
appearance (ID) and external factor (style) conditions. These two conditions
provide a direct way to control the inter-class and intra-class variations. To
this end, we propose a Dual Condition Face Generator (DCFace) based on a
diffusion model. Our novel Patch-wise style extractor and Time-step dependent
ID loss enables DCFace to consistently produce face images of the same subject
under different styles with precise control. Face recognition models trained on
synthetic images from the proposed DCFace provide higher verification
accuracies compared to previous works by $6.11\%$ on average in $4$ out of $5$
test datasets, LFW, CFP-FP, CPLFW, AgeDB and CALFW. Code is available at
https://github.com/mk-minchul/dcface
