  Arbitrary Style Transfer (AST) aims to transform images by adopting the style
from any selected artwork. Nonetheless, the need to accommodate diverse and
subjective user preferences poses a significant challenge. While some users
wish to preserve distinct content structures, others might favor a more
pronounced stylization. Despite advances in feed-forward AST methods, their
limited customizability hinders their practical application. We propose a new
approach, ArtFusion, which provides a flexible balance between content and
style. In contrast to traditional methods reliant on biased similarity losses,
ArtFusion utilizes our innovative Dual Conditional Latent Diffusion
Probabilistic Models (Dual-cLDM). This approach mitigates repetitive patterns
and enhances subtle artistic aspects like brush strokes and genre-specific
features. Despite the promising results of conditional diffusion probabilistic
models (cDM) in various generative tasks, their introduction to style transfer
is challenging due to the requirement for paired training data. ArtFusion
successfully navigates this issue, offering more practical and controllable
stylization. A key element of our approach involves using a single image for
both content and style during model training, all the while maintaining
effective stylization during inference. ArtFusion outperforms existing
approaches on outstanding controllability and faithful presentation of artistic
details, providing evidence of its superior style transfer capabilities.
Furthermore, the Dual-cLDM utilized in ArtFusion carries the potential for a
variety of complex multi-condition generative tasks, thus greatly broadening
the impact of our research.
