  Stylized text-to-image generation focuses on creating images from textual
descriptions while adhering to a style specified by a few reference images.
However, subtle style variations within different reference images can hinder
the model from accurately learning the target style. In this paper, we propose
InstaStyle, a novel approach that excels in generating high-fidelity stylized
images with only a single reference image. Our approach is based on the finding
that the inversion noise from a stylized reference image inherently carries the
style signal, as evidenced by their non-zero signal-to-noise ratio. We employ
DDIM inversion to extract this noise from the reference image and leverage a
diffusion model to generate new stylized images from the "style" noise.
Additionally, the inherent ambiguity and bias of textual prompts impede the
precise conveying of style. To address this, we introduce a learnable style
token via prompt refinement, which enhances the accuracy of the style
description for the reference image. Qualitative and quantitative experimental
results demonstrate that InstaStyle achieves superior performance compared to
current benchmarks. Furthermore, our approach also showcases its capability in
the creative task of style combination with mixed inversion noise.
