  Hairstyle transfer is challenging due to hair structure differences in the
source and target hair. Therefore, we propose Latent Optimization of Hairstyles
via Orthogonalization (LOHO), an optimization-based approach using GAN
inversion to infill missing hair structure details in latent space during
hairstyle transfer. Our approach decomposes hair into three attributes:
perceptual structure, appearance, and style, and includes tailored losses to
model each of these attributes independently. Furthermore, we propose two-stage
optimization and gradient orthogonalization to enable disentangled latent space
optimization of our hair attributes. Using LOHO for latent space manipulation,
users can synthesize novel photorealistic images by manipulating hair
attributes either individually or jointly, transferring the desired attributes
from reference hairstyles. LOHO achieves a superior FID compared with the
current state-of-the-art (SOTA) for hairstyle transfer. Additionally, LOHO
preserves the subject's identity comparably well according to PSNR and SSIM
when compared to SOTA image embedding pipelines. Code is available at
https://github.com/dukebw/LOHO.
