  Photorealistic style transfer entails transferring the style of a reference
image to another image so the result seems like a plausible photo. Our work is
inspired by the observation that existing models are slow due to their large
sizes. We introduce PCA-based knowledge distillation to distill lightweight
models and show it is motivated by theory. To our knowledge, this is the first
knowledge distillation method for photorealistic style transfer. Our
experiments demonstrate its versatility for use with different backbone
architectures, VGG and MobileNet, across six image resolutions. Compared to
existing models, our top-performing model runs at speeds 5-20x faster using at
most 1\% of the parameters. Additionally, our distilled models achieve a better
balance between stylization strength and content preservation than existing
models. To support reproducing our method and models, we share the code at
\textit{https://github.com/chiutaiyin/PCA-Knowledge-Distillation}.
