  Arbitrary style transfer (AST) transfers arbitrary artistic styles onto
content images. Despite the recent rapid progress, existing AST methods are
either incapable or too slow to run at ultra-resolutions (e.g., 4K) with
limited resources, which heavily hinders their further applications. In this
paper, we tackle this dilemma by learning a straightforward and lightweight
model, dubbed MicroAST. The key insight is to completely abandon the use of
cumbersome pre-trained Deep Convolutional Neural Networks (e.g., VGG) at
inference. Instead, we design two micro encoders (content and style encoders)
and one micro decoder for style transfer. The content encoder aims at
extracting the main structure of the content image. The style encoder, coupled
with a modulator, encodes the style image into learnable dual-modulation
signals that modulate both intermediate features and convolutional filters of
the decoder, thus injecting more sophisticated and flexible style signals to
guide the stylizations. In addition, to boost the ability of the style encoder
to extract more distinct and representative style signals, we also introduce a
new style signal contrastive loss in our model. Compared to the state of the
art, our MicroAST not only produces visually superior results but also is 5-73
times smaller and 6-18 times faster, for the first time enabling super-fast
(about 0.5 seconds) AST at 4K ultra-resolutions. Code is available at
https://github.com/EndyWon/MicroAST.
