  We present a novel approach to the problem of text style transfer. Unlike
previous approaches requiring style-labeled training data, our method makes use
of readily-available unlabeled text by relying on the implicit connection in
style between adjacent sentences, and uses labeled data only at inference time.
We adapt T5 (Raffel et al., 2020), a strong pretrained text-to-text model, to
extract a style vector from text and use it to condition the decoder to perform
style transfer. As our label-free training results in a style vector space
encoding many facets of style, we recast transfers as "targeted restyling"
vector operations that adjust specific attributes of the input while preserving
others. We demonstrate that training on unlabeled Amazon reviews data results
in a model that is competitive on sentiment transfer, even compared to models
trained fully on labeled data. Furthermore, applying our novel method to a
diverse corpus of unlabeled web text results in a single model capable of
transferring along multiple dimensions of style (dialect, emotiveness,
formality, politeness, sentiment) despite no additional training and using only
a handful of exemplars at inference time.
