  We introduce a new task, Contextual Text Style Transfer - translating a
sentence into a desired style with its surrounding context taken into account.
This brings two key challenges to existing style transfer approaches: ($i$) how
to preserve the semantic meaning of target sentence and its consistency with
surrounding context during transfer; ($ii$) how to train a robust model with
limited labeled data accompanied with context. To realize high-quality style
transfer with natural context preservation, we propose a Context-Aware Style
Transfer (CAST) model, which uses two separate encoders for each input sentence
and its surrounding context. A classifier is further trained to ensure
contextual consistency of the generated sentence. To compensate for the lack of
parallel data, additional self-reconstruction and back-translation losses are
introduced to leverage non-parallel data in a semi-supervised fashion. Two new
benchmarks, Enron-Context and Reddit-Context, are introduced for formality and
offensiveness style transfer. Experimental results on these datasets
demonstrate the effectiveness of the proposed CAST model over state-of-the-art
methods across style accuracy, content preservation and contextual consistency
metrics.
