  Coherence is one of the critical factors that determine the quality of
writing. We propose writing relevance (WR) training method for neural
encoder-decoder natural language generation (NLG) models which improves
coherence of the continuation by leveraging negative examples. WR loss
regresses the vector representation of the context and generated sentence
toward positive continuation by contrasting it with the negatives. We compare
our approach with Unlikelihood (UL) training in a text continuation task on
commonsense natural language inference (NLI) corpora to show which method
better models the coherence by avoiding unlikely continuations. The preference
of our approach in human evaluation shows the efficacy of our method in
improving coherence.
