  Large pretrained language models can easily produce toxic or biased content,
which is prohibitive for practical use. In order to detect such toxic
generations, existing methods rely on templates, real-world data extraction,
crowdsourcing workers, or automatic generation to construct adversarial
contexts that are likely to induce toxic generations. However, what type of
context is more likely to induce unsafe responses is still under-explored. In
this paper, we identify that context toxicity and context category (e.g.,
\textit{profanity}, \textit{insult}, \textit{drugs}, etc.) are two important
factors to cause safety issues in response generation. Hence, we propose a
method called \emph{reverse generation} to construct adversarial contexts
conditioned on a given response, with the flexibility to control category,
toxicity level, and inductivity of the generated contexts. Via reverse
generation, we augment the existing BAD dataset and construct a new dataset
BAD+ which contains more than 120K diverse and highly inductive contexts in 12
categories. We test three popular pretrained dialogue models (Blender,
DialoGPT, and Plato2) and find that BAD+ can largely expose their safety
problems. Furthermore, we show that BAD+ can greatly enhance the safety of
generation and reveal the key factors of safety improvement. Our code and
dataset is available at \url{https://github.com/thu-coai/Reverse_Generation}.
