  Neural dialog models are known to suffer from problems such as generating
unsafe and inconsistent responses. Even though these problems are crucial and
prevalent, they are mostly manually identified by model designers through
interactions. Recently, some research instructs crowdworkers to goad the bots
into triggering such problems. However, humans leverage superficial clues such
as hate speech, while leaving systematic problems undercover. In this paper, we
propose two methods including reinforcement learning to automatically trigger a
dialog model into generating problematic responses. We show the effect of our
methods in exposing safety and contradiction issues with state-of-the-art
dialog models.
