  Implicit knowledge, such as common sense, is key to fluid human
conversations. Current neural response generation (RG) models are trained to
generate responses directly, omitting unstated implicit knowledge. In this
paper, we present Think-Before-Speaking (TBS), a generative approach to first
externalize implicit commonsense knowledge (think) and use this knowledge to
generate responses (speak). We expect that externalizing implicit knowledge
allows more efficient learning, produces more informative responses, and
enables more explainable models. We analyze different choices to collect
knowledge-aligned dialogues, represent implicit knowledge, and transition
between knowledge and dialogues. Empirical results show TBS models outperform
end-to-end and knowledge-augmented RG baselines on most automatic metrics and
generate more informative, specific, and commonsense-following responses, as
evaluated by human annotators. TBS also generates knowledge that makes sense
and is relevant to the dialogue around 85\% of the time.
