  Large-scale sequence-to-sequence models have shown to be adept at both
multiple-choice and open-domain commonsense reasoning tasks. However, the
current systems do not provide the ability to control the various attributes of
the reasoning chain. To enable better controllability, we propose to study the
commonsense reasoning as a template filling task (TemplateCSR) -- where the
language models fills reasoning templates with the given constraints as control
factors. As an approach to TemplateCSR, we (i) propose a dataset of commonsense
reasoning template-expansion pairs and (ii) introduce POTTER, a pretrained
sequence-to-sequence model using prompts to perform commonsense reasoning
across concepts. Our experiments show that our approach outperforms baselines
both in generation metrics and factuality metrics. We also present a detailed
error analysis on our approach's ability to reliably perform commonsense
reasoning.
