  This paper proposes a hybrid neural network (HNN) model for commonsense
reasoning. An HNN consists of two component models, a masked language model and
a semantic similarity model, which share a BERT-based contextual encoder but
use different model-specific input and output layers. HNN obtains new
state-of-the-art results on three classic commonsense reasoning tasks, pushing
the WNLI benchmark to 89%, the Winograd Schema Challenge (WSC) benchmark to
75.1%, and the PDP60 benchmark to 90.0%. An ablation study shows that language
models and semantic similarity models are complementary approaches to
commonsense reasoning, and HNN effectively combines the strengths of both. The
code and pre-trained models will be publicly available at
https://github.com/namisan/mt-dnn.
