  Deep neural networks, empowered by pre-trained language models, have achieved
remarkable results in natural language understanding (NLU) tasks. However,
their performances can drastically deteriorate when logical reasoning is
needed. This is because NLU in principle depends on not only analogical
reasoning, which deep neural networks are good at, but also logical reasoning.
According to the dual-process theory, analogical reasoning and logical
reasoning are respectively carried out by System 1 and System 2 in the human
brain. Inspired by the theory, we present a novel framework for NLU called
Neural-Symbolic Processor (NSP), which performs analogical reasoning based on
neural processing and logical reasoning based on both neural and symbolic
processing. As a case study, we conduct experiments on two NLU tasks, question
answering (QA) and natural language inference (NLI), when numerical reasoning
(a type of logical reasoning) is necessary. The experimental results show that
our method significantly outperforms state-of-the-art methods in both tasks.
