  To what extent can a neural network systematically reason over symbolic
facts? Evidence suggests that large pre-trained language models (LMs) acquire
some reasoning capacity, but this ability is difficult to control. Recently, it
has been shown that Transformer-based models succeed in consistent reasoning
over explicit symbolic facts, under a "closed-world" assumption. However, in an
open-domain setup, it is desirable to tap into the vast reservoir of implicit
knowledge already encoded in the parameters of pre-trained LMs. In this work,
we provide a first demonstration that LMs can be trained to reliably perform
systematic reasoning combining both implicit, pre-trained knowledge and
explicit natural language statements. To do this, we describe a procedure for
automatically generating datasets that teach a model new reasoning skills, and
demonstrate that models learn to effectively perform inference which involves
implicit taxonomic and world knowledge, chaining and counting. Finally, we show
that "teaching" models to reason generalizes beyond the training distribution:
they successfully compose the usage of multiple reasoning skills in single
examples. Our work paves a path towards open-domain systems that constantly
improve by interacting with users who can instantly correct a model by adding
simple natural language statements.
