  Languages models have been successfully applied to a variety of reasoning
tasks in NLP, yet the language models still suffer from compositional
generalization. In this paper we present Explainable Verbal Reasoner Plus
(EVR+), a reasoning framework that enhances language models' compositional
reasoning ability by (1) allowing the model to explicitly generate and execute
symbolic operators, and (2) allowing the model to decompose a complex task into
several simpler ones in a flexible manner. Compared with its predecessor
Explainable Verbal Reasoner (EVR) and other previous approaches adopting
similar ideas, our framework supports more diverse types of reasoning such as
nested loops and different types of recursion. To evaluate our reasoning
framework, we build a synthetic dataset with five tasks that require
compositional reasoning. Results show that our reasoning framework can enhance
the language model's compositional generalization performance on the five
tasks, using a fine-tuned language model. We also discussed the possibility and
the challenges to combine our reasoning framework with a few-shot prompted
language model.
