  Despite the success of neural models in solving reasoning tasks, their
compositional generalization capabilities remain unclear. In this work, we
propose a new setting of the structured explanation generation task to
facilitate compositional reasoning research. Previous works found that symbolic
methods achieve superior compositionality by using pre-defined inference rules
for iterative reasoning. But these approaches rely on brittle symbolic
transfers and are restricted to well-defined tasks. Hence, we propose a dynamic
modularized reasoning model, MORSE, to improve the compositional generalization
of neural models. MORSE factorizes the inference process into a combination of
modules, where each module represents a functional unit. Specifically, we adopt
modularized self-attention to dynamically select and route inputs to dedicated
heads, which specializes them to specific functions. We conduct experiments for
increasing lengths and shapes of reasoning trees on two benchmarks to test
MORSE's compositional generalization abilities, and find it outperforms
competitive baselines. Model ablation and deeper analyses show the
effectiveness of dynamic reasoning modules and their generalization abilities.
