  We present ReCAT, a recursive composition augmented Transformer that is able
to explicitly model hierarchical syntactic structures of raw texts without
relying on gold trees during both learning and inference. Existing research
along this line restricts data to follow a hierarchical tree structure and thus
lacks inter-span communications. To overcome the problem, we propose a novel
contextual inside-outside (CIO) layer that learns contextualized
representations of spans through bottom-up and top-down passes, where a
bottom-up pass forms representations of high-level spans by composing low-level
spans, while a top-down pass combines information inside and outside a span. By
stacking several CIO layers between the embedding layer and the attention
layers in Transformer, the ReCAT model can perform both deep intra-span and
deep inter-span interactions, and thus generate multi-grained representations
fully contextualized with other spans. Moreover, the CIO layers can be jointly
pre-trained with Transformers, making ReCAT enjoy scaling ability, strong
performance, and interpretability at the same time. We conduct experiments on
various sentence-level and span-level tasks. Evaluation results indicate that
ReCAT can significantly outperform vanilla Transformer models on all span-level
tasks and baselines that combine recursive networks with Transformers on
natural language inference tasks. More interestingly, the hierarchical
structures induced by ReCAT exhibit strong consistency with human-annotated
syntactic trees, indicating good interpretability brought by the CIO layers.
