  Answering complex questions often requires multi-step reasoning in order to
obtain the final answer. Most research into decompositions of complex questions
involves open-domain systems, which have shown success in using these
decompositions for improved retrieval. In the machine reading setting, however,
work to understand when decompositions are helpful is understudied. We conduct
experiments on decompositions in machine reading to unify recent work in this
space, using a range of models and datasets. We find that decompositions can be
helpful in the few-shot case, giving several points of improvement in exact
match scores. However, we also show that when models are given access to
datasets with around a few hundred or more examples, decompositions are not
helpful (and can actually be detrimental). Thus, our analysis implies that
models can learn decompositions implicitly even with limited data.
