  When acquiring syntax, children consistently choose hierarchical rules over
competing non-hierarchical possibilities. Is this preference due to a learning
bias for hierarchical structure, or due to more general biases that interact
with hierarchical cues in children's linguistic input? We explore these
possibilities by training LSTMs and Transformers - two types of neural networks
without a hierarchical bias - on data similar in quantity and content to
children's linguistic input: text from the CHILDES corpus. We then evaluate
what these models have learned about English yes/no questions, a phenomenon for
which hierarchical structure is crucial. We find that, though they perform well
at capturing the surface statistics of child-directed speech (as measured by
perplexity), both model types generalize in a way more consistent with an
incorrect linear rule than the correct hierarchical rule. These results suggest
that human-like generalization from text alone requires stronger biases than
the general sequence-processing biases of standard neural network
architectures.
