  Neural language models have achieved state-of-the-art performances on many
NLP tasks, and recently have been shown to learn a number of
hierarchically-sensitive syntactic dependencies between individual words.
However, equally important for language processing is the ability to combine
words into phrasal constituents, and use constituent-level features to drive
downstream expectations. Here we investigate neural models' ability to
represent constituent-level features, using coordinated noun phrases as a case
study. We assess whether different neural language models trained on English
and French represent phrase-level number and gender features, and use those
features to drive downstream expectations. Our results suggest that models use
a linear combination of NP constituent number to drive CoordNP/verb number
agreement. This behavior is highly regular and even sensitive to local
syntactic context, however it differs crucially from observed human behavior.
Models have less success with gender agreement. Models trained on large corpora
perform best, and there is no obvious advantage for models trained using
explicit syntactic supervision.
